Read This First!
The Talos Principle has not fully optimized its Vulkan integration. The inclusion of the API is presently in early stages. As the Talos Principle team has indicated, Dx9 performance used to outrank Dx11 performance within the game, but later optimizations resolved and switched that performance. The team expects the same to happen with Vulkan, noting in its FAQ:
“Q: Then I can expect some great benchmark results?
A: Not so fast! Sorry. :( This is a beta phase, and there's still work to be done from our (Croteam) side and IHVs from driver side. But please be patient: in time, after optimizations are done, Vulkan will be faster, I'm sure.”
Our benchmarks then will serve as an interesting means to plot the historic progression of the Vulkan API's performance. As of now, performance will be lower than DirectX 11 – but updates to drivers and the game should eventually force a switch-over, something we're all too excited to plot as IHVs and ISVs roll-out those updates.
The results below are from limited scenario runs where we tested only two hardware configurations, both running the 5930K CPU. As Vulkan receives its updates from Croteam and the IHVs, we'll begin benchmarking lower-end CPUs (Core i3, for instance) to determine the impact from off-loaded draw calls. DirectX 12 and Vulkan should both aid low-end and mid-range CPUs more directly than the GPU, in most scenarios, by sending draw call processing to the parallel processors on the GPU.
For now, we've got an initial look at Vulkan benchmark / FPS performance using The Talos Principle's benchmark tool.
We tested using our X99 test bench, detailed in the table below. Our thanks to supporting hardware vendors for supplying some of the test components.
The latest AMD drivers (16.15.1009 Beta 4) were used for testing. NVidia's 356.39 Vulkan beta drivers were used for testing. Game settings were manually controlled for the DUT. All games were run at presets defined in their respective charts. We disable brand-supported technologies in games, like The Witcher 3's HairWorks and HBAO. The Talos Principle is presently the only Vulkan-supported game; we used the built-in benchmark for automated test execution over multiple repeated 30-second test passes. This allows others to replicate our results by studying our bench courses.
Our usual test recording software does not work with Vulkan. For Vulkan testing, we used the built-in FPS logging utilities which report framerates somewhat differently, so rather than the usual “1% LOW” and “0.1% LOW” read-outs, we just have “lows,” which are somewhat ambiguously defined by the built-in benchmark. Lows will represent an average of the worst frame outputs over the course of benchmarking.
|GN Test Bench 2015||Name||Courtesy Of||Cost|
This is what we're testing!
|CPU||Intel i7-5930K CPU||iBUYPOWER
|Memory||Kingston 16GB DDR4 Predator||Kingston Tech.||$245|
|Motherboard||EVGA X99 Classified||GamersNexus||$365|
|Power Supply||NZXT 1200W HALE90 V2||NZXT||$300|
|SSD||HyperX Savage SSD||Kingston Tech.||$130|
|Case||Top Deck Tech Station||GamersNexus||$250|
|CPU Cooler||NZXT Kraken X41 CLC||NZXT||$110|
Vulkan vs. DirectX 11 Benchmark – 1080p, Ultra
At 1080p, Ultra, we observed a performance disparity of 57.89% for the R9 390X, favoring D3D11. That's a significant slow-down on Vulkan, but one which Croteam feels will eventually close and switch as optimizations occur. 75.5FPS for Vulkan and 113.3FPS (averaged over three passes) for DirectX 11 amount to our averages, with the low performance at 41.6FPS vs. 88.1FPS.
For the 980 Ti, performance was marginally closer with a 44.47% disparity, again favoring DirectX 11. The GTX 980 Ti at 1080, Ultra pushed 90.7FPS averages with a 57.7FPS low on Vulkan, or 127.2FPS average and 98.3FPS low on D3D11.
Between the two cards, there's absolutely no reason to run Vulkan on Windows right now – and we'd imagine that scales downward across all configurations, but need to perform further testing for validation. It's the low-end CPUs that should be most interesting once these drivers and software patches get pushed.
Vulkan vs. DirectX 11 FPS Benchmark – 4K Ultra
The next two charts use the same configurations, but run a 4K resolution with a max MP/s throughput of 8.7.
AMD's R9 390X shows a performance disparity of 47.8% between Vulkan and D3D11, favoring Microsoft's established API. The low performance is fairly tightly timed with the averages, at 22.1FPS (low) / 29.9FPS (AVG) vs. 39.2FPS (low) / 48.7FPS (AVG).
NVidia fronts a 31.67% performance disparity between Vulkan and D3D11, favoring DirectX 11. That's a significant lead over AMD in FPS deltas at 4K; we're curious to see whether nVidia can maintain its present delta advantage (between Vulkan and D3D11, not between the 980 Ti and 390X) as the API matures.
Conclusion: Vulkan's Getting There
Vulkan, or “OpenGL Next,” is still young. It's not ready yet. For Windows users, there's currently no reason to play The Talos Principle using the Vulkan API unless the user is a tester or developer of some kind. Vulkan theoretically opens some doors on other operating systems, like Linux, and we may begin investigating a Linux testing platform in the near future.
For now, it looks like it's a game of “wait and see.” Croteam seems to feel confident that Vulkan will overtake D3D11 performance as updates roll-out – and that'd make sense: We're not talking DirectX 12, here, we're talking DirectX 11, an aged, more restrictive API.
We'll follow-up with more benchmarks as the updates roll-out. It should be interesting to see these >40% gaps close, in the cases where they exist.
- Steve “Lelldorianx” Burke.