Testing Thoughts & Procedures
For this testing, we’re operating with a 1050MHz fixed clock on both the Fury X (stock) and Vega: FE (downclocked from ~1440-1600 operating clock). The voltage has been controlled to 1250mv for each card, hopefully eliminating a Vcore limitation, with power target set to +50% on each card (eliminating power starving). Fans on the Vega: FE card were set to a somewhat intolerable ~60-65dBA spin-rate to ensure thermal throttling did not blur the results.
There are architectural differences that we cannot control for, though, so keep that in mind; this is not a perfect test, but an interesting exercise. Of note, the small primitives discarder on Vega: FE should come into play when tessellation is heavier, and we might finally see that in action with these tests.
As for memory, we’re clocked at 500MHz on the Fury X (stock) and still keeping 945MHz on Vega: FE. Vega: FE has two stacks of HBM2 versus the Fury X’s four stacks of HBM1, where the Fury X has double the bus width as the Vega: FE card. This leads to fairly close memory bandwidth, but not perfectly equal: We’re at ~484GB/s on the Vega: FE card and ~512GB/s on the Fury X. For now, we’ve decided to allow this variable to exist within testing. There are a few reasons for that: One, this will help us determine if memory bandwidth could potentially be a factor in some gaming results, after which point we or someone else could investigate further; two, software is still developing for Vega, and we have observed some issues when playing around with various OC settings. We’re trying to limit the latter from interfering. Memory also is part of this "IPC" calculation, anyway, so we're leaving it alone.
Downclocking was done with Afterburner, which does not have the same bug as WattMan does.
Test platforms below:
|GN Test Bench 2017||Name||Courtesy Of||Cost|
|Video Card||This is what we're testing||-||-|
|CPU||Intel i7-7700K 4.5GHz locked||GamersNexus||$330|
|Memory||GSkill Trident Z 3200MHz C14||Gskill||-|
|Motherboard||Gigabyte Aorus Gaming 7 Z270X||Gigabyte||$240|
|Power Supply||NZXT 1200W HALE90 V2||NZXT||$300|
|Case||Top Deck Tech Station||GamersNexus||$250|
|CPU Cooler||Asetek 570LC||Asetek||-|
BIOS settings include C-states completely disabled with the CPU locked to 4.5GHz at 1.32 vCore. Memory is at XMP1.
We communicated with both AMD and nVidia about the new titles on the bench, and gave each company the opportunity to ‘vote’ for a title they’d like to see us add. We figure this will help even out some of the game biases that exist. AMD doesn’t make a big showing today, but will soon. We are testing:
- Ghost Recon: Wildlands (built-in bench, Very High; recommended by nVidia)
- Sniper Elite 4 (High, Async, Dx12; recommended by AMD)
- For Honor (Extreme, manual bench as built-in is unrealistically abusive)
- Ashes of the Singularity (GPU-focused, High, Dx12)
- DOOM (Vulkan, Ultra, 0xAA, Async)
- 3DMark FireStrike
- 3DMark FireStrike Extreme
- 3DMark FireStrike Ultra
- 3DMark TimeSpy
For measurement tools, we’re using PresentMon for Dx12/Vulkan titles and FRAPS for Dx11 titles. OnPresent is the preferred output for us, which is then fed through our own script to calculate 1% low and 0.1% low metrics (defined here).
Power testing is taken at the wall. One case fan is connected, both SSDs, and the system is otherwise left in the "Game Bench" configuration.
For production testing, we are using an i9-7900X & ASUS X299 Prime Deluxe motherboard with 4x8GB GSkill Trident Z Black memory at 3600MHz with an Enermax Platimax 1350W PSU.
FireStrike Ultra – Fury X vs. Vega: FE “IPC” Test
Let’s start with some synthetics to create a baseline, then move on to gaming benchmarks. We scripted 3DMark to execute 5 times on each configuration, as there’s quite some variance in 3DMark, with the software closed between each run. The results are averaged.
With FireStrike Ultra, the stock Radeon Vega: Frontier Edition card operated a graphics score of 4906 points – that’s our baseline. The R9 Fury X with 1050MHz averaged a graphics score of 3974.7 after several runs, with a range of 21 points across all runs. The Vega: FE card at 1050MHz averaged a graphics score of 3889 points, with a range of 14 points across all runs. The difference is 2.2%, favoring the AMD R9 Fury X. This could be a difference of memory bandwidth, at this point. It is hard to say the exact difference. We are confident, however, that this difference is outside of test variance and error – there is about a 2% advantage on the Fury X under our test conditions. If you’re curious, this scaling would indicate that the stock clocked Vega: Frontier Edition is about 28% faster than the stock clocked Fury X.
In our 2015 review of the Fury X, we noted some interesting, non-linear scaling across lower resolutions. Let’s look at FireStrike’s normal and extreme benchmarks to see how that affects things.
FireStrike Extreme – Fury X vs. Vega: FE Clock for Clock
In FireStrike Extreme, the Fury X again is roughly tied or marginally ahead of the Vega 1050MHz card. The lead held by the Fury X cannot be confidently declared as a consistent advantage test to test; the variance is high enough here that we must call these results “effectively identical.” In our original Fury X review, the card closed the gap on some competitors when getting closer to 4K resolutions – we’re seeing some of that here, where the Fury X held a notable and measurable lead of 2.2% at 4K, but no significant difference at 1440p. Let’s see if that continues at 1080p.
In FireStrike’s normal benchmark, the Vega: FE Stock card ran a graphics score of 21355, for our baseline. The R9 Fury X operated a score of 16531.7, with a range of 85 points. Vega operated a score of 16749.3 points, with a range 37 points. The tables have now flipped – the Fury X is now behind by 1.3%, where it was previously tied or ahead. Let’s take a closer look at the individual tests to better understand this performance behavior.
Why We Think This Happens
Here’s the important data: There are two graphics runs in the FireStrike benchmarks – GT1 and GT2. The GT1 benchmark heavily loads the GPU with polys and tessellation, but doesn’t really apply a compute load. GT2 increases compute task workload and can potentially stress memory more.
If you remember our older Polaris architecture discussion, Polaris introduced a small primitive discarder that helps cull polys and quads that the card can get away with removing. This is something that Fury X did not have and, theoretically, the Fury X would be worse in performance regarding tessellation or geometrically complex scenes.
FireStrike FPS Benchmarks – GT1 vs. GT2
Looking at the results for FPS1 and FPS2 in the FireStrike 1080p test, we see that Vega is notably ahead in GT1 – roughly 4.5% -- and about 1.3% behind in GT2. The other tests show the same trend, but with lessened differences at higher resolutions. The FireStrike 4K test, for example, posted Vega: FE in the lead consistently at around 20.41FPS for GT1, but behind consistently at around 14.44FPS for GT2. The behavior is mirrored to 1080p, but the difference is lessened as GPU load increases.
This is interesting data, but we can presently only hypothesize as to what it means. Our educated guess right now is that Vega: FE’s newer geometry and tessellation culling pipeline benefits Vega: FE in GT1, as it is a high polycount benchmark that would theoretically utilize the primitives discarder. Vega: FE should be culling more unnecessary primitives quicker. As for GT2, we’re not positive what that difference derives from – it could be a memory bandwidth benefit for the Fury X, but this could also be an architectural difference.
This is the first step to better understanding Vega’s performance. It looks to do well in high poly, heavily tessellated scenarios, but perhaps not significantly better overall. It’s best if we move on to gaming benchmarks next – we’ll cover production last.
Tessellation Scaling: Fury X vs. Vega FE (Metro: Last Light)
Let’s resurrect a benchmark from another era. Looking at Metro: Last Light gives us more granular control over tessellation specifically, so we’ll have a line plot showing trend of performance in a moment – but we’re starting with the staple benchmarks.
With 4K and Very High quality settings supported by High tessellation, the R9 Fury X operates an AVG FPS of 43, lows at 33FPS and 27.7FPS 0.1%. The Vega: FE 1050MHz card sustains a 39.3FPS AVG, with comparably timed lows to the AVG. This places the Fury X about 9% ahead.
At 1440p with the same settings, the Fury X now runs a 78.3FPS AVG against 1050MHz Vega’s 74.3FPS AVG – so the gap has closed a bit, with the Fury X now 5.4% ahead rather than 9% ahead.
Finally, with 1080p at VH/H, the Fury X operates a 111.7FPS AVG, with the 1050MHz Vega variant at ~112FPS AVG. The cards effectively equalize at this lower resolution.
Let’s plot a performance trendline with all the different tessellation settings, starting with AVG FPS. This is where it gets interesting. Here, with AVG FPS plotted for 1080p, we see that the Fury X retains a stronger lead with tessellation disabled, but loses that lead as tessellation becomes more demanding – ultimately giving way to a very slight bump at the end for Vega (this bump is within STDEV, so we must call it ‘effectively equal’ at this point). If we could further increase tessellation settings in this benchmark, we’d suspect this trend would continue. Normal tessellation starts to close the gap, but not much, and “High” tessellation is the turning point.
Standard deviation for these tests was 0.577FPS.
0.1% low metrics follow mostly the same progression, as seen in this line graph. High is again the turning point.
DOOM – Fury X vs. Vega FE Clock for Clock
Moving to DOOM at 4K/Ultra with Vulkan and Async Compute, the stock Vega: FE card operates an AVG FPS of 64.2, with the Fury X at 58.4FPS AVG and the Vega: FE card at around 57.5FPS AVG. DOOM isn’t an intensive enough game to test at lower resolutions given its FPS cap, so we’ll save that for some of the other titles.
Ghost Recon: Wildlands – Fury X vs. Vega FE (4K, 1440p, 1080p)
With Ghost Recon: Wildlands at 4K first, we’re seeing the stock Vega: FE card operate a framerate of roughly 37.7FPS AVG, with lows at 34 1% low and 33.7FPS 0.1% lows. That’s our baseline. The Fury X at 1050MHz operates a 32FPS AVG, with lows at 29 and 25. This places the Fury X about 15% behind the Vega: FE stock card. At 1050MHz, the Vega: FE card operates its average at 30.3FPS, marking it 5.3% behind the Fury X, or 19.6% behind our baseline card.
Moving to 1440p, the gap now minimizes – just like we saw with FireStrike. Vega: FE stock runs a baseline score of 62FPS AVG, as the card stretches its legs a bit at the lower resolutions, with 1% and 0.1% lows maintained relatively close by. The Fury X operates at 50FPS AVG, with 0.1% low frametimes consistently around 36.3FPS – part of this may be game-level optimization at this point, as the Fury X is old enough to see waning support from developers. More importantly, the Vega: FE card at 1050MHz scored effectively the same – 49.7FPS AVG, with some of its test passes scoring 50FPS AVG. This resolution equalizes the performance of these cards, aside from Vega’s advantage in the 0.1% low department.
Let’s look at 1080p to build some more resolution scaling data.
Our baseline Vega: FE card performed with an FPS of 81 AVG with this resolution. The Fury X runs at 63.3FPS AVG – again, effectively equal in performance, given test-to-test variance. Lows have also evened out in this test as VRAM consumption has gone down and load has lightened.
This particular set of data aligns well with FireStrike, where we see performance equalize at resolutions lower than 4K – the Fury X always did perform better at 4K, even in our original review from years ago.
GTA V – Fury X vs. Vega FE Clock for Clock
We only ran 4K for GTA V. The numbers position the Fury X at 43FPS AVG and the FE card at 1050MHz at 40.7FPS AVG. This is outside of variance and is a measurable and repeatable result, placing the clock-for-clock Vega: FE card 5.3% behind the Fury X in “IPC,” so to speak.
AOTS – Fury X vs. Vega FE
Looking at Ashes of the Singularity with DirectX 12, High settings, and a 4K resolution, the Vega: FE stock card establishes a baseline of 69.4FPS AVG. The Fury X at 1050MHz runs at 65.4FPS AVG, with low-end frametime performance slightly more behind. Setting Vega: FE to 1050MHz produces a 57.3FPS result, with lows behind the Fury X. This makes the Vega: FE 1050MHz card about 12% slower than the Fury X, and makes the Fury X about 5.8% slower than the Vega: FE stock card.
IMPORTANT NOTE: We have uncovered some curious behavior with Ashes of the Singularity in the process of benchmarking Vega: FE for our next feature test. Our FPS numbers steadily increased pass-to-pass on Vega, growing at one point in a couple percent with no changes between runs (other than more runs). We are unclear if this is some sort of caching going on that is Vega-specific or if it’s just how the game works overall; we’ll have to run non-Vega cards through to see if the same performance behavior emerges. This could impact our above results, assuming “more tests = better performance.” For now, we took the first three tests from each card and averaged them, as those numbers were the most consistent with one another (+/- 1FPS).
Sniper Elite 4: Fury X vs. Vega FE
For our last gaming test, we’re using Sniper Elite 4 at 4K with Dx12 and High settings, with Async compute enabled.
The Vega: FE stock card provides a baseline performance of 53.3FPS AVG, with the Fury X 1050MHz card at 51.5FPS AVG – so the Fury X is still behind Vega when stock, but matching clocks places Vega at 51FPS AVG. These two are effectively equal, and are within our standard deviation. We cannot confidently declare that one is faster than the other when clocks are matched. Clock doesn’t seem to be making a linear difference here and, once we get to another overclocking attempt, we wouldn’t expect to see big gains from just a core clock modifier. Regardless, Vega: FE and Fury X with equal clock speeds are roughly equal in performance for Sniper Elite 4.
We’re moving on to some SPECviewperf tests now, and will be ignoring the Titan Xp competition for the time being. If you’re curious in that comparison, check our review; the focus today is to look at Vega and Fury cards at the same clocks.
SPECviewperf – Fury X vs. Vega FE
Here’s what we’ve got: with 3DSMax, the Vega FE card has a stock weighted performance of 149.3FPS, with the 1050MHz version at 121.5FPS – so positive scaling of 23% -- with the Fury X at 92.7FPS. That places the Vega FE 1050MHz card at 31% ahead of the Fury X at 1050MHz.
Moving to Catia, the Vega: FE card stock operates 19% faster than the 1050MHz card, which in turn operates 55.4% faster than the same-clocked Fury X.
The Energy test gains about 2x performance by moving to the Vega card – note, this isn’t power consumption, it’s a specific test named “energy” in the SPECviewperf suite.
Maya posts scaling of 40% for the FE card, which seems to align with other Dx11 tests.
The SNX test is one of the most interesting. This is generated from Siemens NX software, with model sizes that are 7 million to 8.5 million vertices in complexity. The Fury X gets eviscerated here, and is multiplied nearly 7x over in performance by the clock-for-clock Vega: FE card. With this particular pro application, Vega: FE appears to be showing its strengths in vertex processing or potentially in memory capacity.
Conclusion: An Interesting Academic Exercise
This was an academic exercise. We’d strongly advise that readers don’t draw too many conclusions for RX Vega based on these results. That said, looking at Vega: FE, we can theorize that there are a few parts to the performance differences of this card in gaming: (1) There is room for driver improvement – but keep those expectations realistic, (2) Vega’s small primitives discarder is aiding in some tests – but we’re also not convinced it’s fully functional (or working properly in general), (3) memory bandwidth could play a role in the Fury X’s performance at 4K in some games.
We cannot confidently state what the precise scaling would be with RX Vega, and recommend that everyone just wait and see for its performance. There are too many moving parts with GPUs – as you resolve one bottleneck, you encounter another, and it may not be the same on each architecture. It could still go either way.
Editor-in-Chief: Steve Burke
Video Producer: Andrew Coleman