This will be a quick one. There is some required viewing/reading before diving in: Previously, with the FFXV standalone benchmark release, we found significant culling deficiencies of objects in the game, including both GameWorks and non-GameWorks objects. This suggested overall inefficiency and hasty development, as opposed to some sort of malfeasance. Square Enix later tweeted rather direct acknowledgement of the benchmark’s issues, and began work to optimize the game (and the GameWorks integration) for launch.

Today’s test is a quick one. Square Enix launched a playable demo of Final Fantasy XV and, although it’s still not the complete game, we wanted to see if any of the object culling issues had been addressed. We were primarily interested in HairWorks LOD scaling, as that was previously an issue responsible for causing performance loss on both nVidia and AMD hardware – even when no HairWorks objects were anywhere remotely close to the player.

Following Final Fantasy XV’s benchmark launch, which we found to be flawed in a few ways, Square Enix has now launched its playable demo for the first portion of the game. This is the first time that FFXV has been playable on PC, barring some flukes in the benchmark, and is also the first revisit to the game since the benchmark’s launch.

Our primary concerns with the benchmark tool were validated by Square Enix, who noted they’d be addressing the concerns. The primary issue was that no graphics customization options were present without exposing the game’s .ini files via .dlls, which we did, and we later found the other issue: Some objects were being drawn at high LOD when never appearing on screen, something we validated with inspection tools. This included non-GameWorks objects and GameWorks objects, with the latter impacting performance more heavily on both AMD and nVidia devices.

Update: Square Enix is aware of this issue, has acknowledged its existence, and is working on an update for launch.

Although we don't believe this to be intentional, the Final Fantasy XV benchmark is among the most misleading we’ve encountered in recent history. This is likely a result of restrictive development timelines and a resistance to delaying product launch and, ultimately, that developers see this as "just" a benchmark. That said, the benchmark is what's used for folks to get an early idea of how their graphics cards will perform in the game. From what we've seen, that's not accurate to reality. Not only does the benchmark lack technology shown in tech demonstrations (we hope these will be added later, like strand deformation), but it is still taking performance hits for graphics settings that fail to materialize as visual fidelity improvements. Much of this stems from GameWorks settings, so we've been in contact with nVidia over these findings for the past few days.

As we discovered after hours of testing the utility, the FFXV benchmark is disingenuous in its execution, rendering load-intensive objects outside the camera frustum and resulting in a lower reported performance metric. We accessed the hexadecimal graphics settings for manual GameWorks setting tuning, made easier by exposing .INI files via a DLL, then later entered noclip mode to dig into some performance anomalies. On our own, we’d discovered that HairWorks toggling (on/off) had performance impact in areas where no hair existed. The only reason this would happen, aside from anomalous bugs or improper use of HairWorks (also likely, and not mutually exclusive), would be if the single hair-endowed creature in the benchmark were drawn at all times.

The benchmark is rendering creatures that use HairWorks even when they’re miles away from the character and the camera. Again, this was made evident while running benchmarks in a zone with no hairworks whatsoever – zero, none – at which point we realized, by accessing the game’s settings files, that disabling HairWorks would still improve performance even when no hairworks objects were on screen. Validation is easy, too: Testing the custom graphics settings file by toggling each setting, we're able to (1) individually confirm when Flow is disabled (the fire effect changes), (2) when Turf is disabled (grass strands become textures or, potentially, particle meshes), (3) when Terrain is enabled (shows tessellation of the ground at the demo start' terrain is pushed down and deformed, while protrusions are pulled up), and (3) when HairWorks is disabled (buffalo hair becomes a planar alpha texture). We're also able to confirm, by testing the default "High," "Standard," and "Low" settings, that the game's default GameWorks configuration is set to the following (High settings):

• VXAO: Off
• Shadow libs: Off
• Flow: On
• HairWorks: On
• TerrainTessellation: On
• Turf: On

Benchmarking custom settings matching the above results in identical performance to the benchmark launcher window, validating that these are the stock settings. We must use the custom settings approach, as going between Medium and High offers no settings customization, and also changes multiple settings simultaneously. To isolate whether a performance change is from GameWorks versus view distance and other settings, we must individually test each GameWorks setting from a baseline configuration of "High." 

This content piece will explore the performance anomalies and command line options for the Final Fantasy XV benchmark, with later pieces going detailed on CPU and GPU benchmarks. Prior to committing to massive GPU and CPU benchmarks, we always pretest the game to understand its performance behaviors and scaling across competing devices. For FFXV, we’ve already detailed FPS impact of benchmark duration, impact of graphics settings and resolution on scaling, we’ve used command line to automate and custom configure benchmarks, and we’ve discovered poor frametime performance under certain benchmarking conditions.

We started out by testing for run-to-run variance, which would be used to help locate outliers and determine how many test passes we need to conduct per device. In this frametime plot, you can see that the first test pass, illustrated on a GTX 1070 with the settings in the chart, exhibits significantly more volatile frametimes. The frame-to-frame interval occasionally slams into a wall during the first 6-minute test pass, causing noticeable, visible stutters in gameplay.

As everyone begins running the Final Fantasy XV PC benchmark, we’d like to notify the userbase that, on our test platform, we have observed some run-to-run variance in frame-to-frame intervals from one pass to the next. This seems to stem entirely from the first pass of the benchmark, where the game is likely still loading all of the assets into memory. After the first pass, we’ve routinely observed improved performance on runs two, three, and onward. We attribute this to first-time launcher initialization of all the game assets.

Our Destiny 2 GPU benchmark highlighted massive performance uplift vs. beta on some devices, upwards of 50% on Vega, but was conducted in largely GPU-constrained scenarios. For this content piece, we’ll be exploring the opposite: CPU-constrained scenarios to benchmark Destiny 2 performance on AMD Ryzen and Intel Kaby/Coffee Lake parts, including the R7 1700, R5 1600X, R3 1200, and i7-7700K, i5-7600K, i3-8350K, and G4560.

Most of our test notes have already been recapped in the GPU benchmark, and won’t be fully repeated. Again, we ran a wide spread of tests during the beta, which will be informing our analysis for the Destiny 2 launch benchmarks. Find the previous content below:

• Destiny 2 Beta GPU Benchmark (+ graphics optimization guide, PvP scalability)
• Destiny 2 Beta CPU Benchmark (soon replaced by our Destiny 2 launch CPU bench)
• Destiny 2 texture comparison

As stated in the video intro, this benchmark contains some cool data that was exciting to work with. We don’t normally accumulate enough data to run historical trend plots across various driver or game revisions, but our initial Destiny 2 pre-launch benchmarks enabled us to compare that data against the game’s official launch. Bridging our pre-launch beta benchmarks with similar testing methods for the Destiny 2 PC launch, including driver changes, makes it easier to analyze the deviation between CPU, driver, and game code optimizations.

Recapping the previous tests, we already ran a wide suite of Destiny 2 benchmarks that included performance scaling tests in PvP multiplayer, campaign/co-op multiplayer, and various levels/worlds in the game. Find some of that content below:

• Destiny 2 Beta GPU Benchmark (+ graphics optimization guide, PvP scalability)
• Destiny 2 Beta CPU Benchmark (soon replaced by our Destiny 2 launch CPU bench)
• Destiny 2 texture comparison

NOTE: Our Destiny 2 CPU benchmark is now live.

Some of our original graphics optimization work also carried forward, allowing us to better pinpoint Depth of Field on Highest as one of the major culprits to AMD’s performance. This has changed somewhat with launch, as you’ll find below.

We’re sticking with FXAA for testing. Bungie ended up removing MSAA entirely, as the technique has been buggy since the beta, and left only SMAA and FXAA in its place.

As we’ve done in the past for GTA V and Watch_Dogs 2, we’re now taking a look at Destiny 2’s texture resolution settings. Our other recent Destiny 2 content includes our GPU benchmark and CPU benchmark.

All settings other than texture resolution were loaded from the highest preset and left untouched for these screenshots. There are five degrees of quality, but only highest, medium, and lowest are shown here to make differences more obvious. The blanks between can easily be filled in.

UPDATE: We have run new CPU benchmarks for the launch of this game. Please view the Destiny 2 launch CPU benchmarks here.

Our Destiny 2 GPU benchmark was conducted alongside our CPU benchmark, using many of the same learnings from our research for the GPU bench. For GPU testing, we found Destiny 2 to be remarkably consistent between multiplayer and campaign performance, scaling all the way down to a 1050 Ti. This remained true across the campaign, which performed largely identically across all levels, aside from a single level with high geometric complexity and heavy combat. We’ll recap some of that below.

For CPU benchmarking, GN’s Patrick Lathan used this research (starting one hour after the GPU bench began) to begin CPU tests. We ultimately found more test variance between CPUs – particularly at the low-end – when switching between campaign and multiplayer, and so much of this content piece will be dedicated to the research portion behind our Destiny 2 CPU testing. We cannot yet publish this as a definitive “X vs. Y CPU” benchmark, as we don’t have full confidence in the comparative data given Destiny 2’s sometimes nebulous behaviors.

For one instance, Destiny 2 doesn’t utilize SMT with Ryzen, producing utilization charts like this:

UPDATE: We have run benchmarks of the launch version of Destiny 2. Please view the launch Destiny 2 GPU benchmarks here.

The Destiny 2 beta’s arrival on PC provides a new benchmarking opportunity for GPUs and CPUs, and will allow us to plot performance uplift once the final game ships. Aside from being a popular beta, we also want to know if Bungie, AMD, and nVidia work to further improve performance in the final stretch of time prior to the official October 24 launch date. For now, we’re conducting an exploratory benchmark of multiplayer versus campaign test patterns for Destiny 2, quality settings, and multiple resolutions.

A few notes before beginning: This is beta, first off, and everything is subject to change. We’re ultimately testing this as it pertains to the beta, but using that experience to learn more about how Destiny 2 behaves so that we’re not surprised on its release. Some of this testing is to learn about settings impact to performance (including some unique behavior between “High” and “Highest”), multiplayer vs. campaign performance, and level performance. Note also that drivers will iterate and, although nVidia and AMD both recommended their respective drivers for this test (385.41, 17.8.2), likely change for final release. AMD in particular is in need of a more Destiny-specific driver, based on our testing, so keep in mind that performance metrics are in flux for the final launch.

Note also: Our Destiny 2 CPU benchmark will be up not long after this content piece. Keep an eye out for that one.