Game Test Methodology
We tested using our GPU test bench, detailed in the table below. Our thanks to supporting hardware vendors for supplying some of the test components.
Battlefield 1 and Gears of War use the AMD 16.10.1 drivers, which have less than a 1% performance change from 16.10.2 in BF1. The RX 460 was re-run with 16.10.1. AMD 16.8.1 drivers were used for the RX 470. Driver package 375.57 was used for Battlefield 1 and Gears 4 testing on nVidia hardware, with 375.57 used on the GTX 1050 & GTX 1050 Ti. NVidia's 372.54 drivers were used for game (FPS) testing on the GTX 1080 and 1060. The 368.69 drivers were used for other devices. Game settings were manually controlled for the DUT. All games were run at presets defined in their respective charts.
Windows 10-64 Anniversary Edition was used for testing.
Each game was tested for 30 seconds in an identical scenario, then repeated three times for parity.
Average FPS, 1% low, and 0.1% low times are measured. We do not measure maximum or minimum FPS results as we consider these numbers to be pure outliers. Instead, we take an average of the lowest 1% of results (1% low) to show real-world, noticeable dips; we then take an average of the lowest 0.1% of results for severe spikes.
|GN Test Bench 2015||Name||Courtesy Of||Cost|
|Video Card||This is what we're testing!||-||-|
|CPU||Intel i7-5930K CPU 3.8GHz||iBUYPOWER
|Memory||Corsair Dominator 32GB 3200MHz||Corsair||$210|
|Motherboard||EVGA X99 Classified||GamersNexus||$365|
|Power Supply||NZXT 1200W HALE90 V2||NZXT||$300|
|SSD||HyperX Savage SSD||Kingston Tech.||$130|
|Case||Top Deck Tech Station||GamersNexus||$250|
|CPU Cooler||NZXT Kraken X41 CLC||NZXT||$110|
For Dx12 and Vulkan API testing, we use the PresentMon onPresent variable, then use our own Python script to extract the data.
Specific Test Methodologies for Games
Video Cards Tested
- MSI GTX 1050 Ti Gaming X ($165)
- MSI GTX 1050 OC
- Corsair Hydro GFX GTX 1080 ($750)
- Sapphire RX 470 Platinum (~$180?)
- MSI RX 480 Gaming X
- MSI GTX 1060 Gaming X ($290)
- NVIDIA GTX 1060 FE ($300)
- AMD RX 480 8GB ($240)
- NVIDIA GTX 1080 Founders Edition ($700)
- NVIDIA GTX 980 Ti Reference ($650)
- NVIDIA GTX 980 Reference ($460)
- NVIDIA GTX 980 2x SLI Reference ($920)
- AMD R9 Fury X 4GB HBM ($630)
- AMD MSI R9 390X 8GB ($460)
- And more
Thermal Test Methodology
We strongly believe that our thermal testing methodology is among the best on this side of the tech-media industry. We've validated our testing methodology with thermal chambers and have proven near-perfect accuracy of results.
Conducting thermal tests requires careful measurement of temperatures in the surrounding environment. We control for ambient by constantly measuring temperatures with K-Type thermocouples and infrared readers. We then produce charts using a Delta T(emperature) over Ambient value. This value subtracts the thermo-logged ambient value from the measured diode temperatures, producing a delta report of thermals. AIDA64 is used for logging thermals of silicon components, including the GPU diode. We additionally log core utilization and frequencies to ensure all components are firing as expected. Voltage levels are measured in addition to fan speeds, frequencies, and thermals. GPU-Z is deployed for redundancy and validation against AIDA64.
All open bench fans are configured to their maximum speed and connected straight to the PSU. This ensures minimal variance when testing, as automatically controlled fan speeds will reduce reliability of benchmarking. The CPU fan is set to use a custom fan curve that was devised in-house after a series of testing. We use a custom-built open air bench that mounts the CPU radiator out of the way of the airflow channels influencing the GPU, so the CPU heat is dumped where it will have no measurable impact on GPU temperatures.
We use an AMPROBE multi-diode thermocouple reader to log ambient actively. This ambient measurement is used to monitor fluctuations and is subtracted from absolute GPU diode readings to produce a delta value. For these tests, we configured the thermocouple reader's logging interval to 1s, matching the logging interval of GPU-Z and AIDA64. Data is calculated using a custom, in-house spreadsheet and software solution.
Endurance tests are conducted for new architectures or devices of particular interest, like the GTX 1080, R9 Fury X, or GTX 980 Ti Hybrid from EVGA. These endurance tests report temperature versus frequency (sometimes versus FPS), providing a look at how cards interact in real-world gaming scenarios over extended periods of time. Because benchmarks do not inherently burn-in a card for a reasonable play period, we use this test method as a net to isolate and discover issues of thermal throttling or frequency tolerance to temperature.
Our test starts with a two-minute idle period to gauge non-gaming performance. A script automatically triggers the beginning of a GPU-intensive benchmark running MSI Kombustor – Titan Lakes for 1080s. Because we use an in-house script, we are able to perfectly execute and align our tests between passes.
Noise Testing Methodology
Our noise testing methodology is new and still being revised, but has been kept consistent across all tests contained herein. We test noise in a real-world environment and do not presently use an anechoic chamber. The results align with what consumers will encounter in their own rooms.
We use a REED logging dB meter mounted to a tripod, whose mic is positioned 20” from the face of the GPU (mounted in an open bench). The REED meter is approximately 6” above the bench. All open bench fans are disabled. The Kraken X41 CPU cooling fan is configured to its “silent” mode, minimizing its noise output to be effectively imperceptible.
A noise floor measurement is taken prior to each test's execution to determine ambient without any systems running in the room. We then take an idle measurement (GPU & CPU at idle). Our noise floor has a fluctuation of approximately +/-0.6dB.
Noise levels are logarithmic, and are therefore not as simple to perform delta calculations as thermals or framerates. Noise percent differences are calculated using dB=20*log(V2/V1) (where V is amplitude). You cannot perform a simple percent difference calculation to determine the delta. For an example, a 10dB range (50dB vs. 40dB) is not equal to a 22% delta.
After the noise floor is determined, we log idle fan dB, 50% speed dB, and 100% speed dB (configured in Afterburner). We also measure auto fan dB at an identical stepping for every test; we do this by running Kombustor for exactly 5 minutes prior to beginning dB logging, which is useful for fans which use two push fans. Some dual-push fan cards will only trigger the second fan if the VRM is under load.
Power Testing Methodology
Power consumption is measured at the system level. You can read a full power consumption guide and watt requirements here. When reading power consumption charts, do not read them as a GPU-specific requirements – this is a system-level power draw. The CPU is dropped to its stock clock for power tests.
Power draw is measured during a FireStrike Extreme - GFX2 run. We are currently rebuilding our power benchmark.