Titan X Pascal Hybrid Results: Discovering Clock Throttling on Reference

By Published August 19, 2016 at 8:00 am
  •  

The Titan X (Pascal) DIY “Hybrid” project has come to a close, and with that, we've reached our results phase. This project has yielded the most visible swings in clock performance that we've yet seen from a liquid cooling mod, and has revealed significant thermal throttling in the reference nVidia Titan XP design. What's more, this card will not feature the market saturation created by AIB partners with lower end cards, and so more advanced coolers do not seem to be available without going open loop or DIY.

Our liquid-cooled Titan X Pascal Hybrid has increased the card's non-overclocked frequency by an average of nearly 200MHz – again, pre-overclock – because we've removed the thermal throttle point. The card has also improved its clock-rate stability versus temperature and time, provable during our two-hour endurance run.

 

 

The Project

titan-x-hybrid-teardown-1

titan-x-hybrid-teardown-5

titan-x-hybrid-teardown-4

This project follows our GTX 1080 Hybrid, RX 480 Hybrid, and GTX 1060 Hybrid series articles. These articles are meant to achieve three primary objectives: (1) Show the disassembly process and the PCB, (2) learn about the card's potential limitations as related to the default cooling solution, and (3) discover if additional overclocking is possible through increased thermal headroom. We generally conclude, as one would expect, that there becomes a power or VBIOS/voltage limitation once the thermals have been resolved.

Still, the GTX 1080 DIY Hybrid project granted us an additional +100MHz on the OC just by switching to liquid, and stabilized our clock-rate to a perfectly flat (ideal) line, as opposed to the spiky mess of the reference cooler.

We're using an EVGA Hybrid cooler ($100) again, which is an Asetek supplied CLC that we've previously torn down. The coldplate's copper protrusion and densely packed microfins assist in controlling thermals, with a somewhat unimaginative (but effective) impeller pushing the liquid around.

evga-hybrid-dismantle-4

evga-hybrid-dismantle-2

We've found these coolers to outperform most mainstream-ready alternatives for GPU cooling. CPU CLC coldplates are shaped in a manner that benefits an IHS with specifically located hotspots, but does not work quite as effectively for the flat silicon found on a video card.

Game Test Methodology

We tested using our GPU test bench, detailed in the table below. Our thanks to supporting hardware vendors for supplying some of the test components.

NVidia's 372.54 drivers were used for game (FPS) testing on the Titan X Pascal. The 368.69 drivers were used for other devices. Game settings were manually controlled for the DUT. All games were run at presets defined in their respective charts. We disable brand-supported technologies in games, like The Witcher 3's HairWorks and HBAO. All other game settings are defined in respective game benchmarks, which we publish separately from GPU reviews. Our test courses, in the event manual testing is executed, are also uploaded within that content. This allows others to replicate our results by studying our bench courses. In AMD Radeon Settings, we disable all AMD "optimization" of graphics settings, e.g. filtration, tessellation, and AA techniques. This is to ensure that games are compared as "apples to apples" graphics output. We leave the application in control of its graphics, rather than the IHV. In NVIDIA's control panel, we disable G-Sync for testing (and disable FreeSync for AMD).

Windows 10-64 build 10586 was used for testing.

Each game was tested for 30 seconds in an identical scenario, then repeated three times for parity.

Average FPS, 1% low, and 0.1% low times are measured. We do not measure maximum or minimum FPS results as we consider these numbers to be pure outliers. Instead, we take an average of the lowest 1% of results (1% low) to show real-world, noticeable dips; we then take an average of the lowest 0.1% of results for severe spikes.

GN Test Bench 2015 Name Courtesy Of Cost
Video Card This is what we're testing! - -
CPU Intel i7-5930K CPU iBUYPOWER  
$580
Memory Corsair Dominator 32GB 3200MHz Corsair $210
Motherboard EVGA X99 Classified GamersNexus $365
Power Supply NZXT 1200W HALE90 V2 NZXT $300
SSD HyperX Savage SSD Kingston Tech. $130
Case Top Deck Tech Station GamersNexus $250
CPU Cooler NZXT Kraken X41 CLC NZXT $110

 

Video Cards Tested

Thermal Test Methodology

We strongly believe that our thermal testing methodology is the best on this side of the tech-media industry. We've validated our testing methodology with thermal chambers and have proven near-perfect accuracy of results.

Conducting thermal tests requires careful measurement of temperatures in the surrounding environment. We control for ambient by constantly measuring temperatures with K-Type thermocouples and infrared readers. We then produce charts using a Delta T(emperature) over Ambient value. This value subtracts the thermo-logged ambient value from the measured diode temperatures, producing a delta report of thermals. AIDA64 is used for logging thermals of silicon components, including the GPU diode. We additionally log core utilization and frequencies to ensure all components are firing as expected. Voltage levels are measured in addition to fan speeds, frequencies, and thermals. GPU-Z is deployed for redundancy and validation against AIDA64.

All open bench fans are configured to their maximum speed and connected straight to the PSU. This ensures minimal variance when testing, as automatically controlled fan speeds will reduce reliability of benchmarking. The CPU fan is set to use a custom fan curve that was devised in-house after a series of testing. We use a custom-built open air bench that mounts the CPU radiator out of the way of the airflow channels influencing the GPU, so the CPU heat is dumped where it will have no measurable impact on GPU temperatures.

We use an AMPROBE multi-diode thermocouple reader to log ambient actively. This ambient measurement is used to monitor fluctuations and is subtracted from absolute GPU diode readings to produce a delta value. For these tests, we configured the thermocouple reader's logging interval to 1s, matching the logging interval of GPU-Z and AIDA64. Data is calculated using a custom, in-house spreadsheet and software solution.

Endurance tests are conducted for new architectures or devices of particular interest, like the GTX 1080R9 Fury X, or GTX 980 Ti Hybrid from EVGA. These endurance tests report temperature versus frequency (sometimes versus FPS), providing a look at how cards interact in real-world gaming scenarios over extended periods of time. Because benchmarks do not inherently burn-in a card for a reasonable play period, we use this test method as a net to isolate and discover issues of thermal throttling or frequency tolerance to temperature.

Our test starts with a two-minute idle period to gauge non-gaming performance. A script automatically triggers the beginning of a GPU-intensive benchmark running MSI Kombustor – Titan Lakes for 1080s. Because we use an in-house script, we are able to perfectly execute and align our tests between passes.

Titan X (Pascal) Temperature Testing – Peak Average Thermals

This first chart looks at delta T values, a critical means of comparative analysis that allows for control over potential variance. In our endurance charts, we switch back to non-delta values (straight reads from the GPU diode) to get a better understanding of throttle points on the core.

titan-x-pascal-temps-eq

Our Hybrid mod brings us down to 19.85C from 59.4C (delta T over ambient), or a reduction of ~40C for the load temperature. Idle temperatures are a little lower than the 1080 for a few reasons: (1) the die size is 471mm^2 on the GP102 chip, whereas the GP104 chip is 314mm^2, and this extra surface area helps dissipate heat; (2) for liquid testing, we've improved our implementation by keeping the baseplate on the Titan X Hybrid, not done for the 1080 Hybrid.

Keep in mind that these results are for the out-of-box product, so there's not even overclocking and we've already dropped thermals by 40C.

Titan X (Pascal) Endurance Results – Throttle Point Discovered

59.4C delta T is pretty warm – the hottest on the bench, actually – and that puts the GPU Diode value in the 84-85C range. The GTX 1080 throttles at around 83C, as we've extensively shown, and uses normal Boost 3.0 functionality to down-clock the card along the volt-frequency curve, reducing thermals by reducing performance temporarily. Once the GPU has become satisfied with its new resting temperature, it will attempt again to increase the clock-rate. This cycle repeats ad infinitum while under load, and is a normal part of GPU functionality.

What is sub-optimal, however, is heavy throttling that results in drastically reduced clock-rates. Let's look at that.

titan-x-pascal-stock-endurance

The above is an endurance chart for the stock Titan XP, before applying our liquid cooling solution.

During our endurance test, we plotted the Titan XP with its stock cooler as throttling at around 84C. Every hit to 84C caused an immediate drop in clock-rate, and the clock-rate got stuck around 1544MHz, but sometimes would spike to 1670MHz or drop as low as the 1400s. The spec calls for a 1531MHz boost, on paper, and the card mostly achieves this. The chart makes it pretty clear that our clock-rate is spiking hard, and it's a result of thermals – not power limit. We occasionally warmed to 85C or 86C, but the card mostly throttles hard to keep itself at 83C. And, as we'll find in a moment, the card spec calls for well under its actual operating potential.

Here's a look at a small cross-section of raw data to show what's going on.

Time (s) Core MHz GPU Diode
348 1556.5 83
349 1556.5 83
350 1493 83
351 1531 84
352 1657.5 84
353 1645 84
354 1645 84
355 1632.5 84
356 1531 84
    357
1468 83
358 1493 83
359 1544 84
360 1544 83
361 1531 83
362 1569 84
363 1531 84
364 1531 83
365 1518.5 83
366 1506 83
367 1506 83
368 1518.5 83
369 1531 83
370 1531 83
371 1531 83
372 1556.5 83

You can see that the clock has a range of more than 100MHz (in this sample of data, we've got a range of 189.5MHz, with a high of 1657.5MHz and low of 1468MHz). This clock-rate swing presents itself in 0.1% and 1% low values for gamers, but for production, it'll mostly manifest as an overall loss of efficiency and slow-down in render times. Because the card is so fast already, though, it might not be apparent that the slow-down exists – at least, not until after fixing the reference design. Then it's more obvious.

Let's put that into perspective.

titan-x-pascal-hybrid-endurance

Time (s) Core MHz GPU Diode (C)
349 1771.5 41
350 1759 41
351 1784.5 41
352 1759 41
353 1759 41
354 1721 41
355 1759 41
356 1784.5 41
357 1771.5 41
358 1721 41
359 1733.5 41
360 1759 41
361 1721 41
362 1759 41
363 1721 41
364 1746.5 41
365 1721 41
366 1721 41
367 1721 41
368 1733.5 41
369 1721 41
370 1721 41
371 1695.5 40
372 1695.5 41
373 1695.5 41

(Above table: Some raw data from the Hybrid endurance run).

So, the original chart plots us as hovering in the 1468-1657+ range with an 83-84C diode, averaging at around 1531MHz. With our Hybrid mod, we brought the Titan X Pascal up to nearly 1800MHz – and that's with absolutely no overclock at all. Again, this is running stock, which means that the card's spec sheet is under its actual potential performance, and that the cooler is “stealing” speed from the chip. On average, we're moving from 1531MHz with the stock cooler to an average of about 1784MHz with the liquid cooler. We've improved the clock-rate of the stock card by more than 200MHz just by fixing nVidia's poorly performing air cooler.

The only reason we're still seeing that spiky frequency plot is because the card is now choking on power, not thermals. We've resolved the thermal constraint and are now hitting power constraints, which can be resolved simply by increasing the power limit of the card. Of course, applying an OC will re-create the spiked performance, but fixing the cooler and increasing the power limit (with no OC) will flatten overall clock-rate.

Overclocking

There is an impact on overclocking, too. As with the GTX 1080 FE, we were able to extract additional performance from the chip by keeping it more reasonably cooled. It would be possible to cool-down the reference (and only) Titan XP by increasing fan RPM to 100%, but then you're creating a realistically undesirable scenario by dumping noise into the environment. We found the ~70% fan speed range tolerable from a user perspective and from a power perspective.

Titan X (Pascal) Overclocking – Stock Air Cooler

Here's the stepping table for the stock card:

Peak Core CLK Core Clock (MHz) Core Offset (MHz) Mem CLK (MHz) Mem Offset (MHz) Power Target (%) Peak vCore Fan Target (%) 5m Test 60m Endurance
1695.5 ~1620 1251.5 100 0.95 50 P P
1746.5 1733.5 1251.5 120 1.025 65 P P
1822.5 1809.5 100 1251.5 120 1.025 65 P -
1860 1835 125 1251.5 120 1.025 65 P P
1873 1847.5 150 1251.5 120 1.025 66 P -
1923.5 1911 175 1251.5 120 1.025 66 P -
1923.5 1911 175 1325.5 300 120 1.025 66 P -
1923.5 1911 175 1377 500 120 1.025 66 P -
1923.5 1911 175 1400.6 600 120 1.025 66 F
Driver Crash
-
1987 1911 175 1363.5 450 120 1.025 70 P P
- - 200 1363.5 450 120 1.025 70 F
Driver Crash
-

We're approaching 4000RPM, here. Not quite – but close, in the ~3500RPM range. We found stability on the stock card at 1911MHz core (+175MHz) and 1363.5MHz memory (actual clock, comes out to ~10.9GHz effective). Voltage could not be increased with the tools available at this time, so that column is fully under the card's control.

Titan X (Pascal) Hybrid Overclocking – GN's Liquid Cooler

Here's the stepping results for our Hybrid:

Peak Core CLK Core Clock (MHz) Core Offset (MHz) Mem CLK (MHz) Mem Offset (MHz) Power Target (%) Peak vCore Fan target 5m Test 60m Endurance
1987 1923.5 175 1363.5 450 120 1.031 23 (Auto) P P
1987 1936 200 1363.5 450 120 1.031 23 (Auto) P -
1999.5 1961.5 225 1363.5 450 120 1.031 23 (Auto) P -
2012 1974 250 1363.5 450 120 1.031 23 (Auto) P -
2037.5 1999.5 275 1363.5 450 120 1.031 23 (Auto) F -
2012 1974 250 1363.5 500 120 1.031 23 (Auto) P P
2012 1974 250 1363.5 600 120 1.031 23 (Auto) P P
(Worse FPS)
2012 1974 250 1363.5 450 120 1.031 23 (Auto) P P

We were able to overclock higher and with a greater sustained average clock-rate than before, thanks to the improved thermal solution. Still, the Titan XP chokes hard on power availability, and this is demonstrated by the fact that increasing the memory clock beyond 450MHz will actually decrease FPS, because the power is eaten from the core. We actually gained more FPS by dropping the memory from its stable +600MHz value to a +450MHz value, ultimately used on both devices. That seems to be the point beyond which diminishing returns come into play.

We landed at 2012MHz peak, or about 1974MHz resting clock with the Hybrid card. The stock card, with a loud and high RPM fan, was getting stuck at about +175MHz core offset, for a total of 1987MHz peak, or 1911MHz average.

We've managed to decrease thermals, noise, and increase the overclock by about 63MHz. That's not a huge OC add, but there's more to it than just the offset numbers – we've also got to look at stability, and now that thermals are not throttling the card, it's down to just a power limit.

FPS Difference (Briefly)

We'll talk about this more in our imminent Titan X (Pascal) review, now that we've had a generous reader loan us the card (thanks, Sam!), but this condensed FPS OC results chart will provide a quick preview of what's to come:

titan-x-oc-gtav-4k

titan-x-oc-mordor-1080p

(Note: Hitting CPU limit at 1080p.)

titan-x-oc-mordor-1440p

titan-x-oc-mordor-4k

titan-x-oc-mec-1440p

Mirror's Edge Catalyst, 4K/High:

MEC 4K High    
  Avg FPS 1.0% low .1% low
Titan XP Air 1531MHz 60.3 48.3 43.7
Titan XP Air 1911MHz 69.67 53.33 45.67
Titan XP Hybrid 1974MHz 73.67 57.33 50.67

We see a change of ~2% in Shadow of Mordor at 1080p and at 4K, moving just from 91FPS AVG to 93FPS AVG, but the 0.1% low values have been improved from 62FPS to 71FPS as a result of the more stable clock-rate (a change of 14.5%). Our testing methodology is picking up the clock-rate instability and reflecting a real impact, albeit one which is generally not observable with this particular title. Looking at GTA V, we see a performance swing of nearly 4% in the averages at 4K. 1080p doesn't really show any real change, but the 0.1% lows are marginally improved – again imperceptibly.

Mirror's Edge Catalyst posts the biggest swings, with a change of about 5% at 1440p and about 6% at 4K High.

Production workloads would be interesting to test, but we might not be able to get to those before we have to return this card to its rightful owner.

Conclusion: Titan XP Reference Cooler Should be Replaced

titan-xp-hybrid-assembly-1

If you buy a Titan X (Pascal) GPU, we firmly believe that the cooler should be immediately replaced. Operating at ~83~85C for long uptimes – how this card is meant to be used – isn't going to be great for clock-rate stability. The Titan X should have been shipped with a stock CLC or with a better air setup, we think, and the FE cooler is inadequate for a chip of this power.

Very interesting results, for this one. We'll try to research further if we're able to procure a permanent sample.

Editorial: Steve “Lelldorianx” Burke
Video: Andrew “ColossalCake” Coleman

Last modified on August 18, 2016 at 8:00 am
Steve Burke

Steve started GamersNexus back when it was just a cool name, and now it's grown into an expansive website with an overwhelming amount of features. He recalls his first difficult decision with GN's direction: "I didn't know whether or not I wanted 'Gamers' to have a possessive apostrophe -- I mean, grammatically it should, but I didn't like it in the name. It was ugly. I also had people who were typing apostrophes into the address bar - sigh. It made sense to just leave it as 'Gamers.'"

First world problems, Steve. First world problems.

We moderate comments on a ~24~48 hour cycle. There will be some delay after submitting a comment.

Advertisement:

  VigLink badge