What Does OCAT Do?

OCAT is AMD’s addition to PresentMon. AMD has plans to open the source for OCAT shortly, but from preliminary testing, it looks like OCAT should impose minimal overhead and retain the exact dataset that PresentMon outputs. When OCAT works, it adds an overlay that functions brilliantly to aid in test execution by informing testers that the benchmark has started, stopped, and actually captured data. Hotkey, target exe, and time expiry settings can all be configured through the UI or through a bundled-in configuration text file. We find this useful for automation purposes, but it’s generally just nice to have text file access to configurations.

OCAT also provides on-the-fly frametime and framerate numbers in its overlay, further aiding the validation process. We find this live, during-bench feedback to be important for sort of heuristically ensuring accurate data collection. You wouldn’t want to run endurance FPS benchmarks and find that the tool wasn’t capturing correctly, for instance, and that’s theoretically resolved with OCAT’s overlay.

This is entirely a play to expand usability of PresentMon, which is an already trusted, understood, and well-documented tool that has hands-on development from industry veterans. Again, Intel and Microsoft worked together to hook PresentMon into ETW, the tool monitors both onPresent and onDisplay variables, and outputs a ton of frametime data for careful analysis. It’s also fully functional with Dx11, Dx12, UWP, OpenGL, and Vulkan, and we’ve yet to encounter a game where PresentMon fails to perform. Given the nature of shims between applications and APIs in this new world of low-level programming interfaces, PresentMon was a major saving grace that helped get us off the reliance upon aged tools. FRAPS, for instance, simply doesn’t officially work with the newer APIs (though can sometimes be forced to work), and some of the other existing tools just ignore a lot of important frametime data. Alternatively, they may just measure data differently, or calculate percentiles with formulas that seem confusing at times.

OCAT retains both the onPresent and onDisplay measurements for frames, allowing us to better leverage software to analyze performance for different use cases. onDisplay, for instance, has some serious value adds when conducting multi-GPU benchmarks. onPresent is what we prefer for almost everything, but we still collect both types of data to better understand the game’s (or hardware’s) behavior.

Just to re-iterate something, here’s a sample file from PresentMon:

And here’s a sample file from OCAT (built atop PresentMon):

The columns are the same. Nothing moves. Nothing is added. The only reason you are seeing differences is because they’re different test runs – focus on the columns and formatting.

This means that existing tools (like our internal Perl + Python scripting suite) will still function perfectly with data captured by OCAT, while now offering the advantage of being more accessible. From a benchmarker standpoint, and having used OCAT for a little while now, it’s already clear to GN that this interface will speed-up our production time in a non-trivial fashion. Just talking about working with technicians, the training time for OCAT will be lower than for PresentMon through CMD, and that also reduces chance for technician error. From a casual user’s perspective, it’s just easier to use and get a quick, informed understanding of performance.

OCAT Bugs & Waiting for Final Analysis

Now, unfortunately, we’ve been running into bugs. Our team has been in communication with AMD over these issues for a little while now, as we’ve been trying to troubleshoot and isolate root causes. So far, Battlefield 1 throws an error with the newest patch, GTA  V fails to hook, and UWP has some known issues as well. We were able to successfully execute Metro: Last Light tests, rather boringly, but that just wasn’t enough to justify pushing forward with publishing performance results. The end goal is to analyze overhead – of which there should be nearly none – and the quality of the data output. That will wait until some more bug fixes are in place.

A huge production concern would be if game patches or new game releases require manual updating by AMD, since that basically kills pre-launch use by media, but we don’t yet know if that’ll be the case. PresentMon generally won’t break or require manual updates for new games, but we are unsure of OCAT’s pickiness at this time.

A Promising Addition

In the meantime, OCAT thus far has proven exceptionally promising and stands as an exciting addition to the PresentMon tool. We already trust our PresentMon data, understand it, and agree with its implementation. Getting it easier to use hasn’t been a trivial task, and AMD’s trying to take charge on that goal. We hope to see OCAT mature rapidly for deployment in testing sometime shortly. As a value-add, AMD tells GN that OCAT will eventually have its source code opened, further ensuring that vendors are playing fair when providing benchmarking utilities.

Editorial: Steve “Lelldorianx” Burke
Video: Andrew “ColossalCake” Coleman