Polling the Audience

Not long ago, we posted polls to determine what services our viewers prefer when streaming or viewing streams. We also sought to understand the optimization settings used by the audience, H264 settings, and upload bit-rate for streams. These strawpolls led us to understand that our audience primarily uses “Veryfast” and “fast” for the h264 encode settings, and so we opted to split the difference and go with “faster.” We also learned that most of our viewers/readers prefer streaming at 6-10Mbps, which is likely more a limitation of network quality than anything else – but still, an important factor. Our upload speed supports up to ~40Mbps, so we decided to again split the difference and run tests at both 6Mbps and 10Mbps. Encoding difficulty increases as bit-rate does, making bit-rate just one of many potential optimization or benchmark opportunities.

Although we performed some testing with Twitch at 720p60 / 6Mbps, we ultimately scrapped the testing due to account-side issues that made us question numbers. Instead, given that we already have the ability to stream high bit-rates and qualities on YouTube, we went with YouTube streaming at 1080p60 with bit-rates of 6Mbps and 10Mbps. OBS was used as the streaming service (version 19.0.3 6/22/17 branch), with encoder settings configured to H264 and “Faster.” Video output was fixed to 1080p60, the games were played back at 1080p (Windowed), with physical logging tools for current draw and thermals (additional software, like HWINFO, was also used for validation). We are configuring DiRT to Ultra, 1080p, windowed, with 4xMSAA and Advanced Blending. DOTA2 is at max settings, Vsync off.

Each set of tests went through multiple passes over a couple of days (as we had other tests going on). Numbers are pulled from monitoring tools and OBS logs. The OBS UI text for "dropped frames" does not report the same numbers. Additional information is pulled from the output file.

Difficulties of Stream Testing & Multitasking Benchmarks

We’re testing a few things here, and stream benchmarking serves to fulfill our interest in executing a “multitasking” benchmark. Multitasking benchmarking has long been a request of our audience, it’s just been a matter of what to use as a test, and how to best ensure said test remains repeatable without haphazard variables. This test is one of the heaviest possible multitasking workloads for a gamer, as the CPUs will be juggling live gaming (which requires low input latency and fast response), live streaming/encoding, and also video playback. We’re playing back the stream as it airs, adding another repeatable, measurable benchmark component that doesn’t overly complicate our controls/constants.

We’re interested in both the streamer experience and viewer experience with this testing. The viewer receives the output stream, and so our goal is to ensure a stable, reliable output. The streamer is still ultimately playing a game, and so the framerate must be acceptable for the game. Some games require more consistent and lower frametimes than others, like CSGO or DOTA2, and others diminish this requirement. As a general rule, streaming is a balance between player framerate/experience quality and viewer framerate/experience quality. We only have so many CPU resources to spare, and so the divvying up may sometimes require optimizations – like process priority given to the encoding tool, which is something we ultimately did test.

This balance is important. Even if OBS or XSplit can keep up with the game and encode/deliver 100% of frames to the streaming service, it doesn’t mean that the viewer experience is good. It just means frames are being delivered. You could encode & deliver 100% of frames at 20FPS, so the numbers look good – every frame made it out – but the actual playback is choppy.

Disclaimer

Anyway, before getting in to testing, keep in mind that stream benchmarking has an awful lot of variables. We can only speak for our numbers, and won’t be making sweeping statements that should be taken as truth for all games, all encoders, or all streaming services. Still, this gives good insight as to the performance of each processor, and establishes a stronger starting point for our stream benchmarking. These tests began with the i9-7900X review, but we’ve since improved methods and results can no longer be compared to previous tests (for example, the test duration has now been lengthened to 300s, and we’ve increased our iterations per test).

Other Test Methods, Specs, & Information

Most the testing methodology is defined above, but the rest is below:

Fixed typographical error pertaining to video card origin and price. The card used for each bed was the exact same unit.

Continue to the next page for results.