It's never ending, this industry, but the team is keeping up better than ever. Our Synology NAS deployment has given us an additional ~15TB of usable storage for video, test, and photo production, and provides an advanced back-end interface for permissions and privileges management. This alone has increased the sophistication of our internal test processes, and has allowed me to individually document test settings, benchmark courses, pitfalls, and unique-to-game bugs that can interfere with results. Those walkthrough files can now be used by technicians for test execution, which will accelerate media production.
Actually, while on that topic, we've even taken some internal recordings of test execution and setup for future use. This greatly reduces the chance of test error – games are complex, and the susceptibility to configuration errors is high – for future regression tests, which ensures data integrity and reduces overall workload. Retesting is common in any scientific field and should be seen as part of the daily actions, but retesting can and should be limited to more anomalous findings. There is no reason to get into the habit of retesting software due to forgotten options when working with long-ago tested titles.
The immediate shortcomings have been identified and internally resolved, for now, and we're next pushing for more major content generation. To be fair, that hasn't really stopped; but the internal process improvement will be momentarily put on pause while we catch up post-PAX. That process will resume sometime soon, and we hope to begin revamping some of the website's front/back-end tools shortly.
As for Dx12 and Vulkan testing, stay tuned for more discussion on that. We are investigating a switch-over to PresentMon, but may build a custom interface and tool subset for greater data collection and presentation.
- Steve “Lelldorianx” Burke.
