Boneworks IQ and Variable Rate Supersampling (VRSS) Performance Review
Boneworks is an important VR game that has just been added to BTR’s regular benchmark suite so we are going examine its image quality (IQ) and measure its performance using FCAT-VR. In addition, this Boneworks review is a follow-up to our evaluation of Variable Rate Supersampling (VRSS), an important new IQ feature for Turing architecture so we will specifically look at the performance of NVIDIA’s five current RTX cards.
This editor didn’t really like Boneworks yet admires the developers’ high achievement of coupling a fair game with an incredible sandbox and next generation VR physics interactive tour de force. Our original issues with the game included lack of saves, tediousness required for some interactions, nausea issues for some players, and some real difficulty with climbing including shaking. However, most of this editor’s complaints about the game were addressed by a major 592.9MB patch that dropped this week which also included new content featuring a Brawl mode, fixes for mid-point checkpoint saves, and improved climbing and hand responsiveness.
BTR received a Boneworks reviewer’s key from NVIDIA in early January to check out VRSS although we didn’t start to play it until a couple of weeks ago. Boneworks is available as a $29.99 SteamVR game that is playable on multiple HMDs including the Valve Index, Vive HMD, Oculus Rift, and WMR.
Visuals & Physics Requirements
Boneworks has average to very good visuals and its absolute minimum graphics requirements include a GTX 1060 or GTX 970 for the Rift CV1+Touch, the Original Vive at 90hz, and the Rift S at 80hz. A quad-core 3.0GHz CPU is the required minimum since the physics calculations are done on the CPU, and an i7-7700K or equivalent is recommended. Since the game can get very resource intensive as the enemy NPCs and physics interactions increase, a GTX 1080 or GTX 2060 Super or higher are recommended for the Rift CV1+Touch, Original Vive at 90hz, or for the Rift S at 80hz.
BTR’s VR testing platform uses an Intel Core i7-8700K at 4.8GHz, an EVGA Z370 FTW motherboard and 16 GB of T-Force XTREEM DDR4 at 3866MHz on Windows 10 64-bit Home Edition. We use the Vive Pro because image quality differences are more easily noticed on a higher resolution HMD like the Pro or the Valve Index over using a first generation HMD like the Oculus Rift CV-1 or the original Vive.
Boneworks Gameplay, Physics & Interactions
Boneworks looks like a half-finished game with wireframe NPCs, but this is designed to match its setting inside a broken VR game world that the player is sent to fix. The story is told by graffiti-like messages and by 2D screen narrations while the player explores Monogon Industries’ AI operating system, Myth OS in more than a dozen different levels including Warehouse, City Streets, and Sewers. In many ways, Boneworks pays homage to the Half Life and Portal games even to using robotic head crabs and its extensive use of physics to interact with the world and to solve its many puzzles. 
Boneworks uses a custom designed physics sandbox that attempts to approximate a player’s entire body in VR. In most VR games, you interact with your hands, but in Boneworks you also move things with your legs and they can be important for climbing. Unlike most VR games, Boneworks is filled with interactive objects and weapons that attempt to imitate real world mass and interactions. It’s a completely unique game that will endlessly fascinate gamers who wish to experience maximum VR interaction.
You cannot just pick up and properly swing a sledgehammer with one hand; it requires both. You can block bullets by picking up a container and dropping it over your head, for example. A player can find a lot of fun in endless experimentation and joy with success in finding new ways to interact and to solve problems by using physics. There are some incredible parkour stunts possible using physics and experience with the Boneworks game engine that can lead to taking shortcuts.
Some physics interactions work incredibly well. Using smaller objects such as mele weapons feel realistic, accurate, and precise. On the other hand, swinging larger objects lag slightly to simulate mass, but it gives the experience a slowed under water feeling. In contrast, the variety of firearms feel precise and the player has to realistically pick up a weapon, load, cock, aim, and fire it. Some guns are used with one hand, while larger weapons require using both hands to properly steady and fire them.
Boneworks is made up of huge and varied levels that can each take well over an hour to complete. Some of the entire levels are puzzles to figure out, and they are generally forgiving which allows the player to find his own path. There are always multiple ways to solve a puzzle but if you die, you’ll be sent back to the start unless you reached the checkpoint and saved which is now available in the patch. For those prone to VR sickness, having to start over and over may be a deal breaker, and sometimes it takes a very long time to run back to where you died.
Although Boneworks is a physics playground, precisely controlling what you want to do can lead to extreme frustration. For us, the imprecision of the Vive Wand controls were sometimes at odds with the game’s physics engine which while attempting to add object mass often interferes with a player’s real position. In addition, climbing often caused extreme shakiness, but it has been improved by the patch together with easier interactions. However, there are some other controls that are quite counter-intuitive.
To crouch requires a player to press up on the right Vive Wand’s trackpad (pressing down causes you to jump a little; holding down and letting go causes a full jump), but it is only a half-crouch which will not allow you to pass some areas without physically crouching also. In order to fully crouch, you have to first press up and then slide your thumb down to the button of the trackpad. This also works to move your body over a ledge although you may still jump afterwards.
It takes a lot of practice to get the interactions right, but when it works out, the results are pure VR interactivity magic that makes Boneworks a rare and unique game. Although we found the single player campaign to be poorly-paced, tedious, overlong between checkpoints, and lacking a good story, the sandbox mode makes up for it without even considering the Arena wave shooter mode. And Boneworks has a new Brawl mode beside the new single player campaign to complement Arena and Sandbox mode so there is endless replayability.
Some areas are still not well-optimized as others although generally Boneworks runs well on our five tested RTX video cards. In other areas, the sheer number of enemy NPCs together with their physics interactions can drive many players’ framerates into reprojection requiring Motion Smoothing if their CPU is not up to the task.
The Verdict
Boneworks is an incredibly addictive and fun interactive VR sandbox with a fair game attached that has been greatly improved by this week’s patch. The intense player frustration that sometimes resulted by struggling to control interactions may be improved now, and the shakiness and instability that occurred with climbing is better. Above everything else, implementing in-between checkpoints may help gamers play in shorter segments instead of forcing them to play to reach the end of a level.
Boneworks is an endlessly replayable physics based VR sandbox with such unbelievably well-done interaction capabilities that we wish it had a better campaign attached to it. We can’t wait for Half Life: Alyx.
Let’s look at performance.
Performance
Boneworks is made on the Unity engine and it particularly benefits by allowing for high levels of MSAA up to 8X. Boneworks is a poster child for NVIDIA’s VRSS as SSAA (Supersampling antialising) may be used to increase the shading rate on top of the 8xMSAA, and the IQ improvements are very noticeable. For players with a lower-end PC, any of its settings may be adjusted downward, and using Adaptive resolution or subsampling via SteamVR may be good options. 
Boneworks has average to very good visuals and it generally runs well on an i7-8700K overclocked to 4.8GHz and a RTX 2060 with maxed out setting options that include Ambient Occlusion and Shadow detail (BTR’s “Default” for benchmarking in the Frametime Plot below). Setting Max shadows will just default back to Medium as its highest setting will not stick, but the setting that most impacts performance is MSAA and we use 8X for the best default in-game visuals.
Here are all five RTX cards tested by FCAT-VR using the highest in-game settings including 8xMSAA. We used the latest GeForce 442.19 drivers. Please note that Motion Smoothing/reprojection are off.
Boneworks runs very well on its highest settings with plenty of performance headroom for most of the cards in the vast majority of the game where there is no extreme interaction with physics and mass NPCs. Only the RTX 2060 has occasional performance headroom issues delivering at 90Hz and will occasionally require Motion Smoothing to manage a steady framerate even though its unconstrained framerate is above 111.7 FPS.
Let’s look more closely at the RTX 2060 frametime plot, this time comparing default performance with 200% super resolution, and with adaptive and always on VRSS.
Playing Boneworks with a RTX 2060 using maximum in-game settings are playable without Motion Smoothing for most of Boneworks, but using 2xSupersampling or 200% Super Resolution is clearly ill-advised (pun intended) as frames will probably get dropped even with Motion Smoothing.
Very interestingly, using VRSS provides similar performance to using default settings which bolsters NVIDIA’s claim of free performance by tapping a card’s headroom to intelligently apply supersampling for better IQ in the foviated center of the screen where it matters.
The Boneworks devs recommend that their game not be played with constant reprojection or Motion Smoothing applied. Continually delivering frames at half the HMD’s optimum rate refresh using Motion Smoothing tends to cause artifacting and it may even cause some upset for the gamer.
Supersampling vs. Super Resolution vs. VRSS
For players with high-end PCs, what is commonly called Supersampling is an expensive if dramatic way to increase image quality in virtual reality, but VRSS can be used by everyone with similar performance to the default settings.
We were confused some of the technical terms in our original review. “Supersampling” is commonly used as an umbrella for different techniques including for resolution downsampling (super resolution) and for actual supersampling (SSAA), and these terms are often interchangeably used. NVIDIA’s Variable Rate Supersampling (VRSS) isn’t full screen supersampling in the traditional sense where a game’s images are rendered at a resolution above what a monitor can display and then scaled back down to its native resolution.
Supersampling generally refers to up- and down-sampling used to remove aliasing from images. A supersampled image is generally less pixelated with smoother transitions from one line of pixels to the other along objects’ edges. Supersampling can be further subdivided into multi-sampling antialiasing (MSAA) and supersampling antialiasing (SSAA).
MSAA (multi-sample antiliasing) addresses geometry edge aliasing by using specialized multi-sampled buffers in the graphics pipeline’s rasterization stage. A triangle is tested for coverage at its sample points using a coverage mask to identify which portion of a pixel it covers. The pixel shader is executed and the resulting values are shared across the samples and resolved by the coverage mask into a final buffer resulting in a less aliased image.
SSAA (supersampling antialiasing) works similarly except it executes the pixel shader for all of the covered samples thus accurately computing the color value for each sample location. SSAA operates inside the geometry while MSAA only functions along its edges. However, the main problem with SSAA is that it is incredibly performance intensive compared with MSAA.
Supersampling is particularly useful for improving the IQ of high resolution textures, high frequency content, and for alpha channel textures like for text or for fences. However, supersampling doesn’t work so well for improving the IQ of low detail textures or for flat shaded geometry.
To add to the confusion, supersampling is often confused with super resolution since both methods often show similar visual improvements. Supersampling uses additional samples per pixel without increasing the surface resolution while Super resolution uses a higher surface resolution which is resolved to a lower resolution.
Not long ago, SteamVR referred to super resolution as supersampling, and 1.4X (supersample) would equal what is now 140% (super resolution), for example. From now on, BTR will use the term super resolution to refer to the up- and down-scaling of images instead of supersample as previously.
Antialiasing is a byproduct of super resolution. The purpose of using super resolution in VR is to increase the apparent resolution as long as the density of the rendered pixels in the center of the screen is matched to density of the physical pixels. Higher super resolution increases the viewer’s perceived resolution in the HMD because of temporal and spatial downsampling and the images are also less aliased.
Super resolution reduces blur by shifting the pixel grid slightly allowing more resolution data to be extracted by comparing the differences between the frames. These differences become more well-defined when the source image is rendered at a higher resolution which allow for more accurate calculations which gain image sharpness.
Downsampling or compositing from a higher resolution source increases a HMD display’s resolving power. This does not happen in pancake gaming where the rendering resolution has to be matched exactly as a 2x multiplier to the screen resolution. VR barrel lens optics are distorted and do not need to be matched like the desktop for IQ improvement to occur. There is also human perception involved and using Super resolution, can have a very dramatic effect of improving the IQ – the scene overall gets sharper and less aliased, and even the screen door effect may be minimized.
From experience and observation, there is rarely a need to use anything higher than 200% super resolution. Super resolutions below 120% sometimes degrade the IQ due to VR lens optics, but from about 130% to 170% appear to give the best bang for visual buck with diminishing returns after that. 200% Super resolution does not look much better than 170%, and 400% Super Resolution does not look anywhere close to twice as good as 200%. You will be hard pressed to find visual differences although your performance will tank.
NVIDIA’s fastest gaming video card cannot manage to run Boneworks at 400% super resolution and it is not a pleasant experience playing with always on Motion Smoothing and still have dropped frames. 400% super resolution looks good, but not much better than at 200% super resolution where a RTX 2080 Ti delivers 100% of its frames at a steady 90Hz.
From the FCAT-VR details above, the unconstrained framerates show just how demanding high super resolution settings can be. The question becomes, how does VRSS IQ compare since its performance is very similar to default?
VRSS selectively supersamples the central region of a frame by what is called fixed foveated supersampling. Foveated rendering concentrates on the area where the eye focuses within a VR headset as the center of visual complexity. It works outwards towards the edges of the foveated area that is mostly out of the eyes’ focus so the rendering fidelity is reduced giving the GPU less work so it can maintain a steady 90 FPS. It’s somewhat similar to pancake gaming where visual complexity is based on depth instead on the distance from the center of focus for VR foviated rendering.
VRSS uses Turing’s Variable Rate Shading (VRS) abilities to apply different shading rates across its render target concentrating only on supersampling the foviated central region where a player is looking. No additional resources are wasted on a scene’s peripheral region where a player is not looking.
NVIDIA offers VRS and VRS Wrapper for foveated rendering to make it easier for developers to integrate eye tracking with supersampling which may become important especially for the upcoming generation of HMDs. Like VRSS, both VRS and VRS Wrapper use MSAA buffers for supersampling.
VRSS uses VRS technology for foveated supersampling in VR. On a desktop display, VRS intelligently does less work to increase performance. In a VR HMD, VRSS intelligently does more work to increase IQ as long as there is performance headroom.
Since VR has a fixed refresh rate (the Vive Pro is locked to 90Hz without Motion Smoothing), VRSS improves image quality while maintaining a fixed cadence. At 90Hz each frame is rendered in around 11ms. Without VRSS, if a frame is rendered in less time, the GPU waits for the next frame. With VRSS, the GPU uses its waiting time to supersample the center of the scene by increasing the shading rate while still keeping the total frame time delivery at a fixed 11ms.
VRSS is based on the MSAA level set by the player in the game although there are different components in play; the VR game provides 4x or 8x MSAA and VRSS is leveraged on top of it to apply corresponding shading rates of 4x or 8x.
For example, if MSAA is set to 4x, then VRSS in Adaptive mode will create a 4x shading rate region in the center of the screen that will expand and contract based on having sufficient performance headroom. If MSAA is set to 8x, then VRSS in Adaptive mode will create a 8x region in the center of the scene that expands and contracts. If Always On is set, a fixed area in the center of the scene will be created corresponding to the MSAA set in game but it will not expand nor contract.
VRSS vs. super resolution IQ
We use Boneworks in-game text for the IQ comparison example. In lower resolution HMDs like the original Vive and the Rift CV-1, text is especially hard to read. The letters are aliased and blurry even with 8xMSAA applied and if there is performance headroom, super resolution can be used to clean up the entire scene – albeit at a high performance cost.
The highest Boneworks in-game settings including 8xMSAA are used for all three examples. On the left is the Default text which is blurry and pixelated. In the center, ‘Always On’ VRSS, and on the right is 2x or 200% Super Resolution as set by SteamVR. In our opinion, the VRSS IQ is roughly equivalent to or slightly better than 200% Super Resolution, especially when viewed inside VR.
VRSS or super resolution IQ is something that any enthusiast with a HMD and a RTX video card can easily compare for themselves. Switch between the settings on the desktop by positioning your hand on your mouse beforehand so you can click to change settings while inside VR. The IQ improvements from super resolution or from VRSS can be substantial.
VRSS has been enabled in the latest 442.19 driver as a global setting as Adaptive or Always On. To work with VRSS, a MSAA-enabled forward rendered DX11 VR game needs to be profiled by NVIDIA for Turing video cards and there are currently 24 games that use it.
Although IQ based on VRSS, MSAA, or SSAA is hard to capture in VR screenshots, this additional example of Spiderman: Homecoming can show some of the differences between VRSS on versus off.
VRSS uses the Turing architecture to improve the image quality at the center of the screen. By using adaptive VRSS, a VR gamer doesn’t have to worry about having enough headroom to always display the best image quality like he would by using super resolution instead.
Conclusion
After spending a month with VRSS, we conclude that is is an excellent and automatic optimization that offers as much IQ improvement as possible, while not adding extra work that would cause dropped frames by using super resolution. By using adaptive VRSS, a VR gamer can play any VRSS-supported VR game using the maximum MSAA set in-game with adaptive VRSS without worrying about dropping extra frames as long as there is performance headroom.
Since VRSS is added to VR games by NVIDIA with drivers and does not require any dev support, we expect their list of 24 VRSS-supported games will continue to grow. The significant free VRSS IQ improvements may make an RTX card an important purchasing consideration for a VR gamer.
After the latest patch bringing new content, additional saves, and better interactions, we can recommend Boneworks as a superb physics playground with a decent game attached for already experienced VR gamers, and its IQ noticeably benefits from using VRSS without a super resolution performance penalty.
Next up, we’ll review a 1TB Team Group M.2 PCIe SSD before returning to our indie ray tracing series.
Happy VR(SS) Gaming!