In this blog series we’ll be looking at the current status of Virtual Reality with a special focus on mobile VR, where it’s heading in the future and what ARM® can do to help us get there. We’ll be covering the common challenges and pitfalls and how these can be avoided to produce a great VR experience. We’ll also keep you up to date with other blogs and discussions around technical tips, tutorials and FAQ’s; so stay tuned for the future of mobile VR.
Where VR is now
Virtual Reality isn’t new, people have been talking about it since the 90s, so why has the industry never quite taken off in the way we might expect? The quick answer is that the technology simply wasn’t there. The hardware was prohibitively expensive and very bulky and the graphics capabilities were too limited to produce a successful VR user experience - unless you consider motion sickness a success. Now however, lower cost hardware based on existing platforms is changing the game, with mobile platforms offering console-like performance. Not only that, but existing mobile devices already contain many of the sensors VR requires, from gyros to accelerometers, opening up a whole world of mobile VR possibilities.
What’s next for VR
The Virtual Reality industry has a forecast worth of US$30 billion by 2020, and that all has to come from somewhere. 
Fig.1 Digi-Capital VR Revenue Forecast
Gaming is of course a huge industry and a high-end, immersive gamer experience can now be literally at your fingertips. Mobile VR allows you to become fully involved in your chosen game at home, work, or while trying to escape the monotony of public transport; but that’s not all VR can do. Researching a university assignment can be a chore, but how about if you could visit the most relevant museums or seminars without having to leave the dorm? VR allows us to see exhibitions in world class museums and galleries without an expensive trip to London, Paris, or anywhere else. Shopping too, isn’t everyone’s favourite pastime, especially around the Christmas rush. Wouldn’t it be great if you could wander the aisles and compare options for your next car, sofa, TV or even pair of shoes, without tripping over pushchairs or being upsold by pushy assistants? All this is possible with the huge technical advances in VR and it’s only a matter of time until this is our standard way of working.
Fig.2 nDreams® Perfect Beach experience allows you to get away from it all without leaving the sofa
So how does VR actually work?
Technology is the key to VR success and this blog series will talk about exactly what you need to make it happen. VR comes in mobile or desktop options, but according to Oculus® Co-founder Palmer Luckey, desktop VR is seriously compromised by the requirement for a ‘cable servant’ to follow the user around preventing trip hazards. So mobile VR is the quickest way forward, and the simplest of the mobile options allows you to simply slot your smartphone into the headset and get started. The headset provides you with a stereoscopic display, with two marginally different images rendered for the left and right eye, allowing the user to experience depth. Barrel distortion is then applied to the rendered images in post processing to counteract the curvature of the lenses.
Fig.3 Marginally different images for each eye allow the perception of depth and barrel distortion applies curvature to the image to counteract the curvature of the lens
Finally, sensors in the device detect the movement of your head and adjust the scene in real time to render the updated view to the headset and allow realistic visual feedback. Going forward, additional sensors will facilitate live hand-tracking for a truly immersive experience, and this can be combined with the use of an inbuilt or add-on controller to allow you to interact fully with your virtual surroundings.
VR Optimisation with Mali GPUs
As with any emerging technology, there are issues that can stand in the way of a truly successful VR user experience. These include low resolution blurring the image and compromising visual quality, or a low frame rate making the display appear stilted or jerky. A major issue experienced when developing for VR is latency, or the time it takes for the on-screen image to catch up with the user’s head movement, and this is one of the key causes of sickness or dizziness in VR users.
The ARM® Mali™ GPU family is the world’s #1 licensable GPU in terms of shipments and is perfectly positioned to deliver an optimum VR experience. Mali GPU architecture enables high resolution and power saving through various features such as Adaptive Scalable Texture Compression (ASTC); and ARM Frame Buffer Compression (AFBC) dramatically reduces system bandwidth, with performance fully scalable across multiple cores. Mali support for extensions to OpenGL ES and EGL reduce latency and improve overall performance.
What we’re doing now
At events like VRTGO ARM recently demonstrated how great a mobile VR experience can be with the Mali-based Samsung® Gear VR headset, a collaboration from Samsung Mobile and Oculus. The first version was based on the Galaxy Note 4, with the second generation now available for the Galaxy S6, both powered by the Mali-T760. The Ice Cave Demo, featuring Geomerics Enlighten global illumination in collaboration with RealtimeUk; was easily ported to VR on the Samsung Gear VR headset; read about how we did this here.
Superior visuals and a smooth user experience are all possible in mobile VR and throughout this blog series we’ll be discussing the common challenges surrounding developing for VR and how ARM’s technology and Mali GPUs can help you overcome them.
Stay tuned for more on mobile VR!