In a blog post released last week, Microsoft, the company behind the popular HoloLens 2 mixed reality (MR) headset, introduced “Moving Platform” alongside German automotive giant Volkswagen.
The experimental project is an augmented reality (AR) service for the HoloLens 2 device with an accurate heads up display (HUD) to assist with driver navigation and safety while a vehicle is in motion.
The feature is an immersive solution for operating a vehicle using AR without causing drivers motion sickness or creating defective visuals.
Microsoft and Volkswagen Collaboration: The Story Behind HoloLens 2’s Moving Platform Feature – https://t.co/jzStmlIixC pic.twitter.com/Df1NYgMk8c
— MSPoweruser (@mspoweruser) May 6, 2022
The partnership, formed in 2018, had to overcome various technical problems while developing the Moving Platform such as motion sickness and disrupted outward tracking.
According to Microsoft, the immersive application is already receiving attention from global maritime companies, following Volkswagen’s long-term goals of sharing the service across different industries.
Moving Platform is available for training workers to face challenging road conditions or create new user experiences for autonomous vehicles.
A Deeper Look
Volkswagen’s interest in extended reality (XR) automotive technology kicked off in 2015 with the firm’s Race Trainer system that creates AR visuals like arrows and waypoints to assist with driver navigation. The German company also debuted an AR navigational display for its ID brand of cars in 2020.
Volkswagen employed the HoloLens 2 as the hardware foundation for its Moving Platform AR service in its most recent collaboration with Microsoft.
Creating a Rich Solution
Despite this, the Volkswagen research team, led by Dr Andro Kleen, Head of the Data Science Team at Volkswagen Group Innovation and Marc Pollefeys, the Director of Microsoft’s Mixed Reality and AI Lab in Zurich, Switzerland, noted that integrating HoloLens 2 devices in a moving vehicle causes many technical issues.
The Volkswagen team found that the outward-facing sensors lose their tracking ability when HoloLens 2 devices were in moving vehicles, causing holographic visuals to fail to appear, sparking the need for a motion-ready AR solution. The partnership solved this by creating a spatial tracking system that follows the position of a moving vehicle and separately tracks outside positional data.
Michael Wittkämper, an AR Expert at Volkswagen, added,
“We connected a positioning system that tracks the location of the vehicle. This way we were able to also place 3D elements such as information on point of interests outside of the car. This opens up completely new possibilities to not only display holograms within the driver’s forward-facing field of view, but also wherever the user wearing the glasses is looking”
At the moment, Moving Platform only supports use on large ships, although, Microsoft and Volkswagen confirm that it will debut for more vehicle use cases like bus driving, heavy machinery, and lorries. Microsoft also outlines other potentially supported environments, including elevators and trains.
