VW And Microsoft Team Up To Bring Augmented Reality HoloLens To Cars

Volkswagen and Microsoft are envisioning a future where augmented reality glasses will enable drivers to get “holographic displays of traffic information, weather conditions, shopping recommendations, and architectural highlights along the way.”

Since Volkswagen sees augmented reality as “one of the key components of future mobility,” they partnered with Microsoft to enable the latter company’s HoloLens 2 headset to work in a moving vehicle for the first time.

As Microsoft explains, Volkswagen was keen to use the original HoloLens on their Race Trainer Golf R from 2015. However, when they tried to incorporate the headset into the vehicle, “sensors lost tracking” and the holograms it normally displayed disappeared. Volkswagen approached Microsoft about the issue and the two companies begin collaborating to find a solution around 2018.

Also Read: VW’s Augmented Reality Head-Up Display Tech Is Properly Futuristic

This wasn’t easy as the software giant noted HoloLens uses “two main types of sensors that measure its motion – visible light cameras and an inertial measurement unit, or IMU, that gauges acceleration and rotation speed.” Since these sensors were designed to replicate how humans see and move through the world, that setup wasn’t compatible with a fast moving vehicle.

To address the issue, a Moving Platform mode was developed that uses an algorithm to model discrepancies between the sensors and allow the HoloLens to continue tracking. Researchers at Volkswagen then established a bidirectional data connection between the vehicle and the HoloLens, which enables the latter to display real-time information from the vehicle as well as control some of its features.

This opens up interesting possibilities and moves augmented reality from head-up displays to glasses on your face, so you can see information regardless of where you’re looking. Microsoft didn’t say what Volkswagen is envisioning, but pictures show augmented reality navigation as well as a holographic pedestrian warning system, which appears to highlight pedestrians in crosswalks.

The company also showed “mixed reality” climate controls, which apparently enable users to adjust the climate control system without having to touch physical buttons or switches. Of course, there are endless other possibilities and it’s easy to imagine how an assortment of other safety information could be displayed with augmented reality glasses.

There’s no word on when augmented reality glasses could be used in your next vehicle, but Volkswagen’s Dr. Andro Kleen said “We think mixed reality information is the most intuitive information we could provide to enhance our customers’ user experience, because what you see there, and what you need to process, is very close to what humans normally see and process. It’s not so abstract.”