| looking at some state of the art examples of in-car AR systems on the market
MBUX – the newest infotainment system of Mercedes-Benz
2018 was the year when Mercedes Benz introduced their newest infotainment system called MBUX. This uses the front camera (originally used for parking) to create a live stream of the road ahead, combined with graphics of navigation hints or finding adresses. Since then it was continuosly improved and the latest version was revealed in 2021 in the S/EQS-Class models, featuring an AR navigation display and a HUD with distance assist, lane keeping assist and dinamic arrows showing directions.
Video demostrations of the 2021 MBUX system:
https://www.youtube.com/watch?v=hnRbi5UcJnw
https://www.youtube.com/watch?v=DCgy3askMcM
Audi AR HUD
Audi announced their augmented reality HUDs as an optional feature for the newest high-end electric SUV, the Q4 e-tron for 2021. The visual information shown in front of the driver are similar to the MBUX’s content. Audi explicitly defines two areas: the status field (in a visual distance of ca. 3 m) and the AR field (in a visual distance of ca. 10 m), which seems to be bigger than in the German competitor’s solution.
Demo video of the Q4 e-tron HUD:
https://www.youtube.com/watch?v=Ea6o-_smVk8
Hyundai and WayRay
Looking at HUDs further, Hyundai/Genesis was the first brand to implement laser-holographic AR head-up displays in their G80 model, presented by the young AR developer company WayRay in 2019. It is said to have tremendous benefits compared to past HUDs (using reflected LCD screens) in terms of precision and visibility for the driver.
The Swiss startup WayRay claims to be the only company to have implemented holography to HUDs. Their holographic optical elements (HOEs) in their displays should provide unprecedented 3D images while remaining transparent and capable of being bent to windshields. The company presents its uniqueness in the field by covering “deep-tech” holography hardware development (e.g. blue-laser beams) as well as software development, all realised in-house.
They have already received large fundings by Hyundai and Porsche, have presented a 180° AR cockpit experience and offer different add-on solutions for vehicles, boats and airplanes. Their newest project is a shared car concept (“Holograktor”), designed for the “Metaverse” with a complete gaming / working / learning possibility while traveling autonomously. In their cooperation with Pininfarina on a concept car, they proposed solutions of the “True AR” displays also for side windows, providing new ways of passenger infotainment and entertainment experiences.
A report from the FIA Formula E on WayRay’s developments also predicts the use of HUD systems for race cars in the future. The pilots behind the wheels could get visualized ideal racing lines, brake points or a ghost car to chase on the race track.
Hyundai’s In-Car Noise Cancelling
Besides HUDs, Hyundai is pushing the development of AR solutions in cars in other aspects as well. Like in our headphones, noise cancelling also found its way into the car interiors, bringing more comfort to the passengers. According to Hyundai, the earlier systems were only capable of masking steady engine noises, but their newest solution (“Road Noise Active Noise Control”) in the upcoming Genesis GV80 will be capable of deleting different tire noises at different speeds. It uses multiple microphones placed directly into the wheel wells, accelerometers, amplifiers and a digital signal processor. As a result of the complex calculations for each individual wheel, the in-car noise should be reduced by half (3 dB).
Engine sound enhancements
Writing about noises of the car, we also have to take a short look at the opposite effects to noise cancelling – the engine sound enhancement devices. Due to the downsizing of the engine displacements, the roars coming from the combustion got also reduced. To keep the emotions connected to sporty engine sounds though, manufacturers are using additional devices to create compensating sound effects.
These can be pipes from the intake manifold connected to the dashboard walls, in some cases with an extra flap to control the sound throughput only for the sporty driving situations (Toyota, Ford, Porsche).
BMW was known to use engine sound amplification through a synthesised reproduction of the actual engine noise played simply on the car’s speakers.
The Volkswagen Group made it a bit more complicated by adding a special speaker device (“Soundaktor”) below the windshield to produce deep, buzzing tones resembling larger engine sounds. In some models there are also speakers built into the exhaust pipes to alter the natural noises coming from the engine, to make them more emotional or masculin.
3D ADAS system of Arkamys
Beeping noises in a car are existent since many years, with the intention to help drivers. But beeping on itself is not always enough to give an understandable signal about what is happening or dangerous around the driver. The company Arkamys presented an intuitive alerting concept for Advanced Driver Assistance Systems – parking, lane keeping, blind spot and other assistants – by placing many different speakers inside the cabin and generating a 3D sound experience. With this it is possible to signalize the direction where a possible danger can exist, making the recognition and processing of the information easier and more intuitive for the driver.
Electric cars
Electric cars are further good examples where in-car noise generators are used to give the driver and passengers the known feeling of vehicle driving dynamics. Porsche is a perfect example where specific sounds are developed for representing the brand’s identity within the driver experience. They call the system “Porsche Electric Sport Sound” that enhances some natural noises of the drivetrain but also reduces disturbing ones, while implementing sounds to compy with the legal regulations for electric vehicle alerting sounds.
Thinking further about sound augmentation in cars, probably the already most spread system is the parking assistant, giving beeping sound feedback on the remaining distance to obstacles around the car. The design of these systems could probably fill a chapter on its own, but as it is already an everday tool, I won’t go further into detail on it.
The above listed examples are not even close to a complete list of use cases. Therefor I want to further research the current technologies. The next step will then be to look into the reasons for these systems, why they were developed and what practical needs, feelings and experiences are the underlying causes.
Sources
Online article on Wired: With In-Car AR, Drivers Get a New View of the Road Ahead. Retrieved on 05.12.2021
https://www.wired.com/story/in-car-ar-drivers-get-new-view-road-ahead/
Article on Wired: Hyundai’s Luxury SUV Mixes Mics and Math for a Silent Ride. Retrieved on 05.12.2021
https://www.wired.com/story/hyundai-genesis-gv80-suv-noise-cancelling/
Online Article on FIA Formula E: How AR and VR are revolutionising the car industry. Retrieved on 05.12.2021
https://www.fiaformulae.com/en/news/2020/june/ar-vr
WayRay – offical website. Retrieved on 05.12.2021
https://wayray.com/#who-we-are
https://wayray.com/press-area/#media_coverage
Online article on WayRay by CNET and Autocar. Retrieved on 05.12.2021
https://www.hyundai.news/eu/articles/press-releases/hyundai-wayray-unveil-next-generation-visual-technology-at-ces-2019.html
YouTube video by Roadshow: CES 2019: WayRay’s holographic AR windshield is real, hitting the road soon. Retrieved on 05.12.2021
https://www.youtube.com/watch?v=HFIgjQI2E6Y
AutoCar article on the Pininfarina concept car. Retrieved on 05.12.2021
https://www.autocar.co.uk/car-news/new-cars/pininfarina-concept-car-showcased-holographic-ar-display
Online article by AutoZeitung: Mercedes entwickelt MBUX weiter. Retreived on 05.12.2021
https://www.autozeitung.de/mercedes-infotainment-192628.html
Mercedes-Benz MBUX System – online articles and images, retrieved on 05.12.2021
https://www.wired.com/story/in-car-ar-drivers-get-new-view-road-ahead/
https://www.extremetech.com/extreme/314758-2021-mercedes-s-class-2-hud-sizes-level-3-autonomy-4d-sound-5-lcds
https://carbuzz.com/news/new-mercedes-s-class-shows-off-amazing-augmented-reality-display
Audi AR HUD system: online article and Youtube video on Slashgear. Retreived on 05.12.2021
https://www.slashgear.com/the-audi-q4-e-trons-augmented-reality-head-up-display-is-dashboard-genius-09662735/
https://www.audi-technology-portal.de/de/elektrik-elektronik/fahrerassistenzsysteme/audi-q4-e-tron-ar-hud-de/
Online article on GeekDad: Augmented Reality for Your Ears. Retrieved on 01.02.2022
https://geekdad.com/2016/02/arkamys/
Image of Mercedes G350d soundmodule. Retrieved on 01.02.2022
https://www.tuningblog.eu/kategorien/tuning-wiki/soundgenerator-nachruesten-232502/
CarThrottle article on sound enhancers. Retrieved on 05.12.2021
https://www.carthrottle.com/post/5-ways-that-manufacturers-enhance-the-sound-of-their-cars/
The Porsche Sound – online article, retrieved on 05.12.2021
https://newsroom.porsche.com/de/produkte/taycan/sound-18542.html
