The automotive industry is rapidly moving forward with applications of disruptive technologies. The expansion of sensor technology and the huge wave of electrification is enabling advanced driver assistance systems (ADAS) and autonomous driving.
Chipsets in cars are being updated to ensure the performance and connectivity we expect from mobile devices. A new paradigm of software-defined vehicles has been introduced and accepted to be the industry norm. However, until recently, the in-car user experience driven by software improvements has struggled to keep pace.
HERE Technologies and Unity development partnership leverages the strengths of both parties, to provide drivers with richer driver experiences.
Ahead of the game in advanced driver assistance systems
Unity is the leading provider of real-time 3D (RT3D) solutions that is turning its innovation towards creating products for the business sector. “Unity is a company that’s all about, empowering creators,” says Derek Burrows, senior product marketing manager. “It’s done so in the gaming industry for decades now, but we have skills that are easily transferred to other markets. – Unity has seen them explode in the last couple of years.”
Unity is well-known for its leadership in the mobile gaming sector. Approximately 72% of the top 1,000 mobile games are powered by the Unity engine, while it also maintains a dominant position in powering augmented reality (AR) and virtual reality (VR), and mixed reality (MR) experiences. “The core of our technology is a real-time 3D platform, which lets you build experiences in real-time without having to render. You can play it as you’re building it,” says Burrows.
“The visuals in cars have stagnated for a little while,” says Burrows. Unity can focus the innovation it brought to mobile gaming to bring us a more fulfilling automotive experience. Typically, Unity workflows translate particularly well to the world of the human-machine interface (HMI).
“We have a huge opportunity in automotive HMI,” says Burrows. “One of our unique selling points is combining that high-visual fidelity – those beautiful experiences – with our leadership in mobile deployments and deployments to lower powered chipsets.
“ Frequently, you’re not putting a 3090 [a graphics card with an exceptionally powerful graphics processing unit] in your car; you aren’t running the latest graphics, hardware, and the highest quality chips, so you have to find the right balance of system on a chip and an automotive operating system that works.”
Because Unity has been so effective at optimising its experiences for different screens, processors, and graphical capabilities, it is well-suited to the world of embedded deployment and building those experiences on any hardware and software combination that can be thrown at it.
Mapping a course for the future
“We met for the first time at the CES® show. Unity is a market leader in real-time 3D in the mobile gaming world, so HERE had already adopted us for demoing their 3D maps, but, when we saw the demo they showed us at the show, it was really stunning – an astounding 3D city, based on real data from San Francisco,” says Zhai.
That moment triggered a partnership and Unity’s continued growth in the automotive industry. “We thought, ‘Nice looking high-fidelity 3D maps and automotive…. What can we do with that?’ The first thing that popped up in our minds was a 3D navigation application,” says Zhai.
A 3D navigation use case project commenced in March 2019. It took three Unity engineers six months to complete the proof of concept. One of the main challenges, says Zhai, was importing the massive 3D datasets HERE had collated for the city of San Francisco and loading that into the memory of a relatively low-end device in terms of its capacity. “We’re not using a beefy game PC to load everything. We had to challenge ourselves by using real automotive hardware that could still load big datasets so we can render with high fidelity and multiple levels of details.”
Another major challenge using this type of navigation mode was thinking through the user journey and how to leverage the 3D information presented in combination with real-time rendering technology and a high-fidelity map to make the journey better. “We needed to improve the driver experience, by presenting certain information at a certain time,” says Zhai.
“If you look at the latest Mercedes EQS series, for example, it uses huge displays in the cars – you feel the contents of every pixel. So, leveraging real-time 3D technologies is the driving factor that is demanding manufacturers to introduce this kind of innovation.
So, where does Zhai see the next-generation HMI with state-of-the-art rendering capabilities heading? “I foresee more content being introduced into the cars. When Elon Musk tweeted that they were ‘porting Unity’, they were introducing new games via Tesla Arcade [the newly launched Tesla gaming computer, unveiled in the new Model S and Model X]. It’s a game hub for its drivers, with more entertaining content and a more user-friendly interface for innovation such as advanced driver assistance systems and AD [autonomous driving] technologies,” he says.
This, says Zhai, helps to make driver interaction with new technology easier and safer. It also clinches the buying decision, in his opinion, “It’s not the ADAS and AD technology that sells a car and sells the technology, it’s the user interface [UI] that sells it. When you see the UI then you know what the technology really is.”
Memory lane
LUXIA is essentially broken down into three segments. A traditional infotainment application, a main navigation element, and finally an ADAS ‘auto pilot’ segment. The latter can boast a new lane-level advanced driver assistance systems rendering. HERE Lanes uses location intelligence to enhance a car’s sensors, helping the vehicle find and stay in the correct lane and providing the driver with detailed lane-level visual guidance. “It’s ‘micro-navigation’,” says Zhai.
“Let’s say you need to change to the right-most lane; with this technology you can smoothly accept the route, knowing what’s upcoming. This type of manoeuvre and navigation can only be done with this kind of high-fidelity map. It can warn you if there is another car coming up behind you, making your navigation much safer.”
Presenting information selectively is also critical. When a driver begins a journey, and there is still plenty of distance to their destination, they can use LUXIA to zoom out and get an overall view of the journey. When they get closer, the driver can zoom in, and see much more detailed, real-world information to benefit them – for example, when a driver needs to find a parking spot in a city centre, the micro-level navigation can show which specific basement in a parking facility is available.
LUXIA’s adaptive means the 3D display can be used for screens of different sizes and adapted for both high-end vehicles and those of a more affordable level with a more basic display. “One of Unity’s strengths is we’re able to also adjust our virtual cameras that are capturing the navigation images and then basically render them in front of the driver. So, the resolution of the camera can be adjusted according to the screen resolution,” says Zhai.
There is, says Zhai, huge, untapped potential for powering dynamic infotainment immersive augmented geospatial and media experiences all integrated into this seamless, seamless kind of user experience. The potential is there now, he says, to introduce, for the first time, so-called ‘geolocation-based’ services. LUXIA can provide basic level navigation, but it can also be overlayed with additional data, such as information from shopping centres, parking lots, or even coffee shops via Yelp, for example, on a real-time basis.
HERE and Unity’s collaborative relationship has produced outstanding results, says Zhai. “With both of our offerings, it’s a really good match. HERE is dominant in the navigation space, especially for the automotive industry, and has these huge, detailed datasets of cities and highways and all kinds of geographic information. That really complements Unity’s high-end, highly scalable rendering technology. By combining the strengths, we have enabled a lot of very innovative applications.”
And that feeling is entirely mutual. “Unity’s leadership in real-time 3D technology [RT3D] led us to quickly develop a joint IVI proof-of-concept that is created on Unity’s platform and features 3D data from HERE to showcase what can be achieved with the addition of HMI workflow design improvements,” says Ronak Amin, product marketing manager for Connected Driving at HERE Technologies.
“Unity’s RT3D rendering capabilities complement the suite of HERE-connected vehicle solutions for the next generation of automotive HMIs, which are critical to consumer expectations.”
According to Amin, a winning partnership such as this makes for an exceptionally bright future for dynamic infotainment. “The next generation of in-vehicle experiences will move toward digital cockpits, with multiple screens, seamlessly delivering vehicle information and next-generation infotainment to passengers. As the leading location platform company serving the global automotive industry, HERE can continue to partner with companies such as Unity to build the future of connected vehicles and in-vehicle experiences.”
