BMW Natural Interaction presented at Mobile World Congress 2019

By CARSIFU | 27 February 2019


MUNICH: The BMW Group is highlighting a "natural" way for drivers to interact with their vehicles at the Feb 25-28 Mobile World Congress in Barcelona.

The BMW Natural Interaction system combines the most advanced voice command technology available with expanded gesture control and gaze recognition to enable multi-modal operation for the first time. The first BMW Natural Interaction functions will be available in the BMW iNEXT electric car from 2021.

Just as in a dialogue, BMW Natural Interaction allows the driver to use his voice, gestures and gaze at the same time in various combinations to interact with a vehicle.

The preferred mode of operation can be selected intuitively, according to the situation and context. Voice commands, gestures and the direction of gaze can be reliably detected by the vehicle, combined and the desired operation executed.

P90337444_lowRes_bmw-group--mwc-2019-


This free, multi-modal interaction is made possible via speech recognition, optimised sensor technology and context-sensitive analysis of gestures. Through precise detection of hand and finger movements, gesture direction – in addition to gesture type – is also registered for the first time in an extended interaction space that encompasses the driver's entire operating environment.

Spoken instructions are registered and processed using Natural Language Understanding. An intelligent learning algorithm, which is constantly being refined, combines and interprets the complex information so that the vehicle can respond accordingly. This creates a multi-modal interactive experience geared towards the driver's wishes.

By combining different modalities, vehicle functions can be initiated in different ways. The driver decides how he wants to interact, based on his personal preferences, habits or the current situation.

So, when the driver is engaged in conversation, he would probably choose gesture and gaze control; when his eyes are on the road, it is better to rely on speech and gestures.



In this way, for example, car windows or the sunroof can be opened or closed, air vents adjusted or a selection made on the Control Display. If the driver wants to learn more about vehicle functions, he can also point to buttons and ask what they do.

With enhanced gesture recognition and the car’s high level of connectivity, the interaction space is no longer confined to the interior.

For the first time, occupants will be able to interact with their direct surroundings, such as buildings or parking spaces. Even complex queries can be answered quickly and easily by pointing a finger and issuing a voice command. “What’s this building? How long is that business open? What is this restaurant called? Can I park here and what does it cost?”

“Customers should be able to communicate with their intelligent connected vehicle in a totally natural way,” said BMW Group Electronics senior vice-president Christoph Grote.

“People shouldn’t have to think about which operating strategy to use to get what they want. They should always be able to decide freely – and the car should still understand them. BMW Natural Interaction is also an important step for the future of autonomous vehicles, when interior concepts will no longer be geared solely towards the driver’s position and occupants will have more freedom.”

 

Keywords