Human-Machine Interface: "Today more than yesterday and much less than tomorrow"
Download newspaper
Archives 
Human-Machine Interface: "Today more than yesterday and much less than tomorrow"
Human-Machine Interface (HMI) was originally simply the study of the interface between humans and machines. However, it is now at the center of essential research around the cognitive, material, software and social phenomena that affect the implementation of physical and digital tasks. The primary challenge is to build high quality interfaces. However, to do so, research must consider the entire interactive system. As users work in environments that are more and more interactive, this characteristic has become essential to the field. The second important challenge is mobility. The Grenoble Alpes site has a long history with this subject thanks to research initiated by Joëlle Coutaz (see page 4). Her perspective highlights two types of contributions: first, “fine-point” contributions consisting of new techniques for interaction, and second, “large scope” contributions that create new methods of design or evaluation. Each type of contribution has its own set of problems. Technological progress is opening the way for a new phase in the evolution of HMI. What will the third era of HMI hold for society? As the frontiers between physical and digital worlds dissolve, the roles of users, designers and systems are becoming blurred. This is the world of HMI, a fabulous opportunity for research and education that focuses on challenges directly tied to companies and the evolution of our territory.
Gaëlle Calvary
Researcher at LIG and professor at Grenoble INP-Ensimag; Deputy vice-president for the Grenoble INP Scientific Committee
Interacting with machines
The question of human-machine interactions has been around since the invention of the computer. Over the past 50 years, progress is terms of human-machine interfaces (HMI) has rendered IT systems accessible to the general public. Today, machines are part of our daily lives and our modes of interaction are diversifying to match all types of situations.
Since the creation of the screen, mouse and keyboard: “significant progress has been made in terms of understanding the phenomena tied to interactions between humans and systems. To create a quality interaction, you have to understand the user’s goal, the environment in which the user acts, and the physical and social resources available. The engineering challenge is to deliver a product that is useful and easy to use in a given context. In other words, it has to match the needs and abilities of the user in a precise situation all the while controlling development and maintenance costs,” explains Gaëlle Calvary, a researcher at LIG and a professor at Grenoble INP - Ensimag.
The panel of options available for human-machine interactions has continued to expand from the traditional screen, keyboard and mouse to include solutions such as virtual reality goggles or connected objects. Interface are no longer limited to graphic designs. All senses are being explored, and in particular, the voice, vision, touch and even thought!
Blurring the border between virtual and real
The creation of new technologies for augmented and virtual reality have blurred the border between the real and virtual worlds. For example, François Bérard, a Grenoble INP professor and researcher at LIG, is developing very original interfaces known as Handheld Perspective Corrected Displays (HPCD). These devices can create the sensation of holding a virtual 3D object. "The idea is to provide the user with the most natural gesture." With the support of his team, he built a spheric HPCD that displays an image in a 30cm large polystyrene ball. "The image is modified in real time according to the user’s perspective which gives him or her the impression that the object is really inside the ball." Interacting with this spheric display offers advantages over flat displays and even a real object.
Researchers compared user performance when using HPCD, a flat screen and a touch-sensitive screen. The task given was to inspect complex, 3D virtual puzzles. Physical puzzles were also tested as a reference point. Surprisingly, the researchers found that users were more efficient with all of the virtual puzzles than they were with the physical ones.
Frédéric Noël, a researcher at G-SCOP is also exploring the frontier between real and virtual worlds. At the lab, researchers are using powerful tools to be completely immersed in virtual worlds (tools such as virtual goggles and headsets or even a 3D virtual space known as a mini-Cave). "We can record the movement of a user’s head in order to adapt the virtual image they see according to their movements. There are a variety of ways to interact: movement recognition thanks to cameras, accelerometers, voice recognition, etc. This creates new opportunities for professional applications in particular as they can meet the needs of the upcoming industry 4.0," explains Frédéric Noël.
Sharing information to improve our actions
The link between virtual and real worlds opens the door to extraordinary possibilities. Stéphane Ploix, a Grenoble INP researcher and professor at G-SCOP, is coordinating an ANR project with the goal of providing users with relevant information to understand energy consumption and adapt their behavior to limit their impact on the environment. The research carried out will help create systems that can anticipate situations and provide explanations or recommendations. It’s a multidisciplinary approach that requires in-depth reflexion to find the right balance between creating an interactive "tool" and a “compagnon”. While sensors can provide a wide variety of data, it is still difficult to interpret this data. The challenge of interactive visual design is to find the best way to make "data speak". At Inria, Georges-Pierre Bonneau is studying, for example, how to visually represent energy consumption in buildings.
Inventing models and prototypes for the virtual world
To explore a virtual world, you first have to create it. And to do so, you need models that will enable researchers to develop 3D content. At Inria, Edmond Boyer is working on modelizing objects in movement. Using video, he designs graphic models that can reconstruct a 3D scene it order to replay in in virtual worlds from a variety of perspectives. "You can create a model of a person that can be used in virtual worlds and enable you to interact with them in real-time." Several companies already use such tools to see holograms of dancers, fashion models or athletes.
When combined with 3D printing, this virtual exploration offers interesting possibilities for industrial prototyping. "We still have to define the best usage for each method and determine when a virtual model should give way to physical prototype. Our abilities and capacities are continuing to expand, so it’s important we implement these advances in a relevant manner," explains Frédéric Noël.
Céline Coutrix, an Ensimage engineer, is a promising young researcher who was recently awarded a CNRS bronze medal. She dreams of a physical world that can be transformed. In other words, physical objects that can change form. "The challenge is to develop modular physical interfaces that combine the best of both worlds: the sensation of real physical objects and the flexibility of software." The young researcher has developed rotatable buttons that can transform into linear buttons for a slider effect and a surface with tiny motorized nibs that can be raised one after another to mimic the sliding of a button. Thus, the physical interface adapts to the task and the the user’s interaction.
Brain-machine interface: interacting through thoughts
With his team, Marco Congedo (CNRS researcher at GIPSA-lab) designed and developed a video game platform inspired by Space Invaders. The new game, called Brain Invaders enables players to destroy aliens simply by thinking, without the use of any manual or physical object.
Using a headset that amplifies electroencephalographic (EEG) signals, the player focuses his attention on a screen where aliens are displayed. When an alien flashes to indicate it has to be destroyed, the player’s brain can emit a specific wave signal recognized by the system. Thanks to the expertise of the Vision and Brain Signal Processing team, the researchers created tools to extract EEG signals in real-time.
As a pioneer of this process, the team also used Riemannian geometry to classify the signals in a format that would enable the system to properly interpret a subject’s wishes. "This method won 5 international competitions out of 5!" Brain Invaders was published in 2011 and offered a fun and original means of demonstrating these capacities. Since then, another important hurdle was solved: overcoming the need for pre-calibration before use, which has generally been required for these types of interfaces.
As a pioneer of this process, the team also used Riemannian geometry to classify the signals in a format that would enable the system to properly interpret a subject’s wishes. "This method won 5 international competitions out of 5!" Brain Invaders was published in 2011 and offered a fun and original means of demonstrating these capacities. Since then, another important hurdle was solved: overcoming the need for pre-calibration before use, which has generally been required for these types of interfaces.
Grenoble - a French pioneer of multimodal interaction
Joëlle Coutaz: emeritus professor at University Grenoble Alps and a pioneer of HMI in France.
Joëlle Coutaz graduated from Grenoble’s first master’s degree in IT in 1968. In the 80s, she worked in the U.S. This was the beginning of calculators being deployed for non-specialist users. When she returned to France with a Macintosh in her luggage, she decided to launch the field in France. "In the same manner IT was first seen at a tool for physicists, HMI had trouble being seen as a field in and of itself. This was pretty normal because the field is hard to define and borrows from many different fields such as mathematics, IT, electronics, psychology, sociology, robotics, etc. But you can define it as the study of phenomena related to the interactions between man and a digital system. The engineering challenge is to create an interface that is both useful and usable."
Uniting two worlds
At the beginning of the 80s, the creation of the computer mouse and bitmap screens revolutionized HMI by enabling the display of images instead of simply displaying characters. At this time, Grenoble was already home to two fields of study that had little interaction: Graphical User Interfaces (GUI) and natural written or spoken language. Since the 60s, Bernard Vauquois, a CNRS researcher and professor at the Grenoble University for Sciences, had been working on the automatic translation of Russian into French. With the help of Jean Kuntzmann, he founded in 1960 the Center for the Study of Automatique Translation (now known as GETALP). Following a meeting in 1990 with Jean Caelen, who was a researcher at the ICP in Grenoble, Joëlle Coutaz initiated a connection between these two worlds.
This marked the beginning of multimodal interactions, which united graphic interfaces and speech recognition. Today, it also includes tangible interactions which aim to unite digital and physical worlds through the use of "physical objects" that can interact with the digital world. Currently, the trend is heading towards the disappearance of computers which can be integrated into walls of smart houses for example (currently being studied as part of EquipEx Amiqual4Home*).
*https://amiqual4home.inria.fr
Jean Vanderdonckt: professor of HMI invited to Grenoble INP
After a four year stint as president of the Louvain School of Management Research Institute (Belgium), Jean Vanderdonckt took a sabbatical year in 2016, during which he was invited as a professor to the LIG where he worked on HMI. He offered classes at Ensimag on the subject of fast HMI prototyping. The recorded classes are available online*.
"When studying computer science after having studied digital analysis and mathematical optimization, I realized that the most critical part of a software was not its algorithme but the interface to interact with humans. You can have the most powerful software, but it will not be useful if you cannot master the interface.
I decided to expand my knowledge of HMI by spending five months at Grenoble INP and the LIG. This period enabled me to discover new research subjects such as interfaces that can change form or persuasive computer science. The result was several joint publications**. At Grenoble INP, I benefited from the wide variety of complementary subjects as well as the experienced HMI team, several of whose members are pioneers of international renown.
My collaboration with Grenoble INP began thanks to several previous European projects such as ITEA1Emode, FP5 Cameleon, FP6 Similar, ITEA2 UsiXML and MultiPlex. The collaboration continues and has increased thanks to the current submission of an ITEA3 project dedicated to movement interaction. I hope our cooperation will not stop here as there are still many possibilities. In particular, there are interesting possibilities tied to the France-Belgium Tournesol mobility project, the European Twinning project, or post-doctoral grants."
*http://iihm.imag.fr/calvary/Teaching/HCI/Videos/Ensimag_29-03-2016_FR.mp4
**http://iihm.imag.fr/publication/vanderdonckt/
Top (contents) Newspaper
Download the newspaper(PDF - env. 1,5 Mo)
Complete Archives 2012-18
Recueil d'articles
