Since its creation, SoftBank Robotics has pursued the goal of providing the largest number of companion humanoid robots and personal assistants.
After the development of a first generation of small humanoid robots, NAO , in 2006 , SoftBank Robotics took up a new challenge with Romeo in 2009 : the realization of a large robot, adapted to the domestic environment to facilitate Interacting and fulfilling everyday tasks such as opening a door, taking things on a table or helping someone walk.
The different steps :
Romeo Project Logo
Work on Romeo was initiated as part of a "FUI project" (single interministerial fund) in January 2009 . Labeled by the Cap Digital competitiveness cluster, the project was financed by the DGCIS, the Ile de France Region and the city of Paris. The ROMEO Project brought together a dozen French industrial and academic partners.
The ROMEO Project had four objectives covering several major aspects of robotics:
An interactive, open and modular mechatronics and software platform A personal assistant robot, monitoring functions and human-machine interaction A robust platform for research The foundations of a robotic industrial cluster In four years, Romeo has gone from an ambitious vision of robotics assistance to a 1.4m robot, known around the world. The first copies were ordered by French and European laboratories. The founding dynamic of the French service robotics sector, initiated by this FUI project, is already extending into new national and European collaborations.
Romeo Project 2 logo
The Romeo 2 project was launched in November 2012 . Labeled by Cap Digital and supported by Bpifrance as a structuring project of competitiveness clusters (PSPC) of the Investments for the Future Program, this 4-year project brings together 16 industrial and academic partners.
This second step aims to deepen the essential axes for the acceptability of a large humanoid robot in the home of people in a situation of loss of autonomy:
Computer and physical safety The ability to learn the habits of the user to understand their needs and intentions Personal assistance applications évolution.png
During the first phase of the project (2009 - 2012), the Romeo physical platform was fully assembled by SoftBank Robotics. She knew two versions, as imagined during the drafting of the project. Between the two versions, the design of the spine has changed, the final electronics have been integrated into the head, the upper body shells have been made in a stronger material, the electronic wiring of the legs has been improved, the batteries have been integrated ...
All these improvements were nevertheless not considered sufficient to achieve a quality that would meet the requirements of the laboratories that ordered Romeo. A third design cycle was therefore initiated after the end of the project to finalize the hands, the arms, the internal architecture of the torso and the head in particular. A rationalization of the wiring and the implementation of the electronic cards also had to be carried out. Leg actuators based on cable jacks designed by CEA LISThave been optimized with SoftBank Robotics to improve their reliability for use in walking. Their integration into the carbon exoskeleton could not be done during the first stage of the project and has since been realized. SoftBank Robotics had not finished developing an operational actuation system for the spine, but LISV did two different designs for the column, one of which was powered but could not be integrated into the prototype. full. The wrist and the hand of the robot, which were not a priority of phase 1, were not operational on the prototype of the end of 2012 but have since been integrated.
Touch cap Touch cap (CEA LIST) An original system of actuation of the eyes, proposed by the LPPA , was realized during the project and was since optimized by SoftBank Robotics. The mobile eyes are coupled to a vestibular system to allow a stabilization of the eyes of the robot, essential point to access more dynamic walking modes recommended by the LPPA .
The work on the auditory and vocal system, which is crucial for a natural interaction with the robot, was nevertheless well advanced with the thesis realized by Télécom Paris Tech on the source separation and the first integration of the results on the microcontroller that will equip the processing card. audio. For its part, LIMSI recorded and annotated several corpora that were used for the development of the nonverbal recognition functions developed during its two theses: the detection of emotions and the recognition of the speaker. These functions have been integrated into the robot architecture of SoftBank Robotics in cooperation with Voxler . Acapelahas improved its speech recognition function and designed the synthesized voice of Romeo. Voxler has developed a tool to modify the voice of Romeo and has integrated on NAO a musical game exploiting its bricks of musical analysis.
Corpus Registration (LIMSI, Institute of Vision) Corpus Registration (LIMSI, Institute of Vision) The CEA LIST and SoftBank Robotics worked on the visual system of the robot to acquire the ability to recognize objects, gestures and navigate in its environment. On the recognition of object instances, that is to say objects whose images were learned a priori, the CEA LIST had obtained performances that placed it at the second world level for accuracy and the first for the report speed / accuracy, but there was still work to be done on improving the calculation time of this recognition. On the subject of the recognition of classes of objects (it is a matter of recognizing a chair while the learning base contains images of chairs different from that to be recognized), theCEA LIST has developed a method called Fast Shared Boosting that is at or above the best current global performance.
Object Recognition (CEA LIST) Object Recognition (CEA LIST) For the perception of the operator's gesture, SoftBank Robotics focused on the use of several 3D sensor technologies (IR pattern projection, flight time, stereo-vision) to be able to make first interaction tests gestures that have confirmed the importance of this type of interaction. But the choice of 3D sensor technology could not be made at the end of the first project because all the ones that had been tested had advantages and disadvantages and the one that would be chosen should be the subject of a delicate compromise, in which the technical constraints will be added to the industrial constraints. On location and navigation too, SoftBank Robotics has explored several tracks (visual bitters, laser) that have achieved satisfactory results but again,
The partners who had worked on the planning and sensorimotor control of the Romeo movement had unfortunately not been able to work on the complete prototype of Romeo but had been able to develop their algorithms on realistic simulators of Romeo, on NAO, on other humanoid robots available in their lab or on Romeo subsets. The LAAS has worked on generating stable full body movements that allows the robot to generate complex movements while maintaining its balance. Inspired by the LPPA's work on the top-down control paradigm, LAAS proposed an oculocentric control scheme that resulted in much better reactive markets.
The LAAS had also improved its motion planning algorithms to account for complex tasks such as opening a door that requires the coordination of the movement of the door, the robot arm and also his legs. Work had also been undertaken to draw more inspiration from human movements in order to equip robots with "natural" gestures that would facilitate the acceptability of large humanoid robots in our environment. The INRIAhas also taken a step closer to a walk guided by the gaze, a reactive walk capable of taking into account on one or more steps a disturbance detected during the walk. A new walking pace, exploiting the flexibility of Romeo's toes and allowing a vertical oscillation of the pelvis, has been developed. It allows the robot to walk not only more naturally but also faster, thanks to larger steps, while limiting the speed and effort required of the motors. SoftBank Robotics has used some of these principles to improve NAO's workflow. The LPPA worked on a probabilistic equilibrium model that manages the imprecision of NAO (and Romeo) sensors to offer good resistance to abnormal situations such as external flares.
This type of stabilization will be crucial for a large robot like Romeo. Finally, exploring the biomimetic pathway, the LPPA worked on a neural controller for bipedal locomotion by genetic algorithms. This controller has given some promising results, but still requires work to be integrated into a full-featured algorithm.
Neuronal controller Neuronal Controller (LPPA, ISIR) The work on behaviors, dialogue and emotions should exploit all the results of the work described above to allow the robot to adapt the robot's actions to the context in which it is located. Spirops has implemented its decision-making tool in the SoftBank Robotics robot development environment to provide a simple way for the application designer to describe the reasons why the robot needs to take action. rather than another depending on the context. Voxler and Spirops have taken over the work on dialogue, the disappearance of As An Angel(partner at the beginning of the project) left unfinished. They developed a system for describing short dialogues. In order to bring content to this dialogue, Spirops has developed links with Linkedin, Google Contact and Google Calendar. Thanks to these three applications, the robot can use information on the biography, the relations and the agenda of the interlocutor to lead discussions. Spirops has also developed a system of managing the knowledge of its interlocutor that the robot can use to know what emotion to express at the appropriate time. the LIMSIworked on the characterization of the speaker in the emotional interaction with the robot via his vocal behavior to adapt the behavior of the robot. This consisted of determining an emotional profile of the interlocutor according to the emotions detected in his voice, to implement, in the Spirops decision-making tool , this profile in the form of rules and the influence of this profile on the choice. behavior to adopt by the robot. Spirops and SoftBank Robotics have finally integrated all these developments in the framework of the dialogue between man and robot. The integration was partial, since the robot could not have been available to all partners before the end of project 1 and could not be tested with its end users. But models on NAO have validated the feasibility of integrating partner developments on Romeo. The Institute of Visionwas able to test, on NAO, some of the basic applications developed with some of his patients. The results of this test have proved quite positive and have aroused a real interest on the part of the deficient persons for whom it would be a real help at home everyday. A list of recommendations that will be taken into account for future developments of Romeo has been established.