OPAL-RING
Kaneko and Nakamura Laboratory
Analysis of facial features and impressions, computerized facial caricature artists, facial expression recognition, facial similarity search, human-robot interactions, intelligent robotics, ”Sassuru” interfaces, acquisition of concept and meaning of words by robot
| Faculty/Department | Department of Mechanical Engineering and Intelligent Systems Graduate School of Informatics and Engineering |
|---|---|
| Members | Masahide Kaneko, Professor |
| Affiliations | Institute of Electronics, Information and Communication Engineers; Institute of Electrical Engineers of Japan; Institute of Image Information and Television Engineers; Japanese Academy of Facial Studies; Information Processing Society of Japan; Human Interface Society; Robotics Society of Japan; Institute of Electrical and Electronics Engineers (IEEE) |
| Website | http://soybean.ee.uec.ac.jp/kaneko/ |
As of August, 2015
- Masahide KANEKO
- Keyword
-
Processing of facial image information, computerized facial caricature, facial feature analysis, facial impression, facial similarity search, facial expression recognition, human-robot interactions, intelligent robotics, ”Sassuru” interface, intelligent interactions, autonomous mobile robot, integration of visual and audio information, human-like behavior, acquisition of concept and meaning of words
Summary of Research
Research to Develop a Computer Capable of Creating Facial Caricatures
The face plays a key role in communication between people, revealing identity, ethnic origins, and gender and conferring an individuality that distinguishes one person from another. The face also provides information on emotions such as happiness, angry, sadness, and surprise; on physical and psychological states; and on the inside of person. Can computers be programmed to understand all this information? Achieving such capabilities would make computers far better at communicating with humans.
- Facial feature analysis and generation of facial caricatures
Our laboratory analyzes the shapes and configuration of facial features taken from photographs to make quantitative descriptions of features and impressions of the human face, an area never before subjected to rigorous study. We are also exploring ways to apply such systems to real-world applications. One example of a practical application on which we are currently working is the technology required to create a computerized facial caricature artist capable of generating highly expressive facial caricatures. This system would extract and analyze information on the shapes and configuration of various facial features, such as the eyebrows, eyes, nose, and mouth, and the contour of the face. The system begins with an average face generated from many Japanese faces including both males and females. The system determines and processes the differences between the shapes and configuration of the facial features of the average face and the target face by a statistical method known as principal component analysis to produce a quantitative description of the features of the target face. High values indicate that a feature stands out and contributes to the character of a face, perhaps resulting in the capability to generate a facial caricature that emphasizes these features.
Recognizing Facial Expressions
Facial expressions can be categorized into six basic emotions: happiness, sadness, angry, fear, disgust, and surprise. Our system achieves remarkably accurate recognition of expressions by extracting information on characteristics in local regions of the face and the characteristic shape of facial features, and then determines the category of expression for each part before integrating the results to generate a comprehensive decision of the expression represented by the entire face. A random forest classifier is used to form statistical assessments of expression type and intensity to ensure the robustness of the system to slight expressions most frequently exhibited by humans in daily life.
Creating Autonomous Robots That Behave Like Humans
Our laboratory also engages in robotics research. Our main focus here is on components that correspond to human eyes, ears, and brains. These sensory perception skills will allow robots to behave autonomously, like humans, and to coexist with us in our homes and offices. Our robots will obtain information from the outside world using functions corresponding to our eyes and ears to determine what the people around them want them to do, and then autonomously perform the actions.
The first part of our research involves the functions necessary for autonomous decision-making. Audio signals captured by multiple microphones are analyzed to determine the differences in time of arrival to identify the three-dimensional position of the source, or the location of the person speaking to the robot. A camera is then directed toward the source to capture an image, which is processed for human face and body detection based on brightness and color information. Based on this information, the robot can determine whether or not a person is indeed present; identify the person if present; and select the appropriate set of interactions for the person identified.
The second part consists of technologies involving the autonomous motion of robots, including automatic map generation of surroundings and simultaneously self-positioning. The unique feature of our technology lies in the precise map generation results achieved through statistical descriptions that seek to identify whether objects are fixed or mobile-in other words, whether an object should be included as permanent fixtures in the generated map-based on assessments of the potential for movement in objects, such as doors (open/shut) and moving objects, like people in the surroundings.
Autonomous Mobility of Robots in Ordinary Environments
A major characteristic of our robot is autonomous behavior. It draws on information gathered from its surroundings, including information on distance, color images, and sound, to make independent decisions. We began by establishing a decision-making process to allow robots to pass by each other in a narrow corridor, as humans do, and allow robots to adjust their relative position to those whom they happen to be accompanying (parallel or cascade formation), depending on oncoming persons or obstacles. We devised a unique potential method to develop these capabilities: It is assumed that when a robot travels together with a person, the robot demonstrates a strong attractive force that keeps it to the right or left of the person and a strong repulsive force from oncoming persons. A potential field is calculated for the changes in attractive/repulsive forces experienced by the robot for each incremental moment as it approaches an oncoming person. By programming the robot to move towards the point having the lowest potential, the robot can be made to move to the front or behind its accompanied person while avoiding collisions with oncoming persons.
- Display screen for automatic recognition processing of oncoming persons and surroundings
- A robot that switches autonomously between parallel/cascade formation
- Autonomous mobile robot
Advantages
Development of Computerized Facial Caricature Technologies Based on Objective Descriptions of Facial Features
- A robust map generation system to track dynamic changes in the environment
- Shapes of facial contour represented by each of principal components
Our technologies for the computerized facial caricature artist are special in that they numerically convert all facial data to generate an objective description. This unique tool for facial feature analysis and synthesis can treat all information associated with a face, such as shapes and configuration of the facial features, facial expressions, and even facial impressions, to date only subjectively expressed in words, in an objective and comprehensive manner as a combination of quantified data-weighted values of principal components.
Since the various facial features are described numerically and presented in visual manner as a facial caricature, the system permits an efficient retrieval of the facial database when making searches based on features that match those of a given facial caricature, or even based on facial impressions described with words: for example, eyes tilting up; firmly set mouth; or stern visage.
”Sassuru” and Social Skills: Making Robots Behave More Like Humans
Not only will our robots recognize humans in their path as obstacles and move to avoid collisions, they will read our state of mind (“Sassuru”) while doing so by determining what actions the humans they are avoiding expect. The behavior and actions of robots will also depend on the social relationships between them and each person. This will mark a significant step forward in creating robots that behave more like humans.
Our study is currently focusing on the task of a robot accompanying a person in addition to a robot moving about by itself in a crowd. We are studying situations in which a robot accompanies a single person, as well as others in which a robot leads a group of people, a robot joins a person walking apart at some distance. We are also studying functions that will allow the robot to take into account the social attributes of its accompanied person and act accordingly: for example, closely attending elderly persons or children or keeping an appropriate distance in formal situations.
Future Prospects
Developing a Robot by Integrating Two Component Technologies into One
We hope to confer on our computerized facial caricature artist robustness that will make it capable of handling data obtained under various environments for photography, as well as generating facial caricatures that offer near professional levels of expression. Facial images will also play a vital role in communication. It is our hope that the system will find a wide range of applications.
While our current studies of robots, facial recognition, and computerized facial caricature systems are independent efforts, we plan to integrate these technologies to create robots capable of making accurate assessments of three-dimensional surroundings and the activities of multiple people-robots capable of making independent decisions and interacting with us in our daily lives in various different ways.