Human Interface from Keio's Faculty of Science and Technology Bridging humans and computers

Developing Trustworthy and User-friendly Human Interfaces

As its name implies, the term “human interface” denotes the touchpoint where humans and computers interact and exchange information. Common examples include the mouse and keyboard of computers, and smartphone touchscreens. In recent years, as computers have become more compact and telecommunication has become faster, the potential and roles of human interface has expanded immensely.
Against this backdrop, what caught Sugiura’s attention were the “soft” objects in our everyday lives, and his research has been focusing on the integration of human interfaces into unexpected items.
As Sugiura explains, “We are surrounded by soft objects such as sofas, cushions, stuffed toys,1 sheets, curtains, and numerous others in our daily routines. We are exploring various technologies and systems based on the idea of adding human interface functions to these kinds of items, enabling the smooth measurement of human behavior and the transfer of information from computers to individuals, with no conscious operation necessary.”

Along with developing fundamental technologies in human interface, Sugiura’s laboratory is also conducting unique research projects in collaboration with experts in diverse fields.
　One example is a smartphone app designed to instruct people on how to hold babies. It works by attaching a smartphone with this app to a stuffed animal or cushion. Then, the user simulates holding a baby with the object, and the app provides feedback to guide the user on how to hold one properly2.
Being a father himself, Sugiura had a personal experience where he struggled to hold his newborn who had yet to develop the necessary head support. As he puts it, “I felt it would be helpful to have some training before a baby is born to know the right way of holding them. At that time, I was approached by Associate Professor Noriko Aso from Kanagawa University, who had experience as a midwife and is currently conducting research in clinical psychology. She broached the possibility of a joint research project. As infants who aren't held properly may cry incessantly, contributing to additional parental stress, and in some severe cases, even leading to abuse. We hope that the tools we have developed will help make parenting a little easier.”

Sugiura’s research is not limited to his experiences with child-rearing, but also encompasses his hobbies and interests. Since childhood, Sugiura has been playing the violin and he joined an orchestra while he was a student. He states, “It was my dream to someday be in a performance with visually impaired musicians.” The unique challenge faced by visually impaired musicians is that they cannot see the conductor's gestures, which makes it difficult for them to follow the tempo and express the music appropriately. To solve this problem, Sugiura developed a device that transmits the conductor’s gestures to a soft cushion attached to the backrest of a performer’s seat, which captures the movements of the conductor’s baton and vibrates in response, allowing the performer to feel the instruction.3
　Indeed, one visually impaired horn player who tested the device had a positive reaction, stating, “I usually rely on the breathing sound of the people around me when performing, but using this tool made it easier to synchronize with the tempo.” As a result, as an open experiment, we collaborated with them to successfully hold a concert at the Yagami campus in December 2023.

In addition to the above applications, Sugiura has also been collaborating with medical researchers to develop tools for detecting early signs of medical conditions.4 One example is carpal tunnel syndrome, which causes numbness and pain in the fingers due to aging and overuse of the hands. Early detection and treatment are important because it may require surgery when it progresses to a later stage.
To address this, Sugiura developed a smartphone app that can assess whether an individual has carpal tunnel syndrome simply by engaging them in a game. As the condition progresses, carpal tunnel syndrome is characterized by the individual having difficulty moving their thumb. The game app collects data on an individual’s thumb movements as they play the game and uses machine learning models to estimate the likelihood of them having carpal tunnel syndrome. If the app determines a high possibility of the condition, it will display a message urging them to go to a hospital.
Sugiura's future goal is to design an app that can estimate the possibility of having a medical condition by collecting and analyzing data on movements made through daily smartphone activities such as typing with flick input, and other gestures. In addition to smartphones, it may also be possible to detect abnormalities in their walking patterns by attaching a device to their slippers to observe and analyze their gait, which could be applied at places such as senior support centers.
Sugiura has ambitious plans for expanding theses kinds of projects. He emphasizes the importance of not ignoring seemingly non-life-threatening issues, since those signs could lead to a cycle of worsening health, including repeated falls, fractures, and muscle weakness. As he notes, “If we can detect abnormalities and encourage people to seek medical attention through devices that naturally blend into their daily lives, it is possible to treat the illness before their conditions worsen.” Sugiura also envisions developing devices that monitor not only people’s physical movements, but also their emotions and feelings. For example, when you are feeling down and you sit on a sofa, there could be a device that detects how you feel and respond by playing music to cheer you up. As he says, “I hope to create a future where computers like these seamlessly support our daily lives without us consciously realizing their presence.” With Sugiura’s research closely linked to various aspects of our everyday lives, what will be next? We are excited to see what he has in store for us in the future.

I am the oldest of five siblings. My mother worked as an elementary school teacher, and my father was self-employed, specializing in designing various machines. My father had a keen interest in building things as a hobby, and often be found to be building things in the living room during his spare time. It was not uncommon to find tools lying scattered around the floor, our house was filled with large automatic machinery. Growing up in such an environment naturally drew me towards pursuing a career in building and creating things. When I was in elementary school and junior high school, I was really into model kits, mini four-wheel drive cars, and remote-controlled vehicles. The remote-controlled (RC) toys were the real deal. I would make them by cutting out the materials from wood myself and soldering on the motors. I built cars, ships, planes and conquered land, sea, and air with them.
　I used to go to the riverbank near my home to enjoy playing with my RC toys. Interestingly, I recently found out that the place where I spent time playing is now part of the Keio University campus, so I feel a strange sort of connection with the way things worked out.

In high school, I made bipedal robots and participated in robot fighting tournaments. Since working with robots is more difficult than with RC toys, I worked to expand my knowledge by attending workshops and learning more about them on my own. My brothers shared my enthusiasm for building thing, and I teamed up with my immediate younger brother for some tournaments , while my father paired up with my second younger brother. Beyond competitions, I also went to different parts of Japan and overseas to give robot demonstrations in front of live audiences. These performances not only provided me with valuable experience, but I also earned income from them. Just like with my RC toys, my fascination has always revolved around building things and operating them according to my own vision.

I was a member of the kendo club when I was in junior high school, and the fencing club when I was in high school. I also competed in the Inter-High School Championship for fencing. I have good memories of competing in the team matches against Yuki Ota’s team, who won the silver medal at the 2008 Beijing Olympics.
　In addition to fencing, I started learning the violin when I was a child. I kept playing for over 10 years, and joined an orchestra in university. Currently, my wife and I are blessed with two young boys, and we have been talking about how it would be nice to play an instrument with our children, and we’ve been trying to figure out what instrument to introduce them to. Both parenting and being a part of an orchestra serve as topics that intersect with my research. When you are directly involved in something, you really do become more aware of the kinds of issues and challenges it brings.
　I also played a lot of games upon entering university. Video games are an embodiment of human interfaces. Through games, we can learn what entertains people, and the mechanics that facilitate such engagement. Many of the graduates from my lab have gone on to work for video game companies. Through research, we can discover new challenges to tackle while developing our abilities to solve problems. Consequently, this opens various career pathways, ranging from consumer electronics to the consulting industry, that leverage our acquired expertise.

My idea was to have a person control a robot that picks up on certain stimuli, and to have that sensory information relayed back to the operator, aiming to recreate the immersive experience of the robot in the operator's perception. For example, there could be an operator on Earth controlling a robot in space. Even though they’re not physically there, they would feel as they were there themselves, mirroring the experiences of the robot.
　At that time, I was interviewed by a website that focuses on robotics. It was then that I got an unexpected email from Professor Masahiko Inami, who was then a professor of UEC and is now a professor at the University of Tokyo. After reading my interview, he reached out to me and acknowledged, “The idea you wrote about is the very concept of telexistence. It’s an idea that my own professor Susumu Tachi proposed for the first time about 20 years ago. If you’re interested in this field, why don’t you come join my lab?” Telexistence refers to technology that allows one to operate a physical entity in real-time, seamlessly replicating the experience of being in proximity. Thanks to Professor Inami’s opportunity, I was able to join his lab from my second year as an undergraduate, and I thereby began my research on intuitive human interfaces for bipedal robot operation.
　At that time, I only had a vague sense that I wanted to conduct research on robots. However, after listening to Professor Inami, I discovered that what I really wanted to study was the human interface.
　Incidentally, while undergraduate students would usually join a lab in their fourth year, at UEC there was a system where students could begin their research earlier and earn credits. There is also a similar program at Keio University’s Faculty of Science and Technology called JREP (Junior Research Experience Program) where first- and second-year students can participate in a lab’s research activities. This includes my lab as well, where we have accepted students in their second year through JREP. In my case, being able to join a lab from my second year allowed me to broaden my perspective and gain confidence. As such, I think it is wonderful to have these opportunities where one can grow. I strongly urge everyone to proactively challenge themselves.

KMD was founded in 2008 with the purpose of developing leaders and innovators who would be able to excel across various disciplines and make an impact on the global stage. When Professor Inami joined KMD as a founding member, I wanted to enroll as a member of their first cohort of students, so I withdrew from UEC in my third year, took the entrance exam for KMD, and joined the graduate school.
　I believed that if I could enroll as a member of the first class, I, along with my peers, could have the opportunity to shape the graduate school’s culture, so I was very eager to be among the inaugural group of students at KMD. In fact, after enrolling, we were able to create our own workshop space and were given free rein over the facility’s rooms, which brought us invaluable experiences. Especially as KMD was brand new with no reputation preceding it, the first class was very diverse, and even if we pursued different paths, there was no fear of criticism. It was a supportive and accepting environment.
　I pursued my master’s and doctoral degrees at KMD. As a master’s student, I made robots including one that could fold laundry in the way you instructed it to, and one that could cook your favorite recipe when you told them how. At the doctoral level, the main focus of my research changed to “soft” objects. For instance, I developed systems where you could use cushions as controllers. The products we have developed to date are on demonstration for visitors during lab tours so they can partake in research experience firsthand.

In April 2018 I established the lab that I now lead. During the process, I carefully considered the necessary features of the lab space and spent a significant amount of time deciding how to arrange the desks and experimentation space. Recognizing the impact of environment on people's performance, I also focused on arranging the space in such a way that would foster effective communication. I made it so that people could have discussions about research or light chit-chat in the free space, or if they wanted to concentrate on their studies or conduct research they could move to the desk space. We set aside clear distinctive spaces for different activities.
　We also included finer adjustments. For example, we made it so that when using the coffee maker, you would have to pay to drink alone, but if two or more people were to drink together, the coffee would be free of charge. This approach encouraged students to invite someone else to join them for coffee, and there communication would occur. This means that I am the one picking up the bill to provide my students with free coffee, but I believe it is worth it.

I am conducting research on computer output. I chose this laboratory because its research interests me. In addition, the fact that Professor Sugiura is a young researcher with a wealth of experience is also a strong plus. I thought that if I am going to create something new, I would want to study under a young researcher. In addition, there is not much of a hierarchical relationship in the laboratory, and students can work hard, bouncing off and improving one another while remaining on equal footing. I also like the environment where I can have discussions with Professor Sugiura as much as I’d like.
I was given the opportunity to go by myself to the University of Sydney, Australia, to conduct collaborative research for a month. I decided to enter a doctoral program because I found research interesting, and I plan to do research to the fullest for three more years. (2nd year master's student)