The Quest for a new humanity emerging from the integration of humans and
computers: "Cybernetic Humanity"

The open lab of Cybernetic Humanity

Dr. Shunichi Kasahara is currently advancing the pursuit of "Cybernetic Humanity", a research area focused on the new form of humanity emerging from the integration of humans and computers, at the Okinawa Institute of Science and Technology Graduate University (OIST). This is based on his previous research, "Superception", which is centered around using computer technology to enhance and transform human perception and cognition. On April 21, 2023, a public seminar was held at the newly established "Cybernetic Humanity Studio" at OIST. This report will focus on the seminar's content, discussing Cybernetic Humanity and OIST's research environment.

※The text has been appropriately modified to ensure clarity since it includes complex concepts for explanation.

【What is "Cybernetic Humanity"? 】

Where does our true self begin and end? On social media, we may have friends with whom we've been interacting through only text for years, or the relationships we cultivate through avatars in the metaverse. How does those relationships integrate into our real lives?

"In 'this reality,' our sense of self is clear, isn't it? Right now, it is the self that exists here. However, in the metaverse or virtual reality and human augmentation with AI, a different oneself emerges, while still being 'me.' The question then arises: How do we balance our subjective self in 'this reality' with our self in the integration of humans and computers? That is the issue.", Dr. Kasahara says.

Dr. Kasahara has dual roles as both a researcher and an artist. As a researcher, he has been exploring the concept of the "self" when computers and humans intergrate. Additionally, by presenting these findings as artworks, he has been exploring how society receives and interprets the self that combines with computers, possessing characteristics different from "this reality."

"In virtual reality, we can do things that are impossible in reality, such as jumping from great heights without injuries or flying in the sky. It's like being in a video game world. But what if the things happening in that virtual reality were to happen in 'this reality' due to technological advancements? "How should we embrace and integrate the experiences and skills of our virtual reality selves into 'this reality'?" Cybernetic Humanity is a research field that explores this new form of humanity in such realities.", Dr. Kasahara says.

【What is our own action?】

In Cybernetic Humanity, our own "actions" become crucial. In the modern era, where advanced technology coexists, it becomes a challenge to determine the boundaries of our own actions. Dr. Kasahara illustrates this by demonstrating the limits of human action, using a simple yet effective trick: the "pen drop" experiment.

"The aim of the 'Wired Muscle" Project lies in the 'delay' inherent in our bodies. When we see something and move our bodies in response, there is a delay of approximately 250 milliseconds. As long as this delay exists, humans are unable to catch a pen randomly dropped by someone. This represents a biological limitation. Due to this delay, we sometimes cannot avoid accidents happening right in front of us while driving a car.", Dr Kasahara says.

In the "Wired Muscle" project, researchers investigated the extent to which humans perceive actions as their own when they overcome the inherent delay through technology. The approach involved directly connecting the act of dropping a pen with the act of catching it. Essentially, it read the muscle signals of the person dropping the pen and used electrical muscle stimulation to move the muscles of the person catching it (pre-emptive action), thereby shortening the delay to as little as 60 milliseconds. This allowed humans to catch a pen that would otherwise be considered impossible in reality.

"One intriguing point of this project is that the participants who are able to catch the pen using this device exclaim, 'I did it!'. Although it became possible due to the device, they attributed the action to themselves. Then, I conducted research to identify the mechanism behind this phenomenon.", Dr. Kasahara says.

Dr. Kasahara investigated to determine the extent to which humans can recognize an accolated action as their own volitional action in Preemptive Action project. It was found that the optimal value at which pre-emptive actions were recognized as one's own actions is up to 80 milliseconds. Based on this research, Dr. Kasahara has conducted further studies with more complex tasks, including the project titled〈Whose touch is this?

In the study, participants answered a two-choice quiz while their muscles were contracted through electrical-muscle-stimulation, faster than their response. In the series of experiment, participants were deliberately guided by the computer towards either the correct or incorrect answers. As a result, participants attributed their correct answers to themselves, thinking, "I did it," while attributing their incorrect answers to the influence of the machine, concluding, "It was the machine's fault." This indicates that the participants' sense of agency during their responses was biased based on the outcome. Dr. Kasahara and the research team suggested that this bias can be attributed to sense of agency bias.

Furthermore, Dr. Kasahara and the research team brought this mechanism into robotics and attempted to play ping pong with a robotic arm.

"Playing ping pong with a robotic arm is extremely challenging. Due to the inherent delay in the human biological system, we cannot react appropriately to fast-moving balls.", Dr. Kasahara says.

How can we control a robotic arm as if it were our own and play ping pong? Dr. Kasahara and his colleagues have attempted to integrate both the motion of human body and the computational auto-control into a robotic arm for playing ping pong.

"We have successfully managed to play ping pong with a robotic arm while maintaining our sense of agency, achieved through the fine-tuned integration of human motion and computer control under specific conditions. In other words, we preserved the feeling of actively participating while achieving a fusion with the robot.", Dr. Kasahara says.

What this experiment illustrates is the creation of actions that, even though the robot is actually responding automatically, allow humans to perceive that "we are doing it ourselves." When such actions become a reality, how far will we consider them as "our own actions"?

What is our body?

What is our body? It seems that our body is “this body right here”. However, Dr. Kasahara has achieved to transform body perception by replacing the visual representation of our body in spatial and temporal alterations. This is "Malleable Embodiment"

"In Malleable Embodiment , we didn't perform any physical manipulation. What we did was predict the future movements of the body through computer algorithms and provide that as visual feedback to the user. As a result, the users felt their bodies becoming lighter. Furthermore, when we provided feedback that slowed down the perceived body movements, the users felt their bodies becoming heavier. It's almost like being in a 'hangover'. This research has revealed that we can alter our bodily sensations solely through visual information.", Dr. Kasahara says.

Furthermore, Dr. Kasahara embarked on a groundbreaking research. As humans, we adhere to the concept of "one consciousness, one body." However, Dr. Kasahara conducted research aimed at rewriting this restriction. It is "Parallel Ping-Pong." In this experiment, a single user controlled two robotic arms, engaging in a ping pong game on two separate tables simultaneously.

"Parallel Ping-Pong," which was on display at the Cybernetic Humanity Studio's Open Lab.

"The previous research has already demonstrated that humans can play ping pong with a robotic arm while maintaining their sense of agency. Therefore, the next step was to increase the number of robotic arms. The challenge here was to achieve something we haven't yet realized, which is 'adapting to multiple bodies simultaneously.'", Dr. Kasahara says.

However, human consciousness is one. "How can a single consciousness play two games of ping pong simultaneously? The key lies in the technology to master multiple bodies. This pursuit is what the 'Parallel Adaptation' project aimed to explore.", Dr. Kasahara says.

We need to adapt to the characteristics of two bodies (dual adaptation) to play two ping pong games with a single consciousness. However, simultaneously adapting to robot arms positioned at different viewpoints in reality is impossible for humans who adhere to the concept of "one consciousness, one body."

Therefore, Dr. Kasahara and his team made "dual adaptation" possible by providing strong visual cues in virtual reality between these bodies. This research revealed that users can adapt to two bodies visually.

As if fitting puzzle pieces together, these research have made "Parallel Ping-Pong." In "Parallel Ping-Pong," the integration of computer and human enables actions that were previously impossible for a person to perform while still preserving a sense of agency. However, can this robot truly be considered the "user's own body"?

What is the Self?

"What is the self? This is a question that numerous philosophers have grappled with over the years. Researcher Kasahara, in response to this profound question, explored the extent to which the 'self' can be recognized through technology. This is called 'Morphing Identity.'"

Two individuals who entered a space resembling an automatic passport photo booth initially appear to have a normal conversation. However, gradually, their faces begin to blend together, and in the end, they completely exchange facial identities. "The mechanism of 'Morphing Identity' involves analysing the faces of the two individuals, generating interpolated facial images between them using a 'GAN' (Generative Adversarial Network), and transforming them into real-time animation."

"Through 'Morphing Identity,' it has been revealed that there is a 'margin' within self-recognition. We installed 'Morphing Identity' in an exhibition space and had it experienced by over 7,600 individuals. Upon aggregating the data, it was found that, on average, people identify a facial image with up to 20% other and 80% self as 'themselves.'", Dr. Kasahara says.

The photo of Cybernetic Humanity Studio’s open lab.

Research on Pseudo-Haptic Sensation by Dr. Anatole Lécuyer"

This seminar was conducted in collaboration with Dr. Anatole Lécuyer, who was on a short-term stay at OIST. Dr. Lécuyer is affiliated with the French research institute INRIA and has conducted numerous impactful studies and published many papers in the field of 'pseudo-haptic' interactions.

"Interestingly, in our research lab, we enjoy employing low-tech approaches and finding tricks to complement the current technological limitations", Dr. Lécuyer explained as he introduced his research during the seminar talk.

"Pseudo-haptics" is a concept that simulates the sense of touch without directly stimulating the sense of touch. One of the most intriguing and low-tech approaches in this field is the research that allows users to feel sensations of "elevation" and "depression" using only the movement of a computer screen cursor.

"In this research, we have successfully achieved the identification of macro-textures like bumps and holes using input devices like a mouse. This is made possible by adjusting the cursor's speed (in accordance with the displayed images or content), allowing users to perceive these textures.", says Dr. Lécuyer.

Dr. Lécuyer's papers are particularly well-cited among researchers specializing in haptics, and during the seminar, there was even a humorous reference from the audience to 'Anastasia' ('Anatole's Sensation'), a playful play on words inspired by "synaesthesia".

Dr. Lécuyer's research team applies pseudo-haptics in various contexts, including using digital avatars. One of their studies involves creating a tactile illusion in real-life touch based on the visual information of a digital avatar lifting weights.

"This research may resemble something akin to a 'hangover.' By slowing down or accelerating the movements of a digital avatar lifting weights with dumbbells, users can (in reality) feel the weights as heavier or lighter. In the future, applications in exercise and other areas are conceivable, where artificial adjustments in the user's workload can be made."

Dr. Lécuyer's research team has achieved numerous milestones in the field of Brain-Computer Interfaces (BCI). One of their notable contributions is the distribution of free software called "OpenViBE" under an open-source license (with Yann Renard listed as the lead author in the reference paper). BCI is a communication system that allows commands to be sent to computers based on brain activity, and it is currently advancing in various fields, including healthcare.

"We have used OpenViBE to play video games solely through brain activity (2013), and we have achieved control of mobile robots in augmented reality (AR) using Microsoft HoloLens (2020)."

"The reason for opening the Cybernetic Humanity Studio at OIST"

Dr. Kasahara, who has explored actions, the body, and self from a unique perspective, cites one of the motivations behind the pursuit of "Cybernetic Humanity" as the desire to extend individual research into society.

At OIST, there is a vibrant exchange of ideas with researchers from both Japan and around the world, with approximately 1,000 scientists visiting each year through workshops and conferences. The significance of the pursuit of Cybernetic Humanity at OIST lies in the opportunity for interactions with highly accomplished scientists like Dr. Lécuyer. OIST serves as a place where research at the individual level seamlessly connects with society, making it a truly integral hub for these endeavors.

"My research up until now has been focused on individual phenomena. However, as technology is increasingly merging with human beings, integrating into society, I believe it's essential to not only concentrate on individual research but also encompass society as a whole in order to understand the future of humanity. That is what we want to explore by connecting Sony CSL and OIST as part of the Cybenetic Humanity.", Dr. Kasahara says.

※INRIA: Institut national de recherche en sciences et technologies du numérique

The Japanese version of this article was written by science writer Akihico Mori.


"OIST hosts French-Japanese Seminar on "Metaverse and Virtual Reality"

Public Lecture: Metaverse and Virtual Reality by Anatole Lecuyer and Shunichi Kasahara