In an article published in the journal Nature Communications, researchers developed a textile-based wearable human-machine interface capable of capturing, conveying, and adaptively transferring physical tactile information across time and space. The interface consists of smart gloves with integrated tactile sensors and vibrotactile haptic actuators that are digitally designed and rapidly fabricated using a digital embroidery machine. The research also presented a machine-learning pipeline that adaptively optimizes the haptic feedback for each user based on their responses to tactile sensations.
(a) A full-sized textile-based glove (iv) with integrated tactile sensors (ii) and vibrotactile haptics (iii) is digitally designed and automatically fabricated using a digital embroidery machine (i). Leveraging these gloves with integrated sensing and haptic capabilities, we demonstrate physical interaction transfer across people for skill development (b), and physical interaction transfer between humans and robots for teleoperation (c). Study: https://www.nature.com/articles/s41467-024-45059-8
Background
Human-machine interfaces for tactile interaction transfer devices can sense and display physical touch information between humans and machines or among humans. Such instruments have immense potential for healthcare, augmented and virtual reality, human-robot collaboration, and skill development. However, current human-machine interfaces for capturing and conveying tactile information are often limited by their scalability, customizability, and adaptability.
Moreover, the variation in human perception of haptic feedback makes it challenging to achieve effective and reliable tactile interaction transfer among humans and between humans and machines. Therefore, there is a need for a scalable, conformal, and adaptive human-machine interface that can seamlessly integrate tactile sensing and haptic display capabilities into wearable devices.
About the Research
In the present paper, the authors used a digital embroidery machine to embed arrays of vibrotactile actuators and piezoresistive force sensors into textiles in a scalable, customizable, and modular manner. This process was employed to create gloves capable of recording and transferring tactile interactions across different scenarios.
The vibrotactile actuators are based on a linear resonant actuator structure consisting of an embroidered electromagnetic coil and a moving permanent magnet. The tactile sensors are based on a piezoresistive layer sandwiched between two fabrics with embroidered conductive electrodes. Both the sensors and the actuators are serialized in a matrix form factor to minimize the number of connecting wires.
The study proposed a learning-based optimization pipeline that can adaptively model the individual user's reaction to the haptic feedback and then optimize the haptic parameters for each user. The pipeline comprises an inverse optimization process and a forward dynamics model. The inverse optimization process minimizes the disparity between the predicted tactile sequence and the target tactile sequence through gradient descent and the haptic sequence optimization.
The forward dynamics model predicts the user's tactile response to haptic instructions by simulating a sequence of haptic signals. Additionally, an adaptive module processes combined haptic instructions and tactile responses to generate a user-specific latent feature vector representing individual characteristics. This feature vector enhances the accuracy and generalization of the forward dynamics model.
Research Findings
The study evaluated the performance and usability of the interface and pipeline through various experiments and user studies. Initially, an investigation was conducted into how individuals perceive textile-based vibrotactile haptic feedback with different amplitudes and frequencies. It was observed that users perceive the feedback more strongly with higher input amplitudes and at a frequency of 100 Hz. Furthermore, users demonstrated a high accuracy in identifying the location and temporal pattern of the haptic feedback.
After demonstrating the transfer of coarse tactile interactions for a single user to alleviate tactile occlusion, a double-glove interface was developed. This process included the augmentation of a thick protective glove with tactile sensors and the insertion of a glove with vibrotactile actuators inside it. The interface captures tactile interactions outside the protective glove and transmits them to the user’s hand inside the glove in real time. This enables users to identify the location of a contact force with high accuracy when haptic feedback is activated.
Additionally, an adaptive transfer of tactile interactions across users was showcased by guiding them to perform touch sequences. The pipeline was applied to various scenarios such as rhythmic gaming, piano playing, and racing, demonstrating adaptation to individual user responses and optimized haptic sequences for improved performance. Finally, the study exhibited the transfer of tactile information from a robot to a human for teleoperation, enhancing performance in delicate grasping, particularly when visual information is occluded.
Application
The newly developed textile-based human-machine interface and its underlying algorithms can enable various applications that require capturing and conveying tactile information across time and space. For example, the interface can be used for healthcare, augmented and virtual reality, human-robot collaboration, and skill development. It can also be integrated into other wearable devices, such as socks and jackets, to provide haptic feedback to other body parts. Moreover, it can be extended to transfer more complex and rich tactile sensations, such as texture, temperature, and shape, by incorporating different types of sensors and actuators.
Conclusion
In summary, the novel textile-based wearable human-machine interface can capture, display, and adaptively transfer physical tactile interactions. It consists of a smart glove with integrated tactile sensors and vibrotactile haptic actuators that are digitally designed and fabricated using a digital embroidery machine.
The authors also presented a machine-learning pipeline that can optimize the haptic feedback for individual users based on their responses to the tactile sensations. The interface and the pipeline enable various applications, including alleviating tactile occlusion, guiding skill development, and enhancing robot teleoperation. Moreover, the study demonstrates the potential of textile-based human-machine interfaces for enriching technology-mediated interactions between humans and machines.
Article Revisions
- Feb 7 2024 - Changed image
Collaborative AI Research Between China and the U.S. Yields Greater Impact