The hardware and software of the device used in our laboratory are available at the following sites
- Hardware: Currently available only in Japan. See Japanese page
- Software (Open source)
- Q&A
- Paper about AUTD (IEEE Trans. on Haptics)
- Safety use of ultrasound (in Japanese)
＊If you have any questions on the software, please refer to the above Q&A. Please do not directly contact our laboratory or the above manufacturers.
The Airborne Ultrasound Tactile Display (AUTD) produces tactile sensation remotely on a human skin surface. The AUTD does not require users to wear any devices on them.
Ultrasound transducers arrayed on the AUTD surface generate a ultrasound focus at an arbitrary position in the air. At the focal point with high energy density, static pressure called “acoustic radiation pressure” is produced, and it can push the skin surface. The focal position can be electrically controlled based on the phase array technique. In addition to creating a focus, you can produce a more general complex pressure pattern by solving an inverse problem in real-time.
The current limitations of the technology are that the maximum pressure is about 50 mN per square centimeter and that the spatial resolution is comparable to the sound wave length 8.5 mm for 40 kHz ultrasound. But under these constraints, radiation pressure pattern are controlled freely by a computer, which create various tactile sensations on the skin.
Haptics using such non-contact haptic feedback is called “midair haptics.” The initial studies of ultrasonic midair haptics were conducted by Hiroyuki Shinoda’s group from 2008 at the University of Tokyo. The research group demonstrated the basic technologies and haptic effects in the first half of the 2010s. Currently, the field is growing and many universities and companies in the world are moving forward with development.
World-first demonstrations by Univ. of Tokyo
Airborne Ultrasound Tactile Display (AUTD)
The world first study of midair haptics (haptic feedback using ultrasound).
- Takayuki Iwamoto, Mari Tatezono, and Hiroyuki Shinoda, “Non-contact Method for Producing Tactile Sensation Using Airborne Ultrasound,” Haptics: Perception, Devices and Scenarios: 6th International Conference, Eurohaptics 2008 Proceedings (Lecture Notes in Computer Science), pp.504-513, 2008.
- Takayuki Hoshi, Masafumi Takahashi, Takayuki Iwamoto, and Hiroyuki Shinoda, “Noncontact Tactile Display Based on Radiation Pressure of Airborne Ultrasound,” IEEE Trans. on Haptics, Vol. 3, No. 3, pp.155-165, 2010.
Touchable Holography
The first demo of midair haptic feedback synchronized with visual floating images.
Tactile Projector
Projecting visual image on the skin with midair haptic stimulation. A large aperture AUTD synchronizing multiple AUTD-units enabled this system.
- Keisuke Hasegawa, Hiroyuki Shinoda, “Aerial Display of Vibrotactile Sensation with High Spatial-Temporal Resolution using Large-Aperture Airborne Ultrasound Phased Array,” Proc. IEEE World Haptics Conference 2013, Oral, pp.31-36, Daejeon, Korea, April 14-18, 2013.
*Finalist of Best paper award, IEEE World Haptics Cconference (WHC) 2013
*Best Student Paper Award, IEEE World Haptics Conference (WHC) 2013 - Keisuke Hasegawa, Hiroyuki Shinoda, “Aerial Vibrotactile Display Based on Multiunit Ultrasound Phased Array,” IEEE Transactions on Haptics, vol. 11, no. 3, pp. 367-377, 1 July-Sept. 2018.
Haptomime, Mid-Air Haptic Touch Panel
Floating touch-panel with midair haptic feedback.
- Yasuaki Monnai, Keisuke Hasegawa, Masahiro Fujiwara, Seki Inoue, and Hiroyuki Shinoda, “HaptoMime: Mid-Air Haptic Interactions with a Floating Virtual Screen,” Proc. 27th ACM User Interface Software and Technology Symposium (UIST2014), pp.663-667, Hawaii, USA Oct. 5-8, 2014.
*ACM UIST 2014 People’s Choice Best Demo Award
*Special prize by SIGGRAPH, Industry Special prize, Innovative Technologies 2014
3D Haptic Hologram
Computational design of 3D acoustic energy-density field.
Boundary hologram
Advanced scheme of acoustic hologram and application to levitation. Boundary hologram provides acoustic field that produces proper restoring force to any minute displacement and rotation of the object. The algorithm enabled the world first ultrasonic levitation of a macroscopic non-spherical object significantly larger than the wavelength. It is applicable to arbitrary-shape objects considering the diffraction and reflection by the object.
Haptoclone (Haptic & Optical Clone)
Symmetric mutual telexistence by 3D haptic & optical field reconstruction. A pair of people can touch the realistic 3D image of each other with haptic sensation.
Recent important results by Univ. of Tokyo
Producing Static Pressure Sensation
The generated ultrasonic radiation pressure is not very high. If a time-invariant pressure (static pressure) is applied to the skin, the stimulus does not exceed the tactile threshold and cannot be felt by humans. Therefore, the conventional aerial tactile presentation uses time-varying vibrotactile stimuli. However, it is felt as “vibration,” which is different from the sensation of touching an object in a usual way.
However, recent research by Morisaki et al. has revealed that a dynamic pressure distribution can create a sensation similar to static pressure. This finding has shown that a versatile tactile presentation is possible under a certain limitation on the spatial resolution and the maximum force.
- Tao Morisaki, Masahiro Fujiwara, Yasutoshi Makino, and Hiroyuki Shinoda, “Non-Vibratory Pressure Sensation Produced by Ultrasound Focus Moving Laterally and Repetitively with Fine Spatial Step Width,” IEEE Transactions on Haptics, Volume 15, Issue 3, pp. 441-450, 2021.
*Best Student Demonstration Award, SIGGRAPH ASIA 2021 Emerging Technologies
*Best Demo Award 3rd place, Eurohaptics 2022 by a related work
Cooling Sensation
Temperature is an important factor in tactile perception. However, non-contact presentation of temperature, especially cold sensation, has been considered difficult. However, Nakajima et al. recently confirmed that the skin temperature rapidly decreases due to the heat of vaporization when ultrasound waves are focused on the skin in a room-temperature mist of water. A temperature drop of 3 K was observed within 0.5 s immediately after ultrasound irradiation, comparable to cooling using standard Peltier devices. They also confirmed the movement of the cooling spot could be perceived.
- Mitsuru Nakajima, Keisuke Hasegawa, Yasutoshi Makino, and Hiroyuki Shinoda, “Spatiotemporal Pinpoint Cooling Sensation Produced by Ultrasound-Driven Mist Vaporization on Skin,” IEEE Transactions on Haptics, Volume 14, Issue 4, pp. 874-884, 2021. *Best Paper Award, IEEE Transactions on Haptics, 2022
Visualization of Ultrasound
A problem with midair haptics was the measurement of the sound field distribution. The virtually only practical way to measure the distribution was to scan a microphone with an automatic stage or robot arm. In 2021, Onishi et al. discovered a new practical method to measure the details of the sound field of intense ultrasonic waves. Using a thermo-camera, the sound pressure distribution on an arbitrary two-dimensional plane in a 3D space can be measured. The measurement is real-time and with high spatial resolution. For example, a thermal image of a mesh screen inserted in an ultrasonic wave can visualize the two-dimensional ultrasonic field distribution on the mesh in real-time. It is easy to obtain the 3D distribution by moving the screen. It is also possible to visualize the sound field distribution on a skin surface.
- Ryoya Onishi, Takaaki Kamigaki, Shun Suzuki, Tao Morisaki, Masahiro Fujiwara, Yasutoshi Makino, and Hiroyuki Shinoda, “Two-Dimensional Measurement of Airborne Ultrasound Field Using Thermal Images,” Phys. Rev. Applied 18, 044047 (pp. 044047-1-6) Oct. 2022. doi.org/10.1103/PhysRevApplied.18.044047
- arXiv (12 Mar 2022) https://arxiv.org/abs/2203.07862
Other main results by Univ. of Tokyo
Lateral Modulation: A Method to Intensify Haptic Sensation by Ultrasound
Reciprocating motion of a focus along the skin surface (Lateral modulation, LM) can generate stronger sensation than Amplitude Modulation (AM) where the radiation pressure changes vibrationally at a spatially fixed point. The threshold of the radiation force of LM is 10 dB less than that of AM in the displayed maximum force, and LM can create haptic sensation on a hairy skin as well as a glabrous skin.
- Ryoko Takahashi, Keisuke Hasegawa, and Hiroyuki Shinoda, “Lateral Modulation of Midair Ultrasound Focus for Intensified Vibrotactile Stimuli,” International Conference on Human Haptic Sensing and Touch Enabled Computer Applications (Euro Haptics), pp 276-288 ,Pisa, Italy, June 13-16, 2018.
- Ryoko Takahashi, Keisuke Hasegawa, and Hiroyuki Shinoda, “Tactile Stimulation by Repetitive Lateral Movement of Midair Ultrasound Focus,” IEEE Transactions on Haptics, Vol. 12, No 4, pp.652-657, 2019 (Open Access).
Reducing Unnecessary Sound
Quick phase change (as seen in LM) causes unexpected amplitude change. The amplitude change produces unexpected tactile perception and sound noises. The following shows a practical effective method to reduce it. (The sound generation and amplitude change by phase shift was first pointed out by Hoshi in 2016.)
3D Object Manipulation by Fingers
We are clarifying what information is essential to make the user feel the substance of a virtual object. Connected AUTDs surrounding fingers can generate a 5 mm diameter force spot, and create fine pressure distribution on the fingers. The distribution is controlled in real-time reflecting the contact state between the fingers and the virtual object. We have examined experimentally if the fingers can handle an object blindly.
- Atsushi Matsubayashi, Yasutoshi Makino, and Hiroyuki Shinoda, “Rendering Ultrasound Pressure Distribution on Hand Surface in Real-Time,” International Conference on Human Haptic Sensing and Touch Enabled Computer Applications (Euro Haptics), pp. 407-415, Sep. 6-9, 2020, Leiden, Netherlands. (Open Access)
*Finalist (Top3), Best Paper Award, Eurohaptics 2020 - Atsushi Matsubayashi, Yasutoshi Makino, and Hiroyuki Shinoda, “Direct Finger Manipulation of 3D Object Image with Ultrasound Haptic Feedback,” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland UK, May 04 – 09, 2019. (Open Access)
- Atsushi Matsubayashi, Hiroki Oikawa, Saya Mizutani, Yasutoshi Makino and Hiroyuki Shinoda, “Display of Haptic Shape Using Ultrasound Pressure Distribution Forming Cross-Sectional Shape,” Proc. 2019 IEEE World Haptics Conference, pp.419-424, Tokyo, Japan, July 9-12, 2019.
Motion Guidance
Haptic Pursuit
In human vision, smooth pursuit eye movement is the basic ability to visually follow a moving object by keeping it at the sight center. In this study, we validated that a human hand has a similar ability to track a midair haptic stimulus, i.e., a human palm exposed to a point vibration by AUTD can follow the continuous movement of the stimulation point. The ability of motion tracking by hand can be applied to haptic guidance.
Virtual Handrail
There is another strategy where the user find and follow the designated path actively. Consider a situation the path, from start to finish (the ‘goal’), is presented by scanning ultrasound focal points along the path. The scanned focal point produces vibrotactile stimuli on the hand, and users perceive the local path-direction easily. Then user can follow the path without strong concentration.
Guidance along a palm
Guidance perpendicular to a palm
Comments