The University of Tokyo
Shinoda & Makino Lab Logo Shinoda & Makino Lab

Midair Haptics by Airborne Ultrasound Tactile Display (AUTD)

photo of autd3

Hardware and software for the devices used in our laboratory are available at the following sites:

*For inquiries regarding usage, please use the Q&A link above. Please refrain from contacting our laboratory directly. Our laboratory receives no financial benefit from the sale of these devices.

An Airborne Ultrasound Tactile Display (AUTD) uses ultrasonic transducer arrays to create ultrasound focal points or detailed distributions at arbitrary positions in midair. This allows for the presentation of tactile sensations to the human body without requiring the user to wear any equipment.

At the focal point, where energy density is high, the average pressure increases, creating a force known as acoustic radiation force that pushes against the skin. The positions of these focal points are controlled electronically, allowing them to be moved freely and instantaneously in 3D space. Furthermore, complex spatial distributions can be generated in real-time by solving inverse problems.

The field of non-contact tactile stimulation is called “Midair Haptics.” Its most fundamental aspects were proposed and established by Hiroyuki Shinoda’s group at the University of Tokyo between 2008 and the early 2010s. Since then, universities and companies worldwide have entered the field. Entering the 2020s, the University of Tokyo discovered new possibilities, and the field is now moving into its “Second Generation” phase.

The Second Generation: High Fidelity Midair Haptics

For over a decade since the initial proposal, the prevailing wisdom was that “tactile sensations generated by midair haptics are weak and limited to specific vibratory sensations.” This assumption began to shift significantly in the 2020s. Today, it has become possible to reproduce both substantial pressure sensations and sensations of heat and cold. This allows for the realistic transmission of the “physical presence” of objects with their surface textures. While the generated sensations are not yet perfect, social implementation has begun, focused on three pillars: Well-being, Learning Experiences, and Entertainment.

Realistic Reproduction of 3D Characters

SXSW24

Technology that effectively communicates the physical presence of soft 3D characters was exhibited and experienced at SXSW 2024 in the United States. The tactile-visual interaction was developed by Matsubayashi, while the character visuals were designed by the group led by Shinoda at Hakuhodo. In addition to character interaction, realistic sensations of rain and thunder-like effects were also reproduced. Similar tactile interactions were exhibited at Niterra’s place in the Osaka-Kansai Expo Future City Pavilion in 2025.

Reproduction of Diverse Textures

texture

The reproduction of static pressure sensations enables the realistic experience of touching virtual surfaces. A smooth, frictionless sensation is perceived when the pressure component is varied gradually. By adding vibration and force distributions, it becomes possible to render a diverse range of textures. This is a joint research project with NTT.

Rendering Static Pressure and Temperature

Reproduction of Static Pressure Sensation

The generated ultrasonic radiation pressure is not very high. If a time-invariant (static) pressure is applied to the skin, the stimulus remains below the tactile threshold and is imperceptible to humans. Therefore, conventional midair tactile presentation relies on time-varying vibrotactile stimuli. However, this is felt as a “vibration,” which differs from the sensation of touching an object naturally.
Recent research by Morisaki et al. has revealed that a dynamic pressure distribution can create a sensation similar to static pressure. This finding demonstrates that versatile tactile presentation is possible, albeit with certain limitations on spatial resolution and maximum force.

Reproduction of Cold Sensations

cooling

Temperature plays a vital role in perceiving texture, but non-contact temperature presentation—especially cold—was previously considered difficult. However, in 2021, Nakajima et al. confirmed that focusing ultrasound on the skin in an environment with room-temperature mist of water causes the skin temperature to drop rapidly due to the heat of vaporization near the surface. A temperature drop of 3 K was observed within 0.5 s of irradiation, which is comparable to cooling via standard Peltier elements. It was also confirmed that the movement of the cooling spot is perceptible.

Visualization of Ultrasound

thermal

A problem with midair haptics was the measurement of the sound field distribution. Previously, the only practical method was to scan a microphone using an automatic stage or a robot arm. In 2021, Onishi et al. discovered a new practical method to measure the details of intense ultrasonic fields. Using a thermal camera, the sound pressure distribution on an arbitrary two-dimensional plane in 3D space can be measured. This measurement is conducted in real-time with a high spatial resolution. For example, a thermal image of a mesh screen inserted into the ultrasonic wave can visualize the 2D sound field distribution on the mesh instantaneously. The 3D distribution can be easily obtained by moving the screen. Furthermore, it is possible to visualize the sound field distribution directly on the skin surface.

Key Achievements up to 2020

Lateral Modulation: Creating Strong Tactile Sensations

In tactile stimulation using acoustic radiation pressure, the maximum force is determined by the device’s output power. Therefore, it is necessary to employ stimulation methods that feel as strong as possible within this fixed range. It has long been known that when vibrating stimuli are applied to human skin at a constant amplitude, they have a lower threshold than static ones, with sensitivity peaking around 100–200 Hz. For this reason, Amplitude Modulation (AM) at these frequencies has traditionally been used.

However, recent research has shown that keeping the output constant while rapidly vibrating or rotating the focal point along the skin—a method called Lateral Modulation (LM)—makes the stimulus feel significantly stronger. Compared to AM at the same maximum force, the threshold drops by approximately 10 dB. This trend is particularly notable at around 50 Hz, allowing tactile sensations to be clearly felt on hairy skin, such as the arm, as well as on glabrous skin.

Suppression of Unwanted Noise

noise

In methods like LM, rapid changes in ultrasound phase occur when moving the focal point quickly. Such phase changes often cause fluctuations in ultrasound amplitude, leading to changes in tactile feel or the generation of unwanted audible noise. (The occurrence of noise and amplitude fluctuations due to phase changes was first pointed out by Hoshi.) The following paper demonstrates a practical method to suppress these fluctuations.

Distribution Reproduction on Fingertips

mat

Reproducing the sensation of grasping and manipulating 3D images as if they were physical objects is a primary challenge in midair haptics. When touching a real object, the depth and angle of contact on the skin change dynamically; this sensory feedback is crucial for stable grasping. Matsubayashi et al. developed a system that presents the expansion of the contact area in response to contact depth in real-time, utilizing a high-speed rotating focal point. Even with 40 kHz ultrasound (8.5 mm wavelength), surrounding the target with a phased array allows the focal region to be localized to approximately 5 mm. By scanning this focal point, the contact area on the fingertip can be significantly varied according to the contact depth. This research demonstrates the successful execution of tasks such as picking up an object and moving it to a designated location without visual cues.

Motion Guidance via Haptics

guidance

Haptic Pursuit

In vision, when a target object moves, we naturally track it while keeping it at the center of our visual field. This ability is known as “pursuit.” A similar capability, termed “Haptic Pursuit,” has been discovered in the sense of touch.

Consider a scenario in which a point-like tactile stimulus is presented to the palm and moves laterally. With midair haptics, even though no shear force occurs along the skin surface and only the pressure distribution moves, the hand can easily follow this lateral motion. By leveraging this innate human ability, it is possible to guide the hand to any desired location by moving the stimulus point toward the target destination.

Virtual Handrail (Guidance along the palm)

By presenting a point stimulus that scans along a line, the system allows the user to easily perceive the direction and follow a trajectory without visual input.

Guidance perpendicular to the palm

The Early Era: World-First Achievements at the University of Tokyo (2008–2015)

It was demonstrated that even though the force from acoustic radiation pressure is very weak, tactile sensations can be perceived by modulating it. Although different from the sensation of touching a solid object, it was proven that various tactile experiences could be created by overlaying this with visual information.

Airborne Ultrasound Tactile Display (AUTD)

autd iwamoto

Touchable Holography (Interaction with midair 3D images)

touchable_holo

Tactile Projector (Simultaneous presentation of images and tactile sensations on body surface)

fireworks_hand

This research pioneered the use of synchronized multi-unit phased arrays. By aligning midair haptics with visual projections on the skin, it achieved a seamless integration of visual and passive tactile sensations in a live demonstration. (Demonstrated at World Haptics Conference 2013)

youtube Video

Haptomime: Midair Haptic Touch Panel

haptomime

World’s first demonstration of a midair virtual touch panel. It enables users to manipulate icons in midair with the same intuitive feel as operating a physical touch panel. youtube Video

3D Haptic Holograms

horn

Steady 3D Ultrasound Distribution

The first research to calculate and reproduce 3D acoustic energy distributions. Using steady standing waves from surrounding sources, it realized midair tactile presentation of 3D object shapes without the sensation of sound or airflow. youtube Video 1 youtube Video 2

inoue_lev

Acoustical Boundary Hologram

The practical formalization of acoustic boundary holograms was achieved, leading to the world’s first midair levitation of free-form objects significantly larger than the wavelength. This system provides restoring forces in all six degrees of freedom, ensuring stability against external fluctuations.

Haptoclone: Haptic & Optical Clone

haptoclone