Pair an M5AtomS3 head tracker over Bluetooth. Once paired, it stays remembered between launches.
Tell the head tracker which 2.4 GHz WiFi network to join. Credentials travel over the encrypted Bluetooth link and are stored on the device.
First-time setup: pick your head-tracker hardware below, flash it with the firmware, then pair it via Bluetooth in the Devices section above. Each supported device has its own firmware tuned to its IMU chips. WiFi credentials (if you want them) are sent over the paired Bluetooth link — no longer compiled into the firmware.
Enable if the M5AtomS3 is mounted upside-down or the rotation feels inverted.
Display the M5AtomS3 silhouette with USB-C port on top of the 3D dummy head.
Mirrors the on-screen head's yaw rotation without changing the audio. Use this if the dummy head appears to rotate the wrong way while the audio is correct.
Tap Get Firmware to share the .ino source — AirDrop to your Mac is the smoothest path. Open in Arduino IDE and upload to the M5AtomS3. After flashing, the device boots advertising over Bluetooth — pair it from the Devices section above.
Sensory/Multi-Sensory Design Playground
Version 1.0.0
© Shimon Shmueli
Your brain doesn't have a separate "touch" module — it builds the sensation of a surface by integrating vision, audio, and proprioception. When those signals agree on what you're touching, the brain commits to a single percept, even if one of the channels is missing or fake.
1. Cross-modal binding (audio ⇄ touch). Each ping is a micro-impulse. When pings appear at the right spatial density and timbre for a given drag, the auditory cortex triggers activity in somatosensory regions (S1/S2). You literally "feel" what you hear. Jitter, transient bursts, and amplitude variation are essential — perfectly identical pings are flagged as synthetic and the illusion collapses.
2. Control/Display (C:D) gain mismatch. When the puck moves slower than your finger, your brain resolves the conflict by inferring friction. Lécuyer (2004) showed this is enough to make people report "resistance" with zero physical force. C:D < 1 = sticky/heavy, C:D > 1 = slippery/light. Stick-slip motion adds dry-friction realism on top.
3. Visual-motor prediction error. Puck wobble, ripples, and procedural surface patterns give the motor system predictive visual targets. Any mismatch between the expected and actual motion is interpreted as physical interaction with the surface.
Motion Coupling — velocity→brightness mimics friction exciting high harmonics; velocity→density creates washboard acceleration; direction→timbre creates anisotropy (brushed metal).
Pattern (audio) — temporal distribution of pings. Uniform = grid, random = natural, double = corduroy, sparse = gravel.
Transient ping + inharmonicity — the "impact" of contact before the tone. Crucial for crisp surfaces (glass, metal).
Reverberation + early reflections — signals the acoustic distance of the imagined surface.
Visual Feedback panel — all the classic Lécuyer-style cross-modal illusions.
• Use headphones. Laptop speakers destroy the stereo pan and low-end cues.
• Start with a Library preset (Sandpaper, Ice, Gravel) — they're hand-tuned to combine effects that reinforce each other.
• Try A/B: store two surfaces, then press A/B on the keyboard while dragging. The contrast makes each feel more "real."
• The Blind Test is the best way to convince yourself it works.
A random surface is loaded. Drag on the pad, listen and feel — then pick what it is.