A browser-based hand-controlled synthesizer built with p5.js, ml5.js, and Tone.js.
The app uses your webcam to track hand landmarks and maps hand movement to synthesizer controls in real time.
https://frankstop.github.io/HandSynth/
HandSynth displays a live mirrored webcam feed with detected hand landmarks drawn on top.
Once audio is enabled:
- Index finger height controls pitch
- Thumb-to-index distance controls volume
- No detected hand fades the sound out
- Real-time webcam hand tracking
- Hand landmark visualization
- Browser-based audio synthesis
- Pitch control using vertical finger position
- Volume control using thumb-to-index finger distance
- Smooth audio parameter changes
- Mirrored webcam view
- Start button for browser audio permissions
- No build step required
- GitHub Pages compatible
| Input | Action |
|---|---|
| Start Audio button | Enables the synthesizer |
| Move index finger up | Raises pitch |
| Move index finger down | Lowers pitch |
| Move thumb and index finger apart | Increases volume |
| Move thumb and index finger together | Decreases volume |
| Remove hand from camera view | Fades audio out |
HandSynth/
├── index.html
└── README.md