Doodle Tunes

This project lets you turn doodles (drawings) of musical instruments into actual music. A camera looks at your drawing, detects instruments that you have drawn, and begins playing electronic music with those instruments.

Doodle Tunes is a software application, built with openFrameworks, that uses computer vision (OpenCV) and convolutional neural networks (ofxCcv) to analyze a picture of a piece of paper where instruments have been hand drawn, including bass guitars, saxophones, keyboards, and drums.

The classifications are made using a convolutional neural network trained on ImageNet, and sends OSC messages to Ableton Live, launching various clips playing the music for each instrument.

A simplified version of the software is available at

Made at Nabi AI Hackathon 2016 in Seoul in collaboration with Gene Kogan.

Selected press:


Prosthetic Knowledge

Digimag 76


Heinz Nixdorf Museumsforum, Paderborn, Permanent Installation QUT Art Museum, Brisbane, 2017