Doodle Tunes
This project lets you turn doodles (drawings) of musical instruments into actual music. A camera looks at your drawing, detects instruments that you have drawn, and begins playing electronic music with those instruments.
Doodle Tunes is a software application, built with openFrameworks, that uses computer vision (OpenCV) and convolutional neural networks (ofxCcv) to analyze a picture of a piece of paper where instruments have been hand drawn, including bass guitars, saxophones, keyboards, and drums.
The classifications are made using a convolutional neural network trained on ImageNet, and sends OSC messages to Ableton Live, launching various clips playing the music for each instrument.
A simplified version of the software is available at https://ml4a.github.io/guides/DoodleClassifier/
Made at Nabi AI Hackathon 2016 in Seoul in collaboration with Gene Kogan.
Selected press:
Exhibitions:
Heinz Nixdorf Museumsforum, Paderborn, Permanent Installation QUT Art Museum, Brisbane, 2017