OMK: Open Music Kontrollers

24 Sensor Prototype

This is just a first prototype, the next one will have 144 narrowly spaced sensors, corresponding roughly to the dimension of 4 octaves on a standard keyboard, stay tuned...

General idea of the device

The general idea was to develop an expressive music controller. The general characteristics that had to be fulfilled were:

playable with fingers
multi touch(less) sensitive
two continuous dimensions
lightweight
dimensions of a ribbon controller
low latency and high update rates (~2kHz)
entirely based on open hardware, software and communication protocols
designed to be as simple as possible but at the same time as configurable as possible

The outcome was a device that is best described as a polyphonic theremin, based on magnetic distance sensing to permanent magnets atttached to fingers relative to an array of high accuracy linear hall effect sensors. An open source prototyping board with an ARM Cortex M3 MCU acts as the central computing unit, reading out the sensors, computing multi touch recognition and sending output via Ethernet.

The device therefore features two continuous dimensions. The X-dimension tells about the fingers position in the sensor array and most of the time will correspond to pitch. The P-dimension tells about finger proximity to the sensor array and may correspond to amplitude, filter cutoff frequency, modulation, etc.

Math

The magnetic field of a magnet always influences more than one sensor. By fitting a parabolic curve through the center sensor and its two adjacent ones, one can get the exact position of the magnet. Due to the interpolation of three sensor values, we get a higher accuracy for the X and P dimensions for a single touch event than for just one sensor value alone.

$y_{1} = A \cdot x_{1}^{2} + B \cdot x_{1} + C$
$y_{2} = A \cdot x_{2}^{2} + B \cdot x_{2} + C$
$y_{3} = A \cdot x_{3}^{2} + B \cdot x_{3} + C$

$A = \frac{(y_{2} - y_{1}) \cdot (x_{1} - x_{3}) + (y_{3} - y_{1}) \cdot (x_{2} - x_{1})}{(x_{1} - x_{3}) \cdot (x_{2}^{2} - x_{1}^{2}) + (x_{2} - x_{1}) \cdot (x_{3}^{2} - x_{1}^{2})}$
$B = \frac{(y_{2} - y_{1}) - A \cdot (x_{2}^{2} - x_{1}^{2})}{x_{2} - x_{1}}$
$C = y_{1} - A \cdot x_{1}^{2} - B \cdot x_{1}$

$X = \frac{- B}{2 \cdot A}$
$Y = A \cdot X^{2} + B \cdot X + C$

Example video

"Multi Touchless Ribbon" aka "Polymagnetophonic Theremin" on Vimeo.

Example music

Here are some sample tracks created with SuperCollider as the synthesizer backend.

Electronic parts

Those are the electronic parts this prototype was built with:

LeafLabs Maple Mini (1x)
WIZnet ethernet module WIZ820io (1x)
Allegro linear hall effect sensor A1324 with a sensitivity of 5mV/Gauss (24x, SIP package 0.05")
Dual 4-channel analog multiplexer 74HC4052, e.g from NXP (3x, DIP package 0.1")

PCB layout

The PCB was layouted with Fritzing and manufactured at FritzingFab. Get the raw Fritzing files or the Gerber files in the download section. To reduce the costs of the prototype board and have a small width of the PCB, the WIZ820io module is mounted on top of the Maple Mini, which may look a bit awkward.

Magnets

The magnetic fields are measured continuously to get an estimate of its position in and proximity to the sensor array. A handy online tool to calculate magnetic fields of permanent magnets of varying shapes and sizes is provided by K&J Magnetics. There you can buy the magnets that suit your sensors and case mechanics best. The magnets for my project have been bought at supermagnete.ch: circular magnets 8x2mm, N45 quality.

Case

The magnets may just be attached somehow to your finger tips by e.g. sticking them to gloves or steel rings, thereby controlling the magnets directly with your fingers in a touchless way. An other possibility would be to use one magnet for each sensor, attach them to springed keys or to an elastic textile, thereby getting a proper touch surface. I prefer the first way, as its much more theremin like, has less resistence and is less error prone, the less moving parts in a device, the better.

The final case shall be based on parts that all can be produced by a laser cutter out of your desired material (e.g. acrylic or wood) in your local FabLab or from a rapid prototyping provider like Ponoko.

Firmware

The firmware that is flashed onto the Maple Mini can be found in the sources. What it does is reading out all the hall effect sensors, doing the multi touch recognition and writing out the multi touch events via Ethernet. The firmware was highly optimized and parallelized and reaches a maximal update rate of 2kHz. The device uses the TUIO specification to transmit the multi touch events via the Open Sound Control protocol via UDP over Ethernet. This has the advantage of being fast, needing no special drivers, being completely independent on the used operating system and easily transferable over a local network. Check out the master branch of the repo here.

The MIDI protocol is not ideal for this application as you loose precision by using it, both in the time and the space domain. MIDI maximally supports 14bit resolution (if your MIDI device can use double precision control messages, and many only support 7bit). The device however is able to provide much more than just 14bit, which would be a pity to miss. However, direct MIDI support may still be provided in a future prototype, either as raw hardware or via USB. For now, if MIDI is really needed, it can easily be generated by software.

Software

What happens on the software side is up to the user. As the device uses open specifications, it's quite easy to use existing or write own dispatchers for whatever software the user wants to steer with the multi touch ribbon. Provided is an example implementation of a TUIO dispatcher which implements a simple dynamic user interface directly on the multi touchless surface and can dispatch the multi touch events to SuperCollider or MIDI (see section above) for further sound synthesis. It is written in C and dynamically configurable with the Lua scripting language. Check out the master branch of the git repo here.

Network profile

The Maple Mini is configured with a fixed IP of 192.168.1.177 and it sends its OSC messages to the broadcast address 192.168.1.255. On your computer you therefore have to configure a local network with gateway 192.168.1.1 and an IP of e.g 192.168.1.2 with broadcast listening on 192.168.1.255. Please look up the corresponding network configuration steps in the manual of your operating system.

If you would like to have different addresses, you can easily change them directly in the firmware sources, a future firmware release will feature dynamic changing of the addresses, too.

Multi Touchless Ribbon aka Polymagnetophonic Theremin