This was a little prototype I put together during the MIDI hack in Stockholm on the 17th and 18th of May. I’ve been working a lot with color matching lately and I had an idea of translating the color values into MIDI notes. My tools of choice were Quartz Composer and Javascript. I had already finished 4 out of 5 color matching macros for QC and I used them as a base for my hack. I imagined the result sounding really cool but I can’t really say that the outcome was that wonderful. That is possibly a result of that I’m foremost a visual artist and not really a musician but also due to that I spent too much time on the technical and visual aspects of the project and less on the music. I should have done more research on chords, harmonies and MIDI notes in general.
How it works
The synth samples the color of a pixel from the dead center of a video screen. I was using the built in iSight camera but I also tried using a mobile camera connected over AirBeam. The synth selects a number of matching colors depending on the selected color matching method; Complementary = 2 colors, Triad = 3, Tetrad 4, and Monochromatic = 7. All results also include the original color. You have probably seen the colors spectrum presented as a circle. I convert the RGB value of the selected color to HSL to get into a circular color space. In that way I can pick colors on the other opposite side of the circle using some math. The monochromatic matching method works in the HSV color space which is a slightly different from HSL. The end result of my color matching macros are RGB values (this is just a practical aspect since it’s the color space I mostly use for other projects). The RGB values are then once again converted to the HSL color spectrum and the hue of the color is converted to a MIDI note that I pick up in Ableton Live.
I almost gave up
First I had some major issues with the MIDI patch I was using in Quartz Composer. It just wouldn’t output anything. By a coincidence I realized that when I toggled the gate switch the MIDI messages were sent, so I started toggling the gate every time the the color value changed. There is probably a better way of handling this but that was my quick fix. I still had problems; the color reading from the video was really jumpy and I tried to figure out ways to smoothen it. I actually gave up saturday night and went home to get some sleep. I know it’s not true hack style to sleep but I had to pick up my daughter the following evening so I didn’t want to be totally messed up. On my way home I realized that I had been over thinking my value problem. The jumpy values were basically a bunch of zeroes being sent that messed everything up. For the sake of the prototype I could just filter out all zeros and it would probably be fine. So the next morning I fixed it, cleaned up my composition and submitted my hack to the demo session. I also drew some color swatches with my daughter’s crayons to be used during the demo.
The presentation
After a long wait (I was nr 41 out of 50 something) I finally got to present. I managed to borrow a multi colored Lego brick from the Native Instruments team. The clear colors gave a better reading than the swatches I drew with the crayons. The demo went fine and I got some unexpected good critique which made me happy. It was super awesome to see all the other hacks. Some of them were really great, others just smart or plain funny.
What is the future of the Color Theory Synth?
I’m not sure. On one hand I don’t think I will ever use it to make music so maybe I should just put it aside. But on the other hand I feel that the music side of it could be explored much further and the synth could actually be made to sound much better. I just read an article on DJ mag today mentioning the Camelot System. That seem to have a pretty obvious connection to what I’m doing so I’d like to investigate in how I could make use of it in my project. One thing is certain and that is that I will release the color matching macros for Quartz Composer in the near future. I just need to finnish up the Analogous macro and ask the QC community to take a good look at my math. I bet there is room for improvements.
And by the way, the video above is from the whole hack, unedited. Please enjoy all the other brilliant artists.
Mike – I love the craziness of this idea. I have a few ideas, I will email you!