Friday, 10 August 2012

Introduction and Text to Music


Two and a half years ago, March 2010, I was 6 months out of University, with a first class degree in music. I'd been running light and sound at a venue in Crewe, without being paid for it. Also looking for rubbish work through the usual channels, and not getting it. Through a stroke of total serendipity I came across a job which involved being taught how to use Ruby. Even though this was minimum wage work, paid for by the government in order to improve unemployment figures prior to the last election, and I wasn't working in a real development environment, this started my career in Ruby development. It turns out that Ruby's in high demand, and with hard work and diligence I've become a competent, valued developer in a number of companies (two).

When I started with ruby, however, I hadn't done any coding at all. I had, however, done some programming with pure data. This was part of my degree, used for generative music, non-standard musical instruments and synthesizer design, and showed me that I have the kind of mind that can do programming. So it was a natural decision was to make my first ruby project into one which interacts with pure data. So my ongoing side-project text to music was born, by way of an interesting way to start this blog, I thought I'd write out a description of how it works and some of the (possible) future directions.

Text to Music

The on-going text to music project does exactly what it says on the tin. A string of text is dealt with one character at a time, and converted to a message that's sent via a http port to pure data. Pure data then routes this to one of a number of synthesizers and it's played out as a musical sequence. It was a simple idea to work with when I started, and taught me how to use the string manipulation, http ports, classes, methods, external libraries etc. in Ruby.

A few years later, after considerably more development experience, I returned to this first project on GitHub and cleaned up a lot of the code, added comments and started to look at methods of feeding the Text to Music algorithm besides the existing manual input method.

This lead to the project as it currently stands, with a seperate class to deal with the connection to pure data and modules which deal with the string manipulation, shared initialization scripts and twitter authentication.

Wait... twitter authentication?

One of the core selling points of the Text to Music POC is real-time integration with twitter via the real-time streaming and user stream API's, so when a tweet comes in on your user stream, or on a search of your choice, it's relayed to pure data and made into music. Representing real-time notification and a sound in which you can recognise some patterns and what they mean.

Listening to tweets

A hang-over of studying music, including some of the odder, freer, more experimental forms of music, is that I've started to listen to 'just noise' with a critical and analytical ear. Text to Music, with each letter producing the same note every time it's played, and punctuation with recognizable sounds, allows us to hear some of the patterns that are inherent in language and text. The length of a word, between the short gaps provided by spaces, is an obvious example, providing a rhythm to the musical sounds.

The further along the alphabet a letter is, the further along the alphabet a letter is, the higher the resultant note. This acts as emphasis, and a way to spot repeated patterns. The classic example is a well known rhyming sentence, 'How now brown cow'. The repeated use of the letters 'ow' results in a motif, the mid-range note 'o' followed by the higher 'w'. Similarly, the word 'potato' allows you to hear a sequence moving up and then swiftly back down again.

Time to look at tweets, punctuation in Text to Music makes more distinct sounds. A hash tag (e.g. '#bieberfever') will be a word preceded by a piece of white noise, like a walkie talkie crackling into life. The @ symbol is a distinctive whoop, making it clearly audible when a user name is mentioned. and a link (preceded by http://) shows up with the two high-pitched t's and upward whoops from the slashes.  This means that with some time listening, you can start to have some idea of the content of a tweet merely from it's sound.

I've also connected to RSS feeds for a source of text, and other API's are almost certainly possible as source material.

What's next?

A recent discussion over twitter with fellow developer Adam Strawson (@AdamStrawson) has lead me to think about the possibility of using Text to Music as part of a Ruby on Rails application. In short so that a Rails app could send a message to a machine in an office which would make some music to inform people there that something is happening through the app.

The discussion with Adam was about using it as part of an online shop, so when an order is made, the music machine pipes up and lets employees know that there's an order which needs to be processed. They could even learn to recognise products or their categories from the sound.

The use of http connections means it could be set up so a server sends the commands remotely to a computer running pure data elsewhere. However the use of logical delays (the ruby command sleep(0.1) etc.) means that the actual ruby processing would have to be on a seperate thread to stop the server freezing whenever a notification is sent. Another possiblity is that a third application would be set up to recieve http requests containing the text to be sonified, process the characters and then port them to pure data. Either way, progressing with my acts_as_musician plugin is likely to require an increase in my level of in-depth technical knowledge.


If you're reading this and thinking that Text to Music sounds like fun, I have good news for you, It's open source!!! So if you're using Mac OS or just about any linux distribution and can use the terminal, I'd love for you to get involved!

First step is to install Pure Data and Ruby on your machine, there's about a million websites which will tell you about either one of these, so I won't bother using any more space to talk about it.

The second is to go to the Text to Music Github page and follow the instructions from there.

I'm always interested in receiving feedback on how you find it to install, how it could be improved, technical problems you run across or even, if you know any Ruby or Pure Data, any modifications or improvements you want to make to it. Feel free to email me at

Well, I think that's a fairly full first blog post, hopefully I'll be getting back to blogging soon with some pure data recipes.

No comments:

Post a Comment