Cocktail of Sport Test Videos

Finally!

(figure 10.)

Figure 10 is the final version of the sub patches. The left hand side has been changed from having the greater and less than objects in sequence to having them implemented into a logic based format. 

What happens is that a 0 is being outputted from the "expr" box at all times unless both conditions in the greater and the less than boxes are met, in which case, the "expr" box outputs a 1. These numbers then control the volume and filter controls for the patch.

In addition to these alterations, the patch 'cables' were tidied up and put into the right place so that everything works how it is meant to.

There will be a short video shortly to use as an example.

Graphic Score

Figure 10 is the graphic score that is being used for the project.

(Figure 10.)

Basic Maths

Figure 9, below, shows the basic working out of the greater and less than ranges so that the samples could be controlled one at a time when played back. The range was worked out using the number of samples, the data that was outputted from the webcam - maximum of 100 and the two axes. 

(Figure 9.)

A Gentle Reminder

Figure 8 is a gentle reminder to myself to switch the coding when changing form my iMac to my Macbook Pro. This has also reminded me to copy the samples over to the laptop/ designated computer so that the patch can read and play them back.  

(Figure 8.)

Michael Jordan

Michael Jordan is argueable one of the most influential and talented basketball players of all time and having his speech (below) in this project shows what it takes to be the 'best.'

https://www.youtube.com/watch?v=45mMioJ5szc

Muhammed Ali

Muhammed Ali was an incredible athlete and has several motivational speeches.

The video link below provides some of the speech that is used in one of the samples in the project:

https://www.youtube.com/watch?v=7-5_8af3TiY

Problems part 2!

As with any project that you undertake, there are bound to be some problems for several reasons such as human error/ lack of understanding or knowledge, mechanical error or time issues.

As stated in earlier posts, the original concept had to be scrapped because there was no visible way to obtain the results that were sought after - RGBA analytic data. However, with the new concept up and running, new problems arose. The slight lack of knowledge and confusion of certain objects led me to make mistake with patching objects together and making simple rectifiable mistakes within minutes once the realisation had kicked in - however, this often took several days to come to light! This is human error and is all part of the trial and error side of things.

There are two main problems that I encountered, both mechanical. 

The first big problem was the lack of data that was being outputted from the webcam. There was very little data being recorded where it was before, therefore, the samples could not be triggered and affected by the incoming data - the whole point of this project! This, however, could be fixed with a simple reset of the webcam, which could be found in the webcam streaming sub patch. 

Previous Sub Patch and sub patch explanation

Below, is a screen shot of a previous sub patch which was altered to make it simpler as the extra coding ended up being complicated for no extra benefit - making it more straight forward makes less room for error but gives the same desired affect as the later, altered version.

Figure 7 shows the coding for one of the samples and the other than the data parameters (greater and less than values) is identical for every patch. 

 (Figure 7.)

The section on the left hand side, highlighted in green, is the data that is extracted from the webcam's X or Y axis. This data then controls the volume level of the playing sample based a greater than and less than parameters.

The red highlighted, central section in this part of the sub patch takes the data from the webcam, in exactly the same way to the volume section of the sub patch, and was meant to control a high pass and low pass filter based on the same greater than and less than parameters that were set in the volume part of the patch. The data would be received from the webcam and selected 1 or 0 based on wether the data fell within the set data range; this, in turn, would open or close the filter. However, this did not happen as planned and was simplified at a later date to simplify it for the same effect.

The yellow shaded section of this patch is simply the sample loading coding. The coding loads the sample into Pure Data, defines the sample rate, the number of samples and then outputs it to the outlet.

The outlets are shaded in blue in Figure 7. These link the sub patch to the main part of the patch. The way that the outlets are arranged on the screen dictates which side of the sub patch object the outlet appears - this is used for ease of patching in the main patch to prevent confusion of connections.

Samples Examples

Below are a couple of samples that were recorded for the project:

https://soundcloud.com/phil-hutchins-prodctions/sports-sample-1

https://soundcloud.com/phil-hutchins-prodctions/sports-sample-2

Samples

Since the camera input had been already created from the previous patches, that section of the patch was sorted. However, the addition of the X and Y output sections needed to be added. 

The change in data was quite quick and unstable, therefore, I decided to average out the incoming data; the 'average' object in Pure Data averages out 10 of the incoming figures to provide a more stable number. This could be done again to slow down the change  even more. However, I am happy with it the way it is.

The sample had to be recorded before any patching went ahead, otherwise there would be no sound coming out of the patches to see if the coding was working or not.

As a very keen sportsman, sports enthusiast and exercise nut, I chose to use famous, motivational and inspirational speeches/ quotes for the project. This way, each sample would stand out and could have more of an impact rather than general hubbub. It also allows me to relate to my work in a different way other than purely creating it, I can relate to some of the samples and the others are quotes that I deemed important and powerful.

I scoured through a list of 100+ famous and inspirational quotes from various videos, books and websites and settles on using some from the following link:
http://www.keepinspiring.me/100-most-inspirational-sports-quotes-of-all-time/
There are some typically famous athletes in the list such as Arnold Schwarzenegger and Muhammed Ali, as well as some unexpected culprits like Adele and Lil Wayne 

Four people (including myself) were recorded saying five quotes each.

the signal chain was: SE Z3300a microphone -> ART MPA 2 Pre Amp-> Tascam US1800 Interface -> iMac -> Logic Pro 9.

(Figure 6.)

As seen above, the quotes were recorded and then imported into Pure Data through coding.

Idea Change

After researching into the Cocktail Party Effect and the patch failures involving the RGBA data, I decided to use the concept to create a mini installation/ demonstration on selective hearing.
The concept being that motion detection through a camera would be used to control the levels of different samples within Pure Data, based on the output data from the camera (X and Y) - this would be similar to that of the RGBA data that would have been coming from the camera in the first place.

Cocktail Party Links

http://xenia.media.mit.edu/~barons/html/cocktail.html

http://www.dailymail.co.uk/sciencetech/article-2530825/Old-married-couples-easier-hear-partner-crowded-room-ignore-completely-thanks-cocktail-party-effect-new-study-finds.html

Cocktail Party Effect

Below is a link to the BBC World Service audio piece on the Cocktail Party Effect:

http://www.bbc.co.uk/programmes/p012jcyh

The start of Pure Data patch construction!

The first thing that had to be accomplished was a basic understanding of Pure Data, how it works and knowledge of the programming data. When previously attempting to learn Pure Data, I followed some tutorials online by Dr. Rafael Hernandez, which helped to explain the basic functions and different types of objects and boxes that can be found as well as the basic maths and logic that has to be used within the program to make the patches work.

http://www.youtube.com/user/cheetomoskeeto?feature=watch

Once I had a basic understanding of Pure Data programming language, I made a few test patches with simple sine wave and sawtooth wave oscillators to ensure that what I had understood actually worked in practice - this allowed me to get familiar with the shortcuts rather than just watching and understanding.

It then came the time to construct the patch for the project.

In order to get the patch started, the first bit of programming had to be the construction of the camera input. Making sure that the camera was being read by Pure Data and the data could be read correctly was the priority at this stage before even thinking about the audio. Pure Data has a bank of example patches or sections of patches that can be used or used for reference material; one of these is a video section which incorporates the built in webcam on a laptop - extra cameras can be added at a later date with more in depth programming.

(Figure 2.)

Figure 2. (above) shows the help box which appears with all the tutorial settings and examples of what is possible with the program. As seen on the right hand side, there are several video examples that are open to users to learn from and edit for their own use. The one that I have taken most use from is number 03.movement_detection.pd. 

(Figure 3.)

Figure 3. shows one of the first patches that was developed. The 'create', 'destroy' and 'dimen' message boxes at the top are sending messages to the 'gemwin' object to create and destroy the webcam incoming image as well as the size of the image that would be analysed. The 'gemwin' object is what is being controlled by the message boxes; in this instance, this is the webcam. The gemhead is very similar in the instance that it receives information from the webcam which, in turn, sends information to another object which tells Pure Data that it is receiving video (pix_video.)
The problem with this patch is with the (pix_pix2sig~) object. This object transfers data from the signal into audio and there is a matching reversal object as well. In this instance the RGBA data could not be extracted which was attempted in the above figure; instead, audio data was the output measurement; due to this, the patch had to be altered/ deleted.

(Figure 4.) 

(Figure 5.)

The patches in Figure 4. and Figure 5. has similar problems as previous versions. However, in Figure 5, data was outputted to an array which could be readable; it was a shame that it was only possible to obtain one variable of data and not able to get the full RGBA data that was sought after. 

After several failing attempts at tying to obtain the RGBA data, the idea was altered to instead, utilising the motion detection capabilities of Pure Data programming. 

Photo vs. Video

I have always been fascinated by the human body in terms of sports, exercise and movement - being a keen athlete myself adds to the passion. The idea sprung to mind when trying to see how the body could be used to create music in any shape or form. 

The project originally stemmed from an idea of using photography rather than moving image to analyse the pixel content in terms of RGBA (Red, Green, Blue, Alpha) data. The data would then be used to make a piece of music - the picture that would be used for the piece of music would affectively be used as a graphic score for the piece of music.

However, using photography did not really make it a quick process as only one picture would be loaded. There could have been two ways in which the data could have been analysed - short sound bytes of roughly a second long which would be unique depending on the output data or reading the pixel data line by line in the picture and composing music depending on the pixel data. 

It was decided that these techniques would take too long to produce music/ would not create enough music - that was until I discovered Pure Data which is a program for digital music programming. There is a preset on this program however, to analyse the pixel data for pictures/ images so the project would not be creating anything ground breaking or new - to some extent.

Based on the above reasons, I chose to use moving image and analyse the RGBA data through the use of cameras - either a built in webcam, external camera or a mixture of the two. 

Since it was discovered that Pure Data had an analysis patch for image pixels, Pure Data was chosen for the program to be used in this project. Logic Pro would have been useful for the audio editing, EQ/ effects and familiarity; but, the program does not incorporate the depth that was being sought after. However, Pure Data can be linked into Logic Pro so once the patch was complete this could have been an option.

Initial ideas

The following project is set to make music through the use of physical activity rather than the traditional musical instrument setup. This blog helps to show the development of the ideas and project that I am undertaking and coincides as part of a module called 'Integrated Practice' during my final year at the University of Brighton for the 'BA(Hons) Creative Music Production' course.

The purpose of the module is to combine traditional musical theories and practices with another medium such as video. This can be in any format and previous choices have included sound design for film/ radio, inventions and concepts with future technology and art installations.

 Research into art installations was a good starting point for research into ideas. However, several ideas came to light from the get go, one of which, I have decided to stick with.

The first idea that came to mind was to create a marble run (similar to the one below in figure 1.) either out of existing pieces or constructing a marble run out of wood and other materials. Within the marble run there would be pressure pads built in so that when the marble is let go and slides down the marble run it hits the pad and triggers a signal which is received by a computer. In turn, the signal would be converted into sounds and musical notes via Logic Pro or Pure Data.

The aim of the project would have been to produce an interactive art installation which could also potentially be used in a children's learning environment to aid education on music and sound.

 

(Figure 1.)

The second idea that came to mind was to utilise the human body and its actions to create music through the use of medical sensors such as EMG sensors. The EMG sensors would be placed on the muscles to register muscular movement. In addition, heart rate and perspiration monitors would have been used to take readings and use the outputted signals. The signals would have then been converted into musical sounds/ notes via Pure Data or Logic Pro after being processed through Arduino circuit boards.

This concept however, was dismissed due to the fact that I wanted to go into a high level of depth with this project and would not be able to do so in this module in comparison to the 'Major Project' module. Therefore, the Major project for my final year is the Human Body sensor project. The theory behind this project was to create personal soundtracks for people when they are working out and to hear their workout instead of just seeing figures and graphs on a screen which is the trend at the moment.

The idea was inspired by Japanese artist, Daito Manabe (http://www.daito.ws/en/#1) and the Old Spice advert which starred fitness guru and actor, Terry Crews: http://www.youtube.com/watch?v=yZ15vCGuvH0

The final idea is the idea that I have chosen to develop and is the project that I am undertaking.

The concept is an art installation that utilises video cameras and Pure Data. A patch would be designed in Pure Data so that the video camera attached to the computer would be recognised. The signal from the camera would then been analysed within the patch to obtain the RGBA data (Red, Green, Blue and Alpha.) This data, depending on the the numerical values that were outputted, would be associated with different sounds and musical elements that would be used to great an ambient/ avant-garde style piece of music.

The aim would be to have different people walking in front of the camera to set off the musical alterations from the originally programmed sounds. The time of day and lighting, people's clothing, hair colour etc would all have an affect on the characteristics so the sounds that would be coming from the speakers at that time, would be unique and individual.