KineSpace

KineSpace: An Interactive Instrument

COS/MUS 314 Final Project

Samantha Gebb, Travis Henry, Sarah Strenio

May 15th, 2012

http://dl.dropbox.com/u/45828798/plorkFinal/HenryGebbStrenio.zip

http://dl.dropbox.com/u/45828798/plorkFinal/DDR%20finalcopy.ck

For our project, we were interested in creating an instrument for a single person that on its own could create a fully-fledged soundscape and also provide the player with an active and interesting kinesthetic experience.  This manifested itself in using a DDR pad and tether jointly.

Our instrument has many parameters, but is not terribly difficult to operate.  The DDR pad controls the playback of short loops of a given set of instruments in a few different musical styles.  The musical style is selected using three corner buttons:  back-right (X) for latin, back-left (Y) for blues, and front-left (B) for video-game.  The edge buttons toggle different instruments on and off:  left toggles the bass, back toggles the drums, and right toggles the accompaniment, which varies from piano to guitar to synth between the styles.  As well, the front-right (A) button toggles all instruments on or off.  If all instruments are off, it turns them all on;  if only one instrument is on, it turns on the other two; if two instruments are on, it turns them both off; and lastly, if all three instruments are on, it turns them all off.  The front button does nothing, as this is where the body of the tether sits in front of the performer, rendering the button inaccessible.  Finally, the select button toggles between the two modes of the tether instrument, SpaceMode and KinesisMode, which are described below.  In allowing the user to control which instruments are on, as well as which style, we hope the user will create interesting variations of activity levels, and interacts with the tether accordingly.  It is hard to solo by yourself, and easier to solo against someone; we’re providing back-ups to take the tether solo to the next level.

The tether acts as a solo saxophone to be layered on top of the DDR pad loops.  There are two different modes of operation for the tether which have different outputs and control mechanisms.  In SpaceMode, the player has the ability to play through the D-minor blues scale and adjust octave, volume, vibrato, and pitch-bend.  The right tether controls pitch by dividing the unit circle into six pie slices and locating the x- and y-coordinates in that circle to play the note assigned to the given pie slice.  The z-axis, or how high the tether is, controls octave.  Keeping the tether in the midrange will do nothing, moving it high will add one octave to the base note, and taking it low will subtract one octave from the base note.  The left tether controls volume with the z-axis, which is essentially a one-to-one ratio.  The y-axis determines pitch bend, with a center static range and the ability to move up or down an entire wholestep.  The x-axis controls vibrato, with smaller values for x resulting in less vibrato and higher values in more vibrato.

The second mode for playing the tether is KinesisMode, which makes use of the Wekinator and machine learning.  In this mode, the saxophone UGen is constantly playing a D, but the player is able to manipulate Saxofony-specific continuous parameters through the spatial location of the tethers and the speed at which they are being moved.  Speed corresponds to vibrato frequency, with slow speeds resulting in lower frequencies and faster speeds in higher.  Vibrato gain is determined proportionally by the z-axis, blow position the y-axis, and aperture the x-axis.

Unfortunately, as of now, the KinesisMode does not work.  We think the problem lies in our implementation of the Wekinator’s custom feature extractor, since the training was done separately.  As a result, we have chosen to leave out KinesisMode and use the select button on the DDR pad as a mute toggle for the tether instrument.  Our technicians are working on a patch for this problem as we speak.

We feel that we met our goals of creating an interesting kinesthetic instrument that is easy to play and produces interesting sounds fairly well.  Had we more time, We would have made the OSC communication more reliable.  We might have worked on integrating the tether/solo/saxophone/instrument a little more into the functions of the DDR.  Also, we would have done much more training in the Wekinator to generate a very robust learning system file.  .  As for the loops, it would be great for the player to have more creative control.  Perhaps this could be achieved by including more instrument layers and/or styles, so the player can better shape the soundscape.  As well, a third mode for the tether could allow the player to expressively control specific parameters of the individual loops/instruments as they are introduced.  Of course, after a certain point, one must consider the compromise between flexibility as a musical instrument and simplicity as a demonstrative device.  There are many improvements that could make this project more useful and versatile in terms of musicality, but they are often at the expense of ease of operation.  An important goal of ours was for this instrument to easily sound good when played by anyone.  On this aspect, I think we have succeeded, as long as setup goes smoothly.

******tech setup for laptop attached to tether (when KinesisMode works)

1. Make sure you have the new wekinator jar file, and use this to replace the existing one in the wekinator app’s contents/resources/java folder
2. In 314config.ck, change pathways to appropriate folder for sax_physmodEDIT.ck and SpaceModeSend.ck
3. Open Wekinator app
4. Type “chuck 314config.ck” in terminal, hit return
5. Features setup tab in Wekinator: click “load settings from file and load FinalSettingsNew.wfconf and click “Go”
6. Learning setup tab: click “load learning system from file” and load Final1.wlsys and click “Go”
7. Use it tab: go to Run and click Run

******tech setup for laptop attached to tether (when KinesisMode does not work)

1. Connect to the network
2. Connect the tether to the computer
3. Open SpaceModeFinal.ck and SpaceModeSendFinal.ck or run directly from terminal
4. Add the .ck files to the virtual machine

******tech setup for laptop attached to DDR

1. Connect to the network
2. Connect the DDR Pad to the computer
3. Make sure music files pathways are appropriate
4. Make sure the computer name of the OSC receiver is correct (Currently goes to Sam's computer)
5. Add "combineDDROSC" to virtual machine
6. Make music!

(Credit for assistance on the OSC communication goes to Professor Fiebrink, and assistance on the Saxophony toggle goes to Professor Trueman)

We pledge our honor that this represents our own work in accordance with university regulations.
Signed,
Travis Henry, Samantha Gebb, Sarah Strenio


(After uploading the single zip folder, we noticed a sintax error in the DDR file. Please use the separate one, making sure sound files are appropriate).