This is an accelerometer to synth modulator im a P5js editor (using Tone.js) in a browser on an android phone. It can be placed inside an enclosure to create an sonified kinetic sculpture.
I am working on an interactive sonic playground that allows seeing and non-seeing individuals to create - and optionally to collaborate with others on creating - sonic compositions in real time by setting kinetic sculptures into motion either by touching or wearing the sonic objects on their bodies. I began this ongoing project last year at NYU's ITP (Interactive Telecommunications Program) last year and I will work on it again this year.
Last year I learned how to take sensor data from accelerometers and and other motion sensors and feed it into MaxMSP to create real-time generative sound compositions that incorporate sensor data into their algorithm and allow it to impact how the instrument will sound in the future. These ever-changing and impressionable compositions live inside the objects on a raspberry pi and connect to speakers/headphones via bluetooth. Snippets of surrounding space is sometimes captured and fed into the algorithm also, making the space sound part of the permanent DNA of the objects and surfacing again in future compositions. In this way, the sonic objects aren't just instruments but are entities that change over time along with us and affected by us.
So far I have simple prototypes that work and I now have a second year at ITP to create a whole collection of these "instruments", each one with it's own "character" - it's own behaviors and sonic qualities. I intend to use a variety of materials and textures that are interesting to touch and that are durable - plush, inflatable, conducive sting fibers, resin, resin foam, 3d-printed with PLA, and even cement.
The collective soundscapes that result from people engaging with the objects and sometimes with each other are portraits of those interactions and spaces and a testament to collaboration, creativity, and play that are so important for human development and self reflection. These qualities separate us from other animals. This sort of tactile, sonic, mindful making in a space that is highly abstracted from familiarity and removed from the bustle of everyday life is also intended to be a place of meditation that the modern human so desperately needs.
The ideas behind it
Sound is, in my opinion, one of the richest materials to use as feedback because it's it abstract yet accessible to most people. We are naturally inclined to listen to music, for example. We perceive the nuances in the vast variety of sound available to us, which can create a myriad of moods. There is emotion in sound that is often hard to convey using sculpture. Sculpture on the other hand is something that can one can be next to - it's visual and tangible. There is a permanence to it that I find valuable in staying focused on an experience. One can be in the presence of an object and it can be meditative to hold it and focus on it. Sounds can complement this meditative experience. Sounds are fleeting but instruments remain. The sound, in a way, is captured by the object, like a genie in a lamp.
To give the experience dimension and give the objects agency I add code that makes the "instruments'" behavior impossible to predict with certainty yet also intuitive. Certain variables will be easily discoverable while others will change over time so that people who engage with the system of instruments cannot fully impose their will on this system.
In a way these dialogic interactions are simulating what many of us would like to see more of in the world - more cooperation with nature is just one example. Another example is literally creating better relationships between object and people (the opposite of Alexa :) ) I found out from Marina that people have been interested in this already- Actor Network Theory. I'm glad there is a language for what I'm trying to do and the are already many conversations about it. Turns out my mom knows about this too! I told her about giving agency to objects and she said "yes I've read about this. It's become popular recently!" So that's fun. She finally understands now why I'm doing these things. Putting in in context of the movement helps people understand the idea behind this project so that's going to help with thesis as I plan to continue exploring various tangible and sonic interactions with an emphasis on subverting traditional human to object (which can also stand for human to nature, human to other species, human to other human even) relationships.
Another way to look at it is if I'm going to build a set of instruments that are meant to be meditative objects, I don't want them to be servile, predictable, and boring. I would like then to charge and surprise me every time I use them. I'd like to create a sense of wonder that comes with using an instrument for the first time out discovering something new. I think it will be stimulating and encourage creative thinking anf perhaps even self refection after certain types of interactions. What does it mean to you and how do you feel when a series of instruments charge how they play in response to a certain way that you activate them? Things like this. I hope I am able to convey all of this when I make more of these and perhaps I will be able to cover more as language is so limited.
Another way of saying this is that if you just record the same sound and keep pressing play it feels like you've captured the sound and it has to play the same way every time according to your whim. it's the end of the game. Feels a little fascist :) and not as much fun as cooperating with the instruments that allow you - if you listen and pay attention to how they choose to work - to create ever-changing, sonic experiments.