rough prototype


Viet Dung Vo’s “Buddha”


I don’t know how well-known this person is in the art world, but he seems to have done a number of interactive pieces.  In this one, it looks like he used a video in watcher along with a delay and mirror effect, similar to the one in photobooth. If the green figure in the center moved around with the actress, I would say there was a blob counter and eyes++ involved, but it seems to stay put and she makes an effort not to stray too far from it so that it always covers her.

What I’m doing and why it’s significant


I will be creating an interactive musical experience which involves a projection mapping of 6 to 8 light-colored, geometric blocks arranged in a “neat pile,” for lack of a better word. Every visible side of the shapes (except for the tops) will be filled in with different colors by the projection, which I will design in photoshop and edit in Aftereffects. The fronts of the shapes will have faces projected onto them (not live action faces, but cartoon-like faces that I will draw myself.)  I’m going to use Isadora and MakeyMakey to assign a musical note to each block. Every time a block is activated (haven’t decided for sure on my physical interface), that block will “sing” its own note and the face will also emote as it does so.  The user can activate the blocks in any order he wants to make music and even play simple melodies. It will be like a chorus of blocks and you are conductor.  My goal for this project is to teach myself simple projection mapping while creating a fun and uncanny experience for people.  Showmanship is very important to me and my work. I was also looking for an excuse to involve music–as always; but I want to do it in a tasteful and meaningful way. Right now, I’m still in the prototyping phase. There are still some things to work out and troubleshoot, but I’m getting a lot of help from the Isadora forum. I will post pictures of my experiments as soon as possible.

I know what I want to do (but it’s not working)


I had an idea that is similar to what this guy did:

He said he used photoshop to map out the blocks and Processing (which I guess is similar to Isadora) to set up an audio watcher that makes the faces move their mouths depending on audio input levels.  My idea was to map out some colorful shapes and assign a different synth note to each one. Then, I would hook up MakeyMakey to some colored footprints on the floor (color-coordinated to the shapes) and have it so people can step on the footprints and make the blocks “sing” each time a key is activated. The keys would be programmed in Isadora to correspond to each colored shape. W might be a green cube playing the note “A,” for example. or S could be a rectangle playing “E.”  People could make music just by stepping on footprints, and I could even write sheet music for some simple songs for people to follow. The only problem is that when I map out my shapes in a software like paint or photoshop, Isadora distorts the mapping when I load the jpegs into the movie player…  so the mapping does not actually cover the shapes. I can’t seem to get around that problem, but I’ve been trying to think of alternatives so I don’t have to scrap the entire idea.

projection mapping


I was trying out some basic mapping using Paintbrush and loaded the design into Isadora as a jpeg. I want to use actual mapping software, but for now I’m just trying to see what’s possible with the most basic tools available. Plus, I saw a guide where a guy was using this exact technique with microsoft paint and photoshop.  What I did was trace the outline of the box while in paintbrush with the display mirrored. Then I filled in the 2 sides with color and added a black background. I saved it as a jpeg and loaded it into Isadora to project onto the box. Unfortunately, the Isadora stage output made the design smaller than it was in paintbrush, so it did not cover the entire box… I can’t figure out why because I have the stage size set at 1280×800 pixels (the same as my image size in paintbrush).  If anyone has advice on how to make the images match up, I’d appreciate it.  I was thinking of mapping a series of geometric shapes, all in a row, and make it so that each shape corresponds with a musical note.  People can touch the tops of the shapes to make them light up and play music.  It’s a longshot right now, but I’m hoping to make it a reality.

IMG_7168 IMG_7169Screen Shot 2014-11-10 at 4.24.30 PM Screen Shot 2014-11-10 at 4.28.33 PM IMG_7172 IMG_7171

Project 2 follow-up/documentation


I think this project was the result of my not being able to think in terms other than narrative. I’m a story-person by nature, and am fascinated by the combination of narrative and interactivity. My references for this project were not academic–instead, I was inspired by “lay peoples'” entertainment such as choose-your-own adventure books and, to some extent, video gaming culture. One of the biggest challenges I faced was trying to reconcile the project requirements with my own personal vision, while trying to make the technology cooperate with what I wanted to do. I knew I could create 6 total ‘choices’ and unlimited chronological sequences after those 6 letter keys were accounted for.  (Thanks to feedback from the class, I now realize that I can use a timer to potentially have more than 6 controls.) My very simple patch is a reflection of my desire to keep things minimal and yet still explore a meaningful concept.  I struggled a bit with the subject matter of my work because I knew that I was relying a lot on humor; in the past, humor has been a prominent theme of my work. I find that it gives me the license to make every-day, mundane events into something more cinematic.  To have this ‘kitchy’ looking footage try to be something more serious than it is would seem misguided. I also learned a couple things about working with circuits, namely that it is a mistake to wrap a box with foil paper because the system gets confused when all the wires come in contact with it. I learned that the hard way, after spending hours trying to figure out what went wrong. It seems like no matter how careful you think you are, something will always go wrong somewhere. As far as editing went, there was the familiar challenge of taking this ordinary footage and making things happen with it that didn’t actually happen in real life. For instance, how do you imply a character’s death without putting yourself in danger while filming? How do you make a character go to jail? These actions required some appropriated footage to link things together, and it was both difficult and entertaining for me. I know the project was a little too dependent on editing and that the technology was more of a means to an end than an exploration of the different functions. For my final project, I hope to step it up and try to learn more ways to use Isadora (and Arduino) so I can create something that speaks to the technology more than to editing. However, I’m not entirely blocking out the possibility of expanding on this project for my final.