At first, the idea was to exploit the possibility of optical illusions using projection mapping techniques; however, I quickly realized this was not feasible for a someone who is new to this. I was inspired by various youtube videos I’d seen which combine music with projection-mapped objects.  I knew I had to make this interactive, so my mind automatically went to interactive “playable” musical blocks.  At first, I planned to use generic-sounding voices and faces. However, failure to come up with good sound clips led me to the idea of personalizing each block with individual faces and voices. Asking for people from this school to get involved definitely adds a community aspect to the work, as mentioned during critique. Overall, I am happy with the way it turned out, but there are a few things I would do differently. First, I would’ve emphasized the detail on the faces so that they would show up better on the projections. The design looks so much more detailed and “fleshed out” on my computer screen than on the blocks. Secondly, I would’ve put more thought into my interface so that I didn’t end up with such a tacky-looking display. As I mentioned before, I’m not sure what my ideal interface would look like; I toyed with the idea of having 3D objects that people can touch to correspond with the blocks. (like having mini versions of the blocks made out of clay so that it would conduct electricity as needed). I think the interface was more of an afterthought for me because I was busy trying to make a project that works the way it’s supposed to.  Also, in the future I might want to try more diverse animations–maybe bits of speech or extending the animation to the designs on each block.  The challenges I faced include trying to decide between illusion and uncannyness, if that’s a word.  I wanted to play a bit with making the faces seem as though they are jutting out of the blocks, or somehow breaking the 2D surface in a way that goes beyond mapping more than the front-facing sides. I also had to negotiate a lot little glitches, like Isadora moving my projection around. Luckily the zoom actor took care of that. Coordinating my participants was also difficult. Understandably, everyone was busy during this time, so people were hard to reach and get responses from; a few said they were interested and didn’t follow up, but that’s always going to happen.  I had some issues with a few of the recordings being of low quality because everybody records under different conditions, etc. I think the biggest challenge was trying to coordinate the blocks/columns because I knew that once they were mapped, their position couldn’t change. This involved putting blue tape on the floors so I knew where to position each block once they were moved, as well as where to place the projector. The tape got removed once, so I had to work around that and try to get it the way it was. It wasn’t the same, but it was close enough. I feel that I learned a lot from this project, and from the class in general. Isadora/makeymakey have opened up a new world to me, and I want to continue exploring the possibilities.



Screen Shot 2014-12-14 at 10.06.22 PM


I laid out all my blocks today and began the final mapping. I decided that the colors I was using  were too bland, so I tried adding some abstract shapes to the bottoms and making the faces so that they appear to jut out of the blocks, almost like clay. I was afraid the shapes may be too tacky, but it was a solution I came up with to prevent blandness. With some of the elongated columns, there’s too much boring space below the faces (I had been practicing with blocks, so I didn’t anticipate this problem until I settled on columns. I like this arrangement so far, although I still have a lot of work to do on the faces to make them look better.  I’ve put blue tape around the columns and the place where my projector was so I can build everything out again the way it was.  It would be ideal not to have to move the blocks, but there’s a photo class that meets next week before Thursday.


Screen Shot 2014-12-13 at 1.49.45 AM

Screen Shot 2014-12-14 at 2.41.22 AM

singing sample


It’s harder to animate expressive faces when they’re drawn out in more detail, but here’s a rough sample I made relatively quickly. This is not the final one.  I’m unsure about the cartoon blink effect. Might go with something more subtle, but it’s hard to find a “realistic blink sound”

one thing that concerns me


I decided not to use those white pedestals found around the school for the reason that they’re not dependable because everyone grabs them and I can’t just hoard them for my own use (mapping takes a while, and once they’re mapped I have to use those exact same pedestals since the projection is tailor-made for them.  So, I got some cardboard boxes that are all uniform in shape and am wrapping them in white paper. However, I’m finding that it’s extremely difficult wrapping them smoothly so they don’t look sloppy. It’s frustrating, but at least I don’t have to worry about not being able to get all the pedestals I need. I just hope the projection can still look good on these boxes.



Things have changed a bit over the last week or so. I decided that instead of generic faces or character archetypes, I will be mapping 8 students from PNCA, as well as incorporating their voices. I’m having each student sing a single note, and have asked some  of them to recite little phrases like “hurry up!” which I can animate so that they seem impatient with the conductor.  Here’s a rough test of a character block, combined with a real person’s voice.

I think basing these blocks off of students at this school is more meaningful and closer to home than using generic faces and synthesizers.

Prototype 2


Things I’m thinking about:

-turning the blocks into character archetypes rather than just plain, generic faces. (like maybe one can be an opera singer and his voice will have a lot of vibratto, or one can be a robot and have a metallic-sounding voice). However, it’s hard finding good sound bytes. I’ve been using an online synthesizer with the chorus setting, which I don’t like the sound of. Imagine hunting for good, clean sounds that not only match the right note but have the timbres and textures I need.

-even if I don’t go with characters, I’m going to try and add accessories like bow ties/jewelry and more elaborate facial expressions than what you see above. What do people think of the solid colors, though?  Do you want to see more varied patterns like stripes, etc?

-making it so that when the blocks are not being played, they do little things like blink or cough in order to seem like sentient beings.

-interface: I’m probably going to make a tiny control board with mini versions of each block on it (color-coded to match them), wrap each one with colored metallic foil, and give people a conductor’s wand to tap the blocks with and make music. If I can’t find colored foil, I will have to think of something else that conducts electricity.