WEEK 7: PIXELS

INTRO

This week we focused on pixels and creating a video camera experience that would surprise, delight or confuse the user in front of the camera. I decided to create a colored pixel experience where users could dance in front of the camera and depending on their movement, change the speed of the music. 

VIEW THE CODE

PROCESS

Earlier in the week, the class went over pixel arrays and brightness. Our homework was to create a two tone mirror using video pixels, where pixels changed depending on the brightness, or the average of the red, blue and green value of each pixel. The way the brightness value changes is dependent on lighter or darker objects in front of the video camera. In that assignment, we used black and white, so I wanted to experiment with different colors in my creation. I chose blue and orange, where the darker pixels were orange and would generate the shape of the object in front of the camera and the lighter pixels were blue or your surrounding background. As you move in front of the camera,  the ratio of blue to orange squares changes.

Once I got these colors, the movement and solid colored pixel background reminded me of the original iPod commercials, where a silhouette holding an iPod was dancing with a solid background.

Using inspiration from the video, I wanted to incorporate sound into my piece. My favorite thing from these commercials was how the silhouette moved in tune with the music. So, I wanted an interaction where the pixels created from a user in front of the camera changes the speed of the sound. From a user perspective, you would be “dancing” to the music while watching pixels change and in turn, have the music speed up or slow down. I learned how to use the p5 sound library and chose the song, that in my opinion, was the most iconic iPod commercial song : Technologic by Daft Punk.

Originally, I coded the brightness of the pixels to be certain colors using conditionals and was trying to put my song function into the conditional. Working a resident, I realized I was confusing and mixing two concepts: sound and brightness. The way I could set it up was first have my conditional, then map the pixel brightness values to the sound rate. 

I mapped the lightest pixel values to 0.5 which is half the speed of the normal sound played, and darkest pixels values to 1.5 which is 1.5x the normal sound speed. I experimented with not mapping the sound in a range but rather specific speeds, but the large quantity of pixels created a jagged sound experience. So I mapped within a range, which allows for smoother sound. I want to continue looking into playing a specific speed dependent on the brightness of the pixels. One way the resident suggested was trying get average value of all the pixels in the canvas and if the pixel brightness is a certain percentage, then play a certain speed of sound.