This is a blog.

50. Final Reflection

First of all, I would just like to congratulate everyone and their final projects. I am extremely proud with what we’ve all achieved at the end of this studio because truth be told, coming into the studio for the first time, I’m sure most of us, if not everyone was just as lost as I am! So kudos everybody! Be proud!

Now that the mushy part is done. Reflection time.

Overall, I am quite pleased with the development and the final outcome of my project. Throughout the weeks, working on this project has never failed to teach me something new and exciting. Understanding generative and parametric design is just the start, but it is definitely something I’ll look into and even apply into my future design work.

Regarding the project itself, I’m happy with the story/narrative and experience I get to share with everyone. I enjoyed the fact that this studio’s project was an “open” brief allowing me freedom to chose what I want to showcase to everyone. 

The biggest take away from this class, for one, is definitely coding. I wouldn’t say I’m an expert after 13 weeks of browsing exciting codes and stitching them up together, but I would definitely say I understand a bit better than I used to. Coding is not just a tool to create those typical “Hello Word, This Is My Website” html webpages, but it could be used to create art, design, algorithms and so much more. It’s amazing!

One of my biggest regrets in this studio was not being more ambitious with my project. Although I am proud of the outcome I’ve made and the progress I’ve done so far, I wish I could have done even more. The possibilities in this studio are endless. 

With that, I guess it’s safe to say that I am no longer...

a Slave to the Algorithm.

Linsy Alesha Angwyn

Project KAMU (2019)

Interactive Installation

A reflection is a powerful image, to what extent does perception become a reality? Project KAMU is a generative, social and interactive installation that changes the way people perceive themselves by projecting a distorted visual reflection, tracked in a live video feed - essentially creating an experience of catching a glimpse on a mirror only to realise that the figure staring back is not quite what it seems to be. 

This installation is made using several softwares to track and then to trigger different actions. The first software responsible for receiving input/data would be FaceOSC. This software detects facial features as well as its movement in real time. The data is then sent to Processing, equipped with codes, in which will trigger different actions based on the rules, parameters and commands set to each condition. A user’s face and facial motion or gesture will generate a distortion of the live video feed. When the user closes their eyes, the video will revert back to normal - representing the idea that the individual could never see that they are “normal”.

There are several stories this work wishes to express. To start, it aims to raise awareness of issues in regards to body dysmorphia, a mental disorder in which an individual dwell son imperfections and flaws of self appearance that is most often minor, undetected by others or even non-existent. Displaying a sort of “fakeness” and false imagery through the screen.

Through a mirror is the only time we see ourselves, what if that image of ourself is always seemingly defected? This brings up the issue of self identity, testing the concept of The Glass Looking Self, introduced by Charles Horton Cooley, the process in which an individual base their sense of self and identity on how they believe others view them, using social interactions as “mirrors” to measure one’s behaviour, value and worth. 

In this case, the installation takes away the social interactions and replace them with actual mirrors to create the visualisation. What happens to our identity then? In a world such as the one we live in today, where self identity is the strongest asset an individual has, this installation is an experience and gives a glimpse of when self identity is distorted and instead becomes self deprecation. 

48. Project KAMU Display (tired of my face yet?)

On the day of exhibition, things went a little... off plan.

Unfortunately for me, the iMac computer I wanted to use wasn’t able to install the software “FaceOSC” - the base software I use that tracks faces and facial features. In other words, without it, my entire system will NOT function.

BIG OOF am I right?

But after some quick thinking and compromising, I settled with creating 2 setups, one that had this video above on loop to show the system in action, and another that is interactive where audiences could see themselves and mess around with the video.

The mechanics are simple:

You look in the screen and see a distorted image

You move your head which triggers this cursor-like object to distort even more

You open your mouth (as if you were angry), only to find out the video gets glitched and even more disturbing

You close your eyes or look away only NOT to realise that the video is back to normal

47. Setting the Stage (Work in progress)

I’ll admit, being crafty is not my strongest suit. In fact, I’m not sure how many times I’ve accidentally glued my hands together trying to assemble this structure (I’m lying, the answer is: multiple times).

Image on the left is a mockup I put together on photoshop, whilst image on the right is the structure I’ve built. I am still considering the initial concept of being minimal and trendy to reflect the aesthetic that appeals to current generations. 

The reason behind it is to create a sense of contrast and juxtaposition between the clean, beautiful setting/environment and the distorted reflection on the screen. 

In this installation, you are in fact, the artwork.

To what extent does your perception become a reality?

A reflection is powerful image. So please tell me, what do you see when you look into the mirror?

A brief rationale for this design: At its core, this project is a personal story, something I have experienced in the past and want to share - hence, why the model on the logo is in fact, my own face. Acting as a reflection and a distorted image. 

The word “KAMU” simply means “you” in Indonesian language (where I’m originally from). Besides emphasising my identity, I wanted to emphasise that YOU are a part of this installation, YOU are a part of the system in this design.

A week left until the day of exhibition. 

Over the past few weeks of development, I have mainly focused on the technical aspect of the installation - basically, the codes and the system in use. Unfortunately because of this, I have neglected the part where I would actually present the system. 

At the moment, the plan is to create an environment for the “mirror” (displayed on an iMac) that will allow the user/audience to be fully immersed in the experience. The theme I'm going for is something minimal and “instagram worthy”, to reflect on the aesthetic of current today’s generation. 

The sketches below are measurements and rough dimensions of the iMac and the structure I am planning to craft. 

Top - Opening user’s mouth to a certain degree will imitate the motion of anger in which causes the video to turn to a glitchy and red

Middle - Eye closing or when a face is not detect results in the video to be undistorted

Bottom - Face distortion and a head motion controlled cursor that further distorts the video

Currently, the codes in processing are working very well. Although not 100% working to the original concept, it is as close as it could possibly be up to my abilities in coding. 

One of the things I still want to research on and experiment is adding time/duration to the system’s rules - in this case, it is because while closing my eyes will result in the video to revert back to normal, this also happens every time I blink or make any eye closing movement. Although it is not a major issue, it is an improvement that could be made.

Another piece of code discovered! Hurrah! 

But wait!

Of course it wouldn’t be that easy...

Last week, after discovering codes to pixelate a live feed webcam video and have a motion controlled cursor, I now have codes to create a glitch effect on the video. The glitch can also be focused on a particular spot on the video depending on where the cursor is. Initially, the codes were made for moues movements, however, after tweaking for a bit, I was able to link the motion controlled cursor to the mechanics in this code. (The glitch codes were found in Open Processing and created by user Jack Jepson)

The issue I have now is trying to make the glitch effect, less glitchy... ironic. Because the initial codes were simply meant to showcase the glitch, it isn’t very compatible when transferred to a video - in my case, the person’s face become unidentifiable, which will not be as effective for the project.

Besides the glitch aspect of the codes, I have also discovered how to reset the video to its original format when the user closes their eyes or if a face is no longer detected by the software. One of the issues here however is that somethings FaceOSC can detect different parakeets for the facial movements. This needs to be given more attention - ensuring that the user stands in a certain positions or a certain distance from the webcam.

Not too much happened in this week’s class. Once again, we were dealing with the different types of sensors. This time, we were asked to research and categorise them as a class.

Here is an example of the sensors my classmate and I were tasked to identify. The one of the left is an Analog Temperature Sensor (KY-013), it measures ambient temperature based on resistance of the thermistor. The middle is an IR Transmitter Sensor (KY-005), it emits infrared light at 38kHz. And the one of the right is Reed Switch Sensor (KY-021)

 SO recap, I am trying to make a system that works similar to a mirror but instead of reflecting, the user will be morphed and deformed when they look into the reflection - the only way to revert the image back to normal is when they close their eyes. Got that?

Good.

Counter productive?

Yes.

Anymore questions? Where am I with the project right now?

Lost. 

First discovery. So after countless hours of researching and diving deep into GitHub and other code related discussions, I manage to find a line of code that allows processing to detect faces using the laptops webcam. It’s not much... BUT it’s the first step! 

Second discovery. The problem I have now is I can’t seem to detect my eye movement (in order to control the cursor) without purchasing some sort of eye tracking device (ex: Tobii). Being a broke university student, I was not ready to make a big purchase unless I REALLY have to. And so more research and reading until Karen showed me Face OSC. This opened up a while new page and allowed me to do more research on specific codes.

“FaceOSC will track a face and send its pose and gesture data over OSC, as well as the raw tracked points (when selected in the GUI)”

Third discovery. I found some codes that allow the cursor within the screen to be controlled by face movements. (Not the eye, but close enough). Pairing the codes with Face OSC and the previous codes on detecting a face in processing, I now have a running video that displays the user in the webcam’s view, detects their face and moves the cursor according to their movement.

For now, I will try to figure out:

1. How to make the cursor interact with the live video, possibly distorting the image?

2. How to replicate the eye closing in order to reset the video? 

In this week’s studio, we were given the chance to build a device using an Arduino sensor. The devices we created measures the distance of an object from the sensor similar to the concept of Echolocation. Using an Arduino UNO, a couple of wires, a Breadboard and an Ultra Sonic sensor (HS-SR04) this was what we created:

After building the device, and spending a couple of seconds being mesmerised by how it looks SO MUCH like Wall-E, we downloaded the Arduino software and typed in several lines of codes, giving instructions to the device. 

Next, using processing, we were able to link the device and giving it a few more codes to create a visualisation of the distance the sensor detects. In this case, the image of the dog will enlarge or get smaller depending on the distance of the hand. As will the circle, the shape will change size along with the colour.

To be honest, I am still very much lost with the whole idea of creating these devices and typing in codes to make them function properly. It was quite a challenge to figure out where the wires go or what a specific line of code would do. It was very helpful to have Karen guiding us through the process and also searching the Arduino Project Hub for a similar project (which exists). Although I might not use it particularly for my own project, I think it was an insightful activity.

Just before Easter and Mid-term break, we had a pitch for our current projects. And after said pitch, we were asked to group ourselves in terms of similarities within Input, Process, Output and Tools Used of the project in order to make research and development easier. 

As a group, we were tasked to create a system (as absurd as we possibly could) that would be able to one, detect something... two, process it... and finally create some sort of output. Some have created a fart machine, and others create a system that recommends music based on the user’s mood. 

Us? 

We created a discriminatory robot that kicked people who are deemed less than “ideal” to the standards of social media. Wow. We are such NICE people.

Behold our creation.

Okay, maybe it’s not as intimidating as it seems. But still, why would we ever want this to exists? 

And now we describe this system using the MOSCOW method, what is Must, Should, Could and Won’t do. 

We as a class established that his robot MUST be able to identify what is considered “ideal” (machine learning through social media such as Instagram). It MUST be a robot which also has a sensor for self preservation. It MUST be aware of the Asimon Law towards the “ideals”

The system SHOULD have a high success rate. SHOULD have its own defensive mechanism and SHOULD be connected to the network.

It COULD have an app that suggests where an individual could be in (where they’ll be considered “ideal”). It COULD create profiles for night clubs.

And finally, what it won’t do. Which is exist. It won’t exist. Ever.

In this particular studio session, after being introduced to various sensors and electronics, we were given a chance to see some of them in action. One of them being Leap Motion, and one of their devices that tracks hand movement in the real world then visualises it in the virtual scene.

Also, one of the devices Karen brought was a repurposed Jedi Force Training toy. The device now is able to read brain activity and functions, dividing them into whether the user is in a meditative state or attentive one. 

Here are some examples that were shown in class that I found very interesting and inspiring to say the very least. These projects are more than what I typically define as a “design” or “art” project and I am amazed by the impact they have towards society. 

Fish-Bird Circle

“an interactive installation that explores the dialogical possibilities between two autokinetic objects (two robotic wheelchairs) and their audience” 

The interaction between two wheelchairs seems very odd, yet unbelievable interesting at the same time. Maybe bringing inanimate objects to life just brings me joy. 

Space Replay

“a hovering object that explores and manipulates transitional public spaces by constantly recording and replaying their ambient sounds”

Imagine walking down an empty street and hearing an ominous echo of children laughter... fascinating. What is not to love with this project? 

It is almost unreal and extraterrestrial with the way the ball floats and emits it’s own sound, in fact, sentient. 

Eye Writer - Tempt1

Tempt1 is an artist who was diagnosed with ALS in 2003, a disease which has left him almost completely physically paralyzed. However, his eyes are not. The Eye Writer is an eye tracking system that allows him (and many other ALS patients) to create art simple by moving his eyes.

US+

“us+ is a google hangout video chat app that uses audio, facial expression, and linguistic analysis to optimize conversations”

I am conflicted with how I feel about this project. I am fascinated by the ability a machine is able to read human emotion however. also terrified by the fact that a MACHINE can read human emotion... more than us humans are capable of. 

However, I still find this a very powerful project and find it very hard to believe that it existed. I think it gives a sense on how much people are somewhat lacking in social cues or so till the fact that a machine can determine what we say or do in a social situation. 

LA out.

https://www.artspace.org.au/program/exhibitions/2005/fish-bird-circle-b-movement-c/

https://whyj.uk/space-replay

http://www.eyewriter.org