Ji Park ITP Blog

Project  Website featuring machine learning generative design experiments

Link

Library Im using teachable machine, tensorflow and ml5. 

Project title Machine learning generative design website

One sentence description Website featuring machine learning generative design experiments  Project description This website is an interactive site where users can try out different machine learning design experiments

Interaction sketch This will be in a website format. Using user’s mic/camera/motion. Inspiration came from the generative Chair project + many teachable machine projects. Also wanted to expand my idea from last semester’s class assignment.

This week, using Hand pose tracking + KNN Classification model, I made a simple Rock,Scissor, Paper game. (You can never win)

Sketch

User presented ‘paper’ servo responds with ‘Scissor’

Thumb up and down classifier

Input + Output Exercise

Last week in class, we did a miro board exercise thinking about possible input/output for project.

Few ideas I suggested and thinking about developing into projects are

Dog image + dog food feeder

This is a pet food feeder activated by microchip inside the pet. This is very useful for multi pet household, if they have to be on a specific diet. I have three cats and one dog back home and we always had this problem with dogs and cats eating each others’ food. So, I want to expand this idea and make a dog food feeder(bowl) with a camera of some sorts that will identity only my dog. Close when if my cats approach. 

Yoga pose + mood lamp

This week, we learned how to connect arduino with teachable machine dataset.  I choose image to train my model. Simple hand gesture to light LED connected to arduino board. 

This is my model + p5 sketch

Im working from home so I have limited access to tools but I was able to sort of improvise mini light lamp setting with acrylic plastic + LED light bulb.

Icecream land

My goal with this project was to familiarize myself with three.js platform and create a fun icecream land.

https://22fe815795c0.ngrok.io Just like everyone else I was feeling zoom fatigue. I still wanted to hangout with people online but not on zoom. So, I was inspired and wanted to create a platform like Topia. Fun and humane video chat where you can easily move in and out of conversation in the space.

Users can roam around the space

*unlimited* Icecream scoops are stackable when clicked

Playtest & Feedback

Adding raycaster — This week, I added mouse events to objects in my 3d space. 

Inspiration For final project, I want to make a color wheel inspired by Haul’s moving castle door. (*Howl's castle a moving castle has a unique teleportation front door. There is a small dial near the doorknob and where the door leads to when opened, depends on what color the dial is turned to. )

Idea For my project, I wanted to make something similar to the rotating door knob. I want to make a color wheel that responds to a color shown on webcam.

Connecting arduino to p5js sketch.

This week, we covered how to connect arduino BLE with p5js sketch. I made a series of p5js sketches responding to arduino’s accelerometer. Overall, the x/y/z coordinate readings were a bit flaky and unstable but the idea of 3d object transferring into web space was very interesting. Definitely something I’d like to spend more time on and expand.  Sketch

I felt like an iron man controlling his jarvis computer in the lab.

Topia Experience

So far, this is my favorite video chat platform. The environment resembles the feel of animal crossing or among us. 

I think this platform is definitely close to solving the problem of every online meet-up platform. Less seminar/classroom vibe, more party. There are two reasons why I think this was so enjoyable (at least for me!) First, I liked how there was a spatial quality to this chat. You walk up to a group of people or a person to start a chat. Users' avatar/voice fades away when I walk further away from them. I thought this was such a simple but clever solution. Usage of proximity to create a spatial quality. It feels more intimate and natural.  Also, they added a few fun spots throughout the platform such as streams and campfires which I think added to the ‘realness’ of this space.

Three.js

For this week’s continued three.js practice, I changed controls of my environment. Im still getting familiar with basic functions and structure of three.js. Thinking about x/y/z positions and thinking in 3d spaces are still a bit of a struggle for me. To be honest, this environment has no function yet. Its just icecream in spaces floating around and users navigating through giant scoops. Something I thought about adding to this environment is maybe changing how users interact with objects (icecream). Can users move abound those icecream scoops? Stack them up high..? 

Dial up Experience

I was really nervous to try this app. Im very careful and extra cautious on any app that matches stranger on anonymous platform. This app seemed safe since they had an option to report abusive caller and designed it in a way that its completely anonymous.  I signed up for cooking&farming call since both subjects were familiar to me. I liked the design. It was easy to navigate through. I also liked how there were many different and  interesting groups available.

I got a call letting me know that I was matched with and proceeded conversation. I shared my miso eggplant recipe and my call partner shared her favorite pasta recipe. We exchanged our recipe and moved on to just normal conversation for a bit. She told me that this app helped her a lot through anxiety and loneliness during quarantine. We were all at risk of loneliness and mental breakdown during quarantine and I liked how this app provided connectedness with an extra layer of anonymity which I think is extremely beneficial in situations like this. I also wonder since this app is heavily voice centered, if this app is accessible to people with disabilities..?

three.js implementation

This week, we learned how to render a three.js environment on web. I used this template to build off of.  I created an ice cream land, hopefully want to expand this as a fun online platform for sharing new ice cream flavor ideas + recipes!

Goal Build a BLE  service with characteristics with read, write and notify properties. Process First, I explored with read, write and notify properties of BLE. It was so cool to see and control arduino on my phone!!

After I understood the mechanism of BLE basics, I wanted to create a simple version of smart light bulbs — LED based bulbs that can be controlled by a smartphone app—through bluetooth user can set, brightness and even color hues.

I made a mini lamp with acrylic plastic I had. Brightness controllable with bluetooth app!

This week’s reading : Beyond Being There  I appreciate the authors’ effort looking into the inherent limitations of telecommunication and how we should to focus on ‘-communication’ part, not ‘tele-’ part. Unfortunately, almost a year into the pandemic, I’ve tried pretty much all ‘telecommunication’ platforms but I’m suffering from digital fatigue, yet desperately hold onto it because it’s my only form of communication right now. I never felt ‘the sense of being there’ but after I log off from these platforms and come back to reality, I get extremely lonely so does that mean I was at least ‘there’ for a hot second...? 

Live drawing tool

Sketch I designed a real-time drawing game for two people. Each user can draw freely on the canvas until their mouse overlap.  With this experiment, I wanted to express my frustration on how we are always connected and together online but not really..?

BLE discovery, scan available services at home

Im using BlueSee app to scan devices around me. Most of them were long numbers or ‘unknown’. But I was shocked to see how it was able to scan a TV that’s possible our neighbors’..? The connection was unstable (-100 dBm), but I was very surprised that the scan was able to reach out that far.

I was able to test out if it scanned my airpods!

Product discovery

Choose three products that are using wireless communication protocols (BLE, Bluetooth Classic, Wifi, NFC, etc)

List out your assumption of what protocols they are using

Read their product description and compare them with your assumptions

Post your discovery on your blog

I choose these three objects for this assessment: Chromecast, OMNY MTA payment system, Tile tracker.  Just from my experience and shallow knowledge of IOT, I think Chromecast uses wifi because you need to download an app to sync the chromecast. For OMNY mta payment system, Im pretty sure it is an NFC product because it is a contactless payment system. For the Tile tracking system, I think its a BLE but with GPS function as well? Because it tracks the location as well. 

Art work finer

http://www.jipark.org/ml-final/

Using ml image classifier and MET open api, this website identify user uploaded image and find related artwork from MET

Process

Im using ml5 coco-ssd model and MET api.

When the image classifier indentify image with a key word, it passes that keyword to MET api and returns related art work.

Next step

I wish the coco-ssd model could detect more that 80 classes of object. This would extend my project so much broader.

Sheep

Project Title Art work finder  Using ml image classifier and MET open api, this website will identify user uploaded image and find related artwork from MET

Project Title I was inspired by many virtual museum experience projects that came out during covid. I loved the idea of immersive experience with highly accessible platform.  Also, I was inspired by google art selfie project, that matches your selfie with famous portraits.

Audience Anyone whos interested in art! I want users to just have fun and appreciate amazing artworks from the MET

Challenges My biggest technical fear for now is possible hiccup on image classifier part. When I explored it last time, it had trouble recognizing some images clearly.

Sketch

Website

User upload image -> ml recognize image with a keyword -> find artwork accordingly from MET open api