Untitled

Another test, this time to demonstrate the touch trigger starting and stopping a looping GIF in MadMapper. The animations in the final project won’t be looping, but this is just an example

A test of the basic functionality of the touch board, using the built-in audio files

I used a sharpie marker to draw in all of the non-conductive parts of the painting while using the electric paint for the trigger points. (The sharpie was not ideal, but I did not have any black acrylic paint and did not want to take the time to order it online.) A closer look at each canvas would make the distinction between the sharpie and the paint more obvious - these are cheap canvases that did not take all that well to either - but it is good enough for the purposes of this project. (Perhaps I will try to fill in the less consistent areas later.) I have also tested out the touch board to make sure that the animation triggers will work. I’ll have to upload video of this separately, so that will come in the following posts. I am going to hold off on mounting the canvases and hooking them up to the touch board until I have a final answer on whether or not I’ll have a projector. The next step will be preparing the animations.

Well it’s been a bit since the last update, primarily because of everything going on in the country right now due to the coronavirus. I had to leave my student housing and fly across the country. It was a bit disruptive, to say the least. What’s more disruptive is now I no longer have access to the resources I would have had at school, which means that my project will not go forward as intended. Most notably, I will not have access to Madmapper or a projector. This of course defeats the purpose of the project to a great degree, but I’m looking at ways to adjust the plan so I can at least end up with something that demonstrates the original concept well. I have been looking into ways of sending Arduino data into Unreal Engine 4 so that I can create a virtual mockup within a game environment. Nothing too fancy, but perhaps a recreation of what the canvases would look like set up on a wall. I should be able to still trigger the animations using the electric paint and touch board, but it will just require a different process to get there. Hopefully it’s not too much more complicated than triggering animations via MIDI instructions in Madmapper, because that was actually quite simple. In light of having to pivot towards Unreal Engine, I’m considering lessening the complexity of the physical components - that is, the actual paintings. I don’t have a ton of space where I am currently staying so the likelihood of actually painting and displaying four canvases is slim. What I might do instead is paint everything on a large sheet of acrylic painting paper and space it out in a way that resembles the original layout. The important part is seeing the touch triggers activating the animations. So what I will likely end up doing is hooking up the painting to the touch board and then running that to my computer and into UE4, where the animations will play. Upon doing so, I will record some video of me interacting with the canvas next to the computer screen running the project in UE4 to show it in action. It won’t be quite like seeing the canvases light up in person, but it should get the idea across. One potential backup option I have in case this process ends up being a lot more complicated than I intended would be to swap out the animations for sound files, such that I still and up with an interactive painting but with an audio component rather than an animated one. This would be a last resort, because I’m much more excited about the animation aspect, but I’m making note of it here so that a future pivot in that direction doesn’t seem completely out of nowhere. That’s about it for now. I do have my hands on the touch board now, so I will start to experiment with that in the coming days and post some video of it in action.

While I am moving on to the asset creation stage, I think my next step will be to get the canvases and do a test projection of this layout onto them to make sure I like how it all looks. From there, I’ll begin on the initial animations. Hopefully my next update will include a photo of the test projection, so look forward to that!

That’s it for now, this week I begin forging ahead in earnest!

- Various construction/painting materials (paintbrushes, masking tape, stenciling materials, sharpie markers, etc.) One last thing that could be cool would be to have some speakers so I could add an audio component, but due to the nature of how the museum show operates, there may be too much room/crowd noise for this to be viable I’m pretty sure this is about all I’ll need, based on my research and my estimations. The biggest question mark is whether or not I can get a Touch Board, or if I will instead need to work with a standard Arduino board. The first option is preferable since it comes pre-programmed with the code for the touch triggers and has a companion plug-in for use with Arduino, but the same things should be feasible with a different Arduino board, it would just take more work.

Here is a link to the asset list for the completed Phase 3. It also includes a list of assets that might have to be altered or added if I pivot towards Phase 2 instead. The actual amount of art assets shouldn’t be too overwhelming. I’m going to go with a pretty simple style to keep it from eating up too much time. It’s more the animations that might get complicated because there have to be slight variations for different scenarios. Calling the drone over to the delivery platform needs three variations, for example - one where the battery isn’t there at all, one where the battery is there but has not been properly charged, and one where the battery is there and has been charged. If I can reduce some of these variations somehow, I will. I did not specify on the list between which of these assets are projected and which are physically painted on the canvas - but essentially all the animations are projected, whereas the static elements (the isles and buildings, mostly) are painted. As of now I want to keep the painted areas pretty simple stylistically - pretty much just black painted silhouettes or outlines. If I have time, I might paint over with color and detail, but I also want to be careful not to overdo the painted elements because I am not yet sure how it will synthesize with the projected elements.  With all of this laid out, the next steps are to write up a list of the equipment I’ll need and then to get started prototyping the animation structure in Madmapper.

According to this diagram, there are 11 hot spots out of a possible 12. I am leaving one available in case I realize down the road that I need one more for some reason. Here is the full sequence: 1. Touch hot spot 2 to send the robot on a rail car over to Isle #2 2. Touch hot spot 3 to have the robot turn a valve to release oil through the tube, which will fill up the drone 3. Touch hot spot 5 to send the robot down the ladder, onto the rail car, and over to Isle #4 4. Touch hot spot 7 to send the robot over to Isle #6 5. Touch hot spot 10 to send the robot up the ladder and over to the acid tank, where he turns a valve to dump acid into the battery, charging it 6. Touch hot spot 9 to send the robot down the ladder and over to the battery claw lever, where he moves the claw from position A to position B 7. Touch hot spot 8 to drop the battery through the delivery tube, which drops it over on the delivery platform 8. Touch hot spot 7 to send the robot on the rail car over to Isle #4 9. Touch hot spot 6 to send the robot down the ladder and over to the drone beacon, where he pushes the button which calls the drone over to the delivery platform. It picks up the battery, flies it up to the charge station, and inserts it into the charge station 10. Touch hot spot 5 or 7 to send the robot up the ladder and onto a rail car to either Isle #3 or Isle #6 11. Touch hot spot 2 or 11 depending on which Isle the robot is on now, sending him on a rail car back to Isle #1 12. Touch hot spot 1 to send the bot into the charge station to recharge, at which point he retires into his house, day turns to night, and he emerges in need of a new recharge One thing to note is that while in the diagram the hot spots are indicates as little circles, the actual hot spots can be whatever shape I choose to paint them into on the canvas. For the rail stations, for instance, I intend to paint the entire rail line - including the buildings and the connecting wire - using the electric tape, connecting them between canvases using copper wire/tape. This way, you could touch either end of the rail station or the rail itself to trigger the animation sending the robot between two islands. While the above sequence is the “correct” sequence, pressing the “wrong” hot spot would still have some sort of effect in most cases. But for the effect to trigger, the robot would have to currently be on the appropriate canvas. For example, if you pressed hot spot 4 but the robot was on canvas A, nothing would happen. But if the robot was on canvas B, he would walk from wherever he was over to the drone and inspect it. Depending on whether the drone has been filled with oil, he will respond by either indicating (via a thought bubble with a symbol) that either the drone needs oil, or the drone needs to be called by activating the drone beacon on Isle #5. My hope is that I can clue people in to the next step in the sequence, or at least point them in the right direction, by having the robot indicate things through these thought bubbles. Pressing hot spot 1 while on canvas A and the battery has not been replace will trigger a thought bubble with a battery symbol, hopefully indicating to the user that they need to figure out how to get the battery to the charge station. The initial question of how to indicate to the user that they can touch the canvas to interact with it in the first place is a question I’m still pondering, but once they know they can do so, this method should help keep the interaction going. One final note before wrapping up this post - if I don’t actually get to Phase 3 of this project, then Phase 2 will consist of a similar but slightly-altered layout with more or less the same hot spots, but instead of having to activate them in a sequence, they can be triggered at any time and are not connected to each other in any way (except possibly limiting some from playing at the same time). The animations would be slightly different, but as similar as possible for the sake of efficiency. For example, touching hot spot 1 would trigger the robot to walk over to the charge station, charge himself, and simply exit the station and return to his original position. I might still try to have the robot transfer between Isles/canvases if possible, limiting which hot spots are trigger-able based on which canvas the robot is currently on. But I’m still figuring that part out. The goal is ultimately to complete all three phases, with the possibility to pivot towards Phase 2 if necessary. I don’t foresee a situation in which I can’t complete Phase 2, so I am not going too heavy on the design for Phase 1 and will instead be treating it mostly as a proof of concept. That’s it for now - I’ll post the asset list in the next update.

Once I hit the milestone for each phase, I should be able to determine whether or not I can move onto the next one in the time I have left or if I should focus instead on shoring up the current phase. For example, if I’m able to successfully test out the full projection setup across multiple canvases in Phase 2, I should be able to say that I’m completely satisfied with the result before moving onto Phase 3. That way, if I start the next phase but am unable to complete it, I will at least have something complete to show at the museum show. (The painting of the canvases will always happen regardless of if I actually move in to phase three, I realize now that that probably should have been indicated separately.) Note that we still don’t know the day of the museum show, so the final few rows are subject to change. The museum show is of course where this project will be installed at the end of the semester, so whatever date that actually get set as will adjust the timeline accordingly.

This quick sketch is just to demonstrate the idea behind the project as a whole, but the actual content is just placeholder. In this example, the stick man wants to give the stick girl a flower, so he has to take the rail from canvas A over to canvas B. Then he has to use the well to pour water onto the plant down below in canvas D, which causes it to grow into a flower. He then takes the rail to get to canvas C, then to canvas D, where he picks the flower and returns to canvas C to place the flower in the pot. After taking the rail car all the way back to canvas A, he activates the pulley to bring the flower up from canvas C, which he can then present to the stick girl, completing the story. Again, this is just a placeholder story, but it demonstrates the way in which I intend to transfer the story between canvases. In this example, the canvases would all be displayed near each other on the same wall, allowing for some clear visual continuity between them like when the water pours from canvas B to canvas D. It is possible that in my final design will likely have two canvases displayed near each other like this, with one or two other canvases displayed somewhere else in the room. One potential challenge here will of course be communicating to people that the story moves from canvas to canvas and that they need to explore the room to follow it. In terms of scale-able goals for this project, I essentially have three benchmarks: 1. Create a single canvas on which a collection of animations are triggered by hot spots 2. Extend this concept to a series of canvases 3. “Gamify” these animations by modifying the Touch Board’s code in the Arduino IDE In an upcoming post, I will detail how I intend to develop the story in such a way that the animations/assets I create can be adapted to each of these versions.

Hello! This blog will be documenting the process of creating my project for my interactive design class. The ultimate goal for my project is going to be a series of interactive canvases using touch triggers which will interact with projection mapping software to activate a sequence of animations. The initial inspiration for this project came from this project by Sofia Aronov titled “Awake” as highlighted on Bare Conductive’s website. Bare Conductive is a company that produces the electric paint I intend to use for this project, as well as the Touch Board (an Arduino-based board) that I hope to use as well. I am still finalizing my designs, trying to keep scope in mind, but the following posts will get into those in detail. Stay tuned!