# Physical Computing

## The Balance Challenge

### Team Members

Ariana Vassilopoulou

Jesal Trivedi

### Mission

Develop an interactive experience using both the Arduino and p5.

### Project Brief

The Balance Challenge is a game where the user uses a board which they must balance on to move their player on the screen from left to right in order to avoid square objects falling from the top of the screen. The user accumulates points based on how long they avoid hitting any of the square objects.

### Items Used

• Arduino Uno
• Wires
• 6″x6″ wood enclosure
• Gorilla Tape
• 5.75″x”5.75″ black acrylic (3mm thick)
• USB 2 Cable
• 4 2″ screws
• p5
• C++ (Arduino)

### Tools Used

• CNC Routing Machine
• Drill Press
• Sanding sheets
• Power Drill
• VectorWorks
• Laser Printer
• Macbook Pro Laptop

### Process

We plotted out the timeline in which to reach certain milestones by in order to get the project finished in the given amount of time (14 days). We began the brainstorming process and developed many ideas. We then decided on the balance board game (we we eventually titled ‘The Balance Challenge‘) based on the time constraints we had.

We began with coding out the game in p5 first. Once we finished about 95% of the p5 coding, we moved onward to figure out the accelerometer sensor and building a “quick n’ dirty” balance board.

We determined that we only needed one output (direction) from the accelerometer (the y-axis), as the board only needed to tilt tilt in two, opposite but equal directions. We determined the raw values after connecting the prototype board and seeing what the min and max values we were seeing in the serial monitor. The cardboard was too flimsy, so we got wood to help provide us with more accurate figures. We then mapped the raw data from 0-2, with 1 being the ‘Middle’, 0 being ‘Left’ and 2 being ‘Right’.

Once we had the data properly being displayed in the Serial Monitor, we then connected both p5 and Arduino together using Serial Port. We kept running into processing issues, but we able to resolve that by delaying the data being received.

After everything technically worked, we began the process of building the actual balance board. We got the lumber from a wood outlet in the city and began cutting and drilling away. The board is made from just three pieces, help together with super strong wood glue and reinforced with screws.

We then tested it out with a few of our classmates to get their feedback on the sensitivity of the motion and made some adjustments to the responsiveness.

We then attached the enclosure to the top, taser etched the name on the acrylic and finished up the board.

### Things We Would Have Done Differently

Some of the things we would have done differently. If we had a bit more time, we would have added pads to have more friction with the users shoes. We also would have varied the sensitivity based on the angle the board was tilted. For example, the player on the screen would only move in a direction directly proportional to the level of tilt the user was applying on the board. Another consideration was the the board seems to move slightly forwards and backwards because of the user. If there was a way to keep the board in one place, that would be ideal for the gameplay.

Check out our code on GitHub!

A BIG thank you to the amazing students and residents at NYU ITP that provided guidance and help throughout the course of the project in both the workshop and in the code. This project would not have happened without you.

## Conceptualizing, Sketching and Testing Ideas

Learning how to brainstorm properly can exponentially increase your ability to develop something truly fantastic and impactful. Think of this “brainstorming” as a muscle , which you have to run through the paces and make stronger through deliberate practice. I love brainstorming and have actually implemented it into my morning routine, where I create a design concept based on either news or something I’ve been thinking of. I then list out 10 ideas that could solve a particular issue I’m facing or a certain theme, for example “VR/AR Ideas”.

The book Sketching User Experiences: The Workbook gave me even more methods and ways to improve my brainstorming concepts.

I remember in the first class, we has to describe our favorite tools, and mine was my Moleskine and Pen, and have kept these two by my side wherever I go. I can relate a lot to this book. One thing I’d liek to start doing is creating a collection of physical objects that inspire me.

Actually, reading this book gave me a few ideas on the spot that I used their methods to ‘scribble sketch’ them out into my sketchbook.

Tom Igoe’s post on Making Interactive Art discussed an important point about when you have created an interactive piece of art, you should not ‘influence’ how the user should use it. Leave it to them to figure out. If it’s designed properly, the user should be able to figure out quite quickly how it is supposed to function, relying only on feedback loops. This reminds me of Steve Blank’s entrepreneurship concept of ‘Customer Development’. This process is to assess the viability of a particular business idea. Once your idea has conceptualized, and you have listed out your assumptions of your ideal customer, you develop a list of questions and begin to interview people that match that ‘ideal’ as closely as possible. It’s VERY important that you are not asking questions that lead them to answers that reinforce your assumptions. Rather, you want to leave it fairly open ended and have the potential user to discuss their way through frustrations with current products or services, as you might discover an issue with your original idea or find an entirely new problem that you’d want to solve.

## Compliation Day: Digital and Physical Become One

This day did it for me. Melding the worlds of physical and digital was an awesome experience and something I have been looking forward to all semester. My partner and I both worked on our own mini-projects, which simultaneously helping each other figure out how to make everything work properly in p5. Below I have documented what I made that day.

Control RGB Values of a Particle System

Using 3 potentiometers, I connected each one to their own analog pins. I then fed the sensor values from Arduino into p5 and organized the data appropriately (with the help of Craig, Dan, and Kat) to effectively change each value properly (R, G, B) and have it relay that change immediately on the screen. I have posted the code here:

I want state that there was a delay, probably caused by the two sides communicating via the serial port that slowed down the processing of information. What this looked like was me turning the knobs and it changing colors after 15-20 seconds, which is not ideal.

## Wawa: The Next Gen Sandwich Ordering System?

Wawa. It is the one of the places that brings me so much joy when I go back home to the Philly-area. For those who don’t know, Wawa is like a bodega on steroids, packed with snack and foods and a full-service deli, where you order all your items on a machine like this:
Over the weekend, I went to one of my best friend’s weddings and made a veteran decision to stop at a Wawa on the drive back into the city. With fresh eyes, I watched 15-20 people use this amazing little kiosk during my visit there and recorded my observations:
Assumptions
1) Customers want a faster, more accurate way of ordering food
2) Less labor intensive (less staff required)
3) ‘Quick serve’ food is prepared in front of you, giving customer transparency in their order
4) People don’t want to wait in a long line to give an order, they would prefer to wait in a shorter queue, place their order and then wait for their food
Customers who come to Wawa are looking for a delicious meal that they can get between 5-10 minutes. Popular around lunch, but open 24/7, Wawa’s flexibility to provide quality food at all times of the day cater to a wide range of customers.
There are certain aspect of this system that I want to call out. After placing their order, a small ticket is printed, which the customer then takes to the cashier to pay for. Once paid for, the ticket it then validated with a ‘PAID’ stamp. I believe this helps ‘take up time’ that would be otherwise be spent waiting for their food.
There are inherent simplifications and difficulties this brings to the experience at Wawa.
What’s Easy
1) Step up and order a sandwich within 30 seconds (if you know what you’re doing)
2) Margin for error is extremely low. If there is a mess-up, you have a record of everything you ordered on your sandwich as validation.
What’s Difficult
1) First time Wawa customers have to either observe other customers use the kiosks or ask their friends/family on how to order. Eventually they understand the flow.
2) Some options appear to be repetitive “Cold Hoagies” “Hot Hoagies” “Double Meat Cold Hoagies”, etc.., which led to the user having to press cancel and start again
3) The paper print out receipt actually ran out and a customer was left without an order number, leading to confusion and a breakdown of a solid, organized system.
Not only is this a waste of paper, but inefficient. Why shouldn’t the customer be allowed to pay for their food from the order screen without having to print paper and go to another line to pay for their goods?
Quickest observed transaction time (just ordering a sandwich): 0:20 seconds
Quickest observed transaction time (ordering and paying): 4:02 minutes
Slowest observed transaction time (just ordering a sandwich): 3:12 seconds
Slowest observed transaction time (ordering and paying): 6:32 seconds
As someone who has grown up with Wawa, this is the easiest system to order a sandwich with. With a big sign that says, “ORDER HERE” and ‘TOUCH THE SCREEN TO ORDER’ are straightforward indicators to new customers on what to do. From my perspective, and what I’v observed, there is quite a bit of affordance built into the copy and layout of the screens, guiding the user to eventually press, “Complete and print order”.
This system does not align well with Crawford’s definition of interactive, where the user inputs their selections, and the system basically compiles it all together, allows the user to review their selection and then prints out the result. In the backend, the system is sending that order to the sandwich maker, giving them the instructions to build the sandwich that was ordered. The only processing the computer is doing is making sure the queue numbers are sequential. That’s pretty much it, which is why, although the user can interact with the kiosk, it doesn’t fit too well in my opinion to a process-heavy interactive experience.
—-

## Playing with Analog and Digital Ins and Outs

This week we had to apply digital and analog inputs/outputs to our circuits. Here is what I used:
– Arduino 101
– 1 light resistor
– 1 10 Kohm resistor
After connecting the circuits, and adding a bit of code, I was able to print out the range of light the photo resistor was picking up. I then mapped that to a particular range from 0-3, whose variables I assigned globally. I then mapped that range to either print ‘Dark’ if it read, “0” or “1”. It would print ‘Light’ if read ‘”2″ or “3”.
I have it set to delay a reading every 60 seconds.
My original goal was for it to send a notification to the rnF app via Bluetooth if it was ‘Light’ and to not send it if it was ‘Dark’, but as I could not figure out to configure the Bluetooth in time for class. I wanted to create a simple system that would notify you if you left the lights on in a room. I will figure it out and revisit this.

## Switches on Switches

I create a simple switch control. There are three switches that each individually control their own LED (red, green, yellow). The schematic is shown below.

Green LED

Red LED

Yellow LED

All Three LEDs lit up

Close up

Video

Further Thoughts

I wanted to try to see if I could create just one switch that made the light jump from one LED to another, sort of like Christmas lights, but I couldn’t figure out the right amount of resistors and power needed to do something like this. This might be something to consider in the near future, possibly with some Arduino programming.

Don Norman’s The Design of Everyday Things has been one of my favorite books on design. I’m happy that I read the second article on emotions and attractiveness of designs, that Norman ALSO wrote, because that was a criticism of his work that I had previously. I thought Don had a utilitarian view of the world and how products should be designed, so after reading both, I definitely see more balance between his thinking.

I discussed affordance in my last blog post as something that is critical for good interactivity. Your product has to be designed in a way so that the user understands what needs to be done in order to use it. He used a door as a fantastic example to explain this and the concept of ‘visibility’, and how it can go wrong in design. The can also be seen shopping cart

Mapping was another interesting topic (especially since we are learning how to use this function in ICM!). I liked how he distinguished between additive dimensions (adding more of something to show incremental increases) versus substitute dimensions (replacing something of one particular value for another to make change). I kept thinking of color and how it can be both an additive and substitute dimensions. For example, if you are showing the prices and higher prices are indicated with yellow, orange and eventually red, and lower prices range from yellow, to green to blue, you can map color to prices and it can serve an additive utility.

It was fascinating seeing the other student’s projects. I recall seeing some of them when I visited during the Winter and Spring ITP shows over the past couple of years. I do agree that you can add your own spin to the projects. Motion, LEDs, and ‘FIELDS OF GRASS’ are particularly interesting, but I’m excited to see what we can come up with here in this class. I’m curious to see if any of the past projects have been based on actual product development rather than installation designs, which most of the projects that I noticed, are.

P.S. Now I want some teapots!

## What is Physical Interaction?

What is physical interaction? What does it have to consist of to be considered ‘interactive’? Have we misinterpreted what interaction design is at it’s fundamental core?

In Crawford’s The Art of Interactive Design he states the definition as a conversation with two actors who “listen. think, and speak.”

In A Brief Rant on the Future of Interaction Design, Bret Victor uses this video to define physical interaction, not through tasks to complete (human need) or technology, but by our own natural capabilities as humans, such as using our senses of touch, balance, strength, etc. to navigate the world around us.

Using these sources, I would define physical interaction as natural actions that occur with two or more “actors” (as defined by Crawford), where there are clear and immediate feedback loops (responses) to various inputs (either analog or digital).We must be sure to contrast the difference in thought between the two authors. Crawford states that an interaction is the act of listening, thinking and speaking. Victor argues further, stating that physical interaction must emulate natural expressions with our body using all of our senses.

I believe physical interaction does not need to emulate these natural “expressions”, but some of the best examples of great physical interactive experiences have the human element ingrained in their design. Let’s take cars for example:

As we had been pretty nomadic people for a large part of our history, humans have moved around to various geographic locations, using either our own legs as locomotion or riding atop horses or in carriages. This was a fairly passive way to travel (unless the need called to gallop to a certain destination). Very little input needed to be given to the animal to guide them into a certain direction. With the advent of cars at the turn of the 20th century, we has a whole new set of “tools” to interact with to navigate the roadways (which resulted in an astonishing amount of crashes..surprise surprise). We shifted as a human race from passive riders to active drivers.

As technology progresses, and the dawn of the autonomous vehicle is upon us, we will slowly transition back to what was most natural for us, returning as passive riders once again, stepping in only when intervention is needed.

What makes for good physical interaction?

I believe a good physical interaction incorporates three things:

1. Affordance in the product’s design
2. Using your senses to interact with an object, either physically or digitally
3. Feedback from the product to the user

According to Don Norman, author of The Design of Everyday Things, an affordance is the design aspect of an object which suggests how the object should be used. Essentially it’s a visual clue to its function and use. This is important for the user to understand how they should interact. Next, for a good physical interaction, the user must use their senses to interact with the object first. The most common initial action is usually tactile (grabbing, pressing, swiping, tapping, etc…). The product will be thinking (processing) a response, which will then give the user some information that their action is either complete or incomplete, thus completing the feedback loop. When these three things can work together, and learn from each other as they are used, this is what creates good physical interaction.

There are several examples of digital technology that are not interactive. If we follow Crawford’s definition to interaction, TVs, Podcasts, online blogs and media are fairly non-interactive. You can make the argument that online media sites are interactive in the comments section, but not to the extent that it can completely change a written/video/audio piece on content that is already published. TVs are not interactive as you can’t engage with a TV, you can only control it. Podcasts, although new, are not interactive unless they are recorded live with full audience participation.

## Favorite Tools and Fantasy Devices

Class 1 – Wednesday Sep 7, 2016

I know it. This is going to be one of my favorite classes here at ITP. Melding the two worlds, one physical and one digital, is something that I always was allured to. The course description is perfect for my interests and I hope I can bring 100% to it.

I loved the first question that Tom, our professor, asked us, “What is your favorite tool and why?”. A tool, by definition aids in accomplishing a task, often times much more easily than if you did not have said tool. The answers everyone gave were varied and unique. I said my Uniball .7mm Jetstream pen and my Moleskine notebook, as that’s where I develop ideas, sketch, reflect, and think about the world around me. That physical connection I have with my notebook is something that can never replaced by the computer.

After the break, Tom split us into groups gave us an assignment to build a fantasy device. Paired with two bright, creative students, Alex and Shaun, we decided to build a prototype for an ocean vacuum, to suck up all the trash that has unfortunately collected in our oceans.

We went to the shop and started to collect various pieces of items from the junk shelves in the shop to begin building our fantasy device. We had no idea what it was going to look like, but we let the items help inspire the design and quickly, we began to see a vision for our “creation”. We drew a quick sketch of this design and began to pull parts out. We found a stool, some fabric, some light fixtures and began taping and stapling away. Everyone was so eager to pitch in and just work on this, that it felt so automatic and seamless.

We began to think about some of the questions believed would be asked of us in class:

1. How do we prevent animals from getting sucked in?
2. How is the trash disposed of?
3. Where does the robot go after it’s full of trash?
4. how do we prevent clogging the system from algae?
5. Is it completely automated or can their be a manual override?

We developed answers to all these questions (or at least our best attempt at a solution) and once time was called we piled into class, ready to talk about our “creation” and to hear from other teams.

Some of our classmates has some excellent ideas around teleportation, air cleaning, translating dog barks into English, and realtime translation devices. Our device, Tom said, was already being developed by a former ITP student. That’s good that we were on the right track in terms of our thinking, but maybe we could have thought of something even more impractical to design around. I’m not sure, but it was a great exercise in getting our minds to think in this creative, yet very practical way.

If the rest of class is going to be like this, I’m going to be in for a treat.