{"id":606,"date":"2018-09-04T19:39:03","date_gmt":"2018-09-05T00:39:03","guid":{"rendered":"https:\/\/www.tigoe.com\/blog\/?p=606"},"modified":"2020-11-17T07:03:37","modified_gmt":"2020-11-17T12:03:37","slug":"what-do-the-people-do","status":"publish","type":"post","link":"https:\/\/www.tigoe.com\/blog\/category\/physicalcomputing\/606\/","title":{"rendered":"What do the people do?"},"content":{"rendered":"

This essay is a loose collection of principles for physical interaction. Some of this is\u00a0 taken from the book\u00a0Physical Computing<\/em> by Dan O’Sullivan and I, and some of it is material developed over the course of teaching and building interactive systems.\u00a0\u00a0I’ve given variations of this as a talk in various venues, and usually start each new year of Intro to Physical Computing at ITP with it. It seemed well beyond time to write it down.<\/p>\n

A definition of interaction<\/h2>\n

Before we go too much further, I should define what I mean by interactivity. I like Chris Crawford\u2019s definition:<\/p>\n

“Interaction: a cyclic process in which two or more actors alternately listen, think, and speak.”<\/p>\n

-Chris Crawford,<\/span> Designing Interactivity<\/i><\/span><\/p>\n<\/blockquote>\n

The keyword here is cyclic. Conversations don\u2019t end with one exchange, and neither does interaction. There has to be an ongoing dialogue, and there has to be a subject to talk about. The action of one actor provokes a response from another actor. The first listens to that response, considers, and responds with another action. This cycle, repeated over many cycles, is an interactive conversation. The actors do not have to be human, they can be a person and a machine. But their actions have to be understandable to each other in some way.<\/p>\n

Describe the form and the behavior<\/h2>\n

When designers describe interactive installations, they usually describe in great detail the controls, the displays, and the sights and sounds that participants will see when they encounter the installation. Often, they neglect to talk about the people for whom they are designing: who are they, what will they be expected to do with the installation, and why will they be motivated to do it. If you can’t motivate participants to press the button or walk into the room, the interaction won’t ever happen.\u00a0 This is a principle that’s stuck with me from working with actors and directors in college and during my early career: You can’t tell an actor what to do. Actors need to find the motivation for action on their own in order to take any action. The role of directors, playwrights, and designers is to provide the conditions in which the motivation will be found.\u00a0 Similarly, interaction designers can’t tell their audience what to do. We can only provide the setting and context in which they will understand the action that’s expected from the cues given.<\/p>\n

Expressive, Instrumental, Instructional<\/h2>\n

When talking about physical interaction projects, it can be useful to know the goals of the creator of the project. This is particularly helpful when giving critical feedback on the project. There are a few categories of project that occur regularly in the physical computing classes I’ve taught: expressive works, instruments or tools, and instructional works.<\/p>\n

Expressive works are often the least directly interactive, because they\u2019re usually about expressing an artistic point of view. There’s a balance between participation and control of the story: the greater the level of participation, the more the participants, rather than the designer, determine what story emerges.\u00a0The creator of an expressive work generally has something to say. Image, sound, movement and movement are the elements that multimedia tools provide in order to tell that story. Participants can change the direction of the story by doing what you don’t expect or don’t want them to do, or by ignoring certain elements altogether. Expressive projects are\u00a0useful for learning about control of physical systems, the pacing and structure of and control of the story, and about the effects of aesthetic choices, like any expressive work, though.<\/p>\n

Not all art that involves digital technologies has to be interactive. I think it\u2019s hard for a lot of artists to give up the authorial voice and include the viewer. That\u2019s okay. Sometimes you just want to make something beautiful, like the colorfield painting robot shown below, by Matthew Richard. It’s useful to know the difference between interactive works and those made with interactive tools, however.<\/p>\n

\"Photo
Estrella Intersects the Plane<\/a> by Matthew Richard.<\/figcaption><\/figure>\n

Many interactive works are\u00a0 instruments, like Strings, a room-sized musical instrument by Luisa Pereira, Monica Bate Vidal, and Johann Deidrick, shown in the next image. This is an example of interactive work on which the creators have made a clear aesthetic mark, yet the participants complete the work through their participation. The instrument affords a number of different expressive actions by the participants, and at the same time it suggest what they might not do, because of its apparent fragility. You wouldn’t want to sit on the strings, for example.\u00a0<\/span><\/p>\n

\"Photo
Strings, a musical instrument<\/a>. By\u00a0Luisa Pereira, Monica Bate Vidal, and Johann Deidrick.\u00a0<\/span><\/figcaption><\/figure>\n

Some instruments are tools for everyday use, like the custom PS3 controller shown below. It was built to match the particular abilities of the the young man in the photo, who had\u00a0limited mobility of his upper body, but could move his head and one hand.\u00a0\u00a0<\/span><\/p>\n

\"Photo
Single Switch PS3 Controller<\/a>. By John Schimmel, Holly Cohen, Steven, and Tymm Twillman.<\/figcaption><\/figure>\n

Instructional projects are the third popular category. These are works that are designed to convey an idea through direct experience. They rely on the idea that you can learn things through direct action and manipulation that you can learn no other way. Through these, you internalize an idea through direct experience, as seen in Jill Haefele’s Human:Nature<\/a> project, in which the participant wears a pair of headphones containing live crickets, separated from one’s ear by a mesh screen. The direct acoustic experience of the crickets is unlike any recording of the crickets.<\/p>\n

\"Photo<\/a>
Human: Nature by Jill Haefele.<\/figcaption><\/figure>\n

The next exhibit, How You See<\/a>, by Brett Peterson, Xuedi Chen, Tom Arthur, and Hannah Mishin, is another example of an instructional work. Intended as a museum exhibit, it shows<\/span>\u00a0you through video what the rods and cones in your eyes actually see. Participants change the image seen by the eye on the screen by plugging and unplugging phono jacks that represent the connections of the different\u00a0sensory cells of the eye.\u00a0<\/span><\/p>\n

\"A
How you see, by\u00a0by Brett Peterson, Xuedi Chen, Tom Arthur, and Hannah Mishin<\/figcaption><\/figure>\n

Transduction<\/h2>\n

Humans can understand a great deal about each others’ actions and the motivations underlying them because of our shared experiences. Even without language, there are certain assumptions we make about each other that are often correct. When we interact with machines, however, we can’t make those assumptions. Machines\u00a0 don’t understand that we can think, or even what thinking means. They can only read the world through changes in physical energy that our bodies generate through our physical actions. Machines don’t read your intentions, only your actions.<\/p>\n

Transduction<\/dfn> is the conversion of one form of energy into another. In electronics, transducers<\/dfn> are elements that convert, or transduce, other forms of energy to or from electrical energy. Transducers are commonly divided into two groups.\u00a0 Sensors<\/dfn> convert changes in light, heat, air pressure, and other forms of energy into electrical energy.\u00a0 Actuators<\/dfn> convert electrical energy into light, heat, movement, and other forms of energy. Computers can read and write changes in electrical energy, so sensors and actuators are the senses and muscles of a computer.<\/p>\n

Physical interaction involves transduction as well \u2014\u00a0 This is important to keep in mind, because the computers you use in making interactive systems don’t know anything about a participant except what they can sense from the energy changes the participant generates. The computer can\u2019t read their minds,\u00a0 it can only read their actions.<\/p>\n

Digital and Analog<\/h2>\n

Sensors and actuators and the actions they sense and control can be divided into two groups: digital and analog.\u00a0 Digital<\/dfn> sensors and actuators, sometimes called binary transducers<\/dfn>, can sense or control two states, on or off. Analog<\/dfn> sensors and actuators can sense or control a continually varying range of states. A simple way to remember the difference is to think about a cat on a mat. Digital sensors can tell you if the cat is on the mat, or the cat is not on the mat. Analog sensors can tell you how fat is the cat on the mat.<\/p>\n

\"Photo
Digital state: The cat is on the mat<\/figcaption><\/figure>\n
\"Photo
Digital state: the cat is not on the mat<\/figcaption><\/figure>\n
\"Photo
Analog state: how fat is the cat on the mat?<\/figcaption><\/figure>\n

Although analog sensors are often called continuous, in practice the computers that read them measure a discrete number of possible values. For example, a typical microcontroller might be able to read the changing resistance produced by a force-sensing resistor (used under the cat on the mat above) to a resolution of 1,024 possible discrete states. The resolution<\/dfn> of a sensor or actuator determines how finely you can sense or control the action at hand.<\/p>\n

When you describe the action that a participant is expected to take, ask yourself how much you need to know about the action. Will it be enough to know that the action happened or not? If so, then a digital sensor might do the job. Or do you need to know about a range of possible states of the action? If that is the case, then you probably need an analog sensor.<\/p>\n

Explicit and Implicit Interaction<\/h2>\n

Physical interaction is can be implicit or explicit. Thinking this way offers a link between a person’s intention and the physical action they take. An explicit interaction<\/dfn> is one where the participant’s action is primarily intended to send the computer a message. Think of pressing a pushbutton. The physical affordances for an explicit action should be clear and obvious, and the sensing is often limited to a very contained area. An implicit interaction<\/dfn> has some other primary purpose, and sending the computer a message is a secondary effect. Doorway entry sensors, automatic faucet sensors, and other motion detectors can support implicit interaction. The physical affordances may not be obvious, and sensing may be across a wide area. Too wide an area may result in false triggering.<\/p>\n

\"Photo
Explicit interaction. The actor taps the dome to make something happen.<\/figcaption><\/figure>\n
\"Photo
Implicit interaction. The dancer dances to express herself. Sensors capture her motion and translate it into images on the screen.<\/figcaption><\/figure>\n
\n

Examples of sensors for explicit interaction: pushbuttons, knobs, sliders, keys, card swipers<\/p>\n

Examples of sensors for implicit interaction: door entry sensors, f<\/span>loor triggers,\u00a0<\/span>faucet sensors,\u00a0<\/span>motion detectors.\u00a0<\/span><\/p>\n

Wendy Ju<\/a> developed a framework for implicit interaction as part of her dissertation, and has a good book on the design of implicit interactions<\/a>.<\/p>\n

“What should I do here?”<\/h2>\n
\"Photo
Noise Nest, by Jeremy Diamond.<\/figcaption><\/figure>\n

Interactive art breaks the rules of traditional plastic arts. With painting and sculpture, we know our role as viewers. We look but don\u2019t touch. Interactive art is not always so clear. We don\u2019t yet have an historical context for interactive art: should we touch, should we not touch? Should we watch and nod appreciatively? Should we dance and jump around frenetically? What\u2019s acceptable? There\u2019s no commonly expected response that audience members can either follow along with, or defy, as there is in the traditional plastic arts. It is important that artists who make interactive work set the expectations for the audience. If you want them to explore, you can make it vague, but give them something evocative, ike the soft, inviting fur texture of Jeremy Diamond’s\u00a0Noise Nest sculpture<\/a> shown in the previous image.<\/p>\n

Quite often, the default for this kind of work is to leave things very open. This hasn\u2019t changed for a lot of years. For example, consider one of the germinal works of interactive computer video art from Myron Kruger in the early 1970\u2019s in the video below.<\/p>\n

\n