Bill Gaver’s group — The Interaction Research Studio — does design as a means of research into people and technology. At the Media Systems gathering he used examples from their work to illustrate a number of approaches to one of our major topics: “… guidance and evaluation methods from arts, design and the humanities.”
In his talk he focuses on two phases of their work. The first, context setting, happens at the beginning of a project — while the second, evaluation, of course happens toward the end. Both phases are shaped by the somewhat-unusual goal (a key one for the development of computational media) of creating finished, fully-functional projects that can be experienced by everyday people over extended periods. This is in sharp contrast to computational media work that focuses on flashy, momentary demos or even systems that are theoretical “improvements” on elements of media technology that are not integrated with any audience experience.
Gaver is well known for having developed, together with collaborators, the concept of the “cultural probe” as part of guiding initial work. These probes present evocative tasks that can elicit inspiring information from people. Examples include partial storybooks to be completed with drawings and words, one-shot dream recorders, forms for listing house rules or relations, and cameras packaged with lists of images to take. The researchers get massive information back that is difficult to analyze and difficult to compare. It’s hard to draw clear lessons — and that’s the point. The point is to have one foot in empirically looking at people’s lives while the other foot is in struggling to make sense of the mess, in a way that, as Gaver puts it, “we’re always sure that we’re not sure.” It is looking for inspiration, not information. Probes have been taken up in the research community, but often through their surface characteristics, rather than through their deeper approach.
The approaches Gaver presented in the evaluation phase are focused on much longer-term engagements than in most “user studies” of fields like human-computer interaction. Such studies tend to be so short that they only capture the initial experience of novelty, which is not very informative for the design of objects or media intended for more prolonged experience. One technique that Gaver’s group uses is simply look at the trajectory of engagement over time. After the initial novelty wears off, do people keep interacting? If so, is it steady, or bursty, or in some other temporal pattern? Simply understanding whether something is compelling enough to produce a pattern of long-term interaction is a way of establishing if it is a successful design project.
But to Gaver looking at these patterns is not as interesting as trying to get at deeper narratives of what his team’s objects mean to people, understanding how people engage with them, what values are brought into play in the interaction, and related issues. One way they conduct these investigations is through ethnography — and not simply through interviews, but by spending days at a time in the houses of people where their projects have been installed, without establishing the concerns and categories of such ethnographies in advance.
In addition, they also feel it is important to move further from traditional evaluation approaches, in which there are always people in the room who know the agenda of the project, and in which there is always the danger that participants are performing for the researchers. Gaver’s group has repeatedly used strategies like hiring documentary filmmakers to create films about the people and objects in their sites of deployment — or working with journalists, or even poets — with very little instruction from the research team. These provide important complements to the narratives and understandings built up by the research team through ethnographic methods.
If you wish to discuss the ideas in Gaver’s talk further, please leave comments here or take to Twitter with the #MediaSystems hashtag. Also, as with Fox Harrell’s talk posted last week, which also presented insightful ideas about guiding and evaluating computational media projects, PDF slides are available on the main Media Systems page for this talk. And watch for Michael Mateas’s Chad Greene’s talk next week!
This material is based upon a project supported by the National Science Foundation (under Grant Number 1152217), the National Endowment for the Humanities: Exploring the human endeavor, the National Endowment for the Arts, Microsoft Studios, and Microsoft Research.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation, the National Endowment for the Humanities, the National Endowment for the Arts, Microsoft Studios, or Microsoft Research.
