Figuring out what might work

• Nick Rougeux
• May 30, 2008

Although our grant proposal listed features that we expected to be included in the portal, we also outlined methods to figure out what might best address community development practitioners' needs and motivate them to use it.

Our proposal described our approach to design and development:

The process will begin with contextual research—a combination of techniques for collecting relevant and useful data about a domain, in context. . .

Information will be gathered about audiences’ needs, values, and attitudes in relation to the goals of the website. Data about audience habits will also be collected to determine the best methods to integrate the system or program into their environment and to design the structure and content of its components.

The research will include interviews with community development professionals to learn how they currently perform the functions that the portal will support (italics added).

It is the bane of a Web designer's existence that you are asked to describe the features and technologies that will be included in a website's release (and its cost) at the point you know the least you will ever know about its audiences and business goals.

This approach is often called "Waterfall." You speculate, research, design, and plan for a long time, then spend a year, or two or three, working without deviation, executing what has been planned and ignoring how the world has changed. Finally, you release it to (often unreceptive) audiences.

Then you spend the next year frantically making changes in an effort to make it work. At this point  there's lots of code and designs to change, so it's very expensive. And it's all rush work—a "death march" which produces disspirited employees who may leave, taking with them valuable knowledge about the project.

Executives like this approach because it appears predictable and quantifiable and the logic is compelling. They don't seem to care that it often doesn't work. The alternative seems unthinkable. At the beginning of a project, when they ask the question, "What will make it a success?" they don't want to hear, "We don't know," or  the same answer to "How much is it going to cost?". 

An alternate approach, which we prefer, is "Agile." Although you know at the beginning of a project the general business goals and who the audiences are likely to be, you accept that you can't know all that will be included in the final product (and, by implication, its final cost).

Instead, you design, conduct user research, and build features in short iterations, adjusting what you do in the latest iterations based on what you have learned from previous ones.

We summarized this approach in our proposal:

Interview data will inform the design of early prototypes, which will be presented to members of the target audiences and project stakeholders to collect feedback on utility and usability. The feedback will help fine tune the design of further prototypes. This cycle of design and testing can continue throughout the design stage.

In order to shorten the development time, coding will begin very soon after the project begins. When the design and coded functionality are sufficiently advanced, the portal will be released to a limited pool of users for beta testing. During this phase, measurements of performance, clarity, and easy of use will be collected, and changes made as required. (Italics added.)

In an ideal world, designers and developers are at the same location, sharing  design research (including usability testing), thoughts about database structures and object relationships, etc.

We decided to code in Ruby on Rails, a very flexible programming framework that does a good job of separating the backend programming (models) from the frontend layouts (views).

We knew enough about basic required functionality and data structures at the beginning of the project—even though we didn't know what the visual designs would look like—that we were able to start coding within a couple of weeks. We like to joke that this is "Development-driven design."

Contextual research

It's very easy to find out what users say they want—just ask them. Surveys and focus groups are two popular methods. Unfortunately, what users say they want often does not reflect what they need and will use. This is a basic designer's dilemma.

Malcom Gladwell, in a TED talk about spaghetti sauce, expresses this problem well.

(Warning: reading the next paragraph will spoil the conclusion of this video)

The moral of Gladwell's story is that during many years of expensive market research, asking people "What they want," no one ever said, "I would like some chunky spaghetti sauce."

It took a contextual approach—sitting people down in front of lots of kinds of spaghetti sauce and asking them to taste them—to discover that people liked chunky sauce. The sauce was put on the shelves and quickly captured a significant share of the spaghetti sauce market.

Interviews

One of the most useful means to discover what users might need and want to use is to conduct interviews using open-ended questions. From these, as we mentioned earlier, "we wish to learn how they currently perform the functions that the portal will support."

We don’t ask people what they want in a Web portal, we ask them questions about how they do particular tasks, such as, "If you were looking for a partner for a collaborative, how would they go about finding one?"

Since one of the goals of the portal is to facilitate the exchange of knowledge in  areas such as economic development, foreclosure, and affordable housing, we asked people to tell us about something that they had learned recently related to their jobs . 

One answer:

I had been doing financial counseling for a couple of months. At the end of a counseling session I would ask them, "Would you like to meet with another counselor to talk about employment or housing?" About 80% of the time, they would answer something like, "Not right now, I've got to meet my husband."

At a peer group session, I asked more experienced counselors how they got clients to talk with other counselors. One of them said, "It's easy. Just say, 'Now, I'm going to introduce you to Mrs. Xxx, our housing counselor.' "

I tried it the next day and it worked. In fact, it works about 80% of the time now.

As expected, we found that less experienced people want to learn from the more experienced. Perhaps less expected, however, was that there was reluctance on the part of the more experienced to share. Below is a note from an interview.

A couple of restraints on sharing is that they have limited time, but more importantly that they "don't get points" for sharing. The reports that they make to funders don't include a box to fill in such as "How many times did you share your knowledge with a colleague this week."

This suggests to us that if we can design the portal to be a vehicle through which an experienced person can get points for sharing—recognition, the attention of funders, etc—it might have a greater chance of success.

 

In this video I discuss how we use affinity notes (remind myself to lose a little weight). We record interviews, then copy salient points onto sticky notes, then sort them much like you might do on Sesame Street, "Which of these is not like the other one?" Then you label the groups.

One problem with doing lots of interviews and collecting lots of data is that it is impossible to keep all of it in your head. Putting it on an "affinity wall" helps free your mind. Instead of concentrating on remembering individual bits of data you can look for important patterns that might spark good design ideas.

We will be sharing much of what we've learned from research, and the design implications, as we release new beta versions of the portal.