Why doing user observations first is wrong

Originally published in Interactions.

How many times have you had to fight hard for the ability to do field studies and other observations at the very start of the project? How many times have you patiently explained that taking time now would be rewarded by faster time to market overall? And how many times were you successful? The HCI community has long complained about product processes that do not allow time to start with good observations.

The more I examine this issue, the more I think that it is we, the HCI community, who are wrong. This includes me, for I have long championed the "study first, design second" approach. Well, I now suggest that for many projects the order is "design, then study."

Let's face it: once a project is announced, it is too late to study what it should be, that's what the announcement was about. If you want to do creative study, you have to do it before the launching of the project. You have to be on the team that decides what projects to do in the first place, which means you have to be part of the management team. (HCI bug one: not enough HCIers are executives.)

Most projects are enhancements of already existing projects. Why do we have to start studying the users all over again? Haven't we already learned a lot about them? Shouldn't we be studying them all throughout the adoption period? Once a project starts, it is too late. Think about it.

But note too this contradiction: All of us usability theorists have long argued for iterative design, trying to get rid of the lengthy, inflexible linear project schedules that stymie flexibility and change, that slows up projects. Instead, we have championed iterative design, with frequent, rapid prototyping and frequent, rapid test.

But wait a minute, our continual plea for up-front user studies, field observations, and the discovery of true user needs are a step backwards: they are a linear, inflexible process inserted prior to the design and coding stages. We are advocating a waterfall method for us, even as we deny it for others. Yes, folks. By saying we need time to do field studies, observations, rapid paper prototypes and the like, we are contradicting the very methods that we claim to be promoting.

The programming community has long struggled with similar issues. They too are trying to eliminate the lengthy, inflexible linear project schedules that slow up projects. They are experimenting with a variety of new methods, for example Agile, XP (Extreme Programming), and other rapid, iterative programming methods. Hurrah for them.

The linear project procedures (also called "waterfall methods"), with lengthy setting of objectives, followed by design, coding and then test, is dead. Thank goodness. The new programming styles practice iterative design, and promulgate multi-disciplinary teams: everything we should be striving for. Now it is time for the UI community to follow their lead, to do what we ourselves have been preaching.

Field studies, user observations, contextual analyses, and all procedures which aim at determining true human needs are still just as important as ever, but they should all be done outside of the product process. This is the information needed to determine what product to build, which projects to fund. Do not insist on doing them after the project has been initiated. Then it is too late, then you are holding everyone back.

Once the project is underway, it is going to be severely constrained by time and resources. That's a fact of life: all product teams feel these constraints. Our goal is to work in multidisciplinary teams to produce effective, pleasurable designs rapidly and efficiently. Design first, on day one, if you can. Review, test, and redesign. Let the programming and marketing teams know how the product will look and behave at the very start of the project. Have faith in our ability to design well-crafted, understandable interfaces and procedures rapidly, without the need for (lengthy) research. Become an essential part of the team, so our input is provided simultaneously with everyone else's. Fortunately that's a component of the new programming methods.

One correspondent to a debate on this topic on a CHI mailing list said: "Nothing I have see or read suggests that agile programmers are the least interested in slowing down their scrums meeting waiting for usability testing to happen. There is no utilization of user research at all in so-called agile usability practices -- it's just a game of guessing what users need."

This correspondent is confused. Usability testing is like Beta testing of software. It should never be used to determine "what users need." It is for catching bugs, and so this kind of usability testing still fits the new, iterative programming models, just as Beta testing for software bugs fits the models. I have long maintained that any company proud of its usability testing is a company in trouble, just as a company proud of its Beta testing is in trouble. UI and Beta testing are meant simply to find bugs, not to redesign.

So let's separate the field and observational studies, the conceptual design work, and the needs analyses from the actual product project. We need to discover what users need before the project starts, for once started, the direction has already been determined. We need to embrace rapid, iterative methods. We need to fit the new procedures used by the programming teams, we need to become team players. What's especially nice about these new methods is that they have made room for us: they explicitly acknowledge the importance of HCI design. Everyone wants us on the team, but only if we won't slow down the work. More power to them.

Don Norman wears many hats, including co-founder of the Nielsen Norman group, Professor at Northwestern University, and author, his latest book being Emotional Design. He lives at www.jnd.org.Column written for Interactions. © CACM, 2006. This is the author's version of the work. It is posted here by permission of ACM for your personal use. It may be redistributed for non-commercial use only, provided this paragraph is included. The definitive version will be published in Interactions.