(Article originally published on Core77's design Website.)
Every year the world holds many contests for industrial designers. Lots of submissions, lots of time spent by jurors reviewing them, lots of pretty pictures afterwards. Fun to read, wonderful for the winners. What's the problem?
I have been a juror for a number of contests, including the major American yearly contest sponsored by the Industrial Design Society of America, IDSA, and BusinessWeek. Although I always enjoyed the experience and the interaction with talented, hard-working fellow jurors, I have become increasingly dissatisfied with the results.
Why are shows bad? Shouldn't we reward good design? Sure, if that's what the shows accomplish, but they don't. In fact, I believe they do harm to the profession. They reward the visible parts - styling - and ignore the most important, but hardest parts: interaction, experience, truly meeting needs, and even economic success. Oh sure, the rhetoric that accompanies the awards often heaps praise upon these other aspects of design, but that praise is not based upon solid evidence. No tests or studies, no independent evidence. As a result, the contests perpetuate the myth that industrial design is primarily about style and that brilliant styling leads to success in the marketplace. Both statements are false.
Jurors in design contests can only judge the material submitted to them. Invariably, the contest entries consist only of drawings, photographs, and videos accompanied by lush words of praise written by the design team, their company or client, or worse, their PR agency. Most of the time the actual products are not available for the jurors, nor can they be, when some products are tractors or cranes, air-conditioning units or automobiles - things far too large to transport to the jurors. As a result, jurors cannot experience them in use, they can't watch the intended audience use them, they can't assess how well they provide for graceful interaction, what pleasure or pain they provide, what benefits they provide. These problems result in fundamental limits to design contests.
I recently spoke with a founder of one of the largest and most successful design firms in the world who confessed that although his firm consistently wins multiple awards year after year, he is frequently puzzled by the choices. Entries he considered mediocre (from his own company) would at times win top prizes whereas entries he considered exemplary and superior would sometimes get passed over. What does that tell us of the contests?
Well-meaning, well-qualified judges are thwarted by the system. They are doomed to fail, for they lack the information required to make informed, intelligent choices. The only thing that can be judged is appearances. As a result, these shows perpetuate the myth that design is only about appearance.
DESIGN CONTESTS JUDGE THE SUPERFICIAL
Why does this happen? I do not fault the organizers or the judges: the fault lies with the complex logistics of running these contests. The organizers and judges are dedicated professionals, committed to rewarding the best aspects of design. They would prefer to consider the underlying design research, the effectiveness of the product, ergonomics, ease of use, and the ability to satisfy those who purchase and use it. Ideally, the award winning products will also be commercial successes, validating the awards through the marketplace. But in the end, jurors are unable to make the correct decisions because the structure of design contests is fundamentally incapable of providing the appropriate evidence. As a result, the judges must use the material that is provided them - images and self-serving congratulatory product descriptions, buttressed by whatever accidental experience some of the jurors might have had with the product category.
Design for the modern designer is about finding unmet needs, about the way people interact with and use the product or service, about economic and environmental sustainability, about providing utility and pleasure. Design can be applied to services and organizational structure, to financial systems and medical practice. It is not just style and appearance, although these play an important role.
But how can the jurors assess utility, understandability, sustainability, economic factors, or market viability? Style and form can be judged through photographs and videos, but economic and environmental sustainability cannot be photographed. Market success requires months or years to assess. Services, experiences, and utility could be photographed and recorded, but assessing the images would require considerable time and analysis, plus evidence that the tests were run by independent organizations using procedures that assured a lack of bias and adherence to standard testing protocols. Doing all of this would take time and money.
So, how could we make design contests better? We would have to change how they operated.
HOW COULD DESIGN CONTESTS IMPROVE?
Contrast the way design contests are run with the methods used by Consumers Union, publisher of Consumers Reports, to assess the quality of products. Design contests assemble an ad-hoc team of designers, give them more material than they can readily assimilate in the brief time allowed for the judgments, do not provide the actual items under review, and never have time nor money to do the kinds of analyses that the awards require.
Consumers Union only reviews products that they have purchased themselves from stores, avoiding manufacturer's samples for fear that they might be non-representative of the items under sale. The items undergo exhaustive tests in the laboratories and with users. All of this takes considerable time and money.
Of course the goals of a design contest are different from that of Consumers Union. Design contests are intended to recognize creativity and ingenuity, to applaud design teams that moving the profession forward, pushing the envelope on form and function, environmental responsibility and efficiency, economic value and usability. The point of the design contest is to reward innovation and advancements, not to determine which of many products in a category is most worthwhile. As a result, although we can learn from Consumers Union we have no need to copy them.
I proposed to the organizers of the yearly IDSA-BusinessWeek contest that they should prescreen all applications. Each product or service submission would have to be accompanied by an evaluation performed by an independent testing group that evaluates its functionality, usability, durability, and market impact. These would be somewhat in the spirit of the Consumers Union evaluations, although much less exhaustive and thorough, done by agencies certified as both qualified and independent by the contest organizers. The costs should be born by the submitters, but hopefully these could be reasonably low for each submission.
Jurors need to understand each product category if they are to asses the contribution of the entry in that category. Today, we hope that the knowledge of the assembled jurors will cover all categories, but in my experience this is seldom true. After all, the vary of categories covered by industrial design ranges from streetlights to train systems, from organizational structure to cooling systems for high-technology server farms. Fork lifts and tableware: the range is far too great even for a group of jurors.
Contests could prepare brief overviews of each product or service category being judged to help establish the state of the art. This review would be provided by the contest organizers, would provide a brief review of products in the category and the history of the area, thus letting judges unfamiliar with the category set appropriate expectations. Reviews of each product or service category would greatly enhance the panel's ability to judge the novelty of a submission, something that is difficult to do today when a single, relatively small panel must assess submissions from every conceivable field. These reviews could be reused year after year, usually with only simple updating, minimizing cost.
Finally, there is the problem of composition of the jury. In all the contests in which I have participated, the jurors, including the chief juror, the chair, change every year. Presumably this is done to allow for wider industry participation, but it precludes any learning or the benefits of history. I recommend rolling three-year terms, with 1/3 of the jurors appointed each year. Each jurors would have one year to learn the ropes, one year to practice what had been learned, and a final year both to practice and to mentor newer jurors. The chair should be appointed through a similar rule. One possibility is to appoint an experienced juror for a three-year term, the first as chair-in training, the second as chair, and the third, as past chair, acting as advisor and mentor to the current chair and chair in training.
One colleague suggested that perhaps we could make use of crowd-sourcing, the informed feedback of design professionals who could help assess the entries. This would require delaying the assessment of an entry until after it had been on the marketplace for long enough that there would be evidence to support the claims. The prizes would then reflect market success as well as informed expert opinion. There are obvious issues that would need to be addressed, but huge potential benefits as well.
SHOULD WE HAVE DESIGN COMPETITIONS?
Yes, we should have design competitions. They help bring out the best in design. But today, they reward only a tiny part of modern design. The design profession prides itself on its ability to solve wicked, difficult problems. Well, here is a difficult problem crying out for a superior solution. Design professionals, it is time to design the reward structure for the design profession so that the designs that advance the field, that make a real difference in the world of products, services, and all the other problem areas in which modern designers work receive recognition. We should have contests, but we should do them properly.