PUBLISHED IN THE MARCH 2001 ISSUE OF THE COMMUNICATIONS OF THE ACM
How might we interact with computers of the future? Let me list the ways: by gesture, by hand, foot and body motion, by the speed and forcefulness of our activities, by our thoughts, feelings, and emotions, by where, how and when we look, by speech and sound and music and touch. Imagine it and it shall come to pass? Well, not quite, but the potential is staggering, especially in the area of the Cyborg, the implantation of bio-electronic devices to amplify human thought, memory, vision, and muscle power.
To date, the way we interact with computers is incredibly unimaginative and limited. Basically, we sit in front of the box looking and listening, pointing and typing, and occasionally talking. Will this change? Of course, but I believe the change will come about primarily by changes in the computer itself, getting rid of the boxes and embedding them into devices and appliances. After all, we interact with the computers that control our automobile with no awareness that a computer is involved. Instead, we watch the road, evaluate the instruments (including heads-up displays), rotate the steering wheel, depress the pedals, and shift gears. The auto changes state, adjusts the braking, gears and fuel/air ratios all without our awareness, but by interpreting our inputs as intentions, dynamically adjusting itself to the dynamics of our actions and the car's responses. This is one of the ways it will be: new devices will be imbedded in whatever the appropriate physical form factor for the activity with the interaction proceeding naturally, by whatever means is most suitable.
On the one hand, the fundamentals of life and social settings have not changed all that much in the last millennium, perhaps because so much is determined by the environment, by our biology, and by the laws of the physical world. People still are born, go through a relatively slow maturation process of roughly twenty years, lead an adult, active life, then gracefully - or not so gracefully - grow old and die. We need food, shelter, medical care, and human companionship. As a society, we need governing bodies, laws, and social institutions. Our behavior is determined by the culture in which we live, which includes laws, some enforcement agency. We have developed the concept of money as an aid in trading goods and services, a body of law and an enforcement mechanism to restrict our behavior, and the young must be educated to learn the prior knowledge of the society and the history and behavioral norms of the culture.
This much has not changed in a millennium and is unlikely to change in the next millennium.
Human capability has long been limited by biology. We have two legs, two arms, two eyes, two ears. But these are all limited. Athletes have already pushed the limits of strength and endurance. Human working memory has always been limited to a relatively small number of items. Our minds, even of the most talented, are limited in capabilities. And as we age, we go frail, both physically and mentally.
Up to now, our advances in technology have acted as accessories: they have enhanced our abilities but they have not made any changes to our biology. But we can see the beginnings of change. Already we have implanted pacemakers for the heart and artificial organs. We have artificial limbs, plastic lenses in the eyes, auditory implants, and of course glasses and hearing aids. Two pieces of technology are now so much a part of life that many people strap them to their body, never to be without them: watches and cellular telephones. And for some, we might add music players, whether by FM radio, tape, or MP3 player.
For years I resisted the notion that technology would supplant biology, but now I believe the trend is inevitable. We are very close to the point where video cameras and memory chips are tiny enough to be implanted within the body. Add some simple circuitry, solve the power supply problem, and we could supplement hearing and sight along with memory and reasoning.
Why not a TV camera with zoom lens, allowing magnification of distant scenes or vision through infrared, the better to see at night? Why not amplifiers on the ears? And why not recorders capable of saving all that we have heard, seen, or even felt for later recall? Never again does inattentiveness have to mean missing something. Linger over the interesting parts of life, fast forward through the boring parts. Add some pattern recognition and never again need we be at a loss for words, or forget a name or face. Why John, good to see you again. Your wife, Elaine had knee surgery - how did it go? And your son Peter is about to have a birthday. ... ."
The enhancements are apt to be pretty impressive. Memory aids, calculation aids. Decision aids. Imagine implanted dictionaries and translators. Arithmetic calculators.
Recognize that athletes will enhance the body to add strength. More muscle power, more brain power. Today we test athletes in the ever-more difficult attempt to eliminate drug-enhanced performance. Some day we may have to do full x-ray (3-D, tomographic) scans in the ever-more difficult attempt to detect artificial implants.
Why do I think this will happen? Because it is possible. Today, we have not yet reached the levels of miniaturization necessary, but we can see how to get there. Power is still a problem, but that will get solved. At first, enhancements are apt to be implanted for medical reasons, but when people discover that an artificial eye is better than the real one, or that the memory chip that remembers events, names, and facts can be ever-present, the trend will be unstoppable: order your implant today.
The major remaining hurdle is the control circuitry. How does one communicate with an implanted circuit? Imagine it and it shall come to pass? Not really. The brain communicates with itself through a complex, highly parallel communication process involving the firing patterns of neural impulses, bio-chemical stimulation that bathes the brain structures with highly tuned molecular structures, and through methods as yet unknown. Just how information is stored, regenerated, and interpreted within brain circuits remains a major mystery, one unlikely to be solved soon. Sure, we can record neural firings from tens of neurons. Sure we can stimulate neurons to create crude sensory images and coarse motor control. But full-fledged, precise control of the expert performer eludes understanding (which means the behavior of all of us, for we are all experts in doing our everyday activities such as walking, talking, seeing, and feeling).
Note how easy it is for computers to do tasks we find difficult, such as arithmetic calculations and precise memory, how difficult to do tasks we find trivial, such as walking and talking, throwing and seeing, understanding and creating.
But even with these limitations, the future holds major changes in both the dimensions discussed here. Not only will the devices we use have increased power and dynamic responses, but the way they interact with people will become more natural, more complex, more powerful, and more natural. An incredible variety of new devices will emerge, most not yet conceived. And many will find their way into the human body, to amplify and transform our sensory, mental, and motor capacities.
Do you think the current concerns over privacy violations and personal autonomy are large and complex? You haven't seen anything yet.
Short Biographical Sketch
Donald A. Norman says that although he changes careers often, he never stops the previous ones. He is Prof. Emeritus of Cognitive Science at the University of California, San Diego, former Vice President of the Advanced Technology Group at Apple Computer, and an executive at Hewlett Packard.
He is co founder of the Nielsen Norman group, an executive consulting firm that promotes the development of human-centered information appliances and usable, humane websites. He serves as President of the Learning Systems division of UNext, a major player in high-quality distance education. He is the author of thirteen books that have been translated into thirteen languages, including "The Design of Everyday Things," "Things That Make Us Smart," and "The Invisible Computer," a book that Business Week calls "the bible of post-PC thinking."