WHAT'S NEXT
A New Rule of Cursor Control: Just Follow Your Nose
By IAN AUSTEN
Published: October 28, 2004
SING your nose to move the cursor on a computer screen is not the kind of idea that just pops into your head.
"It didn't occur to me in the middle of the night," admitted Dr. Dmitry O. Gorodnichy, a computer vision scientist with the National Research Council of Canada in Ottawa. Rather, Dr. Gorodnichy's camera and software system, which controls a computer by tracking the movement of the nose, had its origins in a contribution to the American space program.
The nouse, as he calls his system, is among the latest in cursor-control methods designed to replace the hand-operated mouse. Most of these systems, which are generally intended for disabled people, track head movements, and often require that the user wear headgear or attach a marker to the face. The nouse and another method that relies on tongue movements inside the mouth represent attempts to make such systems simpler and more useful.
"It's not to replace the mouse," Dr. Gorodnichy said of his invention. "It's to add extra features to the computer."
Dr. Gorodnichy's work on visual recognition of body motion goes back to his days working on upgrading the robotic lifting arm used in the space shuttle. He was one of many scientists at the research council who had worked on that project, which was Canadian designed and built.
Researchers had already explored the possibility of tracking facial movements, but Dr. Gorodnichy said their systems all suffered from limitations. Some researchers had optimistically assumed that users could keep their faces still when using computers. And the systems developed by others were limited also. For example, they could only recognize whether a user was shaking his head from side to side or nodding.
Initially, Dr. Gorodnichy said, his group tried to use a pair of cameras to create a 3-D image of the face, thinking that would make it easier to track facial elements. But aligning images from two cameras, he said, seemed an excessively fussy procedure for most users. As a result, they concentrated on using one camera, a basic computer Webcam costing about $25.
Many face-recognition systems make use of the eyes, but Dr. Gorodnichy said it became apparent early on that the nose had much more going for it, at least from a computer vision standpoint.
To begin with, Dr. Gorodnichy said, "the nose is located very conveniently in the middle of the face." Adding to its charms is the fact that unlike, say, the eyebrows, the position of its tip isn't significantly altered by tilting the head. Ultimately it proved relatively simple to write software that tracked the nose with precision, despite the resolution limitations of the bargain Webcam.
The Canadian researchers rejected using eyes as their marker facial feature, but they did not completely ignore them. To avoid driving users to distraction as the cursor jumps around every time they move their head, the nouse is turned off or on whenever the software detects a double blink, much like the double-click of a conventional mouse. But rather than finding the eyes first, the system finds the eyes by locating the nose. "Once it finds the nose, it becomes quite trivial to locate the eyes," Dr. Gorodnichy said.
Dr. Gorodnichy has experimented with adding blinks for additional functions - the nouse version of the right-click. But, so far at least, winking is out.
"I'm not very good at winking, and it's just very convenient to blink several times," Dr. Gorodnichy said.
Using the nouse to play a computer game in his lab, Dr. Gorodnichy still uses the keyboard for some functions. During play, his head movements are constrained, at least in contrast to the on-screen action. "It takes some practice," he acknowledged.
The final details have yet to be resolved, but Dr. Gorodnichy said it appeared that the first application of his nouse system would be in a system for hospital patients who are largely paralyzed. With the software and an appropriate camera, he said, they should be able to summon help to their bedsides by double-blinking their eyes.
Think-A-Move, a company in Beachwood, Ohio, is also aiming at the medical market. This month it licensed its tongue-tracking technology to Switch-It, a maker of motorized wheelchair control systems in Houston.
Jim West, the chief executive of Think-A-Move and the former chief information officer at U.S. Cellular, was approached to be a financial backer of the company. After hearing that it was trying to control computers through tongue movements and the audio changes they make in the ear canal, he was not initially impressed.
"It sounded a little science fiction-y," he recalled. "I said, 'I don't think so.' "
But after a demonstration and after meeting with researchers, the company persuaded Mr. West not only to put up money but to join its management.
People think of the ear as something that brings signals from the outside into the body, but Mr. West said the ear - more precisely, the ear canal - also effectively broadcasts what's happening inside the body. Moving the tongue around the mouth, for example, subtly alters air pressure in the ear canal and sets up vibrations in the jaw bone, which also make their way to the ear.
Those signals, of course, are very faint. To pick them up, Think-A-Move users insert what amounts to a reverse hearing aid. Its microphone measures sounds inside the ear canal and transmits them to a computer or another electronic device using a Bluetooth wireless connection.
In the case of the wheelchair, Think-A-Move and Switch-It hope to replace controllers that rely on breath signals. But in all applications, Mr. West said, users must train the software in the same way voice recognition programs are trained. That generally involves repeating a motion (touching the tongue to a molar, for example) and assigning it a function (like "next page"
.
Mr. West said fatigue did not affect the system's performance, but he acknowledged that users must be careful to avoid tongue movements they make almost unconsciously throughout the day.
"I don't usually flick my tongue to my rear back molar," Mr. West said. "A bad move for me would be depressing the tongue against my lower front teeth."
Thu Oct 28, 2004 12:30 pm 
