16-264: Computer Vision with Optical Mouse Chips

Humanoids 16-264
Assignment #2: Vision
Tristan Trutna and Mark Schumacher

Optical Flow:
For robot guidance, having as many sources of feedback for finding and confirming position is imperative. One such strategy is using optical flow, which uses vision algorithms to pick key points in a video stream and track their motion. Unfortunately to do a good job of this, it takes good cameras and significant processing power.

The Optical Mouse:
One often overlooked application of optical flow is the optical mouse. An optical mouse actually contains a low resolution black and white camera which takes 'photos' of the mousing surface at 6000Hz and runs a point finding algorithm and compares these points. So rather than using an expensive high resolution camera and a fast computer, the mouse chip operates on a low resolution image and still returns good results when used as a mouse.

Exploded view of an optical mouse based on Agilent optical chips. (C) Agilent Technologies.

In the image above, the camera chip is placed directly above a plastic lens which is illuminated by an LED. The lens focuses the camera onto the mousing surface directly below the mouse and thus an optical mouse only tracks the surface directly touching or very very slightly above the surface it is tracking.

Using Optical Mouse Technology For General Optical Flow:
The optical mouse chip only tethered to its application by the lens used to focus the camera at the mousing surface. By interfacing an optical mouse chip to a microcontroller and replacing the default lens with a lens focusing at a distance away, it should be possible to harness this cheap optical flow technology for many more tasks.

Mouse-Computer Interface:
In order to get maximum resolution, image frame dumps, and get dx dy data on demand, it is necessary to replace the standard interface components with a microcontroller. For this I chose a Parallax Basic Stamp, for simplicity, however any PIC microcontroller is sufficient.

The current implementation of the mouse chip optical flow design.

The interface is a simple 3 pin synchronous serial connection to the Basic Stamp which in turn interfaces to the computer over RS-232 Serial. Because the mouse chip is designed to do all of the optical flow algorithms internally, the software interface is very simple. Requests are sent over serial for the dx and dy data, and the chip returns values for dx and dy since the last request.

Physical Implementation:

Not much to say here. The left half is an analog circuit and is not related to this project.

Software: Focus The Lens
Because the lens is not the stock lens used for focusing the camera on the ground, it is necessary to focus the camera so that it can resolve "key aspects" of a given frame. This can be done two ways: the first is to point the camera at the target and change the focus until an internal Qval reaches a maximum, but it is difficult to know what exactly it is focusing on. The other approach is to get a frame dump.

Above is a photo shot from the approximate position of the mouse cam, and below is a focused frame dump from the camera. This illustrates both the resolution and the narrow field of vision as well as the focus:

The "experimental setup" is poorly lit, however the music stand is clearly visable and is in focus (though the 30x30px resolution does not give an asthetically pleasing image).

Software: Use The Data
The next step is to actually use the data for some purpose. In this case I wanted to test the accuracy of the mouse chip in keeping track of absolute position at a distance significantly greater than the space between the lens and desktop in an optical mouse application. To do this I made a make-shift track to slide the camera on parallel to a wall at a distance of about 5 feet.

Notice the highly scientific setup in clean room conditions.

The clothing and music stand are to add more details for the chip to consider... I assumed a flat white wall at 5 feet would have very few "key aspects" and give poor data/low Qval values.

The "Experiment":
Using the software I wrote I slid the camera 1 inch and calibrated the displacement measurement to the number of pitch/yaw counts. Then I switched to measuring displacement with dx counts and got encouraging results. Calibrating to 1 inch and then displacing 3 inches, the mouse chip read 2.999, 2.94 and 2.87 inches. Because the setup is so primitive, I did not take more data, but instead I will repeat the experiment with a consistent background and a real track/slide, as well as measurements more precise than 1 inch.

A good experimental run.

A demonstration of the displacement graph... sort of. The graph is actually unscaled, so the graph is only a qualitative representation.