Pi Robot Gets Pananvision

One of the hardest things for Pi to comprehend is the nature of the visual world. While it is easy to grab images and video from the webcam on his head, it is much harder to interpret those images as objects, people, animals, locations, etc. Part of this has to do with the limited resolution and field of view that are provided by a typical web camera. For example, Pi's current webcam has a field of view of about 60 degrees and his onboard computer can only handle a resolution of about 320 x 240 pixles per video frame and still be able to react to events in real time (20-60 frames per second). By comparison, the human eyes together have a field of view of about 180 degrees and a resolution of around 10,000 x 10,000 pixels! In other words, the human visual system has about 1,300 times the resolution that Pi has, and the human brain can process all those extra pixels in parallel rather than one at a time like a single-processor computer is forced to do.

So to give Pi a leg up on the visual world, I have equipped him with a homemade omni-directional vision system. Since a picture is worth a thousand words, here's what his new rig looks like:

The mirror is an 8-inch truck rear view mirror that I found on the web and extracted from its metal housing. I got the plexiglass support cylinder from Tap Plastics. The camera is hidden in Pi's body below the bottom of the cylinder (see next picture) and points directly upward at the mirror to get the 360-degree view reflected in the surface.

Here are a few more images to give you some more perspectives. Click on a thumbnail to see a bigger picture:

Some of you may have noticed that I had to completely rearrange Pi's inner circuit layers to accommodate the recessed camera. Instead of horizontal shelves, Pi's circuits are now mounted on two vertically hanging sheets. This design (which I should have thought of in the beginning) gives me much better access to all of the components as well as frees up a nice channel down the middle.

Pi Hasn't Lost His Head!

The omni-directional vision system does not replace Pi's head camera. Here are a couple of pictures with his back on straight:

(Click to enlarge.)

The 360-Degree World of Pi

So what does the world look like through Pi's new omni-vision system? Here are some pictures taken by Pi of a large balcony. First, here is the raw image:

Click on the image to get the full size view. Notice how the sky and the balcony overhang are visible beyond the mirror. I will be putting a black square covering over the top to prevent this in the future, although as you will see below, we can eliminate it with software. On second thought, it might be good to keep those skyward segments in the frame as they could be used to detect motion from above...

The next series of images show how the raw picture is transformed using a remarkable piece of vision software called RoboRealm. Pi has come to depend on this software for much of what he can do using his cameras.

First we rotate the image 90 degrees so the top of the image aligns with Pi's forward looking perspective:

Next, we "unwrap" the image so we can see the whole 360-degrees laid out in a rectangular strip:

Notice the artifacts at the top created from the fact that we are trying force a spherical image into a flat plane. The perforated gray material at the bottom is the top of Pi's chassis. Now let's crop the image at the top and bottom so we can focus on what is important:

And there you have it. A full 360-degree image of the balcony. (Click on the image for a larger view.) Nothing can escape Pi's gaze now!