Pi Robot Trades Webcam for Kinect RGB-D Camera
One of the benefits of being a robot is that you can swap out your
parts more-or-less at will when new gadgets appear on the market.
So when the $149 Kinect camera was released for the XBox game console,
the robotics community took about a week to figure out how to
use the camera to give their robots 3D vision. Pi Robot has
actually had a Kinect since just before Christmas (no, he couldn't
wait), but today he finally got to try it on.
What you are about to see may shock you at first--Pi's familiar face
has been transformed by the new technology. But as we will see,
it is all for the best. Though bigger and heavier than a typical
webcam, the Kinect provides both a regular color video image as well as
a depth measurement at each point. It does this by projecting a
pattern of structured
infrared light out onto the world and uses a second camera tuned to infrared wavelengths to measure
the distortion of this pattern caused by the shapes of objects in the
scene. The result is a full depth map of every pixel in the image
and at 30 frames per second.
The image on the right shows Pi with his new head on. The
Kinect camera is the black horizontal device at the top. It is
wide because it has to accommodate two cameras and one projector.
It also has two microphones, one at either end, that can be used to
locate sounds. The tufts on top of the Kinect came as a little
surprise, first discovered by Mike Ferguson at the University of Albany
when he built his latest creation Maxwell shown on the left.
So what do the 3-dimensional images look like? ROS includes a
visualization tool called RViz which allows us to see both the regular
video image and the 3D map from the Kinect. The 3D map can be
shown in one of three ways: as the original RGB colors overlayed on the
depth measurements; as a grayscale image; or as a color-coded image
with red indicating objects closer to the camera and blue/violet
labeling far away points. Here's what they look like:
RGB image superimposed on depth map:
Flat Color (gray):
Color Coded Depth Map (Axis mode in RViz)
The last image perhaps best illustrates the quality of the depth map
returned by the Kinect, with subtle changes in hue reflecting the
distance of that point in the scene from the camera. Over the next
several weeks and months, Pi will use this new 3D vision ability to
begin recognizing real-world objects by size and shape, much as we
do. Stay tuned!