Visual Object Recognition

Description

Recognizing objects is an important capability for many automated systems. The capability is important for a robot picking parts from a conveyor belt, for automatic inspection, and for an intelligent system trying to understand its environment (for example, robotic hazardous waste cleanup and battlefield robots). Although commercial solutions exist for simple recognition problems, they depend on carefully engineered object presentation to eliminate the confounding effects of background clutter, overlapping objects, and lighting variations.

Sandia National Laboratories has developed an object recognition system that does not require precise object presentation. It is also easier to use, more flexible, and has wider applications than other systems. The software automatically trains itself to locate new objects. We have demonstrated the system using an overhead camera pointed downward at objects on a flat surface, a scenario that matches many industrial parts feeding and inspection applications.

Instead of requiring a skilled operator to reprogram the vision system for new objects, the system learns about the appearance of objects from a sequence of training images. These training images show the object at different angles, and our program automatically picks out visual features that best identify the object at any angle. These visual features are chosen such that they are both easy to locate and are relatively immune to changes in lighting. A typical training sequence produces more than 5,000 such features, which our program compresses using a standard video compression technique. This saves memory and increases the speed of the program.

When the system is presented with a new image, it looks for the visual features learned in the training phase. Each feature that the system recognizes lends evidence to the presence of a particular object in a particular orientation. This evidence is accumulated throughout the image, and the algorithm reports which location and orientation had the most evidence. Not all the visual features need to be found on the object, so partially overlapping objects do not cause failures.

Features

• System trains itself so a skilled operator is not required
• Image processing of visual features makes them less sensitive to illumination changes than original black and white image features
• Background clutter effects are eliminated
• System is tolerant of overlapping objects
• Measures position of object to an accuracy of 0.6 degrees in rotation and 1.4 pixels in translation
• Program runs in less than one minute per image on a personal computer

Applications

We have successfully applied variations of this algorithm to finding military vehicles from spy cameras and to detecting infant seats in the passenger seats of automobiles to prevent air bag deployment. Other variations of the algorithm could be used in manufacturing situations such as locating known objects randomly piled in a bin or finding known, manmade objects scattered about a work area such as barrels, pipe fittings, doors, and vehicles.

Status

We are refining the program to increase its speed and reliability based on theoretical analysis of the algorithm. Published articles are available describing the theory and successful laboratory experiments that simulate difficult, real world visual environments.