Determining the pose of three-dimensional objects by multisensor fusion

This research addresses the problem of determining the position and orientation (pose) of a known object, given partial pose information from diiferent types of sensors. From our viewpoint each sensory measurement provides geometric information constraining the position and orientation of an object feature; e.g., if depth is unknown, then visual sensing of an object vertex reveals that the vertex lies somewhere on a certain line in space (we assume object features have already been matched to sensory data). Different sensors provide different forms of these constraints and also involve different measurement accuracies. The problem is to find a best guess of object pose when given such information from a variety of sensors.

The approach proposed in this research is of the Least Mean Square Error (LMSE) variety. It involves combining the parameters of each sensory measurement with the estimated errors in measurement to form a distance function which provides a measure of error for particular values of the measurement parameters. From the model of each type of sensing, we define a constraint relationship between the measurement parameters and the parameters describing the position and orientation of an object feature. These constraints and the distance function generated by the sensory measurement are used to generate a distance function of the object feature parameters. From this new distance function and the known object model, a third distance function, one of the pose parameters, is created. This local pose distance function describes an error as a function of the pose parameters based on a single sensory measurement. Finally, the global pose distance function is defined as the sum of the local pose distance functions. The minimum of this last function (found numerically) provides our best guess of object pose. The advantage of this approach is in providing a consistent framework for the incorporation of noisy data from a variety of sensors.

The mathematics of this approach were developed for the sensing of point, line, and plane object features by a variety of sensing modalities. The method was implemented in software and tested using the robotics workstation at the University of Tennessee.