Vision-Based Robot Control in the Context of Human-Machine Interactions

This research has explored motion control based on visual servoing – in the context of complex human-machine interactions and operations in realistic environments. Two classes of intelligent robotic systems were studied in this context: operator assistance with a high dexterity telerobotic manipulator performing remote tooling-centric tasks, and a bio-robot for X-ray imaging of lower extremity human skeletal joints during natural walking. The combination of human-machine interactions and practical application scenarios has led to the following fundamental contributions: 1) exploration and evaluation of a new concept of acquiring fluoroscope images of musculoskeletal features of interest during natural human motion, 2) creation of a generalized framework for tracking features of interest in visual data, and 3) creation and experimental evaluation of a vision based concept of object acquisition suitable for efficient teleoperation.

Several methods were proposed for motion control based on image sensing and processing. These methods were implemented and experimentally evaluated in both application contexts. The fluoroscopy tests included emulated joint tracking using a mechanized mannequin, and actual skeletal joint tracking trials conducted on 30 human subjects. The teleoperation assistance framework was tested using a full-scale telerobotic remote manipulator system with realistic task geometries, loads and lighting conditions.

The proposed methods and approaches produced promising results in both cases. It was demonstrated that the vision-based servo control using fluoroscopy images is an effective way to track human skeletal joints during natural movements. The telerobotic demonstration showed that visual servoing is a feasible mechanism to assist operators in acquiring and handling objects of interest in complex work scenes.