Visual interaction through hand gesture recognition using Hausdorff matching

Abstract

For many humans, interacting with a computer is a cumbersome and frustrating experience. Most people would prefer more natural ways of dealing with computer. The PUI paradigm has emerged as a postWIMP interface paradigm in order to cover these preferences. In this paper, a computer vision system is proposed based on fast detection and accurate hand posture commands recognition in color images, that can be executed in a common PC system equipped with an USB webcam such that any user can use it in its office or home. The major contributions of the presented approach are: (1) a fast segmentation process to segment the moving hand from the image background, which is able to deal with a large number of hand shapes against different backgrounds. (2) A recognition process that identifies the hand posture from the temporal sequence of segmented hands, where postures which are really similar are properly classified. The kernel of the recognition process is a robust shape comparison carried out through a Hausdorff distance approach, which operates on edge maps and derive holistic similarity measures. We introduce the use of a visual memory, which allows handling with diverse visual aspects of each one of the stored patterns that composes this memory. This paper includes experimental evaluations of the recognition process of 26 hand postures and it discusses the results. Experiments show that the system can achieve a 90% recognition average rate and is suitable for real-time applications.