template for presentation _cse
DESCRIPTION
template for dayananda sagar college of engineering computer science department presentationsTRANSCRIPT
Department of Computer Science & Engineering, DSCE
WELCOME
Department of Computer Science & Engineering, DSCE
MEMS ACCELEROMETER BASED NON-SPECIFIC USER HAND GESTURE RECOGNITION
Vishal Bhaskar1DS10CS128
Department of Computer Science & Engineering, DSCE
CONTENTS• INTRODUCTION
• GESTURE MOTION ANALYSIS
• SENSING SYSTEM
• SYSTEM WORK FLOW
• GESTURE SEGMENTATION
• MODEL 1 : BASED ON SIGN SEQUENCE AND HOPFIELD NETWORK.
• MODEL 2 : BASED ON VELOCITY INCREMENT
• MODEL 3 : BASED ON SIGN SEQUENCE AND TEMPLATE MATCHING.
• EXPERIMENTAL RESULTS
• ADVANTAGES
• APPLICATIONS
• CONCLUSION
Department of Computer Science & Engineering, DSCE
INTRODUCTION• Human – Machine Interactions
• Gesture Recognition
• Physical Gestures 7 Hand Gestures MEMS Accelerometer3 Models based on time domain
2 Methods o Vision – Based ( Limitation ) o Accelerometer Based
Department of Computer Science & Engineering, DSCE
EXISTING SYSTEM• iRobot Ava 500(Telerobotics)
• Gesture controlled TVs
• Nintendo wii,Xbox Kinect (Gaming consoles)
Department of Computer Science & Engineering, DSCE
• 3 Gesture Recognition Models
• 7 Gestures
• Inputted to MEMS 3 – Accelerometer
• Gesture Segmentation Algorithm
• 100’s of data to 8 number code
• Gesture Recognition
Sign Sequence & Hopfield Based Velocity Increment Based Sign Sequence & Template Matching Based
Up, down, left, right, tick, circle, cross
PROPOSED SYSTEM
Department of Computer Science & Engineering, DSCE
BLOCK DIAGRAM
SENSING CHIP
DATA PROCESSING
GESTURE SEGMENTATION
GESTURE RECOGNITION
GESTURE INPUT
RECOGNISED GESTURE OUTPUT
Department of Computer Science & Engineering, DSCE
GESTURE MOTION ANALYSIS
Fig 2 : Gesture up motion decomposition
• Motion in vertical plane ( x – z plane)
• Accelerations on x-z plane
• Up Gesture
Up Gesture
Circle Gesture
o X axis : no accelerationo Z axis : negative – positive – negative
o X axis : positive – negative – positive o Z axis : negative – positive – negative - positive
Velocity zero at pt. 1 & 2 Sign changes at pt. 3 & 4
Fig 1 : Coordinate System
Department of Computer Science & Engineering, DSCE
Fig 3 : Predicted velocity and acceleration in the z-axis
Fig 4 : Real acceleration plot
• One Axis – up & down, left & right
• Two Axis – tick, circle, cross (complex)
• Acceleration changes in z axis
• Real acceleration is the same with the prediction
• Unique acceleration pattern
1 to 3 :-ve; V changes from 0 to max. at 3 3 to 4 :+ve; V changes from -ve to +ve & max. at pt 4 4 to 1 :-ve; V changes from +ve to zero
Department of Computer Science & Engineering, DSCE
SENSING SYSTEM
Fig 5 : Sensing System
• MEMS 3 – axes acceleration sensing chip
• Data management chip
• Bluetooth Wireless data chip
Department of Computer Science & Engineering, DSCE
• MEMS ACCELEROMETER??? Micro Electro-Mechanical Systems Combination of mechanical functions & electronic functions on same chip Measures acceleration forces
Department of Computer Science & Engineering, DSCE
SYSTEM WORK FLOW
Department of Computer Science & Engineering, DSCE
Fig 6 : Motions of seven gestures
GESTURE SEGMENTATION
• DATA ACQUISITION
Horizontally place sensing device
Time interval not less than 0.2sec
Perform Gestures as shown in Fig 6
Department of Computer Science & Engineering, DSCE
• GESTURE SEGMENTATION
DATA PREPROCESSING
2 Processes
o Remove vertical axis offsets by subtracting data points from mean value
o Filter to eliminate noise
Department of Computer Science & Engineering, DSCE
SEGMENTATIONo Find terminal points
o We need
o 2 x n matrices generated
o Compare max. acceleration b/w terminal points with its mean valueo No. of columns = No. of gestures
Amplitude of points Point separation Mean value Distance from nearest intersection Sign variation b/w 2 successive points
Department of Computer Science & Engineering, DSCE
MODEL ONE : GESTURE RECOGNITION BASED ON SIGN
SEQUENCE AND HOPFIELD NETWORK
Department of Computer Science & Engineering, DSCE
Fig 8 : Sign sequence generation
GESTURE RECOGNITION• FEATURE EXTRACTION
Examine the sign of the first mean point
of a gesture
Store in gesture code
Detect no. of sign changes
Store the alternate signs in sequence in
the gesture code
Code for the gesture in fig 8 is 1, -1, 1, -1
Department of Computer Science & Engineering, DSCE
• GESTURE ENCODING
Max. no. of signs for 1 gesture on 1 axis is 4
Eight numbers in one gesture code
Hopfield network can take only 1 & -1 as inputs
+ve, -ve sign and zero are encoded as “1 1”, “-1 -1” and “1 -1”
Each gesture has a unique 16 - number code
Department of Computer Science & Engineering, DSCE
• HOPFIELD NETWORK AS ASSOCIATIVE MEMORY
Recovery mechanism
Weight matrix is constructed
sp - Pattern to be stored P - Number of patterns I - Identity matrix
npTPp
p
P sPIssw 1,1,)(1
qsv )0(qqTpp
p
p Psssswvu
)()0()1(
1
1
))(sgn()( nunv outputnv )(
Department of Computer Science & Engineering, DSCE
• GESTURE COMPARISON
Gesture code is compared with the standard gesture codes Difference b/w the two codes is calculated Smallest difference indicates the most likely gesture
Fig 7 : Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-
circle-cross.
Department of Computer Science & Engineering, DSCE
Table 1: Standard patterns for the seven gestures
Department of Computer Science & Engineering, DSCE
MODEL TWO: GESTURE RECOGNITION BASED ON VELOCITY INCREMENT
• Model deals with complex gestures
• Area bounded by acceleration curve & x axis
• Partitioned areas with alternate signs
• Normalization of area sequence,
- Normalized area - Original area, - Max. area
maxA
AA originalnorm
Increase/decrease in velocity
normA
originalAFig. 9 : Acceleration partition
Department of Computer Science & Engineering, DSCE
• To avoid misalignment due to noise
• Compare velocity increment
– Two area sequences compared – Comparison result
• Gesture with min. Value recognized
Imagining curve has mass Obtain center of mass Two curves are aligned to coincide their centers of masses
Subtracting 2 area sequence vectors
nnd
nn
nn
AAAAAAA
AAAAAS
AAAAAS
'2
'21
'1
''1
'3
'2
'12
1........3211
.....
......
,,
21, ssdA
Department of Computer Science & Engineering, DSCE
WORK FLOW CHART
Department of Computer Science & Engineering, DSCE
MODEL 3:GESTURE RECOGNITION BASED ON SIGN SEQUENCE &
TEMPLATE MATCHING
• Similar to model one
• No encoding of sign sequence in to combinations of -1s & 1s
• Not limited to specific users
Table 2: Gesture codes for model 3
Department of Computer Science & Engineering, DSCE
EXPERIMENTAL RESULTS
• ACCURACY Model III > I > II
• PERFORMANCE Model III > I > II
• Model III has an overall mean accuracy of 95.6%
Table 3 : Comparison of gesture recognition accuracy(%) of 3 models
Department of Computer Science & Engineering, DSCE
CONCLUSION
Sensor data collection, segmentation & recognition
Sign sequence of gesture is extracted
100’s of data to code of 8 numbers
Code compared with standard patterns
Department of Computer Science & Engineering, DSCE
REFERENCES
• WEBSITES ieeexplore.ieee.org/ www.analog.com
MEMS Accelerometer Based Nonspecific-User Hand Gesture Recognition , IEEE SENSORS JOURNAL, VOL. 12, NO. 5, MAY 2012.
S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, “Hand-written character recognition using MEMS motion sensing technology,” in Proc.IEEE/ASME Int. Conf. Advanced Intelligent Mechatronics, 2008