phonepoint pen: using mobile phones to write in air
DESCRIPTION
PhonePoint Pen: Using Mobile Phones to Write in Air. Romit Roy Choudhury Assistant Professor. Noting small pieces of information, quickly and effortlessly can be useful So, what are the options. Buy Milk. State of the Art. Sticky notes … organization is a nightmare not handy - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/1.jpg)
PhonePoint Pen: Using Mobile Phones to Write
in Air
Romit Roy ChoudhuryAssistant Professor
![Page 2: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/2.jpg)
Noting small pieces of information,
quickly and effortlesslycan be useful
So, what are the options ...
Buy Milk
![Page 3: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/3.jpg)
State of the Art
Sticky notes … organization is a nightmareorganization is a nightmare not handynot handy
Typing on keyboard
tiny keyboard sizestiny keyboard sizes small inter-key spacingsmall inter-key spacing
Audio recording cannot sketch informationcannot sketch information time consuming to browse through voice messagestime consuming to browse through voice messages
![Page 4: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/4.jpg)
So, need a solution that is
Easy to use
Always-with-me
Allows sketching
Searchable
![Page 5: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/5.jpg)
Proposed Approach: PhonePoint Pen
User can write messages in air holding the phone like a penholding the phone like a pen
Use built-in accelerometer
capture hand movementcapture hand movement
Decode text and image sent to user’s email address for future use sent to user’s email address for future use
![Page 6: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/6.jpg)
Use Cases
Mashing with cameras superimposing text on picturessuperimposing text on pictures
Sketching exchanging figures between caller/calleeexchanging figures between caller/callee
One handed use e.g., carrying a suitcase e.g., carrying a suitcase
Duke
University
![Page 7: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/7.jpg)
Prototype
Current prototype implemented on Nokia N95 mobile phones
several design challenges emerge …
![Page 8: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/8.jpg)
Design Challenges (1)
Lack of a Gyroscope Accelerometers only measure linear Accelerometers only measure linear acceleration acceleration Linear Movements – Rotation Ambiguity Linear Movements – Rotation Ambiguity
Proposed Approach: • Hold Phone in Non-rotating GripHold Phone in Non-rotating Grip• Determine Angular Orientation during Determine Angular Orientation during the the
pause pause
P
![Page 9: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/9.jpg)
Design Challenges (2)
Background Vibration (Noise) Sensitive AccelerometersSensitive Accelerometers Significant Jitter by hand vibrationsSignificant Jitter by hand vibrations
Proposed Approach: • Smooth the accelerometer readings with Smooth the accelerometer readings with
moving average over last n (=7) readingsmoving average over last n (=7) readings• Suppress acceleration values < 0.5 Suppress acceleration values < 0.5
P
2/ sm
![Page 10: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/10.jpg)
Design Challenges (3)
Computing Displacement Erroneous Acceleration ReadingErroneous Acceleration Reading Ambiguity when acceleration is zeroAmbiguity when acceleration is zero
Proposed Approach: • Detect Pause Using Moving WindowDetect Pause Using Moving Window• Reset Velocity to Zero in Between StrokesReset Velocity to Zero in Between Strokes
P
![Page 11: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/11.jpg)
Design Challenges (4)
Lifting Pen from the Paper e.g., dotting the “i” and crossing the “t”e.g., dotting the “i” and crossing the “t”
Proposed Approach: • Impulse on the Z axis during the liftImpulse on the Z axis during the lift• Off the Record – On the RecordOff the Record – On the Record
P
![Page 12: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/12.jpg)
Drawing a Rectangle
RawRaw Accelerometer ReadingAccelerometer Reading
![Page 13: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/13.jpg)
Noise Smoothing using Noise Smoothing using Moving AverageMoving Average
Background Noise Background Noise SuppressionSuppression
![Page 14: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/14.jpg)
Velocity Plot after Velocity Plot after Avoiding Velocity DriftsAvoiding Velocity Drifts
Final RectangleFinal Rectangle
![Page 15: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/15.jpg)
Raw Accelerometer Data for M and iRaw Accelerometer Data for M and i
![Page 16: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/16.jpg)
More PhonePen words …
![Page 17: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/17.jpg)
Ongoing Work
Lowercase character recognition
Smaller hand movements in air
Increased robustness and reliability
![Page 18: PhonePoint Pen: Using Mobile Phones to Write in Air](https://reader035.vdocument.in/reader035/viewer/2022062516/56812bd7550346895d904563/html5/thumbnails/18.jpg)
Swipe your credit card and sign in air …
Our vision: