![Page 1: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/1.jpg)
The Escritoire: a personal projected The Escritoire: a personal projected display for interacting with documentsdisplay for interacting with documents
Mark Ashdown
[email protected] Robinson
University of Cambridge Computer Laboratory, UK
![Page 2: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/2.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
The EscritoireThe Escritoire
• Life-sized desk display• Exploits peripheral vision• Uses two projectors• Two-handed input• Remote participants can
share a task space
![Page 3: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/3.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
OverviewOverview
• Motivation• Other projects• Personal projected display• Input• User interface• Single-user tests• Networking• Two-user tests• Conclusion
![Page 4: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/4.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
MotivationMotivation
• Projection technology• Space• Affordances of paper• Input techniques• Collaboration
![Page 5: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/5.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
MotivationMotivation
• Projection technology• Space• Affordances of paper• Input techniques• Collaboration
![Page 6: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/6.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Other projects – visual peripheryOther projects – visual periphery
• Visualization techniques• Multiple monitors• Attentive displays• Head-mounted displays
![Page 7: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/7.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Other projects – paperOther projects – paper
• DigitalDesk• LivePaper• Dog-eared pages• Rotating and peeling back
pages
![Page 8: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/8.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Other projects – projectorsOther projects – projectors
• Multi-projector display walls
• Focus plus context screen
• Augmented objects
• Geometric and photometric calibration
![Page 9: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/9.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Other projects – collaborationOther projects – collaboration
• Krueger’s VIDEODESK
• DoubleDigitalDesk
• Designer’s Outpost
![Page 10: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/10.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Display – hardwareDisplay – hardware
• Foveal display
• 1024x768 portable projectors
• Oblique projection
• Use 3D hardware to warp graphics
![Page 11: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/11.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Display – calibrationDisplay – calibration
• Use a projective transformation
• Obtain point correspondences between projectors and desk
• Warp at 30 frames per second for two projectors.
• Warping is fast, updating textures requires optimization
![Page 12: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/12.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Display – calibrationDisplay – calibration
• Projective transformation is a good model
![Page 13: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/13.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Multiple planesMultiple planes
• Work with Rahul Sukthankar
• Create an interface spanning multiple surfaces
• Uses one projector and a camera
• Three main parts to the calibration:
– Find the boundaries between the surfaces
– Find homographies from projector to camera
– Find homographies from camera to surface
![Page 14: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/14.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Multiple planes – finding planesMultiple planes – finding planes
• Project lines from the projector
• Find ‘kinks’ in the camera image
• Fit a line to the
![Page 15: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/15.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Multiple planes – homographiesMultiple planes – homographies
• Calculate projector-surface homographies from line correspondences
• The method should be robust to outliers
![Page 16: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/16.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Multiple planes – metric rectificationMultiple planes – metric rectification
• Calculate camera-surface homography up to a similarity
• There is a closed-form solution from images of right-angles
• Should be robust
![Page 17: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/17.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Multiple planesMultiple planes
• Get final projector-surface homographies
• Warp images to appear correctly on the two planes
• Use this display for visualizations
![Page 18: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/18.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
InputInput
• Digitizer pen for dominant hand
• Ultrasonic pen for non-dominant hand
![Page 19: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/19.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
User interface - cursorsUser interface - cursors
• Cursor may be turned off
• Cross hair shows current position
• Trace gives a history of past movement
![Page 20: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/20.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
User interface – pilesUser interface – piles
• Add, re-order, and remove items
• Pile splits for browsing
• Like Apple’s pile metaphor
![Page 21: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/21.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Single user – testSingle user – test
• Escritoire has been used for demos
• Task 1: highlight spelling mistakes
• Task 2: put images in piles
![Page 22: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/22.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Single user - resultsSingle user - results
• People could quickly use the system
• Users preferred no cursors
• Sensing of pen buttons should be designed carefully
• Occlusion was not a problem
• Difference in brightness was not a problem
![Page 23: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/23.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Client serverClient server
• Escritoire software is split into client and server
• The client simply displays tiles
• Events are passed to the server for processing
• Protocol switches between client-pull and server-push
• Updates are coalesced at the server
![Page 24: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/24.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Two users – testTwo users – test
• Standard video conference
• Desks for sharing documents
• Initially each participant was shown 30 houses
• Then two remote participants had to pick the best house from groups of 10
![Page 25: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/25.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Two users - resultsTwo users - results
• No extra training was needed
• The trace was preferred
• The audio and desk were much more useful than the video
• Foveas in different positions
• Assignment of functions to pens
• Private workspaces would be useful
![Page 26: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/26.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
ConclusionConclusion
• The foveal display provides an affordable desk-sized interface
• Fits in a normal office
• Participants could use the interface with two pens after only a few minutes of training
• Bimanual input over a large area provides a kinaesthetic sense that allows items to be retrieved rapidly
• Interaction was possible over a normal ADSL link
• A task space can be more useful than a person space
• Gesturing is useful for collaborating users
![Page 27: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/27.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
ReferencesReferences
• Experiences Implementing and Using Personal Projected Displays, Procams 2003, Nice, France, October 2003
• The Escritoire: A Personal Projected Display, Proceedings of WSCG 2003, Pilsen, Czech Republic, February 2003
• www.mark.ashdown.name
![Page 28: The Escritoire: a personal projected display for interacting with documents Mark Ashdown mark@ashdown.name Peter Robinson pr@cl.cam.ac.uk University of](https://reader035.vdocument.in/reader035/viewer/2022062515/56649ca45503460f949641b2/html5/thumbnails/28.jpg)
Personal Projected DisplaysMark Ashdown
University of Cambridge
Foveal display calibrationFoveal display calibration
• There are various 2D co-ordinate spaces
• Use projective transformations between co-ordinate spaces
• Closed-form least-squares solution from point