pervasive computing at nist martin herman, chief information access & user interfaces division...
TRANSCRIPT
PERVASIVE COMPUTINGPERVASIVE COMPUTINGAT NISTAT NIST
Martin Herman, ChiefINFORMATION ACCESS & USER INTERFACES DIVISION
INFORMATION TECHNOLOGY LABORATORYNATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY
VISIONVISION• Small inexpensive computers and sensors
• in every device, appliance and piece of equipment• in buildings, offices, classrooms, homes, cars, factories• people carrying portable and wearable computers • networked to each other and the Internet
• “A trillion devices connected to the Net!"• sensing and reacting intelligently to the environment
• Information instantly accessible anywhere and anytime
• An emerging new paradigm for IT
• Enormous new global market• transforming the way U.S. industry does business• transforming the way people live and learn• requiring the development of a measurement and standards• infrastructure ensuring that U.S. manufacturers are not at a• disadvantage
NIST’s RoleNIST’s Role
Pervasive computing is in infancyPervasive computing is in infancy Many required technologies are immature & have Many required technologies are immature & have
high riskhigh risk NIST’s unique roleNIST’s unique role
– develop metrics, tests and standards to push forward develop metrics, tests and standards to push forward underlying technologies and help pervasive computing underlying technologies and help pervasive computing mature and growmature and grow
– develop test data and test scenarios/tasksdevelop test data and test scenarios/tasks– provide impartial, large-scale evaluations across many provide impartial, large-scale evaluations across many
different systems and companiesdifferent systems and companies– develop interoperability specifications and test methodsdevelop interoperability specifications and test methods– perform & provide tools for conformance testingperform & provide tools for conformance testing
Current Industry EffortsCurrent Industry Efforts Information appliancesInformation appliances
– Web TVs, screen phones, net-enabled smart handheld computers, internet Web TVs, screen phones, net-enabled smart handheld computers, internet gaming consoles, internet automotive systems, electronic booksgaming consoles, internet automotive systems, electronic books
Integration into InternetIntegration into Internet– wireline high-speed access (high-speed data over phone lines; cable)wireline high-speed access (high-speed data over phone lines; cable)– wireless high-speed access (terrestrial/LMDS; satellite services)wireless high-speed access (terrestrial/LMDS; satellite services)– short-range wireless LANs (HomeRF, Bluetooth)short-range wireless LANs (HomeRF, Bluetooth)– networking/communication protocols for information appliances (Jini, Universal networking/communication protocols for information appliances (Jini, Universal
Plug & Play, Inferno, Odyssey, T spaces, JetSend)Plug & Play, Inferno, Odyssey, T spaces, JetSend)
ApplicationsApplications– networked household (TV set-top boxes, handheld communicators, PDAs, PCs, networked household (TV set-top boxes, handheld communicators, PDAs, PCs,
laptops, printers/scanners, digital cameras)laptops, printers/scanners, digital cameras)– networked automobile (GPS, PDAs, cell phones, video games, e-mail, voice networked automobile (GPS, PDAs, cell phones, video games, e-mail, voice
mail, web surfing, navigation assistance)mail, web surfing, navigation assistance)– BodyLANs (e.g., FedEx integration of handheld computers, scanners & printers BodyLANs (e.g., FedEx integration of handheld computers, scanners & printers
to determine status of packages)to determine status of packages)
Proposed NIST EffortsProposed NIST Efforts Measurements and standards for long-term as Measurements and standards for long-term as
well as short-term needs.well as short-term needs. Rich, natural human-computer interactionRich, natural human-computer interaction
Extensive information presentation & visualizationExtensive information presentation & visualization
Dynamic networkingDynamic networking
Sophisticated multi-media information extractionSophisticated multi-media information extraction
Collaborative working environmentsCollaborative working environments
Security & reliabilitySecurity & reliability
System integration and interoperabilitySystem integration and interoperability
Tight integration of IT with electronic, semiconductor, optical, Tight integration of IT with electronic, semiconductor, optical,
manufacturing, chemical, materials, building and fire technologiesmanufacturing, chemical, materials, building and fire technologies
Worldwide Worldwide Information Information
Appliance ForecastAppliance ForecastUnit Shipments (000)Unit Shipments (000)
0
10000
20000
30000
40000
50000
60000
1997 1998 1999 2000 2001 2002
Appliances vs PCsAppliances vs PCsUS Shipments, Consumer Devices,US Shipments, Consumer Devices,
In MillionsIn Millions
0
5
10
15
20
25
1997 1998 1999 2000 2001 2002
PCs
IAs
Applications & Scenarios Personal Assistants
• Extension of today’s pocket organizers, smart cards, smart mobile phones• New functions: electronic money clip, locator, visual communicator• Provide access to personal data, monitor offices & homes, transfer funds, make secure transactions• Increase effectiveness of people’s time; extend the range of their actions
Personal assistant for youth/elderlyProvide ID informationVital dataGPS-based locationRemote transmission of location“Help me” communication functionAccess to medical informationFunds accessible for emergenciesVoice I/OEmbedded camera for visual interaction
Personal assistant for adultSecure access to informationEmbedded biometric sensorsCoordinate user’s scheduleRetrieve & store phone, fax, e-mailPersonal e-business
Electronic purchasing of ticketsElectronic toll paymentsElectronic banking
Integration with office computer, printer, in-car digital maps, WWWVoice I/O, voice-to-text, text-to-voiceEmbedded camera for visual interaction
Applications & Scenarios Smart Spaces
Smart Spaces are work spaces embedded with computers, information appliances, and multi-modal sensors that allow people to work efficiently, together or individually, through unprecedented access to information and help from computers.
Smart Space ExamplesBusiness meeting roomMedical consultation roomTraining/education facilityMilitary command centerCrisis management center
• Identify and perceive users and their actions and goals
• Speech, natural language, computer vision input
• Interaction with information-rich sources
• Provide for the use of mobile devices and receivers
• Provide extensive information-presentation capabilities
• Understand and anticipate user needs during task performance
• Distributed and local collaboration, including collaboration with field personnel & mobile workers
• Provide improved memory and summaries of activities and deliberations for later use
Technologies for ITL Technologies for ITL InvolvementInvolvement
Rich, natural HCIRich, natural HCISpeech & natural language Speech & natural language
understandingunderstandingInteractive dialogueInteractive dialogueText-to-speech generationText-to-speech generationDetecting, localizing, & Detecting, localizing, & tracking tracking peoplepeopleRecognizing peopleRecognizing peopleActivity & gesture recognitionActivity & gesture recognitionSituation awarenessSituation awarenessSummarizing activities & Summarizing activities & events events recorded by sensorsrecorded by sensorsComputer vision processingComputer vision processingDistributed multi-sensor Distributed multi-sensor fusion & fusion & integrationintegration
Integrated information Integrated information presentation & visualizationpresentation & visualization
Dynamic networkingDynamic networkingMobile/wireless communicationsMobile/wireless communicationsReal-time system & Real-time system & communications protocolcommunications protocolData compressionData compressionDistributed databasesDistributed databases
Multi-media information extractionMulti-media information extractionDocument retrievalDocument retrievalQuestion answeringQuestion answering
Collaborative working Collaborative working environmentsenvironmentsDistributed & local collaborationDistributed & local collaboration
SecuritySecurityComputer/Network SecurityComputer/Network SecurityBiometric authenticationBiometric authentication
System integration technologies & System integration technologies & interoperabilityinteroperability
Perceptive Interface Perceptive Interface OverviewOverview
VideoVideoDSPDSP
WorldWorldModelModel
PatternPatternClassifiersClassifiers
AcousticAcousticDSPDSP
DisplayDisplayServicesServices
Smart Space Test BedSmart Space Test Bed
Camera Elements
Large Screen Display
MicrophonePhased Array Array
Beams
EquipmentRoom
64 ADCs/Microphones64 ADCs/Microphones TI 320C32 60MHz with TI 320C32 60MHz with
512k of RAM512k of RAM 3 cameras/video 3 cameras/video
capturecapture Gigabit ethernetGigabit ethernet Servers for dataServers for data Pervasive devicesPervasive devices
– Palm topsPalm tops– TabletsTablets– Wireless LANWireless LAN
One-dimensional Microphone One-dimensional Microphone Array GeometryArray Geometry
Direction ofDirection ofpropagationpropagation
k
SensorSensorLocations Locations
ZZmm
-kIAUI PrototypeIAUI PrototypeAnalogAnalog ArraysArrays
59 Microphones at59 Microphones at15 mm Inter-sensor15 mm Inter-sensor
distancesdistances45 Microphones at45 Microphones at20 mm20 mm
Inter-sensorInter-sensordistancesdistances
Frontal Face Frontal Face DetectionDetection
Skin pixel extraction
Face localization
Template matching
Largest cluster of skin pixel coordinate
Template
CroppingCropping Edge detectorEdge detector
Hough TransformHough Transform
Face feature extractionFace feature extraction
Crop largest skin pixel cluster from frameCrop largest skin pixel cluster from frame
Apply: Apply:
– Edge detector to emphasize circular shapesEdge detector to emphasize circular shapes
– Hough Transform for eye-extractionHough Transform for eye-extraction
– Face Normalization to gallery conditionsFace Normalization to gallery conditions