data processing and semantics for advanced internet of things (iot) applications: modeling,...
DESCRIPTION
This tutorial presents tools and techniques for effectively utilizing the Internet of Things (IoT) for building advanced applications, including the Physical-Cyber-Social (PCS) systems. The issues and challenges related to IoT, semantic data modelling, annotation, knowledge representation (e.g. modelling for constrained environments, complexity issues and time/location dependency of data), integration, analy- sis, and reasoning will be discussed. The tutorial will de- scribe recent developments on creating annotation models and semantic description frameworks for IoT data (e.g. such as W3C Semantic Sensor Network ontology). A review of enabling technologies and common scenarios for IoT applications from the data and knowledge engineering point of view will be discussed. Information processing, reasoning, and knowledge extraction, along with existing solutions re- lated to these topics will be presented. The tutorial summarizes state-of-the-art research and developments on PCS systems, IoT related ontology development, linked data, do- main knowledge integration and management, querying large- scale IoT data, and AI applications for automated knowledge extraction from real world data. Related: Semantic Sensor Web: http://knoesis.org/projects/ssw Physical-Cyber-Social Computing: http://wiki.knoesis.org/index.php/PCSTRANSCRIPT
![Page 1: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/1.jpg)
Data Processing and Semantics for Advanced Internet of Things (IoT)
Applications:modeling, annotation, integration, and perception
Pramod Anantharam1, Payam Barnaghi2, Amit Sheth1
1Kno.e.sis Center, Wright State University2Centre for Communication Systems Research, University of Surrey
Madrid, Spain, June 12-14, 2013http://aida.ii.uam.es/wims13/keynotes.php#tutorials
Special Thanks to: Cory Henson, Kno.e.sis Research Center, Wright State UniversityWei Wang, Centre for Communication Systems Research, University of Surrey
![Page 2: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/2.jpg)
2
Part 1: Introduction to Internet of “Things”
Image source: CISCO
![Page 3: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/3.jpg)
Internet of Things
“sensors and actuators embedded in physical objects — from containers to pacemakers — are linked through both wired and wireless networks to the Internet.”
“When objects in the IoT can sense the environment, interpret the data, and communicate with each other, they become tools for understanding complexity and for responding to events and irregularities swiftly”
source: http://www.iot2012.org/
![Page 4: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/4.jpg)
“Thing” connected to the internet
“In 2013 cumulative shipments of Bluetooth-enabled devices will surpass 10 billion and Wi-Fi enabled devices will surpass 10 billion cumulative shipments in 2015,” - Peter Cooney, wireless analyst with ABI Research
“73% of Countries with 4G Services Have Dropped Their 4G Tariffs by an Average of 30% in the Past 6 Months”1 - ABI Research (07 Feb 2013)
1http://www.abiresearch.com/
![Page 5: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/5.jpg)
5
Network connected Things and Devices
Image courtesy: CISCO
![Page 6: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/6.jpg)
6
Sensor devices are becoming widely available
- Programmable devices- Off-the-shelf gadgets/tools
![Page 7: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/7.jpg)
7
More “Things” are being connected
Home/daily-life devicesBusiness and Public infrastructureHealth-care…
![Page 8: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/8.jpg)
8
People Connecting to Things
Motion sensorMotion sensor
Motion sensor
ECG sensor
Internet
![Page 9: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/9.jpg)
9
Things Connecting to Things
- Complex and heterogeneous resources and networks
![Page 10: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/10.jpg)
10
Wireless Sensor Networks (WSN)
Sinknode Gateway
Core networke.g. InternetCore networke.g. InternetGateway
End-userEnd-user
Computer servicesComputer services
- The networks typically run Low Power Devices- Consist of one or more sensors, could be different type of sensors (or actuators)- The networks typically run Low Power Devices- Consist of one or more sensors, could be different type of sensors (or actuators)
![Page 11: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/11.jpg)
11
Key characteristics of IoT devices
Often inexpensive sensors (actuators) equipped with a radio transceiver for various applications, typically low data rate ~ 10-250 kbps.
Deployed in large numbers The sensors should coordinate to perform the desired task. The acquired information (periodic or event-based) is
reported back to the information processing centre (or sometimes in-network processing is required)
Solutions are application-dependent.
11
![Page 12: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/12.jpg)
12
Beyond conventional sensors
Human as a sensor (citizen sensors) e.g. tweeting real world data and/or events
Virtual (software) sensors e.g. Software agents/services
generating/representing data
Road block, A3Road block, A3
Road block, A3Road block, A3
Suggest a different routeSuggest a different route
![Page 13: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/13.jpg)
13
Cyber, Physical and Social Data
![Page 14: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/14.jpg)
14
Citizen Sensors
Source: How Crisis Mapping Saved Lives in Haiti, Ushahidi Haiti Project (UHP).
![Page 15: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/15.jpg)
15
Cosm- Air Quality Egg
![Page 16: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/16.jpg)
16
Cosm- data readings
Tags
Data formats
Location
![Page 17: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/17.jpg)
17
Making Sense of Data
In the next few years, sensor networks will produce 10-20 time the amount of data generated by social media. (source: GigaOmni Media)
![Page 18: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/18.jpg)
18
Things, Data, and lots of it
image courtesy: Smarter Data - I.03_C by Gwen Vanhee
![Page 19: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/19.jpg)
19
Big Data and IoT
"Big data" is a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target, as of 2012 ranging from a few dozen terabytes to many petabytes of data in a single data set.” (wikipedia)
Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. (source IBM)
![Page 20: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/20.jpg)
20
The seduction of data
Turn 12 terabytes of Tweets created each day into sentiment analysis related to different events/occurrences or relate them to products and services.
Convert (billions of) smart meter readings to better predict and balance power consumption.
Analyze thousands of traffic, pollution, weather, congestion, public transport and event sensory data to provide better traffic management.
Monitor patients, elderly care and much more…
Adapted from: What is Bog Data?, IBM
![Page 21: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/21.jpg)
21
Do we need all these data?
![Page 22: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/22.jpg)
22
“Raw data is both an oxymoron and bad data”
Geoff Bowker, 2005
Source: Kate Crawford, "Algorithmic Illusions: Hidden Biases of Big Data", Strata 2013.
![Page 23: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/23.jpg)
23
IoT Data in the Cloud
Image courtesy: http://images.mathrubhumi.comhttp://www.anacostiaws.org/userfiles/image/Blog-Photos/river2.jpg
![Page 24: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/24.jpg)
24
Perceptions and Intelligence
Data
Information
Knowledge
Wisdom
Raw sensory data
Structured data (with semantics)
Abstraction and perceptions
Actionable intelligence
![Page 25: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/25.jpg)
25
Change in communication paradigm
Sinknode Gateway
Core networke.g. InternetCore networke.g. Internet End-userEnd-user
DataData
SenderSender
DataData
ReceiverReceiver
A sample data communication in conventional networksA sample data communication in conventional networks
A sample data communication in WSNA sample data communication in WSN
Fire!Fire! Some bits01100011100Some bits01100011100
![Page 26: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/26.jpg)
26
Collaboration and in-network processing In some applications a single sensor node is not able to handle
the given task or provide the requested information. Instead of sending the information form various source to an
external network/node, the information can be processed in the network itself. e.g. data aggregation, summarisation and then propagating the
processed data with reduced size (hence improving energy efficiency by reducing the amount of data to be transmitted).
Data-centric Conventional networks often focus on sending data between
two specific nodes each equipped with an address. Here what is important is data and the observations and
measurements not the node that provides it.
Required mechanisms
![Page 27: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/27.jpg)
“People want answers, not numbers” (Steven Glaser, UC Berkley)
Sinknode Gateway
Core networke.g. InternetCore networke.g. Internet
What is the temperature at home?What is the temperature at home?Freezing!Freezing!
![Page 28: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/28.jpg)
28
IoT Data alone is not enough
Domain knowledge Machine interpretable meta data Delivery, sharing and representation services Query, discovery, aggregation services Publish, subscribe, notification, and access
interfaces/services
![Page 29: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/29.jpg)
29
Storing, Handling and Processing the Data
Image courtesy: IEEE Spectrum
![Page 30: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/30.jpg)
30
IoT Data Challenges
Discovery: finding appropriate device and data sources Access: Availability and (open) access to IoT resources and
data Search: querying for data Integration: dealing with heterogeneous device, networks
and data Interpretation: translating data to knowledge usable by
people and applications Scalability: dealing with large number of devices and
myriad of data and computational complexity of interpreting the data.
![Page 31: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/31.jpg)
Energy consumption of the nodes
Batteries have small capacity and recharging could be complex (if not impossible) in some cases.
The main consumers of the energy are: the controller, radio, to some extent memory and depending on the type, the sensor(s).
A controller can go to: “active”, “idle” and “sleep”
A radio modem could turn transmitter, receiver, or both on or off,
sensors and memory can be also turned on and off.
![Page 32: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/32.jpg)
Actuators
Stepper Motor [1]
Image credits:[1] http://directory.ac/telco-motion.html[2] http://bruce.pennypacker.org/category/theater/[3] http://www.busytrade.com/products/1195641/TG-100-Linear-Actuator.html[4] http://www.arbworx.com/services/fencing-garden-fencing/
[2]
[3][4]
![Page 33: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/33.jpg)
Wireless Sensor Networks (WSN)- gateway connection
SunSpots
Information channel
Control channel
Directory server
Gateway
Web user/application
![Page 34: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/34.jpg)
Distributed WSN
![Page 35: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/35.jpg)
What are the main issues?
Heterogeneity Interoperability Mobility Energy efficiency Scalability Security
![Page 36: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/36.jpg)
What is important?
Robustness Quality of Service Scalability Seamless integration Security, privacy, Trust
![Page 37: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/37.jpg)
In-network processing
Mobile Ad-hoc Networks are supposed to deliver bits from one end to the other
WSNs, on the other end, are expected to provide information, not necessarily original bits Gives addition options E.g., manipulate or process the data in the network
Main example: aggregation Applying aggregation functions to a obtain an average
value of measurement data Typical functions: minimum, maximum, average, sum,
… Not amenable functions: median
source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor NetworksHolger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
![Page 38: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/38.jpg)
In-network processing- example
Applying Symbolic Aggregate Approximation (SAX)
SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbolsover the original sensor time-series data (green)
![Page 39: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/39.jpg)
Data-centric networking
In typical networks (including ad hoc networks), network transactions are addressed to the identities of specific nodes A “node-centric” or “address-centric” networking paradigm
In a redundantly deployed sensor networks, specific source of an event, alarm, etc. might not be important Redundancy: e.g., several nodes can observe the same area
Thus: focus networking transactions on the data directly instead of their senders and transmitters ! data-centric networking Principal design change
source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor NetworksHolger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
![Page 40: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/40.jpg)
Implementation options for data-centric networking Overlay networks & distributed hash tables (DHT)
Hash table: content-addressable memory Retrieve data from an unknown source, like in peer-to-peer networking – with
efficient implementation Some disparities remain
Static key in DHT, dynamic changes in WSN DHTs typically ignore issues like hop count or distance between nodes when
performing a lookup operation
Publish/subscribe Different interaction paradigm Nodes can publish data, can subscribe to any particular kind of data Once data of a certain type has been published, it is delivered to all
subscribes Subscription and publication are decoupled in time; subscriber and published
are agnostic of each other (decoupled in identity); There is concepts of Semantic Sensor Networks- to annotate sensor
resources and observation and measurement data!
Adapted from: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor NetworksHolger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
![Page 41: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/41.jpg)
IoT and Semantic technologies
The sensors (and in general “Things”) are increasingly being integrated into the Internet/Web.
This can be supported by embedded devices that directly support IP and web-based connection (e.g. 6LowPAN and CoAp) or devices that are connected via gateway components. Broadening the IoT to the concept of “Web of Things”
There are already Sensor Web Enablement (SWE) standards developed by the Open Geospatial Consortium that are widely being adopted in industry, government and academia.
While such frameworks provide some interoperability, semantic technologies are increasingly seen as key enabler for integration of IoT data and broader Web information systems.
![Page 42: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/42.jpg)
Semantics and IoT resources and data Semantics are machine-interpretable metadata (for mark-up),
logical inference mechanisms, query mechanism, linked data solutions
For IoT this means: ontologies for: resource (e.g. sensors), observation and
measurement data (e.g. sensor readings), domain concepts (e.g. unit of measurement, location), services (e.g. IoT services) and other data sources (e.g. those available on linked open data)
Semantic annotation should also supports data represented using existing forms
Reasoning /processing to infer relationships and hierarchies between different resources, data
Semantics (/ontologies) as meta-data (to describe the IoT resources/data) / knowledge bases (domain knowledge).
![Page 43: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/43.jpg)
43
A Few Words on
Semantic Web
![Page 44: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/44.jpg)
SSW Introduction
lives in
has petis ahas pet
Person
Person
Animal
Animal
Concrete Facts Resource Description Framework
Concrete Facts Resource Description Framework
Semantic Web(according to Farside)
General Knowledge Web Ontology Language General Knowledge Web Ontology Language
“Now! – That should clear up a few things around here!”
is a
![Page 45: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/45.jpg)
Semantic Web Stack
![Page 46: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/46.jpg)
Linked Open Data
![Page 47: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/47.jpg)
Linked Open Data
~ 50 Billion Statements~ 50 Billion Statements
![Page 48: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/48.jpg)
SW is moving from academia to industry
![Page 49: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/49.jpg)
In the last few years, we have seen many successes …
Knowledge Graph
Watson
Apple Siri
![Page 50: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/50.jpg)
Google Knowledge Graph
![Page 51: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/51.jpg)
51
Sensors and the Web
![Page 52: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/52.jpg)
Sensors are ubiquitous
![Page 53: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/53.jpg)
Enabling the Internet of Things
Situational awareness enables:
Devices/things to function and adapt within their environment
Devices/things to work together
![Page 54: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/54.jpg)
Sensor systems are too often stovepiped.
Closed centralized management of sensing resources
Closed inaccessible data and sensors
![Page 55: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/55.jpg)
We want to set this data free
With freedom comes responsibilityDiscovery, access, and searchIntegration and interpretationScalability
![Page 56: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/56.jpg)
Drowning in Data
A cross-country flight from New York to Los Angeles on a Boeing 737 plane generates a massive 240 terabytes of data
- GigaOmni Media
![Page 57: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/57.jpg)
Drowning in Data
![Page 58: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/58.jpg)
Challenges
To fulfill this vision, there are difficult challenges to overcome such as the discovery, access, search, integration, and interpretation of sensors and sensor data at scale
Discovery finding appropriate sensing resources and data sources
Access sensing resources and data are open and available
Search querying for sensor data
Integration dealing with heterogeneous sensors and sensor data
Interpretation translating sensor data to knowledge usable by people and
applications
Scalability dealing with data overload and computational complexity
of interpreting the data
![Page 59: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/59.jpg)
Solution
Semantic Sensor WebInternet Computing, July/Aug. 2008
Uses the Web as platform for managing sensor resources and data
Uses semantic technologies for representing data and knowledge, integration, and interpretation
![Page 60: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/60.jpg)
Solution
Discovery, access, and search Using standard Web services
OGC Sensor Web Enablement
![Page 61: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/61.jpg)
Solution
Integration Using shared domain models / data
representation
OGC Sensor Web Enablement
W3C Semantic Sensor Networks
![Page 62: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/62.jpg)
Solution
Interpretation Abstraction – converting low-level data to high-level
knowledge
Machine Perception – w/ prior knowledge and abductive reasoning
IntellegO – Ontology of Perception
![Page 63: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/63.jpg)
Solution
Scalability Data overload – sensors produce too much data
Computational complexity of semantic interpretation
“Intelligence at the edge” – local and distributed integration and interpretation of sensor data
![Page 64: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/64.jpg)
SSW Adoption and Applications
![Page 65: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/65.jpg)
65
Part 2: Data and knowledge modelling requirements,
semantic annotation
Image source: CISCO
![Page 66: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/66.jpg)
Recall of the Internet of Things
A primary goal of interconnecting devices and collecting/processing data from them is to create situation awareness and enable applications, machines, and human users to better understand their surrounding environments.
The understanding of a situation, or context, potentially enables services and applications to make intelligent decisions and to respond to the dynamics of their environments.
Barnaghi et al 2012, “Semantics for the Internet of Things: early progress and back to the future”
![Page 67: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/67.jpg)
IoT challenges
Numbers of devices and different users and interactions required. Challenge: Scalability
Heterogeneity of enabling devices and platforms Challenge: Interoperability
Low power sensors, wireless transceivers, communication, and networking for M2M Challenge: Efficiency in communications
Huge volumes of data emerging from the physical world, M2M and new communications Challenge: Processing and mining the data, Providing secure access and
preserving and controlling privacy. Timeliness of data
Challenge: Freshness of the data and supporting temporal requirements in accessing the data
Ubiquity Challenge: addressing mobility, ad-hoc access and service continuity
Global access and discovery Challenge: Naming, Resolution and discovery
![Page 68: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/68.jpg)
IoT: one paradigm, many visions
Diagram adapted from L. Atzori et al., 2010, “the Internet of Things: a Survey”
![Page 69: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/69.jpg)
Semantic oriented vision
“The object unique addressing and the representation and storing of the exchanged information become the most challenging issues, bringing directly to a ‘‘Semantic oriented”, perspective of IoT”, [Atzori et al., 2010]
Data collected by different sensors and devices is usually multi-modal (temperature, light, sound, video, etc.) and diverse in nature (quality of data can vary with different devices through time and it is mostly location and time dependent [Barnaghi et al, 2012]
some of challenging issues: representation, storage, and search/discovery/query/addressing, and processing IoT resources and data.
![Page 70: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/70.jpg)
What is expected?
Unified access to data: unified descriptions Deriving additional knowledge (data mining) Reasoning support and association to other entities
and resources Self-descriptive data an re-usable knowledge In general: Large-scale platforms to support discovery
and access to the resources, to enable autonomous interactions with the resources, to provide self-descriptive data and association mechanisms to reason the emerging data and to integrate it into the existing applications and services.
![Page 71: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/71.jpg)
Semantic technologies and IoT
There are already Sensor Web Enablement (SWE) standards developed by the Open Geospatial Consortium that are widely adopted.
While such frameworks provide certain levels of interoperability, semantic technologies are seen as key enabler for integration of IoT data and and existing business information systems.
Semantic technologies provide potential support for: Interoperability and machine automation IoT resource and data annotation, logical inference, query
and discovery, linked IoT data
![Page 72: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/72.jpg)
Identify IoT domain concepts
Users Physical entities Virtual entities Devices Resource Services …
Diagram adapted from IoT-A project D2.1
![Page 73: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/73.jpg)
IoT domain concepts - Entity
Physical entities (or entity of interests): objects in the physical world, features of interest that are of interests to users (human users or any digital artifacts). Virtual entities: virtual representation of
the physical entities. Entities are the main focus of interactions
between humans and/or software agents. This interaction is made possible by a
hardware component called Device.Definition adapted from De et al, 2012, “Service modeling for the Internet of Things”
![Page 74: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/74.jpg)
IoT domain concepts – Device, Resource and Service
A Device mediates the interactions between users and entities.
The software component that provides information on the entity or enables controlling of the device, is called a Resource.
A Service provides well-defined and standardised interfaces, offering all necessary functionalities for interacting with entities and related processes.
Definition adapted from De et al, 2012, “Service modeling for the Internet of Things”
![Page 75: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/75.jpg)
Other concepts need to considered
Gateways Directories Platforms Systems Subsystems … Relationships among them And links to existing knowledge base
and linked data
![Page 76: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/76.jpg)
Don’t forget the IoT data
Sensors and devices provide observation and measurement data about the physical world objects which also need to be semantically described and can be related to an event, situation in the physical world.
The processing of data into knowledge/ perception and using it for decision making, automated control, etc.
Huge amount of data from our physical world that need to be Annotated Published Stored (temporary or for longer term) Discovered Accessed Proceeded Utilised in different applications
![Page 77: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/77.jpg)
Semantics for IoT resources and data Semantics are machine-interpretable metadata, logical
inference mechanisms, query and search mechanism, linked data…
For IoT this means: ontologies for: resource (e.g. sensors), observation and
measurement data (e.g. sensor readings), services (e.g. IoT services), domain concepts (e.g. unit of measurement, location) and other data sources (e.g. those available on linked open data)
Semantic annotation should also supports data represented using existing forms
Reasoning/processing to infer relationships between different resources and services, detecting patterns from IoT data
![Page 78: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/78.jpg)
Characteristics of IoT resources
Extraordinarily large number Limited computing capabilities Limited memory Resource constrained environments
(e.g., battery life, signal coverage) Location is important Dynamism in the physical environments Unexpected disruption of services …
![Page 79: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/79.jpg)
Characteristics of IoT data
Stream data (depends on time) Transient nature Almost always related to a phenomenon or
quality in our physical environments Large amount Quality in many situations cannot be
assured (e.g., accuracy and precision) Abstraction levels (e.g., raw, inferred or
derived) …
![Page 80: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/80.jpg)
Utilise semantics
Find all available resources (which can provide data) and data related to “Room A” (which is an object in the linked data)? What is “Room A”? What is its location? returns “location” data What type of data is available for “Room A” or that “location”?
(sensor category types)
Predefined Rules can be applied based on available data (TempRoom_A > 80°C) AND (SmokeDetectedRoom_A position==TRUE)
FireEventRoom_A
Learning these rules needs data mining or pattern recognition techniques
![Page 81: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/81.jpg)
81
Part 3: Semantics and data modelling for IoT
Image source: CISCO
![Page 82: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/82.jpg)
Semantic modelling
Lightweight: experiences show that a lightweight ontology model that well balances expressiveness and inference complexity is more likely to be widely adopted and reused; also large number of IoT resources and huge amount of data need efficient processing
Compatibility: an ontology needs to be consistent with those well designed, existing ontologies to ensure compatibility wherever possible.
Modularity: modular approach to facilitate ontology evolution, extension and integration with external ontologies.
![Page 83: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/83.jpg)
Existing models for resources and data
W3C Semantic Sensor Network Incubator Group’s SSN ontology (mainly for sensors and sensor networks, observation and measurement, and platforms and systems)
Quantity Kinds and Units Used together with the SSN ontology based on QUDV model OMG SysML(TM) Working group of the SysML 1.2 Revision Task
Force (RTF) and W3C Semantic Sensor Network Incubator Group
![Page 84: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/84.jpg)
Existing models for services
OWL-S and WSMO are heavy weight models: practical use?
Minimal service model Deprecated Procedure-Oriented Service Model (POSM) and Resource-
Oriented Service Model (ROSM): two different models for different service technologies
Defines Operations and Messages No profile, no grounding
SAWSDL: mixture of XML, XML schema, RDF and OWL hRESTS and SA-REST: mixture of HTML and reference
to a semantic model; sensor services are not anticipated to have HTML
![Page 85: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/85.jpg)
W3C’S SSN ontology
Diagram adapted from SSN report
![Page 86: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/86.jpg)
Some existing IoT models and ontologies
FP7 IoT-A project’s Entity-Resource-Service ontology A set of ontologies for entities, resources, devices
and services Based on the SSN and OWL-S ontology
FP7 IoT.est project’s service description framework A modular approach for designing a description
framework A set of ontologies for IoT services, testing and
QoS/QoI Technology independent modelling for services
![Page 87: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/87.jpg)
IoT-A resource model
Diagram adapted from IoT-A project D2.1
![Page 88: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/88.jpg)
IoT-A resource description
Diagram adapted from IoT-A project D2.1
![Page 89: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/89.jpg)
IoT-A service model
Diagram adapted from IoT-A project D2.1
![Page 90: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/90.jpg)
IoT-A service description
Diagram adapted from IoT-A project D2.1
![Page 91: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/91.jpg)
Service modelling in IoT.est
Diagrams adapted from Iot.est D3.1
![Page 92: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/92.jpg)
IoT.est service profile highlight
ServiceType class represents the service technologies: RESTful and SOAP/WSDL services.
serviceQos and serviceQoI are defined as subproperty of serviceParameter; they link to concepts in the QoS/QoI ontology.
serviceArea: the area where the service is provided; different from the sensor observation area
Links to the IoT resources through “exposedBy” property
Future extension: serviceNetwork, servicePlatform and
serviceDeployment Service lifecycle, SLA…
![Page 93: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/93.jpg)
93
Part 4: Linked-data and semantic enabled systems
Image source: CISCO
![Page 94: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/94.jpg)
Linked data principles
using URI’s as names for things: Everything is addressed using unique URI’s.
using HTTP URI’s to enable people to look up those names: All the URI’s are accessible via HTTP interfaces.
provide useful RDF information related to URI’s that are looked up by machine or people;
including RDF statements that link to other URI’s to enable discovery of other related concepts of the Web of Data: The URI’s are linked to other URI’s.
![Page 95: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/95.jpg)
Linked data in IoT
Using URI’s as names for things;- URI’s for naming IoT resources and data (and also streaming
data); Using HTTP URI’s to enable people to look up those names;
- Web-level access to low level sensor data and real world resource descriptions (gateway and middleware solutions);
Providing useful RDF information related to URI’s that are looked up by machine or people;- publishing semantically enriched resource and data descriptions
in the form of linked RDF data; Including RDF statements that link to other URI’s to enable discovery
of other related things of the web of data;- linking and associating the real world data to the existing data on
the Web;
![Page 96: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/96.jpg)
Linked data layer for not only IoT…
Images from Stefan Decker, http://fi-ghent.fi-week.eu/files/2010/10/Linked-Data-scheme1.png; linked data diagram: http://richard.cyganiak.de/2007/10/lod/
![Page 97: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/97.jpg)
Creating and using linked sensor data
http://ccsriottb3.ee.surrey.ac.uk:8080/IOTA/
![Page 98: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/98.jpg)
Sensor discovery using linked sensor data
![Page 99: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/99.jpg)
Semantics in IoT - reality
If we create an Ontology our data is interoperable Reality: there are/could be a number of ontologies for a domain
Ontology mapping Reference ontologies Standardisation efforts
Semantic data will make my data machine-understandable and my system will be intelligent. Reality: it is still meta-data, machines don’t understand it but can interpret it.
It still does need intelligent processing, reasoning mechanism to process and interpret the data.
It’s a Hype! Ontologies and semantic data are too much overhead; we deal with tiny devices in IoT. Reality: Ontologies are a way to share and agree on a common vocabulary and
knowledge; at the same time there are machine-interpretable and represented in interoperable and re-usable forms;
You don’t necessarily need to add semantic metadata in the source- it could be added to the data at a later stage (e.g. in a gateway);
![Page 100: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/100.jpg)
100
Part 5: Semantic Sensor Web and
Perception
Image source: semanticweb.com; CISCO
![Page 101: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/101.jpg)
What is the Sensor Web?
Sensor Web is an additional layer connecting sensor networks to the World Wide Web.
Enables an interoperable usage of sensor resources by enabling web based discovery, access, tasking, and alerting.
Enables the advancement of
cyber-physical applications through
improved situation awareness.
![Page 102: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/102.jpg)
Why is the Sensor Web important?
In general Enable tight coupling of the cyber and
physical world
In relation to IoT Enable shared situation awareness (or
context) between devices/things
![Page 103: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/103.jpg)
Bridging the Cyber-Physical Divide
Psyleron’s Mind-Lamp (Princeton U), connections between the mind and the physical world.
Neuro Sky's mind-controlled headset to play a video game.
MIT’s Fluid Interface Group: wearable device with a projector for deep interactions with the environment
![Page 104: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/104.jpg)
Bridging the Cyber-Physical Divide
Foursquare is an online application which integrates a persons physical location and social network.
Community of enthusiasts that share experiences of self-tracking and measurement.
FitBit Community allows the automated collection and sharing of health-related data, goals, and achievements
![Page 105: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/105.jpg)
Bridging the Cyber-Physical Divide
Tweeting Sensorssensors are becoming social
![Page 106: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/106.jpg)
How do we design the Sensor Web?
Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Semantic Annotation
Interpretation through integration of heterogeneous data and reasoning with prior knowledge
Semantic Perception/Abstraction Linked Open Data as prior knowledge
Scale through distributed local interpretation
“intelligence at the edge”
![Page 107: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/107.jpg)
OGC Sensor Web Enablement
![Page 108: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/108.jpg)
Role of OGC SWE
![Page 109: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/109.jpg)
Vision of Sensor Web
Quickly discover sensors (secure or public) that can meet my needs – location, observables, quality, ability to task
Obtain sensor information in a standard encoding that is understandable by me and my software
Readily access sensor observations in a common manner, and in a form specific to my needs
Task sensors, when possible, to meet my specific needs
Subscribe to and receive alerts when a sensor measures a particular phenomenon
![Page 110: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/110.jpg)
Principles of Sensor Web
Sensors will be web accessible
Sensors and sensor data will be discoverable
Sensors will be self-describing to humans and
software (using a standard encoding)
Most sensor observations will be easily accessible
in real time over the web
![Page 111: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/111.jpg)
OGC SWE Services
Sensor Observation Service (SOS) access sensor information (SensorML) and sensor
observations (O&M
Sensor Planning Service (SPS) task sensors or sensor systems
Sensor Alert Service (SAS) asynchronous notification of sensor events (tasks,
observation of phenomena)
Sensor Registries discovery of sensors and sensor data
![Page 112: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/112.jpg)
OGC SWE Services
![Page 113: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/113.jpg)
OGC SWE Languages
Sensor Model Language (SensorML)
Models and schema for describing sensor
characteristics
Observation & Measurement (O&M)
Models and schema for encoding sensor
observations
![Page 114: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/114.jpg)
OCG SWE Observation
![Page 115: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/115.jpg)
![Page 116: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/116.jpg)
Semantic Sensor Web
RDF OWL
OGC Sensor Web Enablement
![Page 117: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/117.jpg)
Sensor Web + Semantic Web
Semantic Web
The web of data where web content is processed by machines, with human actors at the end of the chain.
The web as a huge, dynamic, evolving database of facts, rather than pages, that can be interpreted and presented in many ways (mashups).
Fundamental importance of ontologies to describe the fact that represents the data. RDF(S) emphasises labelled links as the source of meaning: essentially a graph model . A label (URI) uniquely identifies a concept.
OWL emphasises inference as the source of meaning: a label also refers to a package of logical axioms with a proof theory.
Usually, the two notions of meaning fit.
Goal to combine information and services for targeted purpose and new knowledge
Sensor Web
The internet of things made up of Wireless Sensor Networks, RFID, stream gauges, orbiting satellites, weather stations, GPS, traffic sensors, ocean buoys, animal and fish tags, cameras, habitat monitors, recording data from the physical world.
Today there are 4 billion mobile sensing devices plus even more fixed sensors. The US National Research Council predicts that this may grow to trillions by 2020, and they are increasingly connected by internet and Web protocols.
Record observations of a wide variety of modalities: but a big part is time-series‟ of numeric measurements.
The Open Geospatial Consortium has some web-service standards for shared data access (Sensor Web Enablement).
Goal is to open up access to real-time and archival data, and to combine in applications.
![Page 118: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/118.jpg)
So, what is a Semantic Sensor Web?
Reduce the difficulty and open up sensor networks by:
Allowing high-level specification of the data collection process;
Across separately deployed sensor networks; Across heterogeneous sensor types; and Across heterogeneous sensor network platforms; Using high-level descriptions of sensor network
capability; and Interfacing to data integration methods using similar
query and capability descriptions.
To create a Web of Real Time Meaning!
![Page 119: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/119.jpg)
W3C SSN Incubator Group
SSN-XG commenced: 1 March 2009
Chairs: Amit Sheth, Kno.e.sis Center, Wright State University Kerry Taylor, CSIRO Amit Parashar Holger Neuhaus Laurent Lefort, CSIRO
Participants: 39 people from 20 organizations, including: Universities in: US, Germany, Finland, Spain, Britain,
Ireland Multinationals: Boeing, Ericsson Small companies in semantics, communications,
software Research institutes: DERI (Ireland), Fraunhofer
(Germany), ETRI (Korea), MBARI (US), SRI International (US), MITRE (US), US Defense, CTIC (Spain), CSIRO (Australia), CESI (China)
![Page 120: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/120.jpg)
W3C SSN Incubator Group
Two main objectives:
The development of an ontology for describing sensing resources and data, andThe extension of the SWE languages to support semantic annotations.
![Page 121: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/121.jpg)
Sensor Standards Landscape
![Page 122: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/122.jpg)
SSN Ontology
OWL 2 DL ontology
Authored by the XG participants
Edited by Michael Compton
Driven by Use Cases
Terminology carefully tracked to sources through annotation properties
Metrics Classes: 117 Properties: 148 DL Expressivity:
SIQ(D)
SSN Ontology – http://purl.oclc.org/NET/ssnx/ssn
![Page 123: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/123.jpg)
SSN Use Cases
![Page 124: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/124.jpg)
SSN Use Cases
![Page 125: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/125.jpg)
SSN Ontology
![Page 126: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/126.jpg)
Stimulus-Sensor-Observation
The SSO Ontology Design Pattern is developed following the principle of minimal ontological commitments to make it reusable for a variety of application areas.
Introduces a minimal set of classes and relations centered around the notions of stimuli, sensor, and observations. Defines stimuli as the (only) link to the physical environment.
Empirical science observes these stimuli using sensors to infer information about environmental properties and construct features of interest.
![Page 127: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/127.jpg)
SSN Ontology Modules
![Page 128: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/128.jpg)
SSN Ontology Modules
![Page 129: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/129.jpg)
SSN Sensor
A sensor can do (implements) sensing: that is, a sensor is any entity that can follow a sensing method and thus observe some Property of a FeatureOfInterest.
Sensors may be physical devices, computational methods, a laboratory setup with a person following a method, or any other thing that can follow a Sensing Method to observe a Property.
![Page 130: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/130.jpg)
SSN Measurement Capability
Collects together measurement properties (accuracy, range, precision, etc) and the environmental conditions in which those properties hold, representing a specification of a sensor's capability in those conditions.
![Page 131: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/131.jpg)
SSN Observation
An Observation is a Situation in which a Sensing method has been used to estimate or calculate a value of a Property.
Links to Sensing and Sensor describe what made the Observation and how; links to Property and Feature detail what was sensed; the result is the output of a Sensor; other metadata gives the time(s) and the quality.
Different from OGC’s O&M, in which an “observation” is an act or event, although it also provides the record of the event.
![Page 132: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/132.jpg)
Alignment with DOLCE
![Page 133: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/133.jpg)
What SSN does not model
Sensor types and models
Networks: communication, topology
Representation of data and units of measurement
Location, mobility or other dynamic behaviours
Animate sensors
Control and actuation
….
![Page 134: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/134.jpg)
Semantic Annotation of SWE
Recommended technique via Xlink attributes requires no change to SWE
xlink:href - link to ontology individual
xlink:role - link to ontology class
xlink:arcrole - link to ontology object property
![Page 135: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/135.jpg)
How do we design the Sensor Web?
Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Semantic Annotation
Interpretation through integration of heterogeneous data and reasoning with prior knowledge
Semantic Perception/Abstraction Linked Open Data as prior knowledge
Scale through distributed local interpretation
“intelligence at the edge”
![Page 136: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/136.jpg)
136
Interpretation of data
A primary goal of interconnecting devices and collecting/processing data from them is to create situation awareness and enable applications, machines, and human users to better understand their surrounding environments.
The understanding of a situation, or context, potentially enables services and applications to make intelligent decisions and to respond to the dynamics of their environments.
![Page 137: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/137.jpg)
137
Observation and measurement data
Source: W3C Semantic Sensor Networks, SSN Ontology presentation, Laurent Lefort et al.
![Page 138: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/138.jpg)
138
How to say what a sensor is and what it measures?
Sinknode
Gateway
![Page 139: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/139.jpg)
139
Data/Service description frameworks
There are standards such as Sensor Web Enablement (SWE) set developed by the Open Geospatial Consortium that are widely being adopted in industry, government and academia.
While such frameworks provide some interoperability, semantic technologies are increasingly seen as key enabler for integration of IoT data and broader Web information systems.
![Page 140: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/140.jpg)
140
Sensor Markup Language (SensorML)
Source: http://www.mitre.org/
![Page 141: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/141.jpg)
141
W3C SSN Ontology
makes observations of this type
Where it is
What it measures
units
SSN-XG ontologies
SSN-XG annotations
SSN-XG Ontology Scope
![Page 142: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/142.jpg)
142
Semantics and IoT data
Creating ontologies and defining data models is not enough tools to create and annotate data data handling components
Complex models and ontologies look good, but design lightweight versions for constrained environments think of practical issues make it as compatible as possible and/or link it to the other
existing ontologies Domain knowledge and instances
Common terms and vocabularies Location, unit of measurement, type, theme, …
Link it to other resources Linked-data URIs and naming
![Page 143: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/143.jpg)
143
Semantics and sensor data
Source: W. Wang, P. Barnaghi, "Semantic Annotation and Reasoning for Sensor Data", In proceedings of the 4th European Conference on Smart Sensing and Context (EuroSSC2009), 2009.
![Page 144: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/144.jpg)
144
Semantics and Linked-data
The principles in designing the linked data are defined as: using URI’s as names for things; using HTTP URI’s to enable people to look up
those names; provide useful RDF information related to
URI’s that are looked up by machine or people;
including RDF statements that link to other URI’s to enable discovery of other related concepts of the Web of Data;
![Page 145: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/145.jpg)
145
Linked Sensor data
![Page 146: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/146.jpg)
146
Myth and reality
#1: If we create an Ontology our data is interoperable Reality: there are/could be a number of ontologies for a domain
Ontology mapping Reference ontologies Standardisation efforts
#2: Semantic data will make my data machine-understandable and my system will be intelligent. Reality: it is still meta-data, machines don’t understand it but can interpret it. It still
does need intelligent processing, reasoning mechanism to process and interpret the data.
#3: It’s a Hype! Ontologies and semantic data are too much overhead; we deal with tiny devices in IoT. Reality: Ontologies are a way to share and agree on a common vocabulary and
knowledge; at the same time there are machine-interpretable and represented in interoperable and re-usable forms;
You don’t necessarily need to add semantic metadata in the source- it could be added to the data at a later stage (e.g. in a gateway);
Legacy applications can ignore it or to be extended to work with it.
![Page 147: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/147.jpg)
147
Processing Streaming Sensor Data
![Page 148: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/148.jpg)
148148
Symbolic Aggregate Approximation (SAX)
Variable String Length and Vocabulary size.
Length: 10, VocSize: 10 Length: 10, VocSize: 4
“gijigdbabd” “cdddcbaaab”
Green Curve: consists of 100 Samples, Blue Curve: SAX
![Page 149: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/149.jpg)
149
SAX representation
SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbolsover the original sensor time-series data (green)
P. Barnaghi, F. Ganz, C. Henson, A. Sheth, "Computing Perception from Sensor Data", in Proc. of the IEEE Sensors 2012, Oct. 2012.
fggfffhfffffgjhghfff
jfhiggfffhfffffgjhgi
fggfffhfffffgjhghfff
![Page 150: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/150.jpg)
150
Data Processing Framework
fggfffhfffffgjhghfff dddfffffffffffddd cccddddccccdddccc aaaacccaaaaaaaaccccdddcdcdcdcddasddd
PIR Sensor Light Sensor Temperature Sensor
Raw sensor data stream
Raw sensor data stream
Raw sensor data stream
Attendance
PhoneHot Temperature
Cold Temperature
Bright
Day-time
Night-time
Office room BA0121
On going meeting
Window has been left open
….
Temporal data(extracted from SSN descriptions)
Spatial data(extracted from SSN descriptions)
Thematic data(low level abstractions)
Parsimonious Covering Theory
Observations
Perceptions
Domain knowledge
SAX Patterns
Raw Sensor Data(Annotated with SSN Ontology)
…
….
Perception Computation
High-level Perceptions
![Page 151: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/151.jpg)
151
SensorSAX
F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
![Page 152: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/152.jpg)
152
Evaluation results of abstraction creation
F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
![Page 153: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/153.jpg)
153
Data size reduction
F. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
![Page 154: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/154.jpg)
154
Enabling the Internet of Things
- Diversity range of applications- Interacting with large number of devices with various types-Multiple heterogeneous networks-Deluge of data-Processing and interpretation of the IoT data
![Page 155: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/155.jpg)
155
Challenges and opportunities
Providing infrastructure Publishing, sharing, and access solutions on a global scale Indexing and discovery (data and resources) Aggregation and fusion Trust, privacy and security Data mining and creating actionable knowledge
Integration into services and applications in e-health, the public sector, retail, manufacturing and personalized apps. Mobile apps, location-based services, monitoring control etc.
New business models
![Page 156: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/156.jpg)
156
Part 6: Cognitive aspects of knowledge representation, reasoning and perception
Image source: CISCO
![Page 157: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/157.jpg)
Abstraction provides the ability to interpret and synthesize information in a way that affords effective understanding and communication of ideas, feelings, perceptions, etc. between machines and people.
Abstraction
![Page 158: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/158.jpg)
People are excellent at abstraction; of sensing and interpreting stimuli to understand and interact with the world.
The process of interpreting stimuli is called perception; and studying this extraordinary human capability can lead to insights for developing effective machine perception.
Abstraction
![Page 159: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/159.jpg)
observe perceive
conceptualizationof “real-world”
“real-world”
Abstraction
![Page 160: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/160.jpg)
Semantic Perception/Abstraction
Fundamental Questions
What is perception, and how can we design machines to perceive?
What can we learn from cognitive models of perception?
Is the Semantic Web up to the task of modeling perception?
![Page 161: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/161.jpg)
What is Perception?
Perception is the act of
Abstracting
Explaining
Discriminating
Choosing
![Page 162: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/162.jpg)
What can we learn from Cognitive Models of Perception?
A-priori background knowledge is a key enabler
Perception is a cyclical, active process
Ulric Neisser (1976)Ulric Neisser (1976) Richard Gregory (1997)Richard Gregory (1997)
![Page 163: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/163.jpg)
Is Semantic Web up to the task of modeling perception?
RepresentationHeterogeneous sensors, sensing, and observation recordsBackground knowledge (observable properties, objects/events, etc.)
InferenceExplain observations (hypothesis building)Focus attention by seeking additional stimuli (that discriminate between explanations)
Difficult Issues to OvercomePerception is an inference to the best explanationHandle streaming dataReal-time processing (or nearly)
![Page 164: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/164.jpg)
Both people and machines are capable of observing qualities, such as redness.
* Formally described in a sensor/ontology (SSN ontology)
observesObserver
Quality
![Page 165: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/165.jpg)
The ability to perceive is afforded through the use of background knowledge, relating observable qualities to entities in the world.
* Formally described in domain ontologies
(and knowledge bases)
inheres in
Quality
Entity
![Page 166: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/166.jpg)
With the help of sophisticated inference, both people and machines are also capable of perceiving entities, such as apples.
the ability to degrade gracefully with incomplete
information
the ability to minimize explanations based on new
information
the ability to reason over data on the Web
fast (tractable)
perceivesEntity
Perceiver
![Page 167: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/167.jpg)
Perceptual Inference
minimizeexplanations
degrade gracefully
tractable
Abductive Logic (e.g., PCT)
high complexity
Deductive Logic (e.g., OWL)
(relatively) low complexity
Web reasoning
Perceptual Inference(i.e., abstraction)
![Page 168: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/168.jpg)
The ability to perceive efficiently is afforded through the cyclical exchange of information between observers and perceivers.
Traditionally called the Perceptual Cycle
(or Active Perception)
sendsfocus
sends observation
Observer
Perceiver
![Page 169: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/169.jpg)
Neisser’s Perceptual Cycle
![Page 170: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/170.jpg)
1970’s – Perception is an active, cyclical process of exploration and interpretation.
- Nessier’s Perception Cycle
1980’s – The perception cycle is driven by background knowledge in order to generate and test hypotheses.
- Richard Gregory (optical illusions)
1990’s – In order to effectively test hypotheses, some observations are more informative than others.
- Norwich’s Entropy Theory of Perception
Cognitive Theories of Perception
![Page 171: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/171.jpg)
Key InsightsBackground knowledge plays a crucial role in perception; what we know (or think we know/believe) influences our perception of the world.Semantics will allow us to realize computational models of perception based on background knowledge.
Contemporary IssuesInternet/Web expands our background knowledge to a global scope; thus our perception is global in scopeSocial networks influence our knowledge and beliefs, thus influencing our perception
![Page 172: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/172.jpg)
observes
inheres in
Integrated together, we have an general model – capable of abstraction – relating observers, perceivers, and background knowledge.
perceives
sendsfocus
sends observation
Observer
Quality
EntityPerceive
r
![Page 173: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/173.jpg)
Ontology of Perception – as an extension of SSN
Provides abstraction of sensor data through perceptual inference of semantically annotated data
![Page 174: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/174.jpg)
Prior Knowledge
W3C SSN Ontology Bi-partite Graph
Prior knowledge conformant to SSN ontology (left), structured as a bipartite graph (right)
![Page 175: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/175.jpg)
Explanation is the act of accounting for sensory observations (i.e., abstraction); often referred to as hypothesis building.
Observed Property: A property that has been observed.
ObservedProperty ≡ ∃ssn:observedProperty—.{o1} ⊔ … ⊔ ∃ssn:observedProperty—.{on} Explanatory Feature: A feature that explains the set of observed properties.
ExplanatoryFeature ≡ ∃ssn:isPropertyOf—.{p1} ⊓ … ⊓ ∃ssn:isPropertyOf—.{pn}
Semantics of Explanation
![Page 176: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/176.jpg)
ExampleAssume the properties elevated blood pressure and palpitations have been observed, and encoded in RDF (conformant with SSN): ssn:Observation(o1), ssn:observedProperty(o1, elevated blood pressure)ssn:Observation(o2), ssn:observedProperty(o2, palpitations) Given these observations, the following ExplanatoryFeature class is constructed:
ExplanatoryFeature ≡ ∃ssn:isPropertyOf—.{elevated blood pressure} ⊓ ∃ssn:isPropertyOf—.{palpitations}
Given the KB, executing the query ExplanatoryFeature(?y) can infer the features, Hypertension and Hyperthyroidism, as explanations:
ExplanatoryFeature(Hypertension) ExplanatoryFeature(Hyperthyroidism)
Semantics of Explanation
![Page 177: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/177.jpg)
Discrimination is the act of deciding how to narrow down the multitude of explanatory features through further observation.
Expected Property: A property is expected with respect to (w.r.t.) a set of features if it is a property of every feature in the set.
ExpectedProperty ≡ ∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ∃ssn:isPropertyOf.{fn}
NotApplicable Property: A property is not-applicable w.r.t. a set of features if it is not a property of any feature in the set.
NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ¬∃ssn:isPropertyOf.{fn}
Discriminating Property: A property is discriminating w.r.t. a set of features if it is neither expected nor not-applicable.
DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty
Semantics of Discrimination
![Page 178: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/178.jpg)
Example Given the explanatory features from the previous example, Hypertension and Hyperthyroidism, the following classes are constructed:
ExpectedProperty ≡ ∃ssn:isPropertyOf.{Hypertension} ⊓ ∃ssn:isPropertyOf.
{Hyperthyroidism} NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{Hypertension} ⊓ ¬∃ssn:isPropertyOf.{Hyperthyroidism} Given the KB, executing the query DiscriminatingProperty(?x) can infer the property clammy skin as discriminating: DiscriminatingProperty(clammy skin)
Semantics of Discrimination
![Page 179: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/179.jpg)
How do we design the Sensor Web?
Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Semantic Annotation
Interpretation through integration of heterogeneous data and reasoning with prior knowledge
Semantic Perception/Abstraction Linked Open Data as prior knowledge
Scale through distributed local interpretation
“intelligence at the edge”
![Page 180: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/180.jpg)
Efficient Algorithms for IntellegO
Use of OWL-DL reasoner too resource-intensive for use in resource constrained devices (such as sensor nodes, mobile phones, IoT devices)
Runs out of resources for problem size (prior knowledge) > 20 concepts
Asymptotic complexity: O(n3) [Experimentally determined]
To enable their use on resource-constrained devices, we now describe algorithms for efficient inference of explanation and discrimination.
These algorithms use bit vector encodings and operations, leveraging a-priori knowledge of the environment.
![Page 181: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/181.jpg)
Efficient Algorithms for IntellegO
Semantic (RDF) Encoding
Bit Vector Encoding
Lower
Lift
First, developed lifting and lowering algorithms to translate between RDF and bit vector encodings of observations.
![Page 182: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/182.jpg)
Efficient Algorithms for IntellegO
Explanation Algorithm
Discrimination Algorithm
Utilize bit vector operators to efficiently compute explanation and discrimination
Explanation: Use of the bit vector AND operation to discover and dismiss those features that cannot explain the set of observed properties
Discrimination: Use of the bit vector AND operation to discover and indirectly assemble those properties that discriminate between a set of explanatory features. The discriminating properties are those that are determined to be neither expected nor not-applicable
![Page 183: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/183.jpg)
Efficient Algorithms for IntellegO
Evaluation: The bit vector encodings and algorithms yield significant and necessary computational enhancements – including asymptotic order of magnitude improvement, with running times reduced from minutes to milliseconds, and problem size increased from 10’s to 1000’s.
![Page 184: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/184.jpg)
Adoption of SSN
![Page 185: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/185.jpg)
185
Part 7: Physical-Cyber-Social Computing
Image source: CISCO
![Page 186: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/186.jpg)
186
J. McCarth
y
J. McCarth
y
M. Weiser
M. Weiser
D. Engelba
rt
D. Engelba
rt
J. C. R. LickliderJ. C. R.
Licklider
Similar Visions of Computing
![Page 187: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/187.jpg)
187
If people do not believe that mathematics is simple, it is only because they do not realize how
complicated life is.
- John von Neumann
If people do not believe that mathematics is simple, it is only because they do not realize how
complicated life is.
- John von NeumannComputational paradigms have always
dealt with a simplified representations of the real-world…
Computational paradigms have always dealt with a simplified representations of
the real-world…
Algorithms work on these simplified representations
Algorithms work on these simplified representations
Solutions from these algorithms are transcended back to the real-world by
humans as actions
Solutions from these algorithms are transcended back to the real-world by
humans as actions
http://ngs.ics.uci.edu/blog/?p=1501
Grand challenges in the real-world are complex
![Page 188: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/188.jpg)
What has changed now?
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
There are more devices connected to the internet thanthe entire human population.There are more devices connected to the internet thanthe entire human population.
Today we have around 10 billion devices connected to the internet making it an era of IoT (Internet of Things)Today we have around 10 billion devices connected to the internet making it an era of IoT (Internet of Things)
http://www.cisco.com/web/about/ac79/docs/innov/IoT_IBSG_0411FINAL.pdf
![Page 189: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/189.jpg)
189
What has not changed?
What has not changed?
We need computational paradigms to tap into the rich pulse of the
human populace
We need computational paradigms to tap into the rich pulse of the
human populace
We are still working on the simpler representations of the real-world!
We are still working on the simpler representations of the real-world!
Represent, capture, and compute with richer and fine-grained representations of
real-world problems
Represent, capture, and compute with richer and fine-grained representations of
real-world problems
What should change?What should change?
![Page 190: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/190.jpg)
PCS Computing: Asthma Scenario
190
Sensordrone – for monitoring environmental air quality
Wheezometer – for monitoringwheezing sounds
Can I reduce my asthma attacks at night?
What are the triggers?What is the wheezing level?
What is the propensity toward asthma?
What is the exposure level over a day?
What is the air quality indoors?
Commute to Work
Personal
Public Health
Population Level
Closing the window at homein the morning and taking analternate route to office may
lead to reduced asthma attacks
Actionable Informatio
n
Actionable Informatio
n
![Page 191: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/191.jpg)
Personal, Public Health, and Population Level Signals for Monitoring Asthma
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist ; *consider referral to specialist
Asthma Control and Actionable Information
Sensors and their observations for understanding asthma
![Page 192: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/192.jpg)
Asthma Early Warning Model
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
192
Personal Level
Signals
Personal Level
Signals
Societal Level Signals
Societal Level Signals
(Personal Level Signals)
(Personalized Societal Level Signal)
(Societal Level Signals)
Societal Level Signals Relevant to the Personal
Level
Societal Level Signals Relevant to the Personal
Level
Personal Level Sensors
(mHealth**)
QualifyQualify QuantifyQuantify
Action Recommendati
on
Action Recommendati
on
What are the features influencing my asthma?What is the contribution of each of these features?
How controlled is my asthma? (risk score)What will be my action plan to manage asthma?
StorageStorage
Societal Level Sensors
Asthma Early Warning Model (AEWM)
Query AEWM
Verify & augmentdomain knowledge
Recommended Action
Action Justification
*http://www.slideshare.net/jain49/eventshop-120721, ** http://www.youtube.com/watch?v=btnRi64hJp4
(EventShop*)
![Page 193: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/193.jpg)
Health Signal Extraction to Understanding
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
Population Level
Personal
Wheeze – YesDo you have tightness of chest? –Yes
Observations Physical-Cyber-Social System Health Signal Extraction Health Signal Understanding
<Wheezing=Yes, time, location>
<ChectTightness=Yes, time, location>
<PollenLevel=Medium, time, location>
<Pollution=Yes, time, location>
<Activity=High, time, location>
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
RiskCategory
<PollenLevel, ChectTightness, Pollution,Activity, Wheezing, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory>
.
.
.
Expert Knowledge
Background Knowledge
tweet reporting pollution level and asthma attacks
Acceleration readings fromon-phone sensors
Sensor and personal observations
Signals from personal, personal spaces, and community spaces
Risk Category assigned by doctors
Qualify
Quantify
Enrich
Outdoor pollen and pollution
Public Health
Well Controlled - continueNot Well Controlled – contact nursePoor Controlled – contact doctor
![Page 194: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/194.jpg)
194
J. McCarth
y
J. McCarth
y
M. Weiser
M. Weiser
D. Engelba
rt
D. Engelba
rt
J. C. R. LickliderJ. C. R.
Licklider
![Page 195: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/195.jpg)
195
Part 8: IoT/PCS Systems: Applications
Image source: CISCO
![Page 196: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/196.jpg)
SSN Applications
Applications of SSN
HealthcareWeather Rescue
Traffic Fire Fighting Logistics
![Page 197: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/197.jpg)
SSN Application: Weather
50% savings in sensing resource requirements during the detection of a blizzard
Order of magnitude resource savings between storing observations vs. relevant abstractions
![Page 198: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/198.jpg)
Linked Sensor Data
Linked Sensor Data(~2 Billion Statements)
![Page 199: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/199.jpg)
Sensor Discovery Application
Query w/ location name to find nearby sensors
![Page 200: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/200.jpg)
Real-Time Feature Streams
Demo: http://www.youtube.com/watch?v=_ews4w_eCpg
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
![Page 201: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/201.jpg)
SSN Application: Fire Detection
Weather Application
SECURE: Semantics-empowered Rescue Environment(detect different types of fires)
DEMO: http://www.youtube.com/watch?v=in2KMkD_uqg
![Page 202: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/202.jpg)
SSN Application: Health Care
MOBILEMD: Mobile app to help reduce re-admission of patients with Chronic Heart Failure
![Page 203: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/203.jpg)
SSN Application: Health Care
Passive Monitoring Phase Passive Monitoring Phase
• Abnormal heart rate• Clammy skin
• Panic Disorder• Hypoglycemia• Hyperthyroidism• Heart Attack• Septic Shock
Observed Symptoms Possible Explanations
Passive Sensors – heart rate, galvanic skin response
![Page 204: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/204.jpg)
SSN Application: Health Care
Active Monitoring Phase Active Monitoring Phase
Are you feeling lightheaded?Are you feeling lightheaded?
Are you have trouble taking deep breaths?
Are you have trouble taking deep breaths?
yesyes
yesyes
Have you taken your Methimazole medication?
Have you taken your Methimazole medication?
Do you have low blood pressure?Do you have low blood pressure?
yesyes
• Abnormal heart rate• Clammy skin• Lightheaded• Trouble breathing• Low blood pressure
• Panic Disorder• Hypoglycemia• Hyperthyroidism• Heart Attack• Septic Shock
Observed Symptoms Possible Explanations
nono
Active Sensors – blood pressure, weight scale, pulse oxymeter Demo: http://www.youtube.com/watch?v=btnRi64hJp4
![Page 205: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/205.jpg)
Domain ExpertsDomain Experts
ColdWeatherColdWeather
PoorVisibilityPoorVisibility
SlowTrafficSlowTraffic
IcyRoadIcyRoad
Declarative domain knowledge
Causal knowledge
Linked Open Data
ColdWeather(YES/NO)IcyRoad (ON/OFF) PoorVisibility (YES/NO)SlowTraffic (YES/NO)
1 0 1 1 1 1 1 0 1 1 1 1 1 0 1 0
Domain ObservationsDomain Observations
Domain KnowledgeDomain Knowledge
Structure and parameters
205
WinterSeasonWinterSeason
Otherknowledge
Correlations to causations using Declarative knowledge on the Semantic Web
kTraffic
![Page 206: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/206.jpg)
Traffic jam
Traffic jam
Link Descriptio
n
Link Descriptio
nScheduled
EventScheduled
Event
traffic jamtraffic jam
baseball gamebaseball game
Add missing random variables
Time of day
Time of day
bad weather CapableOf slow traffic
bad weatherbad weather
Traffic data from sensors deployed on road network in San Francisco
Bay Area
time of daytime of day
traffic jamtraffic jam
baseball gamebaseball gametime of daytime of day
slow trafficslow traffic
Three Operations: Complementing graphical model structure extraction
Add missing links bad weatherbad weather
traffic jamtraffic jam
baseball gamebaseball gametime of daytime of day
slow trafficslow traffic
Add link directionbad weatherbad weather
traffic jamtraffic jam
baseball gamebaseball gametime of daytime of day
slow trafficslow traffic
go to baseball game Causes traffic jam
Knowledge from ConceptNet5
traffic jam CapableOfoccur twice each daytraffic jam CapableOf slow traffic
206
![Page 207: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/207.jpg)
207
Scheduled EventScheduled Event
Active EventActive Event
Day of weekDay of week Time of
dayTime of day
delaydelay
Travel timeTravel time
speedspeed
volumevolume
Structure extracted formtraffic observations (sensors + textual) using statistical techniques
Scheduled EventScheduled Event
Active EventActive Event
Day of weekDay of week
Time of dayTime of day
delaydelayTravel
timeTravel time
speedspeed
volumevolume
Bad WeatherBad Weather
Enriched structure which has link directions and new nodes such as “Bad Weather” potentially leading to better delay predictions
kTraffic
![Page 208: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/208.jpg)
SemMOB
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
First responders have limited time to analyzesensor (on and around them) observations. First responders have limited time to analyzesensor (on and around them) observations.
Fire fighters need to combine prior knowledge of fires and its behavior, extinguishers, and floorplan of the building for rescue strategy and operations.
Fire fighters need to combine prior knowledge of fires and its behavior, extinguishers, and floorplan of the building for rescue strategy and operations.
There are a variety of sensors used to monitorvitals of firefighters, location, and poisonous gases.There are a variety of sensors used to monitorvitals of firefighters, location, and poisonous gases.
O2O2
Heart rateHeart rate
COCO
CO2CO2
GPSGPS
AccelerometerAccelerometer CompassCompass
FootstepsFootsteps
There is a team of them makingit further difficult for analysisThere is a team of them makingit further difficult for analysis
Semantic Web allow us to describe the domain, sensors, andfirst responders -- apply reasoning techniques to derive actionableinsights
Semantic Web allow us to describe the domain, sensors, andfirst responders -- apply reasoning techniques to derive actionableinsights
Sensors on each responder provides different view of the event and they needto register dynamically as the firefightersarrive.
Sensors on each responder provides different view of the event and they needto register dynamically as the firefightersarrive.
![Page 209: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/209.jpg)
SemMOB
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
PlatformPlatform SensingDevice
SensingDevice
System_AGTC1System_AGTC1
Sensor_LSM11Sensor_LSM11
SensorOutput
SensorOutput
O2Concentraction_outputO2Concentraction_output
MeasurementCapabili
ty
MeasurementCapabili
ty
MeasurementCapability_LSM11
MeasurementCapability_LSM11
Property
Property
concentrationconcentration
onPlatform
hasOutput
hasMeasurementCapability
Observes
Observation
Observation
O2ConcentrationO2Concentration
ObservedProperty
O2Obs_15052012O2Obs_15052012
observationResultTimehttp://geonames.org/6298640http://geonames.org/6298640
hasLocation
name
xsd:floatxsd:float
latlong
Dayton, Dayton-Wright Brothers Airport^^xsd:string
FeatureFeature
Demo: http://www.youtube.com/watch?v=nX11OqgtQlw
![Page 210: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/210.jpg)
LOKILAS (LOst Key Identification, Localization and Alert System)
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
![Page 211: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/211.jpg)
LOKILAS
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
![Page 212: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/212.jpg)
LOKILAS
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS 2013, Madrid, Spain
![Page 213: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/213.jpg)
Future work
Creating ontologies and defining data models are not enough tools to create and annotate data Tools for publishing linked IoT data
Designing lightweight versions for constrained environments think of practical issues make it as much as possible compatible and/or link it to
the other existing ontologies Linking to domain knowledge and other resources
Location, unit of measurement, type, theme, … Linked-data URIs and naming
![Page 214: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/214.jpg)
Some of the open issues
Efficient real-time IoT resource/service query/discovery Directory Indexing
Abstraction of IoT data Pattern extraction Perception creation
IoT service composition and compensation Integration with existing Web services Service adaptation
![Page 215: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/215.jpg)
Selected references
Payam Barnaghi, Wei Wang, Cory Henson, Kerry Taylor, "Semantics for the Internet of Things: early progress and back to the future", (to appear) International Journal on Semantic Web and Information Systems (special issue on sensor networks, Internet of Things and smart devices), 2012.
Atzori, L., Iera, A. & Morabito, G. , “The Internet of Things: A survey”, Computer Networks, Volume 54, Issue 15, 28 October 2010, 2787-2805.
Suparna De, Tarek Elsaleh, Payam Barnaghi , Stefan Meissner, "An Internet of Things Platform for Real-World and Digital Objects", Journal of Scalable Computing: Practice and Experience, vol 13, no.1, 2012.
Suparna De, Payam Barnaghi, Martin Bauer, Stefan Meissner, "Service modelling for the Internet of Things", in Proceedings of the Conference on Computer Science and Information Systems (FedCSIS), pp.949-955, Sept. 2011.
Cory Henson, Amit Sheth, and Krishnaprasad Thirunarayan, “Semantic Perception: Converting Sensory Observations to Abstractions”, IEEE Internet Computing, Special Issue on Context-Aware Computing, March/April 2012.
Payam Barnaghi, Frieder Ganz, Cory Henson, Amit Sheth, “Computing Perception from Sensor Data”, In proceedings of the 2012 IEEE Sensors Conference, Taipei, Taiwan, October 28-31, 2012.
Michael Compton et al, “The SSN Ontology of the W3C Semantic Sensor Network Incubator Group”, Journal of Web Semantics, 2012.
Harshal Patni, Cory Henson, and Amit Sheth , “Linked Sensor Data”, in Proceedings of 2010 International Symposium on Collaborative Technologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010.
Amit Sheth, Cory Henson, and Satya Sahoo , “Semantic Sensor Web IEEE Internet Computing”, vol. 12, no. 4, July/August 2008, pp. 78-83.
Wei Wang, Payam Barnaghi, Gilbert Cassar, Frieder Ganz, Pirabakaran Navaratnam, "Semantic Sensor Service Networks", (to appear) in Proceedings of the IEEE Sensors 2012 Conference, Taipei, Taiwan, October 2012.
Wang W, De S, Toenjes R, Reetz E, Moessner K, "A Comprehensive Ontology for Knowledge Representation in the Internet of Things", International Workshop on Knowledge Acquisition and Management in the Internet of Things (KAMIoT 2012) in conjunction with IEE IUCC-2012, Liverpool, UK. Liverpool. 25-27 June, 2012.
![Page 216: Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception](https://reader038.vdocument.in/reader038/viewer/2022102805/54c6b4e74a795942138b45e8/html5/thumbnails/216.jpg)
Some useful links related to IoT
Internet of Things, ITU
http://www.itu.int/osg/spu/publications/internetofthings/InternetofThings_summary.pdf
IoT Comic Book
http://www.theinternetofthings.eu/content/mirko-presser-iot-comic-book
Internet of Things Europe, http://www.internet-of-things.eu/
Internet of Things Architecture (IOT-A)
http://www.iot-a.eu/public/public-documents
W3C Semantic Sensor Networks
http://www.w3.org/2005/Incubator/ssn/XGR-ssn-20110628/