query processing in mobile p2p databases igert seminar presentation bo xu joint work with ouri...

Query Processing in Mobile P2P Databases

IGERT Seminar PresentationBo Xu

joint work with Ouri Wolfson

April 18, 2023 IGERT seminar 2

Talk outline

Introduction System Model The MARKET Algorithm Evaluation Extension to CTS Conclusion and Future Work


Query Processing Environments

Query longevity

Continuous Instantaneous

Wireless ad-hocnetworks

Sensor MANET

Mobile Static

Connectivity

Sparse Dense

Query longevity

Continuous Instantaneous

Wireless ad-hocnetworks

Sensor MANET

Mobile Static

Connectivity

Sparse Dense

Motivation: a general purpose query processing strategy mobile disconnected wireless ad-hoc networks

Vehicular Sensor Network (VSN)

GPS receiverchemical spill detectorstill/video cameravibration sensoracoustic detector

Bo xU

In the London bombings, police agents tracked some of the suspects in the subway by using closed-circuit TV cameras, but they had a hard time finding helpful evidence from the double-decker bus; this scenario has motivated the installation of more cameras in fixed locations along London streets. VSNs could be an excellent complement to fixed cameras.

Bo xU

floating car dataenvironmental monitoringdistributed survaillence


Store-and-forward to deal with sparseness

q

Ar

Q

Q

A AqA

QA


Issues with Store-and-forward

How to manage limited memory, power, and bandwidth?

Which reports to save/transmit?


Difficulty of Store-and-forward

Assume that the trajectories of all nodes is known a priori at a central server. If memory, energy, and bandwidth are bounded at mobile nodes, then the problem of determining whether a set of data-items can be disseminated to all the mobile nodes is NP-complete.

Case: Each mobile node is interested in every data-item

Mobile P2P: Trajectories unknown a priori; Heuristics needed


Talk outline



Mobile P2P Database

report 8

query C

query Breport 4report 5

Local database

Local query query Areport 1report 2report 3

Pda’s, cell-phones, sensors, hotspots, vehicles, with short-rangewireless capabilities

A

B

C

• Applications coexist• Variable report sizes• A peer can be a produce, consumer, and broker

Computer

具有计算或者信息处理能力的移动设备正在以前所未有的速度普及着。1、PDA，掌上电脑2、cellphone, smartphone, computer with CPU, operating system, memory3、传感器, computer with CPU, operating system, memory 4、车辆，两个方面：乘员所带移动设备；OBD系统收集和处理车辆状态信息：速度，刹车状态、ABS状态、油门位置等

Computer

W-F，B， Z使用的频率是非管制的，也就是说使用这些频率无须向政府部门申请license。而CELLPHONE使用的频率是受管制的，PROVIDER需要申请LICENSE才能使用这些频率。COST最终转嫁到用户身上。

Computer

几十兆BITS PER SECOND。比使用CELLPHONE带宽高10倍左右。

Computer

移动设备通过短距通信构造MP2P数据库。具体方式如下。每一个设备有一个LOCAL DATABASE存储RESOURCE INFORMATION，什么是资源：加油站，出租车、等候出租车的客户、停车位。比如一个客户正在一环路和建设路的十字路口等车，前方两公里处有加油站。资源信息可以是PEER 自己产生的，也可以是从其他PEER收到的。LOCAL QUERY定义PEER正在搜索的资源，如停车位。


Queries

A query Q maps each report R to a match degree:

Examples: Top parking slots given my current location

Profile with expertise “children-periodontics” Similarity between two images

1),(0 QRmatch

match(R,Q)=e-t-d

Ouri Wolfson

fix comparison symbols


Query/report Dissemination

Two peers within transmission range exchange queries and reports

Least relevant reports that do not fit in local broker database are purged

Exchange not necessarily synchronous (periodic broadcast)

Ouri Wolfson

relevance ranking is critical for 2 reasons:1. it enables purging of old/distant resource reports, otherwise the table size will grow exponentIALLY.2. IT ENABLES EXCHANGING THE MOST IMPORTANT RESOURCES BEFORE A SESSION BREAKS.


Talk outline



Ranking Factors

Rank of a report R is determined by

Demand: What fraction of peers are querying R Probability that a peer is interested in R

Supply: What fraction of peers already have R Probability that a peer has R

Size of R


Rank of a report

expected benefit = demand(R)*(1supply(R))

reports database 0.3

0.8

0.5

0.4

0.5

0.7

reports benefit

0.5

0.7

Rank(R)=size(R)

demand(R)*(1supply(R))


Report Ranking: sample demand

q3q4

q1q7 O1

O3

O2

(a)

O3

O1

O2

(b)

r2r5

r1r4

q2q6

r3r6

r7r8

q5q8

q3q4

r1r4r7r8

q5q8

q1q3

r1r2r5r4

q4q7

q2q4

r4r3r6r1

q3q6

reportsrel ati on

queri esrel ati on

Queries relation is FIFO maintained


Rank of Reports

Demand for R

Qi’s are the members of the queries relation

Size of the queries relation determined based on Hoeffding’s inequality

n

QRmatchn

ii

1

),(

2)width_interval_confidence(221_levelconfidence neE.g., if n=108, then with 95% chance the demand estimation error is smaller than 0.08


How does peer O determine supply(R)?

A parametric formula giving the supply is beyond the state of the art

O machine-learns supply(R) based on meta-data of R: Age of R Number of times O sighted R from other

peers etc.

peersofnumbertotal

RhavethatpeersofnumberRsupply

___

_____)(


Computing Supply by Machine-learning

aro: The age rank order within O’s reports database

fin: The number of times O has sighted the report from other peers

MAchine LEarning based Novelty rAnking (MALENA)

report -idReport

descriptionaro fin

R1 … 1 1

R4 … 2 4

R2 … 3 2

R7 … 4 2

Reports database of O

report -idReport

descriptionaro fin

R1 … 1 1

R4 … 2 4

R2 … 3 2

R7 … 4 2



report-id

Reportdescription

aro fin

R1 … 1 3

R2 … 2 1

TrackingSet of O

R2

R3

R1Advertise to peer

R1, R2

ReceiveREQ((R1,R2), R2)

Examples created by ReceiveREQ

aro fin label

1 3 old

2 1 new

insertExamples set

ES of O

ReceiveRPT(R4)


report-id

Reportdescription

aro fin

R1 … 1 4

R4 … 2 1

TrackingSet of O

R2

R3

R1


report-id

Reportdescription

aro fin

R1 … 1 4

R2 … 2 1

R2

R3

R1

TrackingSet of O

R4

(a)

(b)

(c)


report-id

Reportdescription

aro fin

R1 … 1 3

R2 … 2 1

TrackingSet of O

R2

R3

R1Advertise to peer

R1, R2

ReceiveREQ((R1,R2), R2)

Examples created by ReceiveREQ

aro fin label

1 3 old

2 1 new

insertExamples set

ES of O

ReceiveRPT(R4)


report-id

Reportdescription

aro fin

R1 … 1 4

R4 … 2 1

TrackingSet of O

R2

R3

R1


report-id

Reportdescription

aro fin

R1 … 1 4

R2 … 2 1

R2

R3

R1

TrackingSet of O

R4

(a)

(b)

(c)

MALENA

B

B

Request R2

Examples created

positive

negative


MALENA Implementation Considerations

Minimize overhead No need to actually store examples Model incrementally built

Bayesian learning a simple but effective method


Talk outline



0

20

40

60

80

100

120

140

0 30 60 90 120 150 180 210 240 270 300

thro

ughp

ut (m

atch

es/p

eer)

response-time bound (second)

peer density (D) = 20, energy allocation fraction (F) = 0.15 report production rate (P) = 0.1 reports/second

MARKETRANDI

Ideal Benchmark

mobility model=random way point, average motion speed=1 mile/hourtransmission range=100 meters, mean of reports database size=100Kbytesqueries database size=100 queriesreport size uniformly distributed between 1K and 2K bytes0.1 report produced per second

Comparison with RANDI (MDM’07)

0

20

40

60

80

100

120

140

0 3600 7200 10800 14400 18000 21600 25200 28800

thro

ughp

ut (m

atch

es/p

eer)


peer density (D) = 1, energy allocation fraction (F) = 0.15 report production rate (P) = 0.1 reports/second

MARKETRANDI

Ideal Benchmark

RANDI=MARKET-supply

20 peers within transmission range 1 peer within transmission range

MARKET half as good as ideal benchmarkMARKET twice better than RANDI


Comparison with LRU and LFU

2

2.5

3

3.5

4

4.5

5

5.5

0 10 20 30 40 50

MARKET

RANDI

LRU

LFU


thro

ug

hput

(matc

hes/

peer)

mobility model=iMotes tracesmean of reports database size=150Kbytesqueries database size=10 queriesreport size uniformly distributed between 2K and 20K bytes0.1 report produced per second, transmission size=100Kbytes

(results obtained by Fatemeh Vafaee)


Evaluation of MALENA (TAAS’09)turn-over: peers enter/exit systeminjection: number of peers that have a report initially

mobility model=iMotes traces, reports database size=100 reports2 reports produced per second, transmission size=10 reports

MALENA always follows the best indicator

0

2000

4000

6000

8000

10000

12000

14000

0 5 10 15 20 25 30 35 40 45 50 55 60

thro

ughp

ut (r

epor

ts/p

eer)

response-time bound (minute)

low-turn-over/low-injection Bluetooth, M=100, f=2, q=0.1, flooding

MALENAaro-rankingfin-ranking

low-turn-over/low-injection

0

20

40

60

80

100

120

140

160

180

200

0 5 10 15 20 25 30 35 40 45 50 55 60

thro

ughp

ut (r

epor

ts/p

eer)


high-turn-over/high-injection Bluetooth, M=100, f=2, q=0.1, flooding


low-turn-over/low-injection

0

20

40

60

80

100

120

140

160

180

200

0 5 10 15 20 25 30 35 40 45 50 55 60

thro

ughp

ut (r

epor

ts/p

eer)


high-turn-over/high-injection Bluetooth, M=100, f=2, q=0.1, flooding


high-turn-over/high-injection


Application: K-nearest-neighbors

Query: K-nearest-neighbors of a fixed location (query-point) Reports: current locations of mobile sensors match(Q,R): in reverse proportion to the distance from query

point

sink

query-point


Itinerary based KNN processing

Phase I: Query delivered to the sensor closest to query point

Phase II: Query traverses an itinerary to collect answers

Phase III: Answers returned to sink


Simulation Results

0%

20%

40%

60%

80%

100%

1 2 3 4 5 6

peer density

ac

cu

rac

y

MARKET

Upper bound of itinerary

mobility model=random way point, average motion speed=1 mile/hourtransmission range=100 meters report size=24 bytes, query size=16 bytesmean of reports database size=100 reportsone location report produced at each sensor per second

MARKET is especially suitable for sparse environments


Talk outline



TrafficInfo: Disseminating Traffic Information in VANET’s


What does relevance mean in TrafficInfo

A

B

A

B

A report is relevant if it changes the route


Which factors indicate relevance of report?

Distance to the reported road segment

Type of road segment Speed variance …


Conceptual Learning Procedure

An example is created for a received report

The example is labeled positive if the report changes route and negative otherwise

Individual vs. group How to deal with aggregation?


Query processing

Conclusion

Store-and-forward enables in-network processing in mobile disconnected networks

Ranking is important for dealing with memory, bandwidth, and energy constraints

sensor-richenvironment

short-rangewireless

Mobile P2P+


Future Work

Multimedia reports Utilization of metadata

Integration of stateless and stateful approaches

Starvation/fairness


Thanks!

Questions?


802.11 Basics

3 modes: transmitting, receiving, listening (order of power consumption)

When listening: if detecting a message destined to host receive-mode

Time divided into slots, 20microsecs each

Transmission: Listen for 1 time slot If channel free start broadcast (observe

collision possible) Broadcast may last for many time slots


Energy Efficiency of a Broadcast

X

Throughput (Th) = (expected number of neighbors that successfully receive broadcast) (broadcast size)

Power efficiency (PE) = En

Th

successfully receive the broadcast from x

Collisions occur at neighbor


Computation of Throughput

X Y

1. No green node inside starts to broadcast at the same time slot with X

2. No transmission from any purple node overlaps with that from X

Conditions for successful reception at an arbitrary node Y


Energy Constraints

Energy consumed by a 802.11 network interface for transmitting a message of size M bytes

En=fM+g

For 802.11 broadcast, g=26610-6 Joule, f=5.2710-6Joule/byte


Experimental MP2P Projects (Pedestrians)

7DS – Columbia University (web pages) iClouds – Darmstadt Univ. (incentives) MoGATU – UMBC (specialized query processing,

e.g., collaborative joins) PeopleNet – NUS, IIS-Bangalore (Mobile

commerce, information type location baazar)

MoB – Wisconsin, Cambridge (incentives, information resources e.g. bandwidth)

Mobi-Dik – Univ. of Illinois, Chicago (brokering, physical resources, bandwidth/memory/power management)


Vehicular Projects

Inter-vehicle Communication and Intelligent Transportation:

CarTALK 2000 is a European project VICS (The Vehicle Information and Control System) is a

government-sponsored system in Japan with an 11-year track record

FleetNet, an inter-vehicle communications system, is being developed by a consortium of private companies and universities in Germany

IVI (Intelligent Vehicle Initiative) and VII (Vehicle Infrastructure Integration), the US DOT

MP2P provides data management capabilities on top of these communication systems

Grassroots, TrafficView, SOTIS, V3 – P2P dissemination of traffic info to reduce travel times


RANk-based DIssemination (RANDI)

Ranking of reports Bandwidth/energy aware Exchange enhances

Consumer functionality Broker functionality

Consumer: Answer local query (pull) Broker: Transmit reports most likely requested

by future-encountered peers (push) Transmission trigger:

Encounter New reports


RANDIWhen two peers meet they conduct a two-phase exchange:

local query

answers

more reports

satisfied as a consumer (pull)

enhanced as a broker (push)

Phase 1: Exchange queries and receive answers (pull)Phase 2: Exchange more reports using available energy/bandwidth (push)

Phase 1

Phase 2

Combination of: unicast (thin line) and broadcast (thick lines) to enable overhearing.


RANDI (Cont’d)

To solve problem with static peers:Two interaction modes which combine pull and push

• Query-response: triggered by discovery of new neighbors

• Relay: triggered by receipt of new reports Disseminate to existing neighbors

new reports


query

queryquery

query

reports

reports

7DS

P2P mode: each node periodically broadcasts its query and receivesreports from neighboring peers. No strategy to determine query frequency and transmission size. Cache management based on web-page expiration time.


PeopleNet

before exchange after exchange

Peer A Peer B Peer A Peer B

random-spread



random-swap

Reports are randomly selected for exchanging and saving upon encountering.


query

queryquery

query

reports

reports

7DS

Each peer periodically broadcasts its query and receives reports from neighboring peers. No strategy to determine query frequency and transmission size. Cache management based on web-page expiration time.


PeopleNet



Reports are randomly selected for exchanging and saving upon encountering.


Mobile Local Search: Applications

transportation Announce sudden stop, malfunctioning brake light, patch of ice Floating car data Dissemination of multi-media traffic information (picture, video,

voice) Search close-by taxi customer, parking slot, ride-share

social networking (wearable website) Personal profile of interest at a convention Singles matchmaking Floating BBS

mobile electronic commerce Sale on an item of interest at mall Music-file exchange

emergency response Search for victims in a rubble

asset management and tracking Sensors on containers exchange security information => remote

checkpoints tourist and location-based-services

Closest ATM

Computer

移动物体搜索本地资源。所谓本地资源就是与移动物体距离较近的资源。

Computer

在学术会议上，MP2P自动对与会者的PROFILE进行匹配并且安排面对面的会谈。与会者定义QUERY(INTREST PROFILE) 和RESOURCE DESCRIPTION(EXPERTISE)，如DATABASE专家寻找安全专家，安全专家寻找数据库专家。如何安排见面？在PROFILE中加入IDENTIFICATION INFROMATION，如手机号码、姓名、照片、穿戴等。这样，在MP2P中存储的个人PROFILE成为wearable web-site.


Applications – Common features

Mobile/stationary peers Resources of interest

in a limited geographic area Short time duration

Can be solved by fixed servers, but Unlikely solution Proposed mp2p paradigm can enhance

fixed solution (reliability, performance, coverage)


MARKETWhen two peers meet they conduct a two-phase exchange:

Local query

answers

more reports

satisfied as a consumer (pull)

enhanced as a broker (push)

Phase 1: Exchange subscriptions and receive answers (pull)Phase 2: Exchange more publications using available energy/bandwidth (push)

Phase 1

Phase 2

Combination of: unicast (thin line) and broadcast (thick lines) to enable overhearing.


MARKET (Cont’d)

To solve problem with static peers:Two interaction modes which combine pull and push

• Query-response: triggered by discovery of new neighbors

• Relay: triggered by receipt of new publications Disseminate to existing neighbors

new publications


Query in static disconnected network

q

A

A

r

In-network query processing may not be possible

Q

Q

Q


Query in static connected sensor network

qA

A

A

r

Data transmission delay is 0. Answer can be obtained instantaneously

Q

Q

Q

Q

QA A

A

A

A

qA


Query in static disconnected network

q

A

A

r

In-network query processing may not be possible

Q

Q

Q


Query in mobile disconnected network

q

A

A

r

One hop case

QAqA

Query processing enabled by mobility and store-and-forward


Query in mobile disconnected network

q

Ar

Multil-hop case

Q

Q

Query can be in network processed, but it is delayed

A

Query processing alogrithm doesn’t control motion.

The answer is disseminated only after an answer node receives query

AqA

QA

First stage: query disseminated during encounter

query processing in mobile p2p databases igert seminar presentation bo xu joint work with ouri...

Documents