"an adaptive modular approach to the mining of sensor network
TRANSCRIPT
![Page 1: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/1.jpg)
An adaptive modular approach to the mining of sensor network data
G. Bontempi, Y. Le Borgne (1)
{gbonte,yleborgn}@ulb.ac.be
Machine Learning Group
Université Libre de Bruxelles – Belgium(1) Supported by the COMP2SYS project, sponsored by the HRM program of the European
Community (MEST-CT-2004-505079)
![Page 2: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/2.jpg)
Y. Le Borgne 2
Outline
Wireless sensor networks: Overview
Machine learning in WSN
An adaptive two-layer architecture
Simulation and results
Conclusion and perspective
![Page 3: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/3.jpg)
Y. Le Borgne 3
Sensor networks : Overview
Goal : Allow for a sensing task over an environment
Desiderata for the nodes:Autonomous power
Wireless communication
Computing capabilities
![Page 4: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/4.jpg)
Y. Le Borgne 4
Smart dust project
Smart dust: Get mote size down to 1mm³Berkeley - Deputy dust (2001)
6mm³
Solar powered
Acceleration and light sensors
Optical communication
Low cost in large quantities
![Page 5: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/5.jpg)
Y. Le Borgne 5
Current available sensors
Crossbow : Mica / Mica dotuProc: 4Mhz, 8 bit Atmel RISCRadio: 40 kbit 900/450/300 MHz or
250 kbit 2.5GHz (MicaZ 802.15.4)Memory: 4K RAM / 128 K Program Flash /
512 K Data FlashPower: 2 x AA or coin cell
Intel : iMoteuProc: 12Mhz, 16 bit ARMRadio: BluetoothMemory: 64K SRAM / 512 K Data FlashPower: 2 x AA
MoteIV : TelosuProc: 8Mhz, 16 bit TI RISCRadio: 250 kbit 2.5GHz (802.15.4)Memory: 2 K RAM / 60 K Program Flash /
512 K Data FlashPower: 2 x AA
![Page 6: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/6.jpg)
Y. Le Borgne 6
Applications
Wildfire monitoring
Ecosystem monitoring
Earthquake monitoring
Precision agriculture
Object tracking
Intrusion detection
…
![Page 7: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/7.jpg)
Y. Le Borgne 7
Challenges for…
Electronics
Networking
Systems
Data bases
Statistics
Signal processing
…
![Page 8: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/8.jpg)
Y. Le Borgne 8
Machine learning and WSN
Local scale
Spatio-temporal correlationsLocal predictive model identification
Can be used to:Reduce sensor communication activity
Predict values for malfunctioning sensors
![Page 9: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/9.jpg)
Y. Le Borgne 9
Machine learning and WSN
Global scale
The network as a a whole can achieve high level tasks
Sensor network <-> Image
![Page 10: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/10.jpg)
Y. Le Borgne 10
Supervised learning and WSN
Classification (Traffic type classification)
Prediction (Pollution forecast)
Regression (Wave intensity, population density)
![Page 11: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/11.jpg)
Y. Le Borgne 11
A supervised learning scenario
Ѕ: Network of S sensors
x(t)={s1(t),s2(t),…sS(t)} snapshot at time t
y(t)=f(x(t))+ε(t) the value associated to S at time t (ε standing for noise)
Let DN be a set of N observations (x(t),y(t))
Goal : Find a model that predicts y for any new x
![Page 12: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/12.jpg)
Y. Le Borgne 12
Centralized approach
High transmission overhead
![Page 13: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/13.jpg)
Y. Le Borgne 13
Two-layer approach
Use of compressionReduce transmission overhead
Spatial correlation induces low loss in compression
Reduction of learning problem dimensionality
![Page 14: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/14.jpg)
Y. Le Borgne 14
Two-layer adaptive approach
PAST : Online compression
Lazy learning : Online learning
![Page 15: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/15.jpg)
Y. Le Borgne 15
Compression : PCA
PCA: Transform the set of n input variables , into a set of m variables , m<n.
Linear transformation : ,
Variance preserving maximization
Solution :m first eigenvectors of x correlation matrix, or
Minimization of
![Page 16: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/16.jpg)
Y. Le Borgne 16
PAST – Recursive PCA
Projection approximation subspace tracking [YAN95]
Online formulation:
Low memory requirement and computational complexity :
O(n*m)+O(m²)
![Page 17: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/17.jpg)
Y. Le Borgne 17
PAST AlgorithmRecursive formulation: [HYV01]
![Page 18: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/18.jpg)
Y. Le Borgne 18
Learning algorithm
Lazy learning: K-NN approachStorage of observation set:
When a query q is asked, takes the k nearest neighbours to q:
Builds a local linear model: , such that
Computes the output at by applying
![Page 19: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/19.jpg)
Y. Le Borgne 19
How many neighbours?
•y=sin(x)+e
•e : Gaussian noise with σ=0.1
•What is the y value at x=1.5?
![Page 20: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/20.jpg)
Y. Le Borgne 20
How many neighbours?
•K=2 : Overfitting
![Page 21: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/21.jpg)
Y. Le Borgne 21
How many neighbours?
•K=2 : Overfitting
•K=3 : Overfitting
![Page 22: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/22.jpg)
Y. Le Borgne 22
How many neighbours?
•K=2: Overfitting
•K=3: Overfitting
•K=4: Overfitting
![Page 23: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/23.jpg)
Y. Le Borgne 23
How many neighbours?
•K=2: Overfitting
•K=3: Overfitting
•K=4: Overfitting
•K=5: Good
![Page 24: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/24.jpg)
Y. Le Borgne 24
How many neighbours?
•K=2: Overfitting
•K=3: Overfitting
•K=4: Overfitting
•K=5: Good
•K=6: Underfitting
![Page 25: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/25.jpg)
Y. Le Borgne 25
Automatic model selection([BIR99],[BON99],[BON00])
Starting with a low k, local models are identifiedTheir quality is assessed by a leave one out procedureThe best model(s) are kept for computing the predictionLow computational cost
PRESS statistics (ALL74)Recursive least squares ([GOO84])
![Page 26: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/26.jpg)
Y. Le Borgne 26
Advantages of PAST and lazy
No assumption on the process underlying data
On-line learning capability
Adaptive with non-stationarity
Low computational and memory costs
![Page 27: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/27.jpg)
Y. Le Borgne 27
Simulation
Modeling wave propagation phenomenon
Helmholtz equation:
k is the wave number
•2372 sensors
•30 k values between 1 and 146; 50 time instants
•1500 Observations
•Output k is noisy
![Page 28: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/28.jpg)
Y. Le Borgne 28
Test procedure
Prediction error measurementNormalized Mean Square Error (NMSE)
10-fold cross-validation (1350/150)
Example of learning curve:
![Page 29: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/29.jpg)
Y. Le Borgne 29
Experiment 1
Centralized configuration
Comparison PCA / PAST for 1 to 16 first principal components
![Page 30: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/30.jpg)
Y. Le Borgne 30
Results
0.1150.1240.1320.1960.1830.2230.2570.3630.782NMSE PAST
0.1160.1240.1330.1340.1380.1440.1810.2660.621NMSE
PCA
16128654321m
•Prediction accuracy similar if number of principal components sufficient
![Page 31: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/31.jpg)
Y. Le Borgne 31
Clustering
The number of clusters involves a trade-off between
The routing costs between clusters and gateway
The final prediction accuracy
The robustness of the architecture
![Page 32: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/32.jpg)
Y. Le Borgne 32
Experiment 2Partitioning into geographical clusters
P varies from P(2) to P(7)
2 main components for each cluster
Ten-fold cross-validation – 1500 data
Example of P(2) partitioning
![Page 33: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/33.jpg)
Y. Le Borgne 33
Results
0.1140.1160.1180.1180.1180.140NMSE
P(7)P(6)P(5)P(4)P(3)P(2)
•Comparison of P(2) (Top) and P(5) (bottom) error curves
•As number of cluster increases:
•Better accuracy
•Faster convergence
![Page 34: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/34.jpg)
Y. Le Borgne 34
Experiment 3
Simulation: at each time instantProbability 10% for a sensor failure
Probability 1% for a supernode failure
Recursive PCA and lazy learning deals efficiently with input space dimension variations
Robust with random sensor malfunctioning
![Page 35: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/35.jpg)
Y. Le Borgne 35
Results
0.1170.1160.1160.1190.1320.501NMSE
P(7)P(6)P(5)P(4)P(3)P(2)
•Comparison of P(2) (Top) and P(5) (bottom) error curves
•The number of clusters increases the robustness
![Page 36: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/36.jpg)
Y. Le Borgne 36
Experiment 4
Time varying changes in sensor measures
2700 time instants
Sensor response decreases linearly from a factor 1 to a factor 0.4
A temporal window:Only the last 1500 measures are kept
![Page 37: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/37.jpg)
Y. Le Borgne 37
Results
•Due to the concept drift, the fixed model (in black) becomes outdated
•The lazy characteristic of the proposed architecture can deal with this drift very easily
![Page 38: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/38.jpg)
Y. Le Borgne 38
Conclusion
Architecture:Yielding good results compared to batch equivalent
Computationally efficient
Adaptive with appearing and disappearing units
Handling easily non-stationarity
![Page 39: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/39.jpg)
Y. Le Borgne 39
Future work
Extensions of tests to real-world data
Improvement of clustering strategyTaking costs (routing/accuracy) into consideration
Making use of ad-hoc feature of the network
Test of other compression proceduresRobust PCA
ICA
![Page 40: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/40.jpg)
Y. Le Borgne 40
References
Smart Dust project: http://www-bsac.eecs.berkeley.edu/archive /users/warneke-brett/SmartDust/
Crossbow: http://www.xbow.com/
[BON99] G.Bontempi. Local Techniques for Modeling, Prediction and Control. PhD Thesis, IRIDIA- Université Libre de Bruxelles, 1999.
[YAN95] B. Yang. Projection Approximation Subspace Tracking, IEEE Transactions on Signal Processing, 43(1):95-107,1995.
[ALL74] D.M. Allen. 1974. The relationship between variable and data augmentation and a method of prediction. Technometrics, 16, 125-127
[GOO84] G.C. Goodwin & K.S. Sin. 1984. Adaptive filtering Prediction and Control. Prentice-Hall.
[HYV01] Independent Component Analysis. A. Hyvarinen, J. Karhunen, E. Oja. 2001.
![Page 41: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/41.jpg)
Y. Le Borgne 41
References on lazy learning
[BIR99] M. Birattari, G. Bontempi, and H. Bersini. Lazy learning meets the recursive least square algorithm. In M. S. Kearns, S.a. Solla, and D.a. Cohn, editors, NIPS 11, pages 375-381, Cambridge,1999, MIT Press.
[BON99] G. Bontempi, M.Birattari, and H.Bersini. Local learning for iterated time-series prediction. In I. Bratko and S. Dzeroski, editors, Machine Learning : Proceedings of the 16th International Conference, pages 32-38, San Francisco, CA, 1999. Morgan Kaufmann Publishers.
[BON00] G. Bontempi, M.Birattari, and H. Bersini. A model selection approach for local learning. Artificial Intelligence Communications, 121(1), 2000.
![Page 42: "An adaptive modular approach to the mining of sensor network](https://reader033.vdocument.in/reader033/viewer/2022060206/55a25ce61a28ab962b8b4757/html5/thumbnails/42.jpg)
Y. Le Borgne 42
Thanks for your attention!