a learning-based mac for energy efficient wireless sensor ......a learning-based mac for energy...

A Learning-Based MAC for Energy

Efficient Wireless Sensor Networks

S. Galzarano1,2, Prof. A. Liotta2, Prof. G. Fortino1

(1)University of Calabria, Italy (2)Eindhoven University of Technology, Netherlands

Outline

• WSN & Machine learning

• Learning-based MAC

• Simulation results

• Conclusion & Future work

Stefano Galzarano, Antonio Liotta, Gincarlo Fortino 2

Challenges in WSN


WSN

Wireless Ad Hoc Medium: - unreliable, asymmetric

or unidirectional links - restricted broadband

Topology changes and mobillity: - Mobile sink and/or nodes - failing nodes - new node joining

Harsh environment: - no physical access to

network once deployed - nodes failure

Resource limitations: - battery - processing - memory

Challenges in WSN

Communication Stack Application level


Clustering

Routing and neighborood

management

Medium Access Control

Physical Layer

Data Processing

Data Collection

Security

Event and target

detection

Medium Access Control

• Protocol layer providing a multiple access control mechanism on a shared communication medium.

• A MAC protocol for WSN should use a radio wake-up/sleep scheduling for: – Energy saving

– Reduce collisions (and then also energy and latency)

– Reduce idle listening periods

– Maximizing throughput

– Minimizing latency


Research Objective

• Using Machine Learning to improve MAC performance in terms of energy efficiency, throughput and latency.


Machine Learning in WSN

Machine Learning paradigms have been successfully adopted to address various challenges


- Can adapt and operate efficiently in dynamic environments.

- Disover important correlation in sensor data

- Support more intelligent decision-making and autonomous control

Reinforcement

Learning

Decision

Tree

Genetic

Algorithms Swarm

Intelligence

R. V. Kulkarni, A. Forster, and G. K. Venayagamoorthy,

“Computational Intelligence in Wireless Sensor Networks: A Survey,”

Communications Surveys Tutorials, IEEE, vol. 13, no. 1, pp. 68 –96, quarter 2011.

.......

Reinforcement Learning

• Usually first choice when solving complex distributed problems in WSNs.

• Trial and error: learning by interacting with the environment: – Learning agents

– Pool of possible actions

– Goodness of actions

– A reward function

– Select one action

– Execute the action

– Observe the reward

– Correct the goodness of the executed action


• The achieved total reward (Q-value) of taking a specific action at a given state is computed using:

Q-Learning


• The achieved total reward (Q-value) of taking a specific action at a given state is computed using:

Q-Learning


new Q-Value old Q-Value

learning constant

Immediate reward old Q-Value

Proposed Q-Learning based MAC

Adapt Q-learning to a radio wake-up/sleep scheduling

• Learning agent Each node in the network





• State Slot sk in a the frame f


frame

t

slot sk





• Possible actions Radio ON/OFF, for each slot


frame

t

slot sk





• Possible actions Radio ON/OFF, for each slot

• Goodness of actions Q(sk)


frame

t

slot sk Q(sk)

Reward

Reward signals (per slot)

• Received packets +

• Succesfully transmitted packets +

• Over-heard packets -

• Expected packets -


Simulation results

• Castalia / OMNET++

• Comparison with other 2 different RL-based MAC


Simulation results

• Linear, star and mesh topologies

• Packets inter-arrival time between 1 and 10 seconds

• Max throughput is between 20 and 200 byte/sec (200 bytes length payload)

• Performance metrics: throughput, latency, energy efficiency


Liu, Z., Elhanany, I.: RL-MAC: A reinforcement learning based MAC

protocol for wireless sensor networks. International Journal of

Sensor Networks 1, 117–124 (2006)

Simulation results


Simulation results

• A nodes-to-sink communication pattern has been used.

• 2 pkt/sec

• Multipath ring routing

• Performance metrics: latency, packet delivery, energy efficiency


Mihaylov, M., Tuyls, K., Nowé, A.: Decentralized learning in wireless

sensor networks. In: Taylor, M.E., Tuyls, K. (eds.) ALA 2009. LNCS

(LNAI), vol. 5924, pp. 60–73. Springer, Heidelberg (2010)

Simulation results


Mihaylov, M., Tuyls, K., Nowé, A.: Decentralized learning in wireless

sensor networks. In: Taylor, M.E., Tuyls, K. (eds.) ALA 2009. LNCS

(LNAI), vol. 5924, pp. 60–73. Springer, Heidelberg (2010)

Conclusions

• A Q-Learning approach has been successfully employed for a self-adapting MAC layer on WSNs;

• Simulation results show that it outperforms others RL-based MAC


Future work

• Ongoing work: – implementation on real sensor platforms;

– extensive experiments with varying real deployment.

• Dynamically update both frame length and slot number on the basis of the network traffic.


Thank you!!!

Any Questions?


A Learning-Based MAC for Energy Efficient Wireless

Sensor Networks

S. Galzarano, Prof. A. Liotta, Prof. G. Fortino

University of Calabria, Italy

&

Eindhoven University of Technology, Netherlands

a learning-based mac for energy efficient wireless sensor ......a learning-based mac for energy...

Documents