learning automata with wireless mesh network

1. Rohit Kumar Das M.Tech (IT), 3rd Sem Roll- 031312 No36320137 Assam University

2. Outline Introduction Literature Survey Related Work Motivation Backbone of Project Proposed Method Experimental Result Conclusion References 3/3/2014

4. Wireless Networks Collection of nodes where each mesh node is also a router. WMN is dynamically self-organized, self-configured, self-healing, easy maintenance, high scalability and reliable service with the nodes in the network Implemented with various wireless technology including 802.11(WiFi), 802.15(Wireless PAN ), 802.16 (Wireless Broadband standards), cellular technologies or combinations of more than one type. 3/3/2014

5. Ad-hoc Networks Communication done without any available. Discover their own path for transmission. Relay on the intermediate nodes. Types of Ad-hoc networks: Wireless Mesh Network (WMN) Wireless Sensor Network (WSN) Mobile Ad-hoc Network (MANET) Mesh Networks 3/3/2014 fixed infrastructure

6. Introduction (Conti.) Load Balancing Increase in network traffic cause load imbalance and leading to network degradation. Routing Protocol AODV routing protocol because it use less memory space helping to achieve the goal. Learning Automata Works well with stochastic environment. 3/3/2014

8. Literature Survey 1. Gateway Discovery Protocol through message notification [11]. At a IGW: If the average Q_length > Max_Permissible_Threshold Identify all the active sources For each active source Send a Congest_Notify message to switch the gateway, if possible End for End if If a GW_REQ message arrives from a node If the average Q_length < Max_Permissible_Threshold Admit this node and send a GW_REP to it End if End if 3/3/2014

9. Conti At a source node: Record the gateway information (GW IDs) in the gateway table When a notification message from IGW arrives: For each gateway ID in the gateway table Send a GW_REQ with the nodes estimated traffic End for When a GW_REP message arrives from a gateway: Make the nearest gateway as the primary gateway 3/3/2014

10. 2. The authors of [12] mentioned about three different load balancing scheme using IEEE 802. 11k Admission control and 3. Client driven Cell breathing Balancing of load by using nodes nearer to gateway node. Have low bandwidth blocking rate. Boundary nodes get un-notified [13]. 3/3/2014

11. 4. In [14], load balancing is performed by dividing domain into clusters then selecting gateway by G_value. Parameter for selecting Gateway: a) Power supply b) Velocity of node c) Distance to center of cluster and d) Processing power of node 3/3/2014

12. 5. Learning automaton for routing incoming calls [18]. Virtual link length Combination of packets Reduce packet delay 3/3/2014

14. Related Works 1. LALB (Learning Automata Based Load Balancing) Algorithm proposed by the authors of [5] is an approach for load balancing in Gateway level. 3/3/2014

15. 2. SARA (Stochastic Automata Rate Adaptation) Algorithm[15] for selecting the transmission rate. 3. Randomly selects. R : x = 1, 2, . . . . . , k (bps) Updates from feedback. R (x) should be best possible rate. Multicasting major problem for MANET. Authors of [17] proposed a weighted LA based multicasting protocol most stable multicast route. packets are forwarded along the edges of Steiner tree. Used LA to find the node with less mobility. Routes composed of long duration link are consider weights are assign. 3/3/2014

16. 4. Mehdi Zarei proposed Reverse AODV with Learning Automata (ROADVA) [25] works in similar way with Reverse AODV Reverse route is available. Route is selected based on stability factor. Updates the choice probability of routes stability according to the feedback information form network. 5. A routing protocol for Ad-hoc mobile network (AAODV) Learning Automata AODV Routing protocol was projected by authors of [26] Operates with energy restriction. Packet are routed through best path. Saves energy. 3/3/2014

17. Problems Domain Route Just flapping consider their load Associate each node to its nearest gateway Switching to another domain 3/3/2014

18. Figure 1: Problem Layer By Layer [20] 3/3/2014

20. Motivation Wireless Mesh Network (WMN) emerging topic for research. Problem with balancing of load. Learning Automata (LA) working ability with stochastic environment like WMN. Ad-hoc On demand Distance Vector (AODV) routing protocol. 3/3/2014

22. Wireless Mesh Network (WMN) Figure 2: Wireless Mess Network [6] 3/3/2014

23. Characteristic of WMN Multiple type of Network access. Two types of nodes: Access Points (APs)/ Mesh Routers (MRs) Mobile Clients / Nodes (MNs) Mobility dependence on the type of mesh nodes Mesh routers usually have minimal mobility Mesh clients can be stationary or mobile nodes Multi-hop wireless network Compatibility and interoperability with existing wireless networks 3/3/2014

24. Load Balancing Traffic volume very high Makes scalability and load balancing becomes important issues. Load balancing Optimization of usage of network resources Moving traffic from congested links to less loaded part. Traffic aggregation occurs in paths. Due to the limited wireless link capacity. Potential bottleneck 3/3/2014

25. Why Load Balancing ??? Avoiding congestion Increasing network throughput Providing reliability in case of any failure Three categories: Path-based load balancing Mesh-router-based load balancing and Internet gateway load balancing 3/3/2014

26. Learning Automata (LA) Systems Select possess incomplete knowledge current action based on past experiences from the environment Adaptive decision-making unit probability distribution 3/3/2014

27. Learning Automata in Network Does not require prior knowledge about traffic characteristic Utilized online in different networks Doesn't not require to complex analyze of network during learning phase Keep just one action probability vector Exhibits less memory demands 3/3/2014

28. Integrating LA Stochastic automaton: Six tuple { x, , , p, A, G} 3/3/2014

29. Integrating LA (Conti.) Environment: i/p -> (n) = {1,.,r} o/p -> x Response [0,1] Penalty Probability Ci (i = 1,r) 3/3/2014

30. Integrating LA (Conti.) Learning automaton: Operates in a random environment Figure 5: Learning automaton 3/3/2014

31. Learning Automata Models P-Model: The output can take only two values, 0 or 1 Q-Model: Finite output set with more than two values, between 0 and 1 S-Model: The output is a continuous random variable in the range [0,1] 3/3/2014

32. Operation of LA Four Stages: 1. Sequences of repetitive cycles 2. Chooses action 3. Receives environmental response 4. Based on response from earlier action, next action is determined. 3/3/2014

33. Operation (Conti) During each cycle: i is chosen with probability pi Environment response with Ci , update p. Next action chosen according p(n+1) 3/3/2014

34. Learning Automata Feedback Connection of Automaton and Environment Figure 6: Feedback mechanism of LA 3/3/2014

35. Reinforcement Scheme Choosing the best response based on the rewards or punishments token from environment Lower the (n) the more favorable the response. General Scheme: Pi(n) - ( 1-(n) ) gi( P(n) ) + (n) hi( P(n) ), if a(n)ai Pi(n+1) = Pi (n) + ( 1- (n) ) ji gj( P(n) ) - (n) ji hj( P(n) ), if a(n)=ai 3/3/2014

36. Reinforcement Schemes Different Scheme according to selection made from functions are : 1. The linear RewardPenalty (LRP) scheme 2. The linear RewardInaction (LRI) scheme 3. Nonlinear schemes 3/3/2014

37. Application of LA in Layers Physical Layer: Transmission power Distributed power control problem Network Layer: Multicasting Routing Transport Layer: Congestion window updation Control mechanisms 3/3/2014

38. Routing Protocol Ad-hoc On-Demand Distance Vector Routing Protocol (AODV) Both unicast and multicast routing Builds routes between nodes only as desired It is loop-free, self-starting, low network utilization, no memory overhead, and scales to large numbers of mobile nodes 3/3/2014

39. AODV Properties The route table stores: The basic message set consists of: RREQ Route Request RREP Route Reply RERR Route Error HELLO For link status monitoring 3/3/2014

40. Re-active routing AODV(RFC3561) A wants to communicate with B 3/3/2014

41. Re-active routing AODV(RFC3561) A floods a route request 3/3/2014

42. Re-active routing AODV(RFC3561) A route reply is unicasted back 3/3/2014

43. Route Requests in AODV Y Z S E F B C M L J A G H D K I N Represents a node that has received RREQ for D from S 3/3/2014

44. Route Requests in AODV Y Broadcast transmission Z S E F B C M J A L G H K D N I Represents transmission of RREQ 3/3/2014

45. Route Requests in AODV Y Z S E F B C M J A L G H K D N I Represents links on Reverse Path 3/3/2014

46. Reverse Path Setup in AODV Y Z S E F B C M J A L G H K D N I Node C receives RREQ from G and H, but does not forward it again, because node C has already forwarded RREQ once 3/3/2014

47. Reverse Path Setup in AODV Y Z S E F B C J A L M G H K D N I 3/3/2014

48. Reverse Path Setup in AODV Y Z S E F B C M J A L G H K D N I Node D does not forward RREQ, because node D is the intended target of the RREQ 3/3/2014

49. Forward Path Setup in AODV (contd) Y Z S E F B C J A L M G H K D N I Forward links are setup when RREP travels along the reverse p Represents a link on the forward path 3/3/2014

51. Proposed Method Learning Automata Ad-hoc On Demand Distance Vector (LA-AODV) routing protocol Integrating LA with AODV Find the best available path for packet delivery. Each routers will be employed with LAAODV 3/3/2014

52. Algorithm for Proposed Method Step 1: (Path Discovery) Start Route Discovery Phase by sending RREQ packet. If reach destination initiate RTL phase Else Forward to next node For each RREQ packet, check for same packet Same packet then discard or forward to next End for 3/3/2014

53. Reverse Path Establishment Fig: Reverse Path Formation Fig: Forward Path Formation 3/3/2014

54. Step 2: (Route Table Management by Learning) Receive feedback from neighbors. Construct local forwarding table using Learning Algorithm. Forwarding Table: check for RREQ entry in routing table. If present check RREQ seq_no > Dest seq_no Else Use recorded route for RREQ Create RREP Forward to intermediate nodes 3/3/2014

55. Step 3: (Routing Phase using Learning) Node activates LA Obtain best route from RLT phase. Check for constraint If between 50% to 100% Positive feedback (rewarded) Else Negative feedback (penalized) 3/3/2014

56. Flow Chart Figure 7: Flow chart for Proposed Model 3/3/2014

58. Experimental Results Figure 8: Basic AODV with Performance measurement 3/3/2014

59. Figure 9: Modified AODV with Performance measurement 3/3/2014

61. Conclusion & Future Works Relatively new technology Significant advantages for many applications Load balancing is one of the important area of research in WMN Load can be balanced using different techniques like Learning Automata 3/3/2014

62. Conclusion (Conti.) Collaborating LA with AODV Learning Automata AODV routing protocol (LA-AODV) for WMN LA agent keep running on each node. Provide best available path Lead to the goal Load Balancing 3/3/2014

64. References [1] Subir Kumar Sarkar, T G Basavaraju, C Puttamadappa, Ad-hoc Mobile Wireless Networks Principles, Protocol and Applications Auerbach Publications, ISBN 978-1-4200-6221-2 [2] Ram Ramanathan and Jason Redi, A Brief Overview of Ad-hoc Networks: Challenges and Directions, IEEE Communication Magazine 50th Anniversary Commemorative Issue/May 2002 [3] Bing He, Dongmei Sun, Dharma P. Agrawal Diffusion based Distributed Internet Gateway Load Balancing in a Wireless Mesh Network, In proceedings of IEEE "GLOBECOM" 2009 [4] Ashish Raniwala, Tzi-cker Chiueh. Architecture and algorithms for an IEEE 802.11based multi-channel wireless mesh network In: Infocom 2005. [5] Maryam Kashanaki, Zia Beheshti, Mohammad Reza Meybodi, A Distributed Learning Automata based Gateway Load Balancing Algorithm in Wireless Mesh Networks, Proceedings of IEEE for GLOBECOM 2009 [6] Akyildiz, Ian F., A Survey on Wireless Mesh Networks, Georgia Institute of Technology Xudong Wang, Kiyon, Inc., IEEE Radio Communications, 2005. [7] firetide.com An Introduction to Wireless Mesh Networking, 16795 Lark Avenue, Suite 200 [8] Kumpati S. Narendra, And M. A. L. Thathachar, Learning Automata - A Survey, IEEE Transactions On Systems, Man, And Cybernetics, Vol. Smc-4, No. 4, July 1974 [9] M.S. Obaidat, G.I. Papadimitriou, A.S. Pomportsis,Efficient fast learning automata, 3/3/2014 International journal of Information Science, June 2002.

65. References [11] Deepti Nandiraju, Lakshmi Santhanam, Nagesh Nandiraju, and Dharma P. Agrawal, Achieving Load Balancing in Wireless Mesh Networks through Multiple Gateways, Proceeding of IEEE in 2006. [12] E.Garcia Villegas, R. Vidal Ferr, J. Paradells Aspas, Load Balancing in WLANs through IEEE 802.11k Mechanisms, Proceeding of the 11th IEEE Symposium on Computers and Communications (ISCC'06). [13] P. Hsiao, A. Hwang, H. Kung, D. Vlah, Load-Balancing Routing for Wireless Access Networks, Proceeding of IEEE INFOCOM '01. [14] Mohammad Shahverdy, Misagh Behnami & Mahmood Fathy, A New Paradigm for Load Balancing in WMNs International Journal of Computer Networks (IJCN), Volume (3): Issue (4): 2011 239. [15] Tarun Joshi, Disha Ahuja, Damanjit Singh, and Dharma P. Agrawal, SARA: Stochastic Automata Rate Adaptation for IEEE 802.11 Networks IEEE Transactions On Parallel and Distributed Systems, Vol. 19, No. 11, November 2008 [16] Antonios Sarigiannidis, Petros Nicopolitidis, Georgios Papadimitriou, Using Learning Automata for Adaptively Adjusting the Downlink-to-Uplink Ratio in IEEE 802.16e Wireless Networks [17] Vinodha K, Joydipa Sen, A Weighted Learning Automata-Based Multicast Routing Protocol for Wireless MANET International Journal of Engineering Reasearch & Technology (IJERT) ISSN: 2278-0181, Vol. 2 Issue 6, June 2013 [18] Anastasios A. Economides, Learning Automata Routing In Connection-Oriented Networks, International Journal of Communication System, Vol 8, No 4, pp 225-237, 1995 [19] Anastasios A. Economides, Real-Time Traffic Allocation Using Learning Automata, International Conference on Systems, Man and Cybernetics, pp. 3307- 3312, IEEE, 1997 3/3/2014 [20] Fry, Michael, et al. Challenge identification for network resilience. Next Generation

66. References [21] Nicopolitidis, Petros, et al. Adaptive wireless networks using learning automata. Wireless Communications, IEEE 18.2 (2011): 75-81. [22] S. Das, C. Perkins, and E. Royer, "Ad Hoc On Demand Distance Vector (AODV) Routing," in IETF. RFC 3561, 2003. [23] Usop, Nor Surayati Mohamad, Azizol Abdullah, and Ahmad Faisal Amri Abidin. Performance evaluation of AODV, DSDV & DSR routing protocol in grid environment. IJCSNS International Journal of Computer Science and Network Security 9.7 (2009): 261-268. [24] Prashant Kumar Maurya, Gaurav Sharma, Vaishali Sahu, Ashish Roberts, Mahendra Srivastava, An Overview of AODV Routing Protocol, International Journal of Modern Engineering Research (IJMER), Vol.2, Issue.3, May-June 2012 pp-728-732. [25] Zarei, Mehdi. Reverse AODV routing protocol extension using learning Automata in ad hoc networks. Computer, Control and Communication, 2009. IC4 2009. 2nd International Conference on. IEEE, 2009. [26] Vahid Hosseini, Majid Taghipoor, A Novel Method of Routing for MANETs with Considering the Energy by Learning Automata World Applied Sciences Journal 17 (1): 113-118, 2012, ISSN 1818-4952, IDOSI Publications, 2012 [27] Arnrita Bose Paul, Shantanu Konwar,Upola Gogoi, Angshuman Chakraborty, Nilufar Yeshrnin, Sukurnar Nandi, Implementation and Performance Evaluation of AODV in Wireless Mesh Networks using NS-3, 2010 2nd International Conforence on Education Technology and Computer (ICETC) [28] Ghorbani, Mahdi, Ali Mohammad Saghiri, and Mohammad Reza Meybodi. A novel adaptive version of AODV routing protocol based on learning automata utilizing cognitive networks concept., Technical Journal of Engineering and Applied Sciences, ISSN 20510853, 2013. 3/3/2014

67. 3/3/2014