|
|
Determining node duty cycle using Q-learning and linear regression for WSN |
Han Yao HUANG, Kyung Tae KIM, Hee Yong YOUN( ) |
College of Software, Sungkyunkwan University, Suwon 440-746, Korea |
|
|
Abstract Wireless sensor network (WSN) is effective for monitoring the target environment,which consists of a large number of sensor nodes of limited energy. An efficient medium access control (MAC) protocol is thus imperative to maximize the energy efficiency and performance of WSN. The most existing MAC protocols are based on the scheduling of sleep and active period of the nodes, and do not consider the relationship between the load condition and performance. In this paper a novel scheme is proposed to properly determine the duty cycle of the WSN nodes according to the load,which employs the Q-learning technique and function approximation with linear regression. This allows low-latency energy-efficient scheduling for a wide range of traffic conditions, and effectively overcomes the limitation of Q-learning with the problem of continuous state-action space. NS3 simulation reveals that the proposed scheme significantly improves the throughput, latency, and energy efficiency compared to the existing fully active scheme and S-MAC.
|
Keywords
wireless sensor network
media access control
duty-cycle scheduling
Q-learning
linear regression
|
Corresponding Author(s):
Hee Yong YOUN
|
Just Accepted Date: 26 February 2020
Issue Date: 24 September 2020
|
|
1 |
A Alemdar, M Ibnkahla. Wireless sensor networks: applications and challenges. In: Proceedings of the 9th International Symposium on Signal Processing and Its Applications. 2007, 1–6
https://doi.org/10.1109/ISSPA.2007.4555630
|
2 |
T AlSkaif, B Bellalta, M G Zapata, J M B Ordinas. Energy efficiency of MAC protocols in low data rate wireless multimedia sensor networks: a comparative study. Ad Hoc Networks, 2017, 56: 141–157
https://doi.org/10.1016/j.adhoc.2016.12.005
|
3 |
T Van Dam, K Langendoen. An adaptive energy-efficient MAC protocol for wireless sensor networks. In: Proceedings of the 1st International Conference on Embedded Networked Sensor Systems. 2003, 171–180
https://doi.org/10.1145/958491.958512
|
4 |
R Zheng, R Kravets. On-demand power management for ad hoc networks. Ad Hoc Networks, 2005, 3(1): 51–68
https://doi.org/10.1016/j.adhoc.2003.09.008
|
5 |
R Zheng, J C Hou, L Sha. Asynchronous wakeup for ad hoc networks. In: Proceedings of the 4th ACM International Symposium on Mobile Ad Hoc Networking & Computing. 2003, 35–45
https://doi.org/10.1145/778415.778420
|
6 |
W Ye, J Heidemann, D Estrin. An energy-efficient MAC protocol for wireless sensor networks. In: Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies. 2002, 1567–1576
|
7 |
W Ye, J Heidemann, D Estrin. Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking (ToN), 2004, 12(3): 493–506
https://doi.org/10.1109/TNET.2004.828953
|
8 |
E S Jung, N H Vaidya. An energy efficient MAC protocol for wireless LANs. In: Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies. 2002, 1756–1764
|
9 |
S Liu, KW Fan, P Sinha. CMAC: an energy-efficient MAC layer protocol using convergent packet forwarding for wireless sensor networks. ACM Transactions on Sensor Networks (TOSN), 2009, 5(4): 29
https://doi.org/10.1145/1614379.1614381
|
10 |
F L Lewis, K G Vamvoudakis. Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2011, 41(1): 14–25
https://doi.org/10.1109/TSMCB.2010.2043839
|
11 |
S Kosunalp, Y Chu, P D Mitchell, D Grace, T Clarke. Use of Q-learning approaches for practical medium access control in wireless sensor networks. Engineering Applications of Artificial Intelligence, 2016, 55: 146–154
https://doi.org/10.1016/j.engappai.2016.06.012
|
12 |
J Polastre, J Hill, D Culler. Versatile low power media access for wireless sensor networks. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems. 2004, 95–107
https://doi.org/10.1145/1031495.1031508
|
13 |
S Du, A K Saha, D B Johnson. RMAC: a routing-enhanced duty-cycle MAC protocol for wireless sensor networks. In: Proceedings of IEEE INFOCOM 2007 — the 26th IEEE International Conference on Computer Communications. 2007, 1478–1486
https://doi.org/10.1109/INFCOM.2007.174
|
14 |
F Tong, W Tang, R Xie, L Shu, Y C Kim. P-MAC: a cross-layer duty cycle MAC protocol towards pipelining for wireless sensor networks. In: Proceedings of IEEE International Conference on Communications (ICC). 2011, 1–5
https://doi.org/10.1109/icc.2011.5962446
|
15 |
P Lin, C Qiao, X Wang. Medium access control with a dynamic duty cycle for sensor networks. In: Proceedings of IEEE Wireless Communications and Networking Conference. 2004, 1534–1539
|
16 |
J Polastre, J Hill, D Culler. Versatile low power media access for wireless sensor networks. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems. 2004, 95–107
https://doi.org/10.1145/1031495.1031508
|
17 |
M Buettner, G V Yee, E Anderson, R Han. X-MAC: a short preamble MAC protocol for duty-cycled wireless sensor networks. In: Proceedings of the 4th International Conference on Embedded Networked Sensor Systems. 2006, 307–320
https://doi.org/10.1145/1182807.1182838
|
18 |
Y Sun, O Gurewitz, D B Johnson. RI-MAC: a receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks. In: Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems. 2008, 1–14
https://doi.org/10.1145/1460412.1460414
|
19 |
J Niu, Z Deng. Distributed self-learning scheduling approach for wireless sensor network. Ad Hoc Networks, 2013, 11(4): 1276–1286
https://doi.org/10.1016/j.adhoc.2010.11.004
|
20 |
R S Sutton, A G Barto. Reinforcement Learning: An Introduction. MIT Press, 2018
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|