Determining node duty cycle using Q-learning and linear regression for WSN

doi:10.1007/s11704-020-9153-6

Front. Comput. Sci.

2021, Vol. 15

Issue (1) : 151101 https://doi.org/10.1007/s11704-020-9153-6

RESEARCH ARTICLE

Determining node duty cycle using Q-learning and linear regression for WSN

Han Yao HUANG, Kyung Tae KIM, Hee Yong YOUN(

)

College of Software, Sungkyunkwan University, Suwon 440-746, Korea

Download: PDF(401 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Wireless sensor network (WSN) is effective for monitoring the target environment,which consists of a large number of sensor nodes of limited energy. An efficient medium access control (MAC) protocol is thus imperative to maximize the energy efficiency and performance of WSN. The most existing MAC protocols are based on the scheduling of sleep and active period of the nodes, and do not consider the relationship between the load condition and performance. In this paper a novel scheme is proposed to properly determine the duty cycle of the WSN nodes according to the load,which employs the Q-learning technique and function approximation with linear regression. This allows low-latency energy-efficient scheduling for a wide range of traffic conditions, and effectively overcomes the limitation of Q-learning with the problem of continuous state-action space. NS3 simulation reveals that the proposed scheme significantly improves the throughput, latency, and energy efficiency compared to the existing fully active scheme and S-MAC.

Keywords wireless sensor network media access control duty-cycle scheduling Q-learning linear regression

Corresponding Author(s): Hee Yong YOUN

Just Accepted Date: 26 February 2020 Issue Date: 24 September 2020

Cite this article:

Han Yao HUANG,Kyung Tae KIM,Hee Yong YOUN. Determining node duty cycle using Q-learning and linear regression for WSN[J]. Front. Comput. Sci., 2021, 15(1): 151101.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-020-9153-6
https://academic.hep.com.cn/fcs/EN/Y2021/V15/I1/151101

1	A Alemdar, M Ibnkahla. Wireless sensor networks: applications and challenges. In: Proceedings of the 9th International Symposium on Signal Processing and Its Applications. 2007, 1–6 https://doi.org/10.1109/ISSPA.2007.4555630
2	T AlSkaif, B Bellalta, M G Zapata, J M B Ordinas. Energy efficiency of MAC protocols in low data rate wireless multimedia sensor networks: a comparative study. Ad Hoc Networks, 2017, 56: 141–157 https://doi.org/10.1016/j.adhoc.2016.12.005
3	T Van Dam, K Langendoen. An adaptive energy-efficient MAC protocol for wireless sensor networks. In: Proceedings of the 1st International Conference on Embedded Networked Sensor Systems. 2003, 171–180 https://doi.org/10.1145/958491.958512
4	R Zheng, R Kravets. On-demand power management for ad hoc networks. Ad Hoc Networks, 2005, 3(1): 51–68 https://doi.org/10.1016/j.adhoc.2003.09.008
5	R Zheng, J C Hou, L Sha. Asynchronous wakeup for ad hoc networks. In: Proceedings of the 4th ACM International Symposium on Mobile Ad Hoc Networking & Computing. 2003, 35–45 https://doi.org/10.1145/778415.778420
6	W Ye, J Heidemann, D Estrin. An energy-efficient MAC protocol for wireless sensor networks. In: Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies. 2002, 1567–1576
7	W Ye, J Heidemann, D Estrin. Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking (ToN), 2004, 12(3): 493–506 https://doi.org/10.1109/TNET.2004.828953
8	E S Jung, N H Vaidya. An energy efficient MAC protocol for wireless LANs. In: Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies. 2002, 1756–1764
9	S Liu, KW Fan, P Sinha. CMAC: an energy-efficient MAC layer protocol using convergent packet forwarding for wireless sensor networks. ACM Transactions on Sensor Networks (TOSN), 2009, 5(4): 29 https://doi.org/10.1145/1614379.1614381
10	F L Lewis, K G Vamvoudakis. Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2011, 41(1): 14–25 https://doi.org/10.1109/TSMCB.2010.2043839
11	S Kosunalp, Y Chu, P D Mitchell, D Grace, T Clarke. Use of Q-learning approaches for practical medium access control in wireless sensor networks. Engineering Applications of Artificial Intelligence, 2016, 55: 146–154 https://doi.org/10.1016/j.engappai.2016.06.012
12	J Polastre, J Hill, D Culler. Versatile low power media access for wireless sensor networks. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems. 2004, 95–107 https://doi.org/10.1145/1031495.1031508
13	S Du, A K Saha, D B Johnson. RMAC: a routing-enhanced duty-cycle MAC protocol for wireless sensor networks. In: Proceedings of IEEE INFOCOM 2007 — the 26th IEEE International Conference on Computer Communications. 2007, 1478–1486 https://doi.org/10.1109/INFCOM.2007.174
14	F Tong, W Tang, R Xie, L Shu, Y C Kim. P-MAC: a cross-layer duty cycle MAC protocol towards pipelining for wireless sensor networks. In: Proceedings of IEEE International Conference on Communications (ICC). 2011, 1–5 https://doi.org/10.1109/icc.2011.5962446
15	P Lin, C Qiao, X Wang. Medium access control with a dynamic duty cycle for sensor networks. In: Proceedings of IEEE Wireless Communications and Networking Conference. 2004, 1534–1539
16	J Polastre, J Hill, D Culler. Versatile low power media access for wireless sensor networks. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems. 2004, 95–107 https://doi.org/10.1145/1031495.1031508
17	M Buettner, G V Yee, E Anderson, R Han. X-MAC: a short preamble MAC protocol for duty-cycled wireless sensor networks. In: Proceedings of the 4th International Conference on Embedded Networked Sensor Systems. 2006, 307–320 https://doi.org/10.1145/1182807.1182838
18	Y Sun, O Gurewitz, D B Johnson. RI-MAC: a receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks. In: Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems. 2008, 1–14 https://doi.org/10.1145/1460412.1460414
19	J Niu, Z Deng. Distributed self-learning scheduling approach for wireless sensor network. Ad Hoc Networks, 2013, 11(4): 1276–1286 https://doi.org/10.1016/j.adhoc.2010.11.004
20	R S Sutton, A G Barto. Reinforcement Learning: An Introduction. MIT Press, 2018

[1]

Article highlights

Download

[1]	Bartłomiej PŁACZEK. Decision-aware data suppression in wireless sensor networks for target tracking applications[J]. Front. Comput. Sci., 2017, 11(6): 1050-1060.
[2]	Chengliang WANG,Yayun PENG,Debraj DE,Wen-Zhan SONG. DPHK: real-time distributed predicted data collecting based on activity pattern knowledge mined from trajectories in smart environments[J]. Front. Comput. Sci., 2016, 10(6): 1000-1011.
[3]	Bin WANG,Xiaochun YANG,Guoren WANG,Ge YU,Wanyu ZANG,Meng YU. Energy efficient approximate self-adaptive data collection in wireless sensor networks[J]. Front. Comput. Sci., 2016, 10(5): 936-950.
[4]	Adnan AHMED,Kamalrulnizam ABU BAKAR,Muhammad Ibrahim CHANNA,Khalid HASEEB,Abdul Waheed KHAN. A survey on trust based detection and isolation of malicious nodes in ad-hoc and sensor networks[J]. Front. Comput. Sci., 2015, 9(2): 280-296.
[5]	Defu CHEN, Zhengsu TAO. An adaptive polling interval and short preamble media access control protocol for wireless sensor networks[J]. Front Comput Sci Chin, 2011, 5(3): 300-307.
[6]	Jun XU, Xuehai ZHOU, Feng YANG. Traceback in wireless sensor networks with packet marking and logging[J]. Front Comput Sci Chin, 2011, 5(3): 308-315.
[7]	Jiannong CAO, Xuefeng LIU, Yi LAI, Hejun WU, . iSensNet: an infrastructure for research and development in wireless sensor networks[J]. Front. Comput. Sci., 2010, 4(3): 339-353.
[8]	Ye LI, Honggang LI, Yuwei ZHANG, Dengyu QIAO, . Packet transmission policies for battery operated wireless sensor networks[J]. Front. Comput. Sci., 2010, 4(3): 365-375.
[9]	Wei QU, Zhe LI, . Research of localization approach for the new comer in wireless sensor networks[J]. Front. Comput. Sci., 2009, 3(4): 543-549.
[10]	XU Jianliang, TANG Xueyan, LEE Wang-Chien. Distributed query processing in flash-based sensor networks[J]. Front. Comput. Sci., 2008, 2(3): 248-256.
[11]	Réean Plamondon, Moussa Djioua, Xiaolin Li. Extraction of delta-lognormal parameters from handwriting strokes[J]. Front. Comput. Sci., 2007, 1(1): 106-113.

Viewed

Full text

Abstract

Cited

Shared

Discussed