An efficient deep learning-assisted person re-identification solution for intelligent video surveillance in smart cities

doi:10.1007/s11704-022-2050-4

Front. Comput. Sci.

2023, Vol. 17

Issue (4) : 174329 https://doi.org/10.1007/s11704-022-2050-4

RESEARCH ARTICLE

An efficient deep learning-assisted person re-identification solution for intelligent video surveillance in smart cities

Muazzam MAQSOOD¹, Sadaf YASMIN¹, Saira GILLANI², Maryam BUKHARI¹, Seungmin RHO³(

), Sang-Soo YEO⁴(

)

¹. Department of Computer Science, COMSATS University Islamabad, Attock Campus, Attock 43600, Pakistan
². Department of Computer Science, Bahria University, Lahore 54600, Pakistan
³. Department of Industrial Security, Chung-Ang University, Seoul 06974, Republic of Korea
⁴. Department of Computer Engineering, Mokwon University, Daejeon 35349, Republic of Korea

Download: PDF(17519 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Innovations on the Internet of Everything (IoE) enabled systems are driving a change in the settings where we interact in smart units, recognized globally as smart city environments. However, intelligent video-surveillance systems are critical to increasing the security of these smart cities. More precisely, in today’s world of smart video surveillance, person re-identification (Re-ID) has gained increased consideration by researchers. Various researchers have designed deep learning-based algorithms for person Re-ID because they have achieved substantial breakthroughs in computer vision problems. In this line of research, we designed an adaptive feature refinement-based deep learning architecture to conduct person Re-ID. In the proposed architecture, the inter-channel and inter-spatial relationship of features between the images of the same individual taken from nonidentical camera viewpoints are focused on learning spatial and channel attention. In addition, the spatial pyramid pooling layer is inserted to extract the multiscale and fixed-dimension feature vectors irrespective of the size of the feature maps. Furthermore, the model’s effectiveness is validated on the CUHK01 and CUHK02 datasets. When compared with existing approaches, the approach presented in this paper achieves encouraging Rank 1 and 5 scores of 24.6% and 54.8%, respectively.

Keywords Internet of Everything (IoE) visual surveillance systems big data security systems person re-identification (Re-ID) deep learning

Corresponding Author(s): Seungmin RHO,Sang-Soo YEO

Just Accepted Date: 12 July 2022 Issue Date: 12 December 2022

Cite this article:

Muazzam MAQSOOD,Sadaf YASMIN,Saira GILLANI, et al. An efficient deep learning-assisted person re-identification solution for intelligent video surveillance in smart cities[J]. Front. Comput. Sci., 2023, 17(4): 174329.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-022-2050-4
https://academic.hep.com.cn/fcs/EN/Y2023/V17/I4/174329

Fig.1 Samples of pair images of the same person under different camera viewpoints

Fig.2 Pictorial representation of the internal working of the person re-identification system

Fig.3 Internal architecture of the proposed deep learning model for re-identification of individuals

Fig.4 Determining feature inter-channel and inter-spatial relationships

Fig.5 Parallel arrangement of attention mechanisms in the adaptive feature refinement block

Fig.6 Spatial pyramid pooling layer

Tab.1 Subject-wise division of the dataset per camera pair

Fig.7 (a) Ranks 1 to 15 for the identification rates of the overall camera view pairs in the series setting of the adaptive feature refinement block, (b) Ranks 1 to 15 for the identification rates of the overall camera view pairs in the parallel setting of the adaptive feature refinement block, and (c) Ranks 1 to 100 for the identification rates over the P1 camera view pairs in both series and parallel settings of the adaptive feature refinement

Fig.8 (a, b) Accuracy and loss curves during training for all camera views in both attention mechanism arrangements

Fig.9 (a, b, c) Ranks 1 to 15 of the identification rates for the overall camera view pairs in series and parallel settings for the adaptive feature refinement block with the stochastic gradient descent optimizer (

l r = 0.001)

. (d, e, f) Ranks 1 to 15 of the identification rates for the overall camera view pairs in mixed configurations for sequential and parallel arrangements of adaptive feature refinement blocks

Tab.2 Results of the model with unknown subject and viewpoint

Tab.3 Dataset division with an unknown subject and viewpoints

Fig.10 Top five matches against query image from the gallery set

Tab.4 Rank-wise comparison with some existing approaches

1	P, Neirotti Marco A, De A C, Cagliano G, Mangano F Scorrano . Current trends in smart city initiatives: some stylised facts. Cities, 2014, 38: 25–36
2	P, Vlacheas R, Giaffreda V, Stavroulaki D, Kelaidonis V, Foteinos G, Poulios P, Demestichas A, Somov A R, Biswas K Moessner . Enabling smart cities through a cognitive management framework for the internet of things. IEEE Communications Magazine, 2013, 51( 6): 102–111
3	P, Singh A, Nayyar A, Kaur U Ghosh . Blockchain and fog based architecture for internet of everything in smart cities. Future Internet, 2020, 12( 4): 61
4	L, Zheng Y, Yang A G Hauptmann . Person re-identification: past, present and future. 2016, arXiv preprint arXiv: 1610.02984
5	D, Wu S J, Zheng X P, Zhang C A, Yuan F, Cheng Y, Zhao Y J, Lin Z-Q, Zhao Y L, Jiang D S Huang . Deep learning-based methods for person re-identification: a comprehensive review. Neurocomputing, 2019, 337: 354–371
6	A, Zahra N, Perwaiz M, Shahzad M M Fraz . Person re-identification: a retrospective on domain specific open challenges and future trends. 2022, arXiv preprint arXiv: 2202.13121
7	M, Ye J, Shen G, Lin T, Xiang L, Shao S C H Hoi . Deep learning for person re-identification: a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44( 6): 2872–2893
8	W, Wu D, Tao H, Li Z, Yang J Cheng . Deep features for person re-identification on metric learning. Pattern Recognition, 2021, 110: 107424
9	X, Chen H, Xu Y, Li M Bian . Person re-identification by low-dimensional features and metric learning. Future Internet, 2021, 13( 11): 289
10	R, Li B, Zhang Z, Teng J Fan . A divide-and-unite deep network for person re-identification. Applied Intelligence, 2021, 51( 3): 1479–1491
11	Z, Ming M, Zhu X, Wang J, Zhu J, Cheng C, Gao Y, Yang X Wei . Deep learning-based person re-identification methods: a survey and outlook of recent works. Image and Vision Computing, 2022, 119: 104394
12	S, Lin C T Li . Person re-identification with soft biometrics through deep learning. In: Jiang R, Li C T, Crookes D, Meng W, Rosenberger C, eds. Deep Biometrics. Cham: Springer, 2020, 21–36
13	N, Shoukry El Ghany MA, Abd M A M Salem . Multi-modal long-term person re-identification using physical soft bio-metrics and body figure. Applied Sciences, 2022, 12( 6): 2835
14	A, Nambiar A, Bernardino J C Nascimento . Gait-based person re-identification: a survey. ACM Computing Surveys, 2020, 52( 2): 33
15	S, Woo J, Park J-Y, Lee I S Kweon . CBAM: convolutional block attention module. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 3–19
16	W, Li R, Zhao T, Xiao X DeepReID: deep filter pairing neural network for person re-identification Wang . In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014, 152–159
17	E, Ahmed M, Jones T K Marks . An improved deep learning architecture for person re-identification. In: Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3908–3916
18	S, Chen J, Qin X, Ji B, Lei T, Wang D, Ni J-Z Cheng . Automatic scoring of multiple semantic attributes with multi-task feature leverage: a study on pulmonary nodules in CT images. IEEE Transactions on Medical Imaging, 2017, 36( 3): 802–814
19	Y, Huang H, Sheng Y, Zheng Z Xiong . DeepDiff: learning deep difference features on human body parts for person re-identification. Neurocomputing, 2017, 241: 191–203
20	H, Zhao M, Tian S, Sun J, Shao J, Yan S, Yi X, Wang X Tang . Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 907–915
21	A, Hermans L, Beyer B Leibe . In defense of the triplet loss for person re-identification. 2017, arXiv preprint arXiv: 1703.07737
22	Z, He C, Jung Q, Fu Z Zhang . Deep feature embedding learning for person re-identification based on lifted structured loss. Multimedia Tools and Applications, 2019, 78( 5): 5863–5880
23	L, Wu Y, Wang X, Li J Gao . What-and-where to match: deep spatially multiplicative integration networks for person re-identification. Pattern Recognition, 2018, 76: 727–738
24	K, Chatfield K, Simonyan A, Vedaldi A Zisserman . Return of the devil in the details: delving deep into convolutional nets. In: Proceedings of British Machine Vision Conference. 2014
25	K, Simonyan A Zisserman . Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations. 2015
26	L, Wu R, Hong Y, Wang M Wang . Cross-entropy adversarial view adaptation for person re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30( 7): 2081–2092
27	X, Zhu J, Liu H, Xie Z-J Zha . Adaptive alignment network for person re-identification. In: Proceedings of the 25th International Conference on Multimedia Modeling. 2019, 16–27
28	A, Wu W-S, Zheng J-H Lai . Robust depth-based person re-identification. IEEE Transactions on Image Processing, 2017, 26( 6): 2588–2603
29	Z, Imani H Soltanizadeh . Histogram of the node strength and histogram of the edge weight: two new features for RGB-D person re-identification. Science China Information Sciences, 2018, 61( 9): 092108
30	L, Ren J, Lu J, Feng J Zhou . Multi-modal uniform deep learning for RGB-D person re-identification. Pattern Recognition, 2017, 72: 446–457
31	A, Wu W-S, Zheng H-X, Yu S, Gong J Lai . RGB-infrared cross-modality person re-identification. In: Proceedings of 2017 IEEE International Conference on Computer Vision. 2017, 5390–5399
32	A, Møgelmose C, Bahnsen T B, Moeslund A, Clapes S Escalera . Tri-modal person re-identification with RGB, depth and thermal features. In: Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2013, 301–307
33	B N, Silva M, Khan K Han . Towards sustainable smart cities: a review of trends, architectures, components, and open challenges in smart cities. Sustainable Cities and Society, 2018, 38: 697–713
34	U, Majeed L U, Khan I, Yaqoob S M A, Kazmi K, Salah C S Hong . Blockchain for IoT-based smart cities: recent advances, requirements, and future challenges. Journal of Network and Computer Applications, 2021, 181: 103007
35	F, Ullah F, Al-Turjman A Nayyar . IoT-based green city architecture using secured and sustainable android services. Environmental Technology & Innovation, 2020, 20: 101091
36	J, Li J, Wang F Ullah . An end-to-end task-simplified and anchor-guided deep learning framework for image-based head pose estimation. IEEE Access, 2020, 8: 42458–42468
37	D H, Hubel T N Wiesel . Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology, 1968, 195( 1): 215–243
38	M, Bukhari K B, Bajwa S, Gillani M, Maqsood M Y, Durrani I, Mehmood H, Ugail S Rho . An efficient gait recognition method for known and unknown covariate conditions. IEEE Access, 2021, 9: 6465–6477
39	R, Ashraf S, Afzal A U, Rehman S, Gul J, Baber M, Bakhtyar I, Mehmood O Y, Song M Maqsood . Region-of-interest based transfer learning assisted framework for skin cancer detection. IEEE Access, 2020, 8: 147858–147871
40	M, Maqsood M, Bukhari Z, Ali S, Gillani I, Mehmood S, Rho Y-A Jung . A residual-learning-based multi-scale parallel-convolutions- assisted efficient CAD system for liver tumor detection. Mathematics, 2021, 9( 10): 1133
41	M, Maqsood S, Yasmin I, Mehmood M, Bukhari M Kim . An efficient DA-net architecture for lung nodule segmentation. Mathematics, 2021, 9( 13): 1457
42	Z, Niu G, Zhong H Yu . A review on the attention mechanism of deep learning. Neurocomputing, 2021, 452: 48–62
43	M H, Guo T X, Xu J J, Liu Z N, Liu P T, Jiang T J, Mu S H, Zhang R R, Martin M M, Cheng S M Hu . Attention mechanisms in computer vision: a survey. Computational Visual Media, 2022, 8( 3): 331–368
44	K, He X, Zhang S, Ren J Sun . Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37( 9): 1904–1916
45	S, Lazebnik C, Schmid J Ponce . Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 2169–2178
46	W, Li X Wang . Locally aligned feature transforms across views. In: Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3594–3601
47	W, Li R, Zhao X Wang . Human reidentification with transferred metric learning. In: Proceedings of the 11th Asian Conference on Computer Vision. 2012, 31–44
48	M, Köstinger M, Hirzer P, Wohlhart P M, Roth H Bischof . Large scale metric learning from equivalence constraints. In: Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2288–2295
49	L, Zheng L, Shen L, Tian S, Wang J, Wang Q Tian . Scalable person re-identification: a benchmark. In: Proceedings of 2015 IEEE International Conference on Computer Vision. 2015, 1116–1124
50	H, Fan L, Zheng C, Yan Y Yang . Unsupervised person re-identification: clustering and fine-tuning. ACM Transactions on Multimedia Computing, Communications, and Applications, 2018, 14( 4): 83
51	G, Feng W, Liu D, Tao Y Zhou . Hessian regularized distance metric learning for people re-identification. Neural Processing Letters, 2019, 50( 3): 2087–2100

[1]

FCS-22050-OF-MM_suppl_1

Download

[1]	Yulin HE, Jiaqi CHEN, Jiaxing SHEN, Philippe FOURNIER-VIGER, Joshua Zhexue HUANG. Density estimation-based method to determine sample size for random sample partition of big data[J]. Front. Comput. Sci., 2024, 18(5): 185322-.
[2]	Lingling ZHAO, Shitao SONG, Pengyan WANG, Chunyu WANG, Junjie WANG, Maozu GUO. A MLP-Mixer and mixture of expert model for remaining useful life prediction of lithium-ion batteries[J]. Front. Comput. Sci., 2024, 18(5): 185329-.
[3]	Enes DEDEOGLU, Himmet Toprak KESGIN, Mehmet Fatih AMASYALI. A robust optimization method for label noisy datasets based on adaptive threshold: Adaptive-k[J]. Front. Comput. Sci., 2024, 18(4): 184315-.
[4]	Hengyu LIU, Tiancheng ZHANG, Fan LI, Minghe YU, Ge YU. A probabilistic generative model for tracking multi-knowledge concept mastery probability[J]. Front. Comput. Sci., 2024, 18(3): 183602-.
[5]	Mingzhi YUAN, Kexue FU, Zhihao LI, Manning WANG. Decoupled deep hough voting for point cloud registration[J]. Front. Comput. Sci., 2024, 18(2): 182703-.
[6]	Mingzhen LI, Changxi LIU, Jianjin LIAO, Xuegui ZHENG, Hailong YANG, Rujun SUN, Jun XU, Lin GAN, Guangwen YANG, Zhongzhi LUAN, Depei QIAN. Towards optimized tensor code generation for deep learning on sunway many-core processor[J]. Front. Comput. Sci., 2024, 18(2): 182101-.
[7]	Ashish SINGH, Abhinav KUMAR, Suyel NAMASUDRA. DNACDS: Cloud IoE big data security and accessing scheme based on DNA cryptography[J]. Front. Comput. Sci., 2024, 18(1): 181801-.
[8]	Hanadi AL-MEKHLAFI, Shiguang LIU. Single image super-resolution: a comprehensive review and recent insight[J]. Front. Comput. Sci., 2024, 18(1): 181702-.
[9]	Yamin HU, Hao JIANG, Zongyao HU. Measuring code maintainability with deep neural networks[J]. Front. Comput. Sci., 2023, 17(6): 176214-.
[10]	Yufei ZENG, Zhixin LI, Zhenbin CHEN, Huifang MA. Aspect-level sentiment analysis based on semantic heterogeneous graph convolutional network[J]. Front. Comput. Sci., 2023, 17(6): 176340-.
[11]	Tian WANG, Jiakun LI, Huai-Ning WU, Ce LI, Hichem SNOUSSI, Yang WU. ResLNet: deep residual LSTM network with longer input for action recognition[J]. Front. Comput. Sci., 2022, 16(6): 166334-.
[12]	Yi WEI, Mei XUE, Xin LIU, Pengxiang XU. Data fusing and joint training for learning with noisy labels[J]. Front. Comput. Sci., 2022, 16(6): 166338-.
[13]	Donghong HAN, Yanru KONG, Jiayi HAN, Guoren WANG. A survey of music emotion recognition[J]. Front. Comput. Sci., 2022, 16(6): 166335-.
[14]	Pinzhuo TIAN, Yang GAO. Improving meta-learning model via meta-contrastive loss[J]. Front. Comput. Sci., 2022, 16(5): 165331-.
[15]	Tian WANG, Shiye LEI, Youyou JIANG, Choi CHANG, Hichem SNOUSSI, Guangcun SHAN, Yao FU. Accelerating temporal action proposal generation via high performance computing[J]. Front. Comput. Sci., 2022, 16(4): 164317-.

Viewed

Full text

Abstract

Cited

Shared

Discussed