Remote heart rate measurement using low-cost RGB face video: a technical literature review

doi:10.1007/s11704-016-6243-6

Front. Comput. Sci.

2018, Vol. 12

Issue (5) : 858-872 https://doi.org/10.1007/s11704-016-6243-6

REVIEW ARTICLE

Remote heart rate measurement using low-cost RGB face video: a technical literature review

Philipp V. ROUAST¹, Marc T. P. ADAM², Raymond CHIONG²(

), David CORNFORTH², Ewa LUX¹

¹. Institute of Information Systems and Marketing, Karlsruhe Institute of Technology, Karlsruhe 76133, Germany
². School of Electrical Engineering and Computing, The University of Newcastle, Callaghan NSW 2308, Australia

Download: PDF(2256 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Remote photoplethysmography (rPPG) allows remote measurement of the heart rate using low-cost RGB imaging equipment. In this study, we review the development of the field of rPPG since its emergence in 2008. We also classify existing rPPG approaches and derive a framework that provides an overview of modular steps. Based on this framework, practitioners can use our classification to design algorithms for an rPPG approach that suits their specific needs. Researchers can use the reviewed and classified algorithms as a starting point to improve particular features of an rPPG algorithm.

Keywords affective computing heart rate measurement remote non-contact camera-based photoplethysmography

Corresponding Author(s): Raymond CHIONG

Just Accepted Date: 23 August 2016 Online First Date: 20 December 2017 Issue Date: 21 September 2018

Cite this article:

Philipp V. ROUAST,Marc T. P. ADAM,Raymond CHIONG, et al. Remote heart rate measurement using low-cost RGB face video: a technical literature review[J]. Front. Comput. Sci., 2018, 12(5): 858-872.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-016-6243-6
https://academic.hep.com.cn/fcs/EN/Y2018/V12/I5/858

1	Zhang Z L, Pi Z Y, Liu B Y. Troika: a general framework for heart rate monitoring using wrist-type photoplethysmographic signals during intensive physical exercise. IEEE Transactions on Biomedical Engineering, 2015, 62(2): 522–531 https://doi.org/10.1109/TBME.2014.2359372
2	Adam M T P, Krämer J, Weinhardt C. Excitement up! Price down! Measuring emotions in dutch auctions. International Journal of Electronic Commerce, 2012, 13(2): 7–39 https://doi.org/10.2753/JEC1086-4415170201
3	Adam M T P, Krämer J, Müller M B. Auction fever! How time pressure and social competition affect bidders’ arousal and bids in retail auctions. Journal of Retailing, 2015, 91(3): 468–485 https://doi.org/10.1016/j.jretai.2015.01.003
4	Astor P J, Adam M T P, Jeřcíc P, Schaaff K, Weinhardt C. Integrating biosignals into information systems: a neurois tool for improving emotion regulation. Journal of Management Information Systems, 2013, 30(3): 247–278 https://doi.org/10.2753/MIS0742-1222300309
5	Riedl R. On the biology of technostress: literature review and research agenda. ACM SIGMIS Database, 2013, 44(1): 18–55 https://doi.org/10.1145/2436239.2436242
6	Verkruysse W, Svaasand L O, Nelson J S. Remote plethysmographic imaging using ambient light. Optics Express, 2008, 16(26): 21434–21445 https://doi.org/10.1364/OE.16.021434
7	McDuff D J, Estepp J R, Piasecki A M, Blackford E B. A survey of remote optical photoplethysmographic imaging methods. In: Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2015, 6398–6404 https://doi.org/10.1109/EMBC.2015.7319857
8	Liu H, Wang Y D, Wang L. A review of non-contact, low-cost physiological information measurement based on photoplethysmographic imaging. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012, 2088–2091
9	Kranjec J, Beguš S, Geršak G, Drnovšek J. Non-contact heart rate and heart rate variability measurements: a review. Biomedical Signal Processing and Control, 2014, 13(1): 102–112 https://doi.org/10.1016/j.bspc.2014.03.004
10	Poh M Z, McDuff D J, Picard R W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics Express, 2010, 18(10): 10762–10774 https://doi.org/10.1364/OE.18.010762
11	Allen J. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement, 2007, 28(3): R1–R39 https://doi.org/10.1088/0967-3334/28/3/R01
12	Lindberg L G, Öberg P A. Optical properties of blood in motion. Optical Engineering, 1993, 32(2): 253–257 https://doi.org/10.1117/12.60688
13	Balakrishnan G, Durand F, Guttag J. Detecting pulse from head motions in video. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2013, 3430–3437 https://doi.org/10.1109/CVPR.2013.440
14	Starr I, Rawson A J, Schroeder H A, Joseph N R. Studies on the estimation of cardiac ouptut in man, and of abnormalities in cardiac function, from the heart’s recoil and the blood’s impacts; the ballistocardiogram. American Journal of Physiology, 1939, 127(1): 1–28
15	Hertzman A B, Spealman C R. Observations on the finger volume pulse recorded photoelectrically. American Journal of Physiology, 1937, 119(2): 334–335
16	Poh M Z, McDuff D J, Picard R W. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Transactions on Biomedical Engineering, 2011, 58(1): 7–11 https://doi.org/10.1109/TBME.2010.2086456
17	Lewandowska M, Ruminski J, Kocejko T. Measuring pulse rate with a webcam: a non-contact method for evaluating cardiac activity. In: Proceedings of the Federated Conference on Computer Science and Information Systems (FedCSIS). 2011, 405–410
18	Kwon S, Kim H, Park K S. Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2012, 2174–2177
19	Lee K Z, Hung P C, Tsai L W. Contact-free heart rate measurement using a camera. In: Proceedings of the 9th Conference on Computer and Robot Vision. 2012, 147–152 https://doi.org/10.1109/CRV.2012.27
20	Wei L, Tian Y H, Wang Y W, Ebrahimi T, Huang T. Automatic webcam-based human heart rate measurements using laplacian eigenmap. In: Proceedings of Asian Conference on Computer Vision. 2013, 281–292 https://doi.org/10.1007/978-3-642-37444-9_22
21	Wu H Y, Rubinstein M, Shih E, Guttag J V, Durand F, Freeman W T. Eulerian video magnification for revealing subtle changes in the world. ACM Transactions on Graphics, 2012, 31(4): 1–8 https://doi.org/10.1145/2185520.2185561
22	Wu H Y. Eulerian video processing and medical applications. Dissertation for the Master Degree. Cambridge, MA: Massachusetts Institute of Technology, 2012
23	Shan L, Yu M H. Video-based heart rate measurement using head motion tracking and ICA. In: Proceedings of the 6th International Congress on Image and Signal Processing. 2013, 160–164 https://doi.org/10.1109/CISP.2013.6743978
24	Irani R, Nasrollahi K, Moeslund T B. Improved pulse detection from head motions using DCT. In: Proceedings of the 9th International Conference on Computer Vision Theory and Applications. 2014, 118–124
25	De Haan G, Jeanne V. Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering, 2013, 60(10): 2878–2886 https://doi.org/10.1109/TBME.2013.2266196
26	De Haan G, Van Leest A. Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiological Measurement, 2014, 35(9): 1913–1926 https://doi.org/10.1088/0967-3334/35/9/1913
27	Lempe G, Zaunseder S, Wirthgen T, Zipser S, Malberg H. ROI selection for remote photoplethysmography. In: Meinzer H P, Deserno M T, Handels H, et al, eds. Bildverarbeitung für die Medizin 2013. Berlin: Springer, 2013, 99–103 https://doi.org/10.1007/978-3-642-36480-8_19
28	Datcu D, Cidota M, Lukosch S, Rothkrantz L. Noncontact automatic heart rate analysis in visible spectrum by specific face regions. In: Proceedings of the 14th International Conference on Computer Systems and Technologies. 2013, 120–127 https://doi.org/10.1145/2516775.2516805
29	Li X B, Chen J, Zhao G Y, Pietikäinen M. Remote heart rate measurement from face videos under realistic situations. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2014, 4264–4271 https://doi.org/10.1109/CVPR.2014.543
30	McDuff D, Gontarek S, Picard R W. Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera. IEEE Transactions on Biomedical Engineering, 2014, 61(12): 2948–2954 https://doi.org/10.1109/TBME.2014.2340991
31	Kumar M, Veeraraghavan A, Sabharwal A. DistancePPG: robust noncontact vital signs monitoring using a camera. Biomedical Optics Express, 2015, 6(5): 1565–1588 https://doi.org/10.1364/BOE.6.001565
32	Tasli H E, Gudi A, Den Uyl M. Remote PPG based vital sign measurement using adaptive facial regions. In: Proceedings of IEEE International Conference on Image Processing. 2014, 1410–1414 https://doi.org/10.1109/ICIP.2014.7025282
33	Feng L T, Po L M, Xu X Y, Li Y M. Motion artifacts suppression for remote imaging photoplethysmography. In: Proceedings of the 19th International Conference on Digital Signal Processing. 2014, 18–23 https://doi.org/10.1109/ICDSP.2014.6900813
34	Feng L T, Po L M, Xu X Y, Li Y M, Ma R Y. Motion-resistant remote imaging photoplethysmography based on the optical properties of skin. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(5): 879–891 https://doi.org/10.1109/TCSVT.2014.2364415
35	Wang W J, Stuijk S, De Haan G. Exploiting spatial redundancy of image sensor for motion robust rPPG. IEEE Transactions on Biomedical Engineering, 2015, 62(2): 415–425 https://doi.org/10.1109/TBME.2014.2356291
36	McDuff D, Gontarek S, Picard R W. Improvements in remote cardiopulmonary measurement using a five band digital camera. IEEE Transactions on Biomedical Engineering, 2014, 61(10): 2593–2601 https://doi.org/10.1109/TBME.2014.2323695
37	Chwyl B, Chung A G, Deglint J, Wong A, Claus i D A. Remote heart rate measurement through broadband video via stochastic bayesian estimation. Vision Letters, 2015, 1(1): 5 https://doi.org/10.15353/vsnl.v1i1.43
38	Monkaresi H, Calvo R A, Yan H. A machine learning approach to improve contactless heart rate monitoring using a webcam. IEEE Journal of Biomedical and Health Informatics, 2014, 18(4): 1153–1160 https://doi.org/10.1109/JBHI.2013.2291900
39	Hsu Y, Lin Y L, Hsu W. Learning-based heart rate detection from remote photoplethysmography features. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2014, 4433–4437 https://doi.org/10.1109/ICASSP.2014.6854440
40	Tran D N, Lee H, Kim C. A robust real time system for remote heart rate measurement via a camera. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2015, 1–6
41	Li M C, Lin Y H. A real-time non-contact pulse rate detector based on smartphone. In: Proceedings of International Symposium on Next- Generation Electronics. 2015, 1–3 https://doi.org/10.1109/ISNE.2015.7132025
42	Yu Y P, Kwan B H, Lim C L, Wong S L, Raveendran P. Video-based heart rate measurement using short-time fourier transform. In: Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems. 2013, 704–707 https://doi.org/10.1109/ISPACS.2013.6704640
43	Holton B D, Mannapperuma K, Lesniewski P J, Thomas J C. Signal recovery in imaging photoplethysmography. Physiological Measurement, 2013, 34(11): 1499–1511 https://doi.org/10.1088/0967-3334/34/11/1499
44	Xu S C, Sun L Y, Rohde G K. Robust efficient estimation of heart rate pulse from video. Biomedical Optics Express, 2014, 5(4): 1124–1135 https://doi.org/10.1364/BOE.5.001124
45	Feng L T, Po L M, Xu X Y, Li Y M, Cheung C-H, Cheung K-W, Yuan F. Dynamic ROI based on K-means for remote photoplethysmography. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2015, 1310–1314 https://doi.org/10.1109/ICASSP.2015.7178182
46	Han B, Ivanov K, Wang L, Yan Y. Exploration of the optimal skincamera distance for facial photoplethysmographic imaging measurement using cameras of different types. In: Proceedings of the 5th EAI International Conference onWireless Mobile Communication and Healthcare. 2015, 186–189
47	Zhang K H, Zhang L, Yang M H. Real-time compressive tracking. In: Proceedings of European Conference on Computer Vision. 2012, 864–877 https://doi.org/10.1007/978-3-642-33712-3_62
48	Fernández A, Carúz J L, Usamentiaga R, Alvarez E, Casado R. Unobtrusive health monitoring system using video-based physiological information and activity measurements. In: Proceedings of International Conference on Computer, Information and Telecommunication Systems. 2012, 1–5
49	Danelljan M, Häger G, Felsberg M. Accurate scale estimation for robust visual tracking. In: Proceedings of the British Machine Vision Conference. 2014, 1–10 https://doi.org/10.5244/C.28.65
50	Hoffmann K P. Biosignale erfassen und verarbeiten. In: Kramme R, ed. Medizintechnik. Berlin: Springer, 2011, 667–688 https://doi.org/10.1007/978-3-642-16187-2_38
51	Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2001, 511–518 https://doi.org/10.1109/CVPR.2001.990517
52	Cootes T F, Edwards G J, Taylor C J. Active appearance models. In: Proceedings of European Conference on Computer Vision. 1998, 484–498 https://doi.org/10.1007/BFb0054760
53	Asthana A, Zafeiriou S, Cheng S Y, Pantic M. Robust discriminative response map fitting with constrained local models. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2013, 3444–3451 https://doi.org/10.1109/CVPR.2013.442
54	Saragih J M, Lucey S, Cohn J F. Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 2011, 91(2): 200–215 https://doi.org/10.1007/s11263-010-0380-4
55	Martinez B, Valstar M F, Binefa X, Pantic M. Local evidence aggregation for regression-based facial point detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(5): 1149–1163 https://doi.org/10.1109/TPAMI.2012.205
56	Shi J B, Tomasi C. Good features to track. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1994, 593–600
57	Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence. 1981, 674–679
58	Bay H, Tuytelaars T, Van Gool L. SURF: speeded up robust features. In: Proceedings of European Conference on Computer Vision. 2006, 404–417 https://doi.org/10.1007/11744023_32
59	Comaniciu D, Ramesh V, Meer P. Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(5): 564–577 https://doi.org/10.1109/TPAMI.2003.1195991
60	Henriques J F, Caseiro R, Martins P, Batista J. Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of European Conference on Computer Vision. 2012, 702–715 https://doi.org/10.1007/978-3-642-33765-9_50
61	Farnebäck G. Two-frame motion estimation based on polynomial expansion. In: Proceedings of Scandinavian Conference on Image Analysis. 2003, 363–370 https://doi.org/10.1007/3-540-45103-X_50
62	Tarvainen M P, Ranta-Aho P O, Karjalainen P A. An advanced detrending method with application to HRV analysis. IEEE Transactions on Biomedical Engineering, 2002, 49(2): 172–175 https://doi.org/10.1109/10.979357
63	Rouast P V, Adam M T P, Cornforth D J, Lux E, Weinhardt C. Using contactless heart rate measurements for real-time assessment of affective states. In: Davis F D, Riedl R, Vom Brocke J, et al, eds. Information Systems and Neuroscience. Springer International Publishing, 2016, 157–163
64	Monkaresi H, Hussain M S, Calvo R A. Using remote heart rate measurement for affect detection. In: Proceedings of the 27th International Florida Artificial Intelligence Research Society Conference. 2014, 118–123
65	Zhao F, Li M, Qian Y, Tsien J Z. Remote measurements of heart and respiration rates for telemedicine. PLoS ONE, 2013, 8(10): e71384 https://doi.org/10.1371/journal.pone.0071384
66	McDuff D, Gontarek S, Picard R. Remote measurement of cognitive stress via heart rate variability. In: Proceedings of the 36th IEEE Annual International Conference of Engineering in Medicine and Biology Society. 2014, 2957–2960 https://doi.org/10.1109/EMBC.2014.6944243
67	Rahman M A, Barai A, Islam M A, Hashem M M A. Development of a device for remote monitoring of heart rate and body temperature. In: Proceedings of the 15th International Conference on Computer and Information Technology. 2012, 411–416 https://doi.org/10.1109/ICCITechn.2012.6509783

[1]

Download

[1]	Xingyue CHEN, Tao SHANG, Feng ZHANG, Jianwei LIU, Zhenyu GUAN. Dynamic data auditing scheme for big data storage[J]. Front. Comput. Sci., 2020, 14(1): 219-229.
[2]	Lu YU,Jun XIE,Songcan CHEN,Lei ZHU. Generating labeled samples for hyperspectral image classification using correlation of spectral bands[J]. Front. Comput. Sci., 2016, 10(2): 292-301.
[3]	Peng JIANG,Qiaoyan WEN,Wenmin LI,Zhengping JIN,Hua ZHANG. An anonymous and efficient remote biometrics user authentication scheme in a multi server environment[J]. Front. Comput. Sci., 2015, 9(1): 142-156.

Viewed

Full text

Abstract

Cited

Shared

Discussed