StreamTune: dynamic resource scheduling approach for workload skew in video data center

doi:10.1007/s11704-016-5438-1

Front. Comput. Sci.

2018, Vol. 12

Issue (4) : 669-681 https://doi.org/10.1007/s11704-016-5438-1

RESEARCH ARTICLE

StreamTune: dynamic resource scheduling approach for workload skew in video data center

Yihong GAO(

), Huadong MA

Beijing Key Lab of Intelligent Telecommunications Software and Multimedia, School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

Download: PDF(786 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Video surveillance applications need video data center to provide elastic virtual machine (VM) provisioning. However, the workloads of the VMs are hardly to be predicted for online video surveillance service. The unknown arrival workloads easily lead to workload skew among VMs. In this paper, we study how to balance the workload skew on online video surveillance system. First, we design the system framework for online surveillance service which consists of video capturing and analysis tasks. Second, we propose Stream Tune, an online resource scheduling approach for workload balancing, to deal with irregular video analysis workload with the minimum number of VMs. We aim at timely balancing the workload skew on video analyzers without depending on any workload prediction method. Furthermore, we evaluate the performance of the proposed approach using a traffic surveillance application. The experimental results show that our approach is well adaptive to the variation of workload and achieves workload balance with less VMs.

Keywords video data center load balancing stream computing online video analysis scheduling algorithm

Corresponding Author(s): Yihong GAO

Just Accepted Date: 14 June 2016 Online First Date: 22 September 2017 Issue Date: 14 June 2018

Cite this article:

Yihong GAO,Huadong MA. StreamTune: dynamic resource scheduling approach for workload skew in video data center[J]. Front. Comput. Sci., 2018, 12(4): 669-681.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-016-5438-1
https://academic.hep.com.cn/fcs/EN/Y2018/V12/I4/669

1	Hu H, Wen Y G, Chua T S, Li X L. Toward scalable systems for big data analytics: a technology tutorial. IEEE Access, 2014, 2: 652–687 https://doi.org/10.1109/ACCESS.2014.2332453
2	Ma H D, Liu L, Zhou A F, Zhao D. On networking of Internet of things: explorations and challenges. IEEE Internet of Things Journal, 2016, 3(4): 441–452 https://doi.org/10.1109/JIOT.2015.2493082
3	Ma H D. Internet of things: objectives and scientific challenges. Journal of Computer Science and Technology, 2011, 26(6): 919–924 https://doi.org/10.1007/s11390-011-1189-5
4	Zhu W W, Luo C, Wang J F, Li S P. Multimedia cloud computing. IEEE Signal Processing Magazine, 2011, 28(3): 59–69 https://doi.org/10.1109/MSP.2011.940269
5	Ahlgren B, Aranda P A, Chemouil P, Oueslati S, Correia L M, Karl H, Sollner M, Welin A. Content, connectivity and cloud: ingredients for the network of the future. IEEE Communication Magazine, 2011, 49(7): 62–70 https://doi.org/10.1109/MCOM.2011.5936156
6	Yang L, Cao J N, Yuan Y, Li T, Han A, Chan A. A framework for partitioning and execution of data stream applications in mobile cloud computing. ACM SIGMETRICS Performance Evaluation Review, 2013, 40(4): 23–32 https://doi.org/10.1145/2479942.2479946
7	Gao Y H, Ma H D, Zhang H T, Kong X Q, Wei W Y. Concurrency optimized task scheduling for workflows in cloud. In: Proceedings of the 6th IEEE International Conference on Cloud Computing. 2013, 709–716
8	Qian Z P, He Y, Su C Z, Wu Z J, Zhu H Y, Zhang T Z, Zhou L D, Yu Y, Zhang Z. TimeStream: reliable stream computation in the cloud. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 1–14 https://doi.org/10.1145/2465351.2465353
9	Zhao X M, Ma H D, Zhan H T, Tang Y, Kou Y. HVPI: extending Hadoop to support video analytic applications. In: Proceedings of the 8th IEEE International Conference on Cloud Computing. 2014, 789–796
10	Liu W, Mei T, Zhang Y D, Che C, Luo J B. Multi-task deep visualsemantic embedding for video thumbnail selection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3707–3715
11	Liu W, Zhang Y D, Tang S, Tang J H, Hong R, Li J T. Accurate estimation of human body orientation from RGB-D sensors. IEEE Transactions on Cybernetics, 2013, 43(5): 1442–1452 https://doi.org/10.1109/TCYB.2013.2272636
12	Liu W, Mei T, Zhang Y D. Instant mobile video search with layered audio-video indexing and progressive transmission. IEEE Transactions on Multimedia, 2014, 16(8): 2242–2255 https://doi.org/10.1109/TMM.2014.2359332
13	Saini M, Wang X, Atrey P K, Kankanhalli M. Adaptive workload equalization in multi-camera surveillance systems. IEEE Transactions on Multimedia, 2012, 14(3): 555–562 https://doi.org/10.1109/TMM.2012.2186957
14	Gao G Y, Zhang W W, Wen Y G, Wang Z, Zhu W W, Tan Y P. Cost optimal video transcoding in media cloud: insights from user viewing pattern. In: Proceedings of IEEE International Conference on Multimedia & Expo. 2014, 1–6 https://doi.org/10.1109/ICME.2014.6890255
15	Ren S L, Van der Schaar M. Efficient resource provisioning and rate selection for stream mining in a community cloud. IEEE Transactions on Multimedia, 2013, 15(4): 723–734 https://doi.org/10.1109/TMM.2013.2240673
16	Kwon Y C, Balazinska M, Rolia J. Skew-resistant parallel processing of feature-extracting scientific user-defined functions. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 75–86 https://doi.org/10.1145/1807128.1807140
17	Pavlo A, Curino C, Zdonik S. Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems. In: Proceedings of ACM SIGMOD International Conference on Management of Data. 2012, 61–72 https://doi.org/10.1145/2213836.2213844
18	Ramakrishnan S R, Swart G, Urmanov A. Balanceing reducer skew in MapReduce workloads using progressive smapling. In: Proceedings of the 3rd ACM Symposium on Cloud Computing. 2012, 1–13
19	Kwon Y C, Balazinska M, Howe B, Rolia J. SkewTune: mitigating skew in MapReduce applications. In: Proceedings of ACM SIGMOD International Conference on Management of Data. 2012, 25–36 https://doi.org/10.1145/2213836.2213840
20	Chen Q, Liu C, Xiao Z. Improving MapReduce performance using smart speculative execution strategy. IEEE Transactions on Computers, 2014, 63(4): 954–967 https://doi.org/10.1109/TC.2013.15
21	Ananthanarayanan G, Agarwal S, Kandula S, Greenberg A, Stoica I, Harlan D, Harris E. Scarlett: coping with skewed content popularity in MapReduce clusters. In: Proceedings of the 6th ACM European Conference on Computer Systems. 2011, 287–300 https://doi.org/10.1145/1966445.1966472
22	Le Y F, Liu J C, Ergün F. Wang D. Online load balancing for MapReduce with skewed data input. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2014, 2004–2012 https://doi.org/10.1109/INFOCOM.2014.6848141
23	Tang J H, Tay W P, Wen Y G. Dynamic request redirection and elastic service scaling in cloud-centric media networks. IEEE Transactions on Multimedia, 2014, 16(5): 1434–1445 https://doi.org/10.1109/TMM.2014.2308726
24	Zhao X M, Ma H D, Zhang H T, Tang Y, Fu G P. Metadata extraction and correction for large-scale traffic surveillance videos. In: Proceedings of IEEE International Conference on Big Data. 2014, 412–420 https://doi.org/10.1109/BigData.2014.7004258
25	Neely M J. Stochastic network optimization with application to communication and queueing system. Synthesis Lectures on Communication Networks, 2010, 3(1): 1–211 https://doi.org/10.2200/S00271ED1V01Y201006CNT007
26	Feris R S, Siddiquie B, Petterson J, Zhai Y, Datta A, Brown L M, Pankanti S. Large-scale vehicle detection, indexing, and search in urban surveillance videos. IEEE Transactions on Multimedia, 2012, 14(1): 28–42 https://doi.org/10.1109/TMM.2011.2170666
27	Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters. In: Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. 2004, 137–150
28	Zaharia M, Das T, Li H Y, Hunter T, Shenker S, Stoica I. Discretized streams: fault-tolerant streaming computation at scale. In: Proceedings of the 23th ACM Symposium on Operating Systems Principles. 2013, 423–438 https://doi.org/10.1145/2517349.2522737
29	Maguluri S T, Strikant R. Scheduling jobs with unkonwn duration in clouds. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2013, 1935–1943
30	Huang F, Anandkumar A. FCD: fast-concurrent-distributed load balanceing under switch costs and imperfect observation. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2013, 1944–1952 https://doi.org/10.1109/INFCOM.2013.6566989
31	Khayyat Z, Awara K, Alonazi A, Jamjoom H, Williams D, Kalnis P. Mizan: a system for dynamic load balancing in large-scale graph processing. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 169–182 https://doi.org/10.1145/2465351.2465369

[1]

Download

[1]	Xiong FU, Juzhou CHEN, Song DENG, Junchang WANG, Lin ZHANG. Layered virtual machine migration algorithm for network resource balancing in cloud computing[J]. Front. Comput. Sci., 2018, 12(1): 75-85.
[2]	Cheqing JIN, Jie CHEN, Huiping LIU. MapReduce-based entity matching with multiple blocking functions[J]. Front. Comput. Sci., 2017, 11(5): 895-911.
[3]	Quanqing XU,Rajesh Vellore ARUMUGAM,Khai Leong YONG,Yonggang WEN,Yew-Soon ONG,Weiya XI. Adaptive and scalable load balancing for metadata server cluster in cloud-scale file systems[J]. Front. Comput. Sci., 2015, 9(6): 904-918.
[4]	Yaobin HE, Haoyu TAN, Wuman LUO, Shengzhong FENG, Jianping FAN. MR-DBSCAN: a scalable MapReduce-based DBSCAN algorithm for heavily skewed data[J]. Front. Comput. Sci., 2014, 8(1): 83-99.
[5]	Xuejun YANG, Xiangke LIAO, Weixia XU, Junqiang SONG, Qingfeng HU, Jinshu SU, Liquan XIAO, Kai LU, Qiang DOU, Juping JIANG, Canqun YANG, . TH-1: China’s first petaflop supercomputer[J]. Front. Comput. Sci., 2010, 4(4): 445-455.
[6]	Wenchao JIANG, Matthias BAUMGARTEN, Yanhong ZHOU, Hai JIN, . A bipartite model for load balancing in grid computing environments[J]. Front. Comput. Sci., 2009, 3(4): 503-523.
[7]	YANG Xuejun, WANG Panfeng, DU Yunfei, ZHOU Haifang. A data-distributed parallel algorithm for wavelet-based fusion of remote sensing images[J]. Front. Comput. Sci., 2007, 1(2): 231-240.

Viewed

Full text

Abstract

Cited

Shared

Discussed