|
|
StreamTune: dynamic resource scheduling approach for workload skew in video data center |
Yihong GAO( ), Huadong MA |
Beijing Key Lab of Intelligent Telecommunications Software and Multimedia, School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China |
|
|
Abstract Video surveillance applications need video data center to provide elastic virtual machine (VM) provisioning. However, the workloads of the VMs are hardly to be predicted for online video surveillance service. The unknown arrival workloads easily lead to workload skew among VMs. In this paper, we study how to balance the workload skew on online video surveillance system. First, we design the system framework for online surveillance service which consists of video capturing and analysis tasks. Second, we propose Stream Tune, an online resource scheduling approach for workload balancing, to deal with irregular video analysis workload with the minimum number of VMs. We aim at timely balancing the workload skew on video analyzers without depending on any workload prediction method. Furthermore, we evaluate the performance of the proposed approach using a traffic surveillance application. The experimental results show that our approach is well adaptive to the variation of workload and achieves workload balance with less VMs.
|
Keywords
video data center
load balancing
stream computing
online video analysis
scheduling algorithm
|
Corresponding Author(s):
Yihong GAO
|
Just Accepted Date: 14 June 2016
Online First Date: 22 September 2017
Issue Date: 14 June 2018
|
|
1 |
Hu H, Wen Y G, Chua T S, Li X L. Toward scalable systems for big data analytics: a technology tutorial. IEEE Access, 2014, 2: 652–687
https://doi.org/10.1109/ACCESS.2014.2332453
|
2 |
Ma H D, Liu L, Zhou A F, Zhao D. On networking of Internet of things: explorations and challenges. IEEE Internet of Things Journal, 2016, 3(4): 441–452
https://doi.org/10.1109/JIOT.2015.2493082
|
3 |
Ma H D. Internet of things: objectives and scientific challenges. Journal of Computer Science and Technology, 2011, 26(6): 919–924
https://doi.org/10.1007/s11390-011-1189-5
|
4 |
Zhu W W, Luo C, Wang J F, Li S P. Multimedia cloud computing. IEEE Signal Processing Magazine, 2011, 28(3): 59–69
https://doi.org/10.1109/MSP.2011.940269
|
5 |
Ahlgren B, Aranda P A, Chemouil P, Oueslati S, Correia L M, Karl H, Sollner M, Welin A. Content, connectivity and cloud: ingredients for the network of the future. IEEE Communication Magazine, 2011, 49(7): 62–70
https://doi.org/10.1109/MCOM.2011.5936156
|
6 |
Yang L, Cao J N, Yuan Y, Li T, Han A, Chan A. A framework for partitioning and execution of data stream applications in mobile cloud computing. ACM SIGMETRICS Performance Evaluation Review, 2013, 40(4): 23–32
https://doi.org/10.1145/2479942.2479946
|
7 |
Gao Y H, Ma H D, Zhang H T, Kong X Q, Wei W Y. Concurrency optimized task scheduling for workflows in cloud. In: Proceedings of the 6th IEEE International Conference on Cloud Computing. 2013, 709–716
|
8 |
Qian Z P, He Y, Su C Z, Wu Z J, Zhu H Y, Zhang T Z, Zhou L D, Yu Y, Zhang Z. TimeStream: reliable stream computation in the cloud. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 1–14
https://doi.org/10.1145/2465351.2465353
|
9 |
Zhao X M, Ma H D, Zhan H T, Tang Y, Kou Y. HVPI: extending Hadoop to support video analytic applications. In: Proceedings of the 8th IEEE International Conference on Cloud Computing. 2014, 789–796
|
10 |
Liu W, Mei T, Zhang Y D, Che C, Luo J B. Multi-task deep visualsemantic embedding for video thumbnail selection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3707–3715
|
11 |
Liu W, Zhang Y D, Tang S, Tang J H, Hong R, Li J T. Accurate estimation of human body orientation from RGB-D sensors. IEEE Transactions on Cybernetics, 2013, 43(5): 1442–1452
https://doi.org/10.1109/TCYB.2013.2272636
|
12 |
Liu W, Mei T, Zhang Y D. Instant mobile video search with layered audio-video indexing and progressive transmission. IEEE Transactions on Multimedia, 2014, 16(8): 2242–2255
https://doi.org/10.1109/TMM.2014.2359332
|
13 |
Saini M, Wang X, Atrey P K, Kankanhalli M. Adaptive workload equalization in multi-camera surveillance systems. IEEE Transactions on Multimedia, 2012, 14(3): 555–562
https://doi.org/10.1109/TMM.2012.2186957
|
14 |
Gao G Y, Zhang W W, Wen Y G, Wang Z, Zhu W W, Tan Y P. Cost optimal video transcoding in media cloud: insights from user viewing pattern. In: Proceedings of IEEE International Conference on Multimedia & Expo. 2014, 1–6
https://doi.org/10.1109/ICME.2014.6890255
|
15 |
Ren S L, Van der Schaar M. Efficient resource provisioning and rate selection for stream mining in a community cloud. IEEE Transactions on Multimedia, 2013, 15(4): 723–734
https://doi.org/10.1109/TMM.2013.2240673
|
16 |
Kwon Y C, Balazinska M, Rolia J. Skew-resistant parallel processing of feature-extracting scientific user-defined functions. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 75–86
https://doi.org/10.1145/1807128.1807140
|
17 |
Pavlo A, Curino C, Zdonik S. Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems. In: Proceedings of ACM SIGMOD International Conference on Management of Data. 2012, 61–72
https://doi.org/10.1145/2213836.2213844
|
18 |
Ramakrishnan S R, Swart G, Urmanov A. Balanceing reducer skew in MapReduce workloads using progressive smapling. In: Proceedings of the 3rd ACM Symposium on Cloud Computing. 2012, 1–13
|
19 |
Kwon Y C, Balazinska M, Howe B, Rolia J. SkewTune: mitigating skew in MapReduce applications. In: Proceedings of ACM SIGMOD International Conference on Management of Data. 2012, 25–36
https://doi.org/10.1145/2213836.2213840
|
20 |
Chen Q, Liu C, Xiao Z. Improving MapReduce performance using smart speculative execution strategy. IEEE Transactions on Computers, 2014, 63(4): 954–967
https://doi.org/10.1109/TC.2013.15
|
21 |
Ananthanarayanan G, Agarwal S, Kandula S, Greenberg A, Stoica I, Harlan D, Harris E. Scarlett: coping with skewed content popularity in MapReduce clusters. In: Proceedings of the 6th ACM European Conference on Computer Systems. 2011, 287–300
https://doi.org/10.1145/1966445.1966472
|
22 |
Le Y F, Liu J C, Ergün F. Wang D. Online load balancing for MapReduce with skewed data input. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2014, 2004–2012
https://doi.org/10.1109/INFOCOM.2014.6848141
|
23 |
Tang J H, Tay W P, Wen Y G. Dynamic request redirection and elastic service scaling in cloud-centric media networks. IEEE Transactions on Multimedia, 2014, 16(5): 1434–1445
https://doi.org/10.1109/TMM.2014.2308726
|
24 |
Zhao X M, Ma H D, Zhang H T, Tang Y, Fu G P. Metadata extraction and correction for large-scale traffic surveillance videos. In: Proceedings of IEEE International Conference on Big Data. 2014, 412–420
https://doi.org/10.1109/BigData.2014.7004258
|
25 |
Neely M J. Stochastic network optimization with application to communication and queueing system. Synthesis Lectures on Communication Networks, 2010, 3(1): 1–211
https://doi.org/10.2200/S00271ED1V01Y201006CNT007
|
26 |
Feris R S, Siddiquie B, Petterson J, Zhai Y, Datta A, Brown L M, Pankanti S. Large-scale vehicle detection, indexing, and search in urban surveillance videos. IEEE Transactions on Multimedia, 2012, 14(1): 28–42
https://doi.org/10.1109/TMM.2011.2170666
|
27 |
Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters. In: Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. 2004, 137–150
|
28 |
Zaharia M, Das T, Li H Y, Hunter T, Shenker S, Stoica I. Discretized streams: fault-tolerant streaming computation at scale. In: Proceedings of the 23th ACM Symposium on Operating Systems Principles. 2013, 423–438
https://doi.org/10.1145/2517349.2522737
|
29 |
Maguluri S T, Strikant R. Scheduling jobs with unkonwn duration in clouds. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2013, 1935–1943
|
30 |
Huang F, Anandkumar A. FCD: fast-concurrent-distributed load balanceing under switch costs and imperfect observation. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2013, 1944–1952
https://doi.org/10.1109/INFCOM.2013.6566989
|
31 |
Khayyat Z, Awara K, Alonazi A, Jamjoom H, Williams D, Kalnis P. Mizan: a system for dynamic load balancing in large-scale graph processing. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 169–182
https://doi.org/10.1145/2465351.2465369
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|