Large-scale virtual machines provisioning in clouds:challenges and approaches

doi:10.1007/s11704-015-4420-7

Front. Comput. Sci.

2016, Vol. 10

Issue (1) : 2-18 https://doi.org/10.1007/s11704-015-4420-7

REVIEW ARTICLE

Large-scale virtual machines provisioning in clouds:challenges and approaches

Zhaoning ZHANG¹,Dongsheng LI^1,^*(

),Kui WU²

1. National Laboratory for Parallel and Distributed Processing, School of Computer Science,National University of Defense Technology, Changsha 410073, China
2. Computer Science Department, University of Victoria, Victoria V8W 2Y2, Canada

Download: PDF(748 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

The scale of global data center market has been explosive in recent years. As the market grows, the demand for fast provisioning of the virtual resources to support elastic, manageable, and economical computing over the cloud becomes high. Fast provisioning of large-scale virtual machines (VMs), in particular, is critical to guarantee quality of service (QoS). In this paper, we systematically review the existing VM provisioning schemes and classify them in three main categories. We discuss the features and research status of each category, and introduce two recent solutions, VMThunder and VMThunder+, both of which can provision hundreds of VMs in seconds.

Keywords cloud computing IaaS large scale virtual machine provisioning

Corresponding Author(s): Dongsheng LI

Just Accepted Date: 22 April 2015 Issue Date: 06 January 2016

Cite this article:

Zhaoning ZHANG,Dongsheng LI,Kui WU. Large-scale virtual machines provisioning in clouds:challenges and approaches[J]. Front. Comput. Sci., 2016, 10(1): 2-18.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-015-4420-7
https://academic.hep.com.cn/fcs/EN/Y2016/V10/I1/2

1	Lu X, Wang H, Wang J, Xu J, Li D. Internet-based virtual computing environment: beyond the data center as a computer. Future Generation Computer Systems, 2013, 29(1): 309–322 https://doi.org/10.1016/j.future.2011.08.005
2	Ahmed W, Wu Y. Estimation of cloud node acquisition. Tsinghua Science and Technology, 2014, 19(1): 1–12 https://doi.org/10.1109/TST.2014.6733202
3	Mao M, Humphrey M. A performance study on the VM startup time in the cloud. In: Proceedings of IEEE International Conference on Cloud Computing (CLOUD). 2012, 423–430 https://doi.org/10.1109/cloud.2012.103
4	Zhang Z, Li Z,Wu K, Li D, Li H, Peng Y, Lu X. VMThunder: fast provisioning of large-scale virtual machine clusters. IEEE Transactions on Parallel and Distributed System, 2014, 25(12): 3328–3338 https://doi.org/10.1109/TPDS.2014.7
5	S otomayor B, Keahey K, Foster I. Combining batch execution and leasing using virtual machines. In: Proceedings of the 17th International Symposium on High Performance Distributed Computing. 2008, 87–96 https://doi.org/10.1145/1383422.1383434
6	Sotomayor B, Montero R S, Llorente I M, Foster I. Virtual infrastructure management in private and hybrid clouds. IEEE Internet Computing,2009, 13(5): 14–22 https://doi.org/10.1109/MIC.2009.119
7	Li J, Li D, Ye Y, Lu X. Efficient multi-tenant virtual machine allocation in cloud data centers. Tsinghua Science and Technology, 2015, 20(1):81–89 https://doi.org/10.1109/TST.2015.7040517
8	Le D, Huang H, Wang H. Understanding performance implications of nested file systems in a virtualized environment. In: Proceedings of USENIX Conference on File and Storage Technologies. 2012, 8
9	Bellard F. Qemu, a fast and portable dynamic translator. In: Proceedings of USENIX Annual Technical Conference. 2005, 41–46
10	Nicolae B, Bresnahan J, Keahey K, Antoniu G. Going back and forth:efficient multideployment and multisnapshotting on clouds. In: Proceedings of ACM Symposium on High Performance Distributed Computing.2011, 147–158
11	Xiao W, Liu Y, Yang Q, Ren J, Xie C. Implementation and performance evaluation of two snapshot methods on iSCSI target storages.In: Proceedings of IEEE Conference on Mass Storage Systems and Technologies. 2006
12	Jayaram K R, Peng C, Zhang Z, Kim M, Chen H, Lei H. An empirical analysis of similarity in virtual machine images. In: Proceedings of the Middleware. 2011, 6 https://doi.org/10.1145/2090181.2090187
13	Peng C, Kim M, Zhang Z, Lei H. Vdn: Virtual machine image distribution network for cloud data centers. In: Proceedings of IEEE Infocom.2012, 181–189
14	Razavi K, Ion A, Kielmann T. Squirrel: Scatter hoarding VM image contents on IaaS compute nodes. In: Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing.2014, 265–278 https://doi.org/10.1145/2600212.2600221
15	Jin K, Miller E L. The effectiveness of deduplication on virtual machine disk images. In: Proceedings of SYSTOR. 2009, 7 https://doi.org/10.1145/1534530.1534540
16	Ng C H, Ma M, Wong T Y, Lee P P C, Lui J C S. Live deduplication storage of virtual machine images in an open-source cloud. In:Proceedings of the 12th International Middleware Conference. 2011,80–99 https://doi.org/10.1007/978-3-642-25821-3_5
17	Srinivasan K, Bisson T, Goodson G, Voruganti K. iDedup: Latencyaware,inline data deduplication for primary storage. In: Proceedings of the 10th USENIX Conference on File and Storage Technologies. 2012, 12: 1–14
18	Ammons G, Bala V, Mummert T, Reimer D, Zhang X. Virtual machine images as structured data: the mirage image library. In: Proceedings of USENIX HotCloud. 2011
19	Reimer D, Thomas A, Ammons G, Mummert T, Alpern B, Bala V. Opening black boxes: using semantic information to combat virtual machine image sprawl. In: Proceedings of International Conference on Virtual Execution Environments. 2008, 111–120 https://doi.org/10.1145/1346256.1346272
20	Tang C. Fvd: a high-performance virtual machine image format for cloud. In: Proceedings of USENIX Annual Technical Conference.2011
21	Papadopoulos P. Extending clusters to Amazon EC2 using the rocks toolkit. International Journal of High Performance Computing Applications,2011, 25(3): 317–327 https://doi.org/10.1177/1094342011414747
22	Nurmi D, Wolski R, Grzegorczyk C, Obertelli G, Soman S, Youseff L, Zagorodnov D. The eucalyptus open-source cloud-computing system. In: Proceedings of the 9th IEEE/ACM International Symposium on CCGrid. 2009, 124–131 https://doi.org/10.1109/ccgrid.2009.93
23	Li D,Cao J, Lu X, Chen K. Efficient range query processing in peer-topeer systems. IEEE Transactions on Knowledge and Data Engineering, 2009, 21(1): 78–91 https://doi.org/10.1109/TKDE.2008.99
24	Zhang Z, Lu X, Peng Y, Li H. A reality check of multiple snowball tree file dissemination in large scale cloud cluster. In: Proceedings of the 15th IEEE International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing Workshops. 2012, 76–80
25	Wartel R, Cass T, Moreira B, Roche E, Guijarro M, Goasguen S, Schwickerath U. Image distribution mechanisms in large scale cloud providers. In: Proceedings of IEEE CloudCom. 2010, 112–117 https://doi.org/10.1109/cloudcom.2010.73
26	Chen Z, Zhao Y, Miao X, Chen Y, Wang Q. Rapid provisioning of cloud infrastructure leveraging peer-to-peer networks. In: Proceedings of the 29th IEEE International Conference on Distributed Computing Systems Workshops. 2009, 324–329 https://doi.org/10.1109/icdcsw.2009.35
27	O’Donnell C M. Using bittorrent to distribute virtual machine images for classes. In: Proceedings of the 36th Annual ACM SIGUCCS Fall Conference: Moving Mountains, Blazing Trails. 2008, 287–290 https://doi.org/10.1145/1449956.1450040
28	Reich J, Laadan O, Brosh E, Sherman A, Misra V, Nieh J, Rubenstein D. Vmtorrent: scalable P2P virtual machine streaming. In: Proceedings of ACM Conference on Emerging Network Experiment and Technology.2012, 289–300 https://doi.org/10.1145/2413176.2413210
29	Morgan Jr T. Drbl: Diskless remote boot in Linux. NETWORK, 2006, 192
30	Weil S A, Brandt S A, Miller E L, Long D D, Maltzahn C. Ceph: A scalable, high-performance distributed file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation. 2006, 307–320
31	Shamma M, Meyer D T,Wires J, Ivanova M, Hutchinson N C,Warfield A. Capo: recapitulating storage for virtual desktops. In: Proceedings of USENIX Conference on File and Storage Technologies. 2011
32	Liao X, Xiong X, Jin H, Hu L. Lvd: A lightweight virtual desktop management architecture. Systems and Virtualization Management. Standards and New Technologies, 2008, 25–36
33	Wo T,Wang H, Hu C, Cui Y. Dvce: the virtual computing environment supported by distributed VM images. In: Proceedings of ISORC iVCE Workshop. 2012 https://doi.org/10.1109/isorcw.2012.25
34	Carns P H, Ligon III W B, Ross R B, Thakur R. PVFS: A parallel file system for linux clusters. In: Proceedings of the 4th Annual Linux Showcase Conference. 2000, 391–430
35	Flouris M D, Lachaize R, Bilas A. Orchestra: Extensible block-level support for resource and data sharing in networked storage systems. In: Proceedings of the 14th IEEE International Conference on Parallel and Distributed Systems. 2008, 237–244 https://doi.org/10.1109/icpads.2008.110
36	Flouris M D, Bilas A. Violin: a framework for extensible block-level storage. In: Proceedings of the 22nd IEEE Goddard Conference on Mass Storage Systems and Technologies. 2005, 128–142 https://doi.org/10.1109/msst.2005.41
37	Meyer D, Aggarwal G, Cully B, Lefebvre G, Feeley M, Hutchinson N, Warfield A. Parallax: virtual disks for virtual machines. In: Proceedings of EuroSys. 2008 https://doi.org/10.1145/1352592.1352598
38	Razavi K, Kielmann T. Scalable virtual machine deployment using vm image caches. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. 2013, 65 https://doi.org/10.1145/2503210.2503274
39	Zhao X, Zhang Y, Wu Y, Chen K, Jiang J, Li K. Liquid: A scalable deduplication file system for virtual machine images. IEEE Transactions on Parallel and Distributed System, 2014, 25(5): 1257–1266 https://doi.org/10.1109/TPDS.2013.173
40	Morrey III C B, Grunwald D. Content-based block caching. In: Proceedings of the 23rd IEEE Conference on Mass Storage Systems and Technologies. 2006
41	Ng C H, Ma M, Wong T Y, Lee P P C, Lui J C S. Live deduplication storage of virtual machine images in an open-source cloud. In:Proceedings of the 12th International Middleware Conference. 2011,80–99 https://doi.org/10.1007/978-3-642-25821-3_5
42	Lagar-Cavilla H A, Whitney J A, Scannell A M, Patchin P, Rumble S M, De Lara E, Brudno M, Satyanarayanan M. Snowflock: rapid virtual machine cloning for cloud computing. In: Proceedings of the 4th ACM European Conference on Computer Systems. 2009, 1–12 https://doi.org/10.1145/1519065.1519067
43	Zhu J, Jiang Z, Xiao Z. Twinkle a fast resource provisioning mechanism for internet services. In: Proceedings of IEEE Infocom. 2011, 802–810 https://doi.org/10.1109/infcom.2011.5935302
44	Cui L, Li J, Li B, Huai J, Hu C, Wo T,Al-Aqrabi H, Liu L.VMScatter: Migrate virtual machines to many hosts. In: Proceedings of International Conference on Virtual Execution Environments. 2013, 63–72
45	Merkel D. Docker: Lightweight linux containers for consistent development and deployment. Linux Journal, 2014, 2014(239): 2
46	Zhao Y,Wu J, Liu <?Pub Caret1?> C. On peer-assisted data dissemination in data center networks: Analysis and implementation. Tsinghua Science and Technology, 2014, 19(1): 51–64 https://doi.org/10.1109/TST.2014.6733208
47	Zhang Z, Wu K, Li H, Feng J, Peng Y, Lu X. Raflow: Read ahead accelerated I/O flow through multiple virtual layers. In: Proceedings of the 9th IEEE International conference on networking, architecture and storage. 2014, 33–42 https://doi.org/10.1109/nas.2014.13
48	Zhang P, Chu R, Wang H. Swapcached: An effective method to promote guest paging performance on virtualization platform. In: Proceedings of the 7th IEEE International Symposium on Service Oriented System Engineering. 2013, 379–384

[1]

Supplementary Material-Highlights in 3-page ppt

Download

[1]	Wei ZHENG, Ying WU, Xiaoxue WU, Chen FENG, Yulei SUI, Xiapu LUO, Yajin ZHOU. A survey of Intel SGX and its applications[J]. Front. Comput. Sci., 2021, 15(3): 153808-.
[2]	Najme MANSOURI, Mohammad Masoud JAVIDI, Behnam Mohammad Hasani ZADE. Hierarchical data replication strategy to improve performance in cloud computing[J]. Front. Comput. Sci., 2021, 15(2): 152501-.
[3]	Jiayang LIU, Jingguo BI, Mu LI. Secure outsourcing of large matrix determinant computation[J]. Front. Comput. Sci., 2020, 14(6): 146807-.
[4]	Meysam VAKILI, Neda JAHANGIRI, Mohsen SHARIFI. Cloud service selection using cloud service brokers: approaches and challenges[J]. Front. Comput. Sci., 2019, 13(3): 599-617.
[5]	Kun SU, Gongping YANG, Lu YANG, Peng SU, Yilong YIN. Non-negative locality-constrained vocabulary tree for finger vein image retrieval[J]. Front. Comput. Sci., 2019, 13(2): 318-332.
[6]	Qiang LIU, Xiaoshe DONG, Heng CHEN, Yinfeng WANG. IncPregel: an incremental graph parallel computation model[J]. Front. Comput. Sci., 2018, 12(6): 1076-1089.
[7]	Fei TIAN, Tao QIN, Tie-Yan LIU. Computational pricing in Internet era[J]. Front. Comput. Sci., 2018, 12(1): 40-54.
[8]	Xiong FU, Juzhou CHEN, Song DENG, Junchang WANG, Lin ZHANG. Layered virtual machine migration algorithm for network resource balancing in cloud computing[J]. Front. Comput. Sci., 2018, 12(1): 75-85.
[9]	Jian-Hao LUO,Wang ZHOU,Jianxin WU. Image categorization with resource constraints: introduction, challenges and advances[J]. Front. Comput. Sci., 2017, 11(1): 13-26.
[10]	Najme MANSOURI. Adaptive data replication strategy in cloud computing for performance improvement[J]. Front. Comput. Sci., 2016, 10(5): 925-935.
[11]	Haibao CHEN,Song WU,Hai JIN,Wenguang CHEN,Jidong ZHAI,Yingwei LUO,Xiaolin WANG. A survey of cloud resource management for complex engineering applications[J]. Front. Comput. Sci., 2016, 10(3): 447-461.
[12]	Bing YU,Yanni HAN,Hanning YUAN,Xu ZHOU,Zhen XU. A cost-effective scheme supporting adaptive service migration in cloud data center[J]. Front. Comput. Sci., 2015, 9(6): 875-886.
[13]	Huiju WANG,Xiongpai QIN,Xuan ZHOU,Furong LI,Zuoyan QIN,Qing ZHU,Shan WANG. Efficient query processing framework for big data warehouse: an almost join-free approach[J]. Front. Comput. Sci., 2015, 9(2): 224-236.
[14]	Xiong FU,Chen ZHOU. Virtual machine selection and placement for dynamic consolidation in Cloud computing environment[J]. Front. Comput. Sci., 2015, 9(2): 322-330.
[15]	Lei XIE,Sanglu LU,Yingchun CAO,Daoxu CHEN. Towards energy-efficient storage placement in large scale sensor networks[J]. Front. Comput. Sci., 2014, 8(3): 409-425.

Viewed

Full text

Abstract

Cited

Shared

Discussed