|
|
Towards application-level elasticity on shared cluster: an actor-based approach |
Donggang CAO1,2(), Lianghuan KANG1,2, Hanglong ZHAN1,2, Hong MEI1 |
1. Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing 100871, China 2. Beida (Binhai) Information Research, Tianjin 300450, China |
|
|
Abstract In current cluster computing, several distributed frameworks are designed to support elasticity for business services adapting to environment fluctuation. However, most existing works support elasticity mainly at the resource level, leaving application level elasticity support problem to domain-specific frameworks and applications. This paper proposes an actor-based general approach to support application-level elasticity for multiple cluster computing frameworks. The actor model offers scalability and decouples language-level concurrency from the runtime environment. By extending actors, a new middle layer called Unisupervisor is designed to “sit” between the resource management layer and application framework layer. Actors in Unisupervisor can automatically distribute and execute tasks over clusters and dynamically scale in/out. Based on Unisupervisor, high-level profiles (MasterSlave, MapReduce, Streaming, Graph, and Pipeline) for diverse cluster computing requirements can be supported. The entire approach is implemented in a prototype system called UniAS. In the evaluation, both benchmarks and real applications are tested and analyzed in a small scale cluster. Results show that UniAS is expressive and efficiently elastic.
|
Keywords
elasticity
elastic scaling
actor programming model
cluster computing
concurrent and parallel processing
|
Corresponding Author(s):
Donggang CAO
|
Just Accepted Date: 25 February 2016
Online First Date: 31 October 2016
Issue Date: 26 September 2017
|
|
1 |
LuX C, WangH M, WangJ, Xu J, LiD S .Internet-based virtual computing environment: beyond the data center as a computer. Future Generation Computer Systems, 2013, 29(1): 309–322
https://doi.org/10.1016/j.future.2011.08.005
|
2 |
ToshniwalA, TanejaS, ShuklaA, Ramasamy K, PatelJ M , KulkarniS, Jackson J, GadeK , FuM S, DonhamJ, BhagatN, Mittal S, RyaboyD . Storm@twitter. In:Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. 2014, 147–156
https://doi.org/10.1145/2588555.2595641
|
3 |
HindmanB, Konwinski A, ZahariaM , GhodsiA, JosephA D, KatzR, Shenker S, StoicaI . Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation. 2011, 295–308
|
4 |
VavilapalliV K, MurthyA C, DouglasC, Agarwal S, KonarM , EvansR, GravesT, LoweJ, Shah H, SethS , SahaB, CurinoC, O’MalleyO , RadiaS, ReedB, BaldeschwielerE . Apache Hadoop YARN: yet another resource negotiator. In: Proceedings of the 4th ACM Symposium on Cloud Computing. 2013
https://doi.org/10.1145/2523616.2523633
|
5 |
ColeM. Algorithmic Skeletons: Structured Management of Parallel Computation. Cambridge, Massachusetts: MIT Press, 1991
|
6 |
BahgaA, Madisetti V K. Rapid prototyping of multitier cloud-based services and systems. Computer, 2013, 46(11): 76–83
https://doi.org/10.1109/MC.2013.154
|
7 |
KächeleS, HauckF J. Component-based scalability for cloud applications. In: Proceedings of the 3rd International Workshop on Cloud Data and Platforms. 2013, 19–24
https://doi.org/10.1145/2460756.2460760
|
8 |
CaromelD, LeytonM. Fine tuning algorithmic skeletons. Lecture Notes in Computer Science, vol 4641. Berlin: Springer-Verlag, 2007, 72–81
https://doi.org/10.1007/978-3-540-74466-5_9
|
9 |
AghaG A. Actors: a model of concurrent computation in distributed systems. Dissertation for the doctoral Degree. Cambridge: Massachusetts Institute of Technology, 1985
|
10 |
ZhanH L, KangL H, CaoD G. DETS: a dynamic and elastic task scheduler supporting multiple parallel schemes. In: Proceedings of the 8th IEEE International Symposium on Service Oriented System Engineering. 2014, 278–283
https://doi.org/10.1109/sose.2014.39
|
11 |
MateiZharia, DasT, LiH Y, Hunter T, ShenkerS , StoicaI. Discretized streams: fault-tolerant streaming computation at scale. In: Proceedings of the 24th ACM Symposium on Operating Systems Principles (OSDI). 2013, 423–438
https://doi.org/10.1145/2517349.2522737
|
12 |
LinJ, ZhaL, XuZ W. Consolidated cluster systems for data centers in the cloud age: a survey and analysis. Frontiers of Computer Science, 2013, 7(1): 1–19
https://doi.org/10.1007/s11704-012-2086-y
|
13 |
LiuL. Computing infrastructure for big data processing. Frontiers of Computer Science, 2013, 7(2): 165–170
https://doi.org/10.1007/s11704-013-3900-x
|
14 |
ZachariaF, Govindaraju M.Delma: dynamically elastic mapreduce framework for CPU-intensive applications. In: Proceedings of the 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2011, 454–463
|
15 |
GordonA W, LuP. Elastic phoenix: malleable mapreduce for sharedmemory systems. Network and Parallel Computing, Vol 6985. Berlin: Springer, 2011
|
16 |
ShiW, HongB. Clotho: an elastic MapReduce workload/runtime codesign. In: Proceedings of the 12th International Workshop on Adaptive and Reflective Middleware. 2013
https://doi.org/10.1145/2541583.2541588
|
17 |
WaldemarH, Satzger B, DustdarS . Elastic stream processing in the Cloud. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2013, 3(5): 333–345
https://doi.org/10.1002/widm.1100
|
18 |
QianZ, HeY, SuC Z, Wu Z J, ZhuH Y , ZhangT Z, ZhouL D, YuY, ZhangZ. Timestream: reliable stream computation in the cloud. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 1–14
https://doi.org/10.1145/2465351.2465353
|
19 |
BugraG, Schneider S, HirzelM , WuK L. Elastic scaling for data stream processing. IEEE Transcations on Parallel and Distributed Systems, 2014, 25(6): 1447–1463
https://doi.org/10.1109/TPDS.2013.295
|
20 |
HuangC, ZhengG B, KaleL, Kumar S. Performance evaluation of adaptive MPI. In: Proceedings of the 11th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 2006, 12–21
https://doi.org/10.1145/1122971.1122976
|
21 |
WeiY, Sukumar K, VecchiolaC , KarunamoorthyD, BuyyaR. Aneka cloud application platform and its integration with Windows Azure. 2011, arXiv preprint arXiv: 1103.2590
|
22 |
BykovS, GellerA, KliotG, Larus J R, PandyaR , ThelinJ. Orleans: cloud computing for everyone. In: Proceedings of the 2nd ACM Symposium on Cloud Computing. 2011
https://doi.org/10.1145/2038916.2038932
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|