With the demand of agile development and management, cloud applications today are moving towards a more fine-grained microservice paradigm, where smaller and simpler functioning parts are combined for providing end-to-end services. In recent years, we have witnessed many research efforts that strive to optimize the performance of cloud computing system in this new era. This paper provides an overview of existing works on recent system performance optimization techniques and classify them based on their design focuses. We also identify open issues and challenges in this important research direction.
. [J]. Frontiers of Computer Science, 2022, 16(6): 166106.
Rong ZENG, Xiaofeng HOU, Lu ZHANG, Chao LI, Wenli ZHENG, Minyi GUO. Performance optimization for cloud computing systems in the microservice era: state-of-the-art and research opportunities. Front. Comput. Sci., 2022, 16(6): 166106.
Achieving stable tail latency, Saving CPU energy consumption
[103]
Extending simultaneously multi-threaded cores to schedule short requests
Reducing tail latency under moderate and high load
[46]
Automatically tuning the threading model for mid-tier microservice
1.9 × tail latency speedup over static threading choices
Invocation latency
[107]
Using language-level isolation and fast preemption
Achieving s scale latency
[108]
Using lightweight container isolation and package-aware caching
Achieving significant speedup over Docker
Communication overhead
[109]
Using application sandboxing and a hierarchical message bus
Achieving 43 speedup over OpenWhisk
[110]
Using a userspace networking stack
Achieving high throughput and low latency
Tab.4
Fig.2
1
Gan Y, Zhang Y, Hu K, Cheng D, He Y, Pancholi M, Delimitrou C. Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices. In: Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. 2019, 19–33
2
Y Chen, T Luo, S Liu, S Zhang, L He, J Wang, L Li, T Chen, Z Xu, N Sun, O Temam. DaDianNao: a machine-learning supercomputer. In: Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture. 2014, 609– 622
3
N P Jouppi, C Young, N Patil, D Patterson, G Agrawal. In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture. 2017, 1– 12
4
E Chung , J Fowers , K Ovtcharov , M Papamichael , A Caulfield . Serving DNNs in real time at datacenter scale with project brainwave. IEEE Micro, 2018, 38( 2): 8– 20
5
Nitu V, Teabe B, Tchana A, Isci C, Hagimont D. Welcome to zombieland: practical and energy-efficient memory disaggregation in a datacenter. In: Proceedings of the 13th EuroSys Conference. 2018, 16
6
K Lim, J Chang, T Mudge, P Ranganathan, S K Reinhardt, T Wenisch. Disaggregated memory for expansion and sharing in blade servers. In: Proceedings of the 36th Annual International Symposium on Computer Architecture. 2009, 267– 278
7
D Taibi, V Lenarduzzi, C Pahl. Architectural patterns for microservices: a systematic mapping study. In: Proceedings of the 8th International Conference on Cloud Computing and Services Science. 2018, 221– 232
8
Alshuqayran N, Ali N, Evans R. A systematic mapping study in microservice architecture. In: Proceedings of the 9th IEEE International Conference on Service-Oriented Computing and Applications. 2016, 44–51
9
Aguiar L, Almeida W, Hazin R, Lima A, Ferraz F. Survey on microservice architecture-security, privacy and standardization on cloud computing environment. In: Proceedings of the 12th International Conference on Software Engineering Advances. 2017, 210
10
T Yarygina, A B Bagge. Overcoming security challenges in microservice architectures. In: Proceedings of 2018 IEEE Symposium on Service-Oriented System Engineering. 2018, 11– 20
11
Villamizar M, Garcés O, Castro H, Verano M, Salamanca L, Casallas R, Gil S. Evaluating the monolithic and the microservice architecture pattern to deploy Web applications in the cloud. In: Proceedings of the 10th Computing Colombian Conference. 2015, 583–590
12
H Vural, M Koyuncu, S Guney. A systematic literature review on microservices. In: Proceedings of the 17th International Conference on Computational Science and its Applications. 2017, 203– 217
13
Gouigoux J P, Tamzalit D. From monolith to microservices: lessons learned on an industrial migration to a Web oriented architecture. In: Proceedings of 2017 IEEE International Conference on Software Architecture Workshops. 2017, 62–65
14
P Di Francesco, P Lago, I Malavolta. Migrating towards microservice architectures: an industrial survey. In: Proceedings of 2018 IEEE International Conference on Software Architecture. 2018, 29– 2909
15
S S Manvi , G K Shyam . Resource management for Infrastructure as a Service (IaaS) in cloud computing: a survey. Journal of Network and Computer Applications, 2014, 41 : 424– 440
16
L M Vaquero , F Cuadrado , Y Elkhatib , J Bernal-Bernabe , S N Srirama , M F Zhani . Research challenges in nextgen service orchestration. Future Generation Computer Systems, 2019, 90 : 20– 38
17
C Pahl, P Jamshidi. Microservices: a systematic mapping study. In: Proceedings of the 6th International Conference on Cloud Computing and Services Science. 2016, 137– 146
18
S Hassan, R Bahsoon. Microservices and their design trade-offs: a self-adaptive roadmap. In: Proceedings of 2016 IEEE International Conference on Services Computing. 2016, 813– 818
19
G Toffetti, S Brunner, M Blöchlinger, F Dudouet, A Edmonds. An architecture for self-managing microservices. In: Proceedings of the 1st International Workshop on Automated Incident Management in Cloud. 2015, 19– 24
20
B Familiar. Microservices, IoT, and Azure: Leveraging DevOps and Microservice Architecture to Deliver SaaS Solutions. Berkeley: Apress, 2015
21
P Jamshidi , C Pahl , N C Mendonça , J Lewis , S Tilkov . Microservices: the journey so far and challenges ahead. IEEE Software, 2018, 35( 3): 24– 35
22
I Baldini, P Castro, K Chang, P Cheng, S Fink, V Ishakian, N Mitchell, V Muthusamy, R Rabbah, A Slominski, P Suter. Serverless computing: current trends and open problems. In: Chaudhary S, Somani G, Buyya R, eds. Research Advances in Cloud Computing. Singapore: Springer, 2017, 1– 20
23
G C Fox, V Ishakian, V Muthusamy, A Slominski. Status of serverless computing and function-as-a-service (FaaS) in industry and research. 2017, arXiv preprint arXiv: 1708.08028
24
P Castro, V Ishakian, V Muthusamy, A Slominski. Serverless programming (function as a service). In: Proceedings of the IEEE 37th International Conference on Distributed Computing Systems. 2017, 2658−2659
25
M Yan, P Castro, P Cheng, V Ishakian. Building a chatbot with serverless computing. In: Proceedings of the 1st International Workshop on Mashups of Things and APIs. 2016, 5
26
V Ishakian, V Muthusamy, A Slominski. Serving deep learning models in a serverless platform. In: Proceedings of 2018 IEEE International Conference on Cloud Engineering. 2018, 257– 262
27
P Castro , V Ishakian , V Muthusamy , A Slominski . The rise of serverless computing. Communications of the ACM, 2019, 62( 12): 44– 54
28
K Kritikos, P Skrzypek. A review of serverless frameworks. In: Proceedings of IEEE/ACM International Conference on Utility and Cloud Computing Companion. 2018, 161– 168
29
B M Michelson . Event-driven architecture overview. Patricia Seybold Group, 2006, 2( 12): 10– 1571
30
J Thalheim, A Rodrigues, I E Akkus, P Bhatotia, R Chen, B Viswanath, L Jiao, C Fetzer. Sieve: actionable insights from monitored metrics in distributed systems. In: Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference. 2017, 14– 27
31
W Cui, D Richins, Y Zhu, V J Reddi. Tail latency in node.js: energy efficient turbo boosting for long latency requests in event-driven web services. In: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. 2019, 152– 164
32
B Terzic, V Dimitrieski, S Kordić, I Luković. A model-driven approach to microservice software architecture establishment. In: Proceedings of 2018 Federated Conference on Computer Science and Information Systems. 2018, 73– 80
33
F Rademacher, S Sachweh, A Zündorf. Differences between model-driven development of service-oriented and microservice architecture. In: Proceedings of 2017 IEEE International Conference on Software Architecture Workshops. 2017, 38– 45
34
S Mellor, K Scott, A Uhl, D Weise. Model-driven architecture. In: Proceedings of International Conference on Object-Oriented Information Systems. 2002, 290– 297
35
A R Da Silva . Model-driven engineering: a survey supported by the unified conceptual model. Computer Languages, Systems & Structures, 2015, 43 : 139– 155
36
E Seidewitz . What models mean. IEEE Software, 2003, 20( 5): 26– 32
37
Vale S, Hammoudi S. Model driven development of context-aware service oriented architecture. In: Proceedings of the 11th IEEE International Conference on Computational Science and Engineering-Workshops. 2008, 412– 418
38
D Ameller , X Burgués , O Collell , D Costal , X Franch , M P Papazoglou . Development of service-oriented architectures using model-driven development: a mapping study. Information and Software Technology, 2015, 62 : 42– 66
39
M Fazio , A Celesti , R Ranjan , C Liu , L Chen , M Villari . Open issues in scheduling microservices in the cloud. IEEE Cloud Computing, 2016, 3( 5): 81– 88
40
X Zhou, X Peng, T Xie, J Sun, C Xu, C Ji, W Zhao. Benchmarking microservice systems for software engineering research. In: Proceedings of the 40th IEEE/ACM International Conference on Software Engineering: Companion. 2018, 323–324
41
Aderaldo C M, Mendonça N C, Pahl C, Jamshidi P. Benchmark requirements for microservices architecture research. In: Proceedings of the 1st IEEE/ACM International Workshop on Establishing the Community-Wide Infrastructure for Architecture-Based Software Engineering. 2017, 8–13
42
X Zhou , X Peng , T Xie , J Sun , C Ji , W Li , D Ding . Fault analysis and debugging of microservice systems: industrial survey, benchmark system, and empirical study. IEEE Transactions on Software Engineering, 2021, 47( 2): 243– 260
43
Gan Y, Zhang Y, Cheng D, Shetty A, Rathi P, et al. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In: Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. 2019, 3−18
44
A Sriraman, T F Wenisch. µ suite: a benchmark suite for microservices. In: Proceedings of 2018 IEEE International Symposium on Workload Characterization. 2018, 1−12
45
N Kratzke, P C Quint. ppbench-a visualizing network benchmark for microservices. In: Proceedings of the 6th International Conference on Cloud Computing and Services Science. 2016, 223−231
46
A Sriraman, T F Wenisch. µtune: auto-tuned threading for OLDI microservices. In: Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation. 2018, 177−194
47
I Papapanagiotou, V Chella. NDBench: benchmarking microservices at scale. 2018, arXiv preprint arXiv: 1807.10792
48
T Ueda, T Nakaike, M Ohara. Workload characterization for microservices. In: Proceedings of 2016 IEEE International Symposium on Workload Characterization. 2016, 1−10
49
Y Gan , C Delimitrou . The architectural implications of cloud microservices. IEEE Computer Architecture Letters, 2018, 17( 2): 155– 158
50
A Sriraman, A Dhanotia, T F Wenisch. SoftSKU: optimizing server architectures for microservice diversity @scale. In: Proceedings of the 46th International Symposium on Computer Architecture. 2019, 513−526
51
L Liu. Qos-aware machine learning-based multiple resources scheduling for microservices in cloud environment. 2019, arXiv preprint arXiv: 1911.13208
52
B Xavier, T Ferreto, L Jersak. Time provisioning evaluation of KVM, docker and unikernels in a cloud platform. In: Proceedings of the 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2016, 277−280
53
P Saha, A Beltre, P Uminski, M Govindaraju. Evaluation of docker containers for scientific workloads in the cloud. In: Proceedings of the Practice and Experience on Advanced Research Computing. 2018, 11
54
D Jaramillo, D V Nguyen, R Smart. Leveraging microservices architecture by using docker technology. In: Proceedings of 2016 IEEE SoutheastCon. 2016, 1−5
55
H Kang, M Le, S Tao. Container and microservice driven design for cloud infrastructure DevOps. In: Proceedings of 2016 IEEE International Conference on Cloud Engineering. 2016, 202−211
56
T Lynn, P Rosati, A Lejeune, V Emeakaroha. A preliminary review of enterprise serverless cloud computing (function-as-a-service) platforms. In: Proceedings of 2017 IEEE International Conference on Cloud Computing Technology and Science. 2017, 162−169
57
C Esposito , A Castiglione , K K R Choo . Challenges in delivering software in the cloud as microservices. IEEE Cloud Computing, 2016, 3( 5): 10– 14
58
Villamizar M, Garcés O, Ochoa L, Castro H, Salamanca L, Verano M, Casallas R, Gil S, Valencia C, Zambrano A, Lang M. Infrastructure cost comparison of running Web applications in the cloud using AWS lambda and monolithic and microservice architectures. In: Proceedings of the 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2016, 179−182
59
V Friðriksson. Container overhead in microservice systems. KTH Royal Institute of Technology, Dissertation, 2018
60
Amaral M, Polo J, Carrera D, Mohomed I, Unuvar M, Steinder M. Performance evaluation of microservices architectures using containers. In: Proceedings of the 14th IEEE International Symposium on Network Computing and Applications. 2015, 27−34
61
N Kratzke. About microservices, containers and their underestimated impact on network performance. 2017, arXiv preprint arXiv: 1710.04049
62
Osses F, Márquez G, Astudillo H. Poster: exploration of academic and industrial evidence about architectural tactics and patterns in microservices. In: Proceedings of the 40th IEEE/ACM International Conference on Software Engineering: Companion. 2018, 256−257
63
D Shadija, M Rezai, R Hill. Microservices: granularity vs. performance. In: Proceedings of the10th International Conference on Utility and Cloud Computing. 2017, 215−220
64
W Lloyd, S Ramesh, S Chinthalapati, L Ly, S Pallickara. Serverless computing: an investigation of factors influencing microservice performance. In: Proceedings of 2018 IEEE International Conference on Cloud Engineering. 2018, 159−169
65
H Baek, A Srivastava, J Van der Merwe. CloudSight: a tenant-oriented transparency framework for cross-layer cloud troubleshooting. In: Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2017, 268−273
66
G Da Cunha Rodrigues, R N Calheiros, V T Guimaraes, G L Dos Santos, M B De Carvalho, L Granville, L M R Tarouco, R Buyya. Monitoring of cloud computing environments: concepts, solutions, trends, and future directions. In: Proceedings of the 31st Annual ACM Symposium on Applied Computing. 2016, 378−383
67
J Nicol, C Li, P Chen, T Feng, H Ramachandra. ODP: an infrastructure for on-demand service profiling. In: Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering. 2018, 139−144
68
Cinque M, Della Corte R, Pecchia A. Microservices monitoring with event logs and black box execution tracing. IEEE Transactions on Services Computing, 2019, DOI:
69
Sambasivan R, Shafer I, Mace J, Sigelman B, Fonseca R, Ganger G R. Principled workflow-centric tracing of distributed systems. In: Proceedings of the 7th ACM Symposium on Cloud Computing. 2016, 401−414
70
L Wu, J Bogatinovski, S Nedelkoski, J Tordsson, O Kao. Performance diagnosis in cloud microservices using deep learning. In: Proceedings of International Conference on Service-Oriented Computing. 2020, 85– 96
71
Ravichandiran R, Bannazadeh H, Leon-Garcia A. Anomaly detection using resource behaviour analysis for Autoscaling systems. In: Proceedings of the 4th IEEE Conference on Network Softwarization and Workshops. 2018, 192−196
72
Á Brandón , M Solé , A Huélamo , D Solans , M S Pérez , V Muntés-Mulero . Graph-based root cause analysis for service-oriented and microservice architectures. Journal of Systems and Software, 2020, 159 : 110432–
73
J Lin, P Chen, Z Zheng. Microscope: pinpoint performance issues with causal graphs in micro-service environments. In: Proceedings of the 16th International Conference on Service-Oriented Computing. 2018, 3−20
74
X Zhang, E Tune, R Hagmann, R Jnagal, V Gokhale, J Wilkes. CPI2: CPU performance isolation for shared compute clusters . In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 379−391
75
A Margaritov, S Gupta, R Gonzalez-Alberquilla, B Grot. Stretch: balancing QoS and throughput for colocated server workloads on SMT cores. In: Proceedings of 2019 IEEE International Symposium on High Performance Computer Architecture. 2019, 15−27
76
L Bao , C Wu , X Bu , N Ren , M Shen . Performance modeling and workflow scheduling of microservice-based applications in clouds. IEEE Transactions on Parallel and Distributed Systems, 2019, 30( 9): 2114– 2129
77
A Jindal, V Podolskiy, M Gerndt. Performance modeling for cloud microservice applications. In: Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering. 2019, 25−32
78
H Khazaei, N Mahmoudi, C Barna, M Litoiu. Performance modeling of microservice platforms. 2019, arXiv preprint arXiv: 1902.0338
79
Gribaudo M, Iacono M, Manini D. Performance evaluation of massively distributed microservices based applications. In: Proceedings of the 31st European Conference on Modelling and Simulation. 2017, 598−604
80
Kannan R S, Subramanian L, Raju A, Ahn J, Mars J, Tang L. GrandSLAm: guaranteeing SLAs for jobs in microservices execution frameworks. In: Proceedings of the 14th EuroSys Conference. 2019, 34
81
Correia J, Ribeiro F, Filipe R, Arauio F, Cardoso J. Response time characterization of microservice-based systems. In: Proceedings of the 17th IEEE International Symposium on Network Computing and Applications. 2018, 1−5
82
Y Yu , J Yang , C Guo , H Zheng , J He . Joint optimization of service request routing and instance placement in the microservice system. Journal of Network and Computer Applications, 2019, 147 : 102441–
83
C Yan , N Chen , Z Shuo . High-performance elastic management for cloud containers based on predictive message scheduling. Future Internet, 2017, 9( 4): 87–
84
X Hou, J Liu, C Li, M Guo. Unleashing the scalability potential of power-constrained data center in the microservice era. In: Proceedings of the 48th International Conference on Parallel Processing. 2019, 10
85
C Guerrero , I Lera , C Juiz . Resource optimization of container orchestration: a case study in multi-cloud microservices-based applications. The Journal of Supercomputing, 2018, 74( 7): 2956– 2983
86
X Leng, T H Juang, Y Chen, H Liu. AOMO: an AI-aided optimizer for microservices orchestration. In: Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos. 2019, 1−2
87
S Klock, J M E M van der Werf, J P Guelen, S Jansen. Workload-based clustering of coherent feature sets in microservice architectures. In: Proceedings of 2017 IEEE International Conference on Software Architecture. 2017, 11−20
88
Monteiro D, Gadelha R, Maia P H M, Rocha L S, Mendonça N C. Beethoven: an event-driven lightweight platform for microservice orchestration. In: Proceedings of the 12th European Conference on Software Architecture. 2018, 191−199
89
D Guo, W Wang, G Zeng, Z Wei. Microservices architecture based cloudware deployment platform for service computing. In: Proceedings of 2016 IEEE Symposium on Service-Oriented System Engineering. 2016, 358−363
90
J Rufino, M Alam, J Ferreira, A Rehman, K F Tsang. Orchestration of containerized microservices for IIoT using Docker. In: Proceedings of 2017 IEEE International Conference on Industrial Technology. 2017, 1532−1536
91
P J Meulenhoff, D R Ostendorf, M Živković, H B Meeuwissen, B M M Gijsen. Intelligent overload control for composite web services. In: Proceedings of the 7th International Joint Conference on Service-Oriented Computing. 2009, 34−49
92
Welsh M, Culler D, Brewer E. SEDA: an architecture for well-conditioned, scalable internet services. In: Proceedings of the 18th ACM Symposium on Operating Systems Principles. 2001, 230−243
93
H Yang, A Breslow, J Mars, L Tang. Bubble-flux: precise online QoS management for increased utilization in warehouse scale computers. In: Proceedings of the 40th Annual International Symposium on Computer Architecture. 2013, 607−618
94
C Delimitrou, C Kozyrakis. Quasar: resource-efficient and QoS-aware cluster management. In: Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems. 2014, 127−144
95
X Hou, C Li, J Liu, L Zhang, S Ren, J Leng, Q Chen, M Guo. AlphaR: learning-powered resource management for irregular, dynamic microservice graph. In: Proceedings of 2021 IEEE International Parallel and Distributed Processing Symposium. 2021, 797−806
96
H Alipour, Y Liu. Online machine learning for cloud resource provisioning of microservice backend systems. In: Proceedings of 2017 IEEE International Conference on Big Data. 2017, 2433−2441
97
M A Chang, A Panda, Y C Tsai, H Wang, S Shenker. ThrottleBot - performance without insight. 2017, arXiv preprint arXiv: 1711.00618
98
H Zhou, M Chen, Q Lin, Y Wang, X She, S Liu, R Gu, B C Ooi, J Yang. Overload control for scaling WeChat microservices. In: Proceedings of the ACM Symposium on Cloud Computing. 2018, 149−161
99
L Suresh, P Bodik, I Menache, M Canini, F Ciucu. Distributed resource management across process boundaries. In: Proceedings of 2017 Symposium on Cloud Computing. 2017, 611−623
100
M Xu , A N Toosi , R Buyya . iBrownout: an integrated approach for managing energy and brownout in container-based clouds. IEEE Transactions on Sustainable Computing, 2019, 4( 1): 53– 66
101
X Hou, C Li, J Liu, L Zhang, Y Hu, M Guo. ANT-man: towards agile power management in the microservice era. In: Proceedings of International Conference for High Performance Computing, Networking, Storage and Analysis. 2020, 78
102
C H Chou, L N Bhuyan, D Wong. µDPM: dynamic power management for the microsecond era. In: Proceedings of 2019 International Symposium on High Performance Computer Architecture. 2019, 120−132
103
A Mirhosseini, A Sriraman, T F Wenisch. Enhancing server efficiency in the face of killer microseconds. In: Proceedings of 2019 IEEE International Symposium on High Performance Computer Architecture. 2019, 185−198
104
H Kasture, D B Bartolini, N Beckmann, D Sanchez. Rubik: fast analytical power management for latency-critical systems. In: Proceedings of the 48th Annual IEEE/ACM International Symposium on Microarchitecture. 2015, 598−610
105
Lo D, Cheng L, Govindaraju R, Barroso L A, Kozyrakis C. Towards energy proportionality for large-scale latency-critical workloads. In: Proceedings of the 41st ACM/IEEE International Symposium on Computer Architecture. 2014, 301−312
106
Y Liu, S C Draper, N S Kim. SleepScale: runtime joint speed scaling and sleep states management for power efficient data centers. In: Proceedings of the 41st Annual International Symposium on Computer Architecuture. 2014, 313−324
107
S Boucher, A Kalia, D G Andersen, M Kaminsky. Putting the “micro” back in microservice. In: Proceedings of 2018 USENIX Annual Technical Conference. 2018, 645−650
108
E Oakes, L Yang, D Zhou, K Houck, T Harter, A Arpaci-Dusseau, R H Arpaci-Dusseau. SOCK: rapid task provisioning with serverless-optimized containers. In: Proceedings of 2018 USENIX Annual Technical Conference. 2018, 57−69
109
I Akkus, R Chen, I Rimac, M Stein, K Satzke, A Beck, P Aditya, V Hilt. SAND: towards high-performance serverless computing. In: Proceedings of 2018 USENIX Conference on Usenix Annual Technical Conference. 2018, 923−935
110
X Luo, F Ren, T Zhang. High performance userspace networking for containerized microservices. In: Proceedings of the 16th International Conference on Service-Oriented Computing. 2018, 57−72
