|
|
Resource abstraction and data placement for distributed hybrid memory pool |
Tingting CHEN, Haikun LIU( ), Xiaofei LIAO, Hai JIN |
National Engineering Research Center for Big Data Technology and System, Services Computing Technology and System Lab/Cluster and Grid Computing Lab, School of Computing Science and Technology, Huazhong University of Science and Technology,Wuhan 430074, China |
|
|
Abstract Emerging byte-addressable non-volatile memory (NVM) technologies offer higher density and lower cost than DRAM, at the expense of lower performance and limited write endurance. There have been many studies on hybrid NVM/DRAMmemory management in a single physical server. However, it is still an open problem on how to manage hybrid memories efficiently in a distributed environment. This paper proposes Alloy, a memory resource abstraction and data placement strategy for an RDMA-enabled distributed hybrid memory pool (DHMP). Alloy provides simple APIs for applications to utilize DRAM or NVM resource in the DHMP, without being aware of the hardware details of the DHMP. We propose a hotness-aware data placement scheme, which combines hot data migration, data replication and write merging together to improve application performance and reduce the cost of DRAM. We evaluate Alloy with several micro-benchmark workloads and public benchmark workloads. Experimental results show that Alloy can significantly reduce the DRAM usage in the DHMP by up to 95%, while reducing the total memory access time by up to 57% compared with the state-of-the-art approaches.
|
Keywords
load balance
distributed hybrid memory
clouds
|
Corresponding Author(s):
Haikun LIU
|
Just Accepted Date: 26 March 2020
Issue Date: 27 January 2021
|
|
1 |
R Kapoor, G Porter, M Tewari, G M Voelker, A Vahdat. Chronos: predictable low latency for data center applications. In: Proceedings of the 3rd ACM Symposium on Cloud Computing. 2012, 1–14
https://doi.org/10.1145/2391229.2391238
|
2 |
J K Ousterhout, P Agrawal, D Erickson, C Kozyrakis, J Leverich, D Mazières, S Mitra, A Narayanan, G M Parulkar, M Rosenblum, S M Rumble, E Stratmann, R Stutsman. The case for ramclouds: scalable high-performance storage entirely in DRAM. Operating Systems Review, 2009, 43(4): 92–105
https://doi.org/10.1145/1713254.1713276
|
3 |
Y Xiao, S Nazarian, P Bogdan. Prometheus: processing-in-memory heterogeneous architecture design from a multi-layer network theoretic strategy. In: Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition. 2018, 1387–1392
https://doi.org/10.23919/DATE.2018.8342229
|
4 |
C Wu, G Zhang, K Li. Rethinking computer architectures and software systems for phase-change memory. ACM Journal on Emerging Technologies in Computing Systems, 2016, 12(4): 1–40
https://doi.org/10.1145/2893186
|
5 |
I B Peng, R Gioiosa, G Kestor, J S Vetter, P Cicotti, E Laure, S Markidis. Characterizing the performance benefit of hybrid memory system for HPC applications. Parallel Computing, 2018, 76: 57–69
https://doi.org/10.1016/j.parco.2018.04.007
|
6 |
I B Peng, R Gioiosa, G Kestor, P Cicotti, E Laure, S Markidis. RTHMS: a tool for data placement on hybrid memory system. In: Proceedings of the 2017 ACM SIGPLAN International Symposium onMemory Management. 2017, 82–91
https://doi.org/10.1145/3092255.3092273
|
7 |
K Wu, Y Huang, D Li. Unimem: runtime data management on nonvolatile memory-based heterogeneous main memory. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2017, 1–14
https://doi.org/10.1145/3126908.3126923
|
8 |
SM Lee, S K Yoon, J G Kim, S D Kim. Adaptive correlated prefetch with large-scale hybrid memory system for stream processing. The Journal of Supercomputing, 2018, 74(9): 4746–4770
https://doi.org/10.1007/s11227-018-2466-7
|
9 |
Y Li, S Ghose, J Choi, J Sun, H Wang, O Mutlu. Utility-based hybrid memory management. In: Proceedings of the 2017 IEEE International Conference on Cluster Computing. 2017, 152–165
https://doi.org/10.1109/CLUSTER.2017.130
|
10 |
Y Xie, D Feng, F Wang, L Zhang. Non-sequential striping encoder from replication to erasure coding for distributed storage system. Frontiers of Computer Science, 2019, 13(6): 1356–1358
https://doi.org/10.1007/s11704-019-8403-y
|
11 |
J O Gutierrez-Garcia, A Ramirez-Nafarrate. Collaborative agents for distributed load management in cloud data centers using live migration of virtual machines. IEEE Transactions on Services Computing, 2015, 8(6): 916–929
https://doi.org/10.1109/TSC.2015.2491280
|
12 |
M R Desai, H B Patel. Efficient virtual machine migration in cloud computing. In: Proceedings of the 15th International Conference on Communication Systems and Network Technologies. 2015, 1015–1019
https://doi.org/10.1109/CSNT.2015.263
|
13 |
J Yin, J Wang, J Zhou, T Lukasiewicz, D Huang, J Zhang. Opass: analysis and optimization of parallel data access on distributed file systems. In: Proceedings of the 2015 IEEE International Parallel and Distributed Processing Symposium. 2015, 623–632
https://doi.org/10.1109/IPDPS.2015.55
|
14 |
A Kalia, M Kaminsky, D G Andersen. FaSST: fast, scalable and simple distributed transactions with two-sided (RDMA) datagram RPCs. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation. 2016, 185–201
|
15 |
Z Duan, H Liu, X Liao, H Jin. HME: a lightweight emulator for hybrid memory. In: Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition. 2018, 1375–1380
https://doi.org/10.23919/DATE.2018.8342227
|
16 |
A Dragojević, D Narayanan, M Castro, O Hodson. FaRM: fast remote memory. In: Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation. 2014, 401–414
|
17 |
Y Lu, J Shu, Y Chen, T Li. Octopus: an RDMA-enabled distributed persistent memory file system. In: Proceedings of the 2017 USENIX Annual Technical Conference. 2017, 773–785
|
18 |
P Stuedi, A Trivedi, J Pfefferle, A Klimovic, A Schüpbach, B Metzler. Unification of temporary storage in the nodekernel architecture. In: Proceedings of the 2019 USENIX Annual Technical Conference. 2019, 767–782
|
19 |
J Nelson, B Holt, B Myers, P Briggs, L Ceze, S Kahan, M Oskin. Latencytolerant software distributed shared memory. In: Proceedings of the 2015 USENIX Annual Technical Conference. 2015, 291–305
|
20 |
Y Shan, S Y Tsai, Y Zhang. Distributed shared persistent memory. In: Proceedings of the 2017 Symposium on Cloud Computing. 2017, 323–337
https://doi.org/10.1145/3127479.3128610
|
21 |
V K Vavilapalli, A C Murthy, C Douglas, S Agarwal, M Konar, R Evans, T Graves, J Lowe, H Shah, S Seth, B Saha, C Curino, O O’Malley, S Radia, B Reed, E Baldeschwieler. Apache hadoop YARN: yet another resource negotiator. In: Proceedings of the 4th ACM Symposium on Cloud Computing. 2013, 1–16
https://doi.org/10.1145/2523616.2523633
|
22 |
S Ghemawat, H Gobioff, S T Leung. The google file system. In: Proceedings of the 19th ACM Symposium on Operating Systems Principles. 2003, 29–43
https://doi.org/10.1145/1165389.945450
|
23 |
M Munetomo, Y Takai, Y Sato. A genetic approach to dynamic load balancing in a distributed computing system. In: Proceedings of the 1st IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence. 1994, 419–421
|
24 |
X Peng, X Liu, H Cao, Z Zhang. An efficient energy aware virtual network migration based on genetic algorithm. Frontiers of Computer Science, 2019, 13(2): 440–442
https://doi.org/10.1007/s11704-019-8084-6
|
25 |
X Chen, J Lin, Y Ma, B Lin, H Wang, G Huang. Self-adaptive resource allocation for cloud-based software services based on progressive QoS prediction model. Science China Information Sciences, 2019, 62(11): 1–3
https://doi.org/10.1007/s11432-018-9750-2
|
26 |
Y Xia, R Ren, H Cai, A V Vasilakos, Z Lv. Daphne: a flexible and hybrid scheduling framework in multi-tenant clusters. IEEE Transactions on Network and Service Management, 2018, 15(1): 330–343
https://doi.org/10.1109/TNSM.2017.2777885
|
27 |
B Hindman, A Konwinski, M Zaharia, A Ghodsi, A D Joseph, R H Katz, S Shenker, I Stoica. Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation. 2011, 295–308
|
28 |
A Ghodsi, M Zaharia, B Hindman, A Konwinski, S Shenker, I Stoica. Dominant resource fairness: fair allocation of multiple resource types. In: Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation. 2011, 323–336
|
29 |
V A B Sanchez, W Kim, Y Eom, K Jin, M Nam, D Hwang, J Kim, B Nam. EclipseMR: distributed and parallel task processing with consistent hashing. In: Proceedings of the 2017 IEEE International Conference on Cluster Computing. 2017, 322–332
https://doi.org/10.1109/CLUSTER.2017.12
|
30 |
L Shen, J Wu, Y Wang, L Liu. Towards load balancing for LSH-based distributed similarity indexing in high-dimensional space. In: Proceedings of the 20th IEEE International Conference on High Performance Computing and Communications; the 16th IEEE International Conference on Smart City; the 4th IEEE International Conference on Data Science and Systems. 2018, 384–391
https://doi.org/10.1109/HPCC/SmartCity/DSS.2018.00082
|
31 |
C Kulkarni, A Kesavan, T Zhang, R Ricci, R Stutsman. Rocksteady: fast migration for low-latency in-memory storage. In: Proceedings of the 26th Symposium on Operating Systems Principles. 2017, 390–405
https://doi.org/10.1145/3132747.3132784
|
32 |
K Psychas, J Ghaderi. Randomized algorithms for scheduling multiresource jobs in the cloud. IEEE/ACM Transactions on Networking, 2018, 26(5): 2202–2215
https://doi.org/10.1109/TNET.2018.2863647
|
33 |
N S Islam, M Wasi-ur-Rahman, X Lu, D K Panda. Efficient data access strategies for hadoop and spark on HPC cluster with heterogeneous storage. In: Proceedings of the 2016 IEEE International Conference on Big Data. 2016, 223–232
https://doi.org/10.1109/BigData.2016.7840608
|
34 |
P Zhou, J Huang, X Qin, C Xie. PaRS: a popularity-aware redundancy scheme for in-memory stores. IEEE Transactions on Computers, 2019, 68(4): 556–569
https://doi.org/10.1109/TC.2018.2876827
|
35 |
N Zhao, A Anwar, Y Cheng, M Salman, D Li, J Wan, C Xie, X He, F Wang, A R Butt. Chameleon: an adaptive wear balancer for flash clusters. In: Proceedings of the 2018 IEEE International Parallel and Distributed Processing Symposium. 2018, 1163–1172
https://doi.org/10.1109/IPDPS.2018.00125
|
36 |
B Atikoglu, Y Xu, E Frachtenberg, S Jiang, M Paleczny. Workload analysis of a large-scale key-value store. In: Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems. 2012, 53–64
https://doi.org/10.1145/2318857.2254766
|
37 |
W Zhang, J Hwang, T Wood, K K Ramakrishnan, H H Huang. Load balancing of heterogeneous workloads in memcached clusters. In: Proceedings of the 9th International Workshop on Feedback Computing. 2014, 1–8
|
38 |
R Salkhordeh, H Asadi. An operating system level data migration scheme in hybrid DRAM-NVMmemory architecture. In: Proceedings of the 2016 Design, Automation & Test in Europe Conference & Exhibition. 2016, 936–941
https://doi.org/10.3850/9783981537079_0605
|
39 |
Y Zhou, R Alagappan, A Memaripour, A Badam, D Wentzlaff. HNVM: hybrid NVM enabled datacenter design and optimization. Microsoft Research Technical Report, 2017
|
40 |
R N Calheiros, R Ranjan, A Beloglazov, C A F D Rose, R Buyya. Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software: Practice and Experience, 2011, 41(1): 23–50
https://doi.org/10.1002/spe.995
|
41 |
J Izraelevitz, J Yang, L Zhang, J Kim, X Liu, A Memaripour, Y J Soh, Z Wang, Y Xu, S R Dulloor, J Zhao, S Swanson. Basic performance measurements of the intel optane DC persistent memory module. 2019, arXiv preprint arXiv:1903.05714
|
42 |
Y J Hong, M Thottethodi. Understanding and mitigating the impact of load imbalance in the memory caching tier. In: Proceedings of the ACM Symposium on Cloud Computing. 2013, 1–17
https://doi.org/10.1145/2523616.2525970
|
43 |
J Ou, J Shu, Y Lu, L Yi, W Wang. EDM: an endurance-aware data migration scheme for load balancing in SSD storage clusters. In: Proceedings of the 28th IEEE International Parallel and Distributed Processing Symposium. 2014, 787–796
https://doi.org/10.1109/IPDPS.2014.86
|
44 |
B F Cooper, A Silberstein, E Tam, R Ramakrishnan, R Sears. Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 143–154
https://doi.org/10.1145/1807128.1807152
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|