Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front. Comput. Sci.    2021, Vol. 15 Issue (3) : 153103    https://doi.org/10.1007/s11704-020-9448-7
RESEARCH ARTICLE
Resource abstraction and data placement for distributed hybrid memory pool
Tingting CHEN, Haikun LIU(), Xiaofei LIAO, Hai JIN
National Engineering Research Center for Big Data Technology and System, Services Computing Technology and System Lab/Cluster and Grid Computing Lab, School of Computing Science and Technology, Huazhong University of Science and Technology,Wuhan 430074, China
 Download: PDF(591 KB)  
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Emerging byte-addressable non-volatile memory (NVM) technologies offer higher density and lower cost than DRAM, at the expense of lower performance and limited write endurance. There have been many studies on hybrid NVM/DRAMmemory management in a single physical server. However, it is still an open problem on how to manage hybrid memories efficiently in a distributed environment. This paper proposes Alloy, a memory resource abstraction and data placement strategy for an RDMA-enabled distributed hybrid memory pool (DHMP). Alloy provides simple APIs for applications to utilize DRAM or NVM resource in the DHMP, without being aware of the hardware details of the DHMP. We propose a hotness-aware data placement scheme, which combines hot data migration, data replication and write merging together to improve application performance and reduce the cost of DRAM. We evaluate Alloy with several micro-benchmark workloads and public benchmark workloads. Experimental results show that Alloy can significantly reduce the DRAM usage in the DHMP by up to 95%, while reducing the total memory access time by up to 57% compared with the state-of-the-art approaches.

Keywords load balance      distributed hybrid memory      clouds     
Corresponding Author(s): Haikun LIU   
Just Accepted Date: 26 March 2020   Issue Date: 27 January 2021
 Cite this article:   
Tingting CHEN,Haikun LIU,Xiaofei LIAO, et al. Resource abstraction and data placement for distributed hybrid memory pool[J]. Front. Comput. Sci., 2021, 15(3): 153103.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-020-9448-7
https://academic.hep.com.cn/fcs/EN/Y2021/V15/I3/153103
1 R Kapoor, G Porter, M Tewari, G M Voelker, A Vahdat. Chronos: predictable low latency for data center applications. In: Proceedings of the 3rd ACM Symposium on Cloud Computing. 2012, 1–14
https://doi.org/10.1145/2391229.2391238
2 J K Ousterhout, P Agrawal, D Erickson, C Kozyrakis, J Leverich, D Mazières, S Mitra, A Narayanan, G M Parulkar, M Rosenblum, S M Rumble, E Stratmann, R Stutsman. The case for ramclouds: scalable high-performance storage entirely in DRAM. Operating Systems Review, 2009, 43(4): 92–105
https://doi.org/10.1145/1713254.1713276
3 Y Xiao, S Nazarian, P Bogdan. Prometheus: processing-in-memory heterogeneous architecture design from a multi-layer network theoretic strategy. In: Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition. 2018, 1387–1392
https://doi.org/10.23919/DATE.2018.8342229
4 C Wu, G Zhang, K Li. Rethinking computer architectures and software systems for phase-change memory. ACM Journal on Emerging Technologies in Computing Systems, 2016, 12(4): 1–40
https://doi.org/10.1145/2893186
5 I B Peng, R Gioiosa, G Kestor, J S Vetter, P Cicotti, E Laure, S Markidis. Characterizing the performance benefit of hybrid memory system for HPC applications. Parallel Computing, 2018, 76: 57–69
https://doi.org/10.1016/j.parco.2018.04.007
6 I B Peng, R Gioiosa, G Kestor, P Cicotti, E Laure, S Markidis. RTHMS: a tool for data placement on hybrid memory system. In: Proceedings of the 2017 ACM SIGPLAN International Symposium onMemory Management. 2017, 82–91
https://doi.org/10.1145/3092255.3092273
7 K Wu, Y Huang, D Li. Unimem: runtime data management on nonvolatile memory-based heterogeneous main memory. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2017, 1–14
https://doi.org/10.1145/3126908.3126923
8 SM Lee, S K Yoon, J G Kim, S D Kim. Adaptive correlated prefetch with large-scale hybrid memory system for stream processing. The Journal of Supercomputing, 2018, 74(9): 4746–4770
https://doi.org/10.1007/s11227-018-2466-7
9 Y Li, S Ghose, J Choi, J Sun, H Wang, O Mutlu. Utility-based hybrid memory management. In: Proceedings of the 2017 IEEE International Conference on Cluster Computing. 2017, 152–165
https://doi.org/10.1109/CLUSTER.2017.130
10 Y Xie, D Feng, F Wang, L Zhang. Non-sequential striping encoder from replication to erasure coding for distributed storage system. Frontiers of Computer Science, 2019, 13(6): 1356–1358
https://doi.org/10.1007/s11704-019-8403-y
11 J O Gutierrez-Garcia, A Ramirez-Nafarrate. Collaborative agents for distributed load management in cloud data centers using live migration of virtual machines. IEEE Transactions on Services Computing, 2015, 8(6): 916–929
https://doi.org/10.1109/TSC.2015.2491280
12 M R Desai, H B Patel. Efficient virtual machine migration in cloud computing. In: Proceedings of the 15th International Conference on Communication Systems and Network Technologies. 2015, 1015–1019
https://doi.org/10.1109/CSNT.2015.263
13 J Yin, J Wang, J Zhou, T Lukasiewicz, D Huang, J Zhang. Opass: analysis and optimization of parallel data access on distributed file systems. In: Proceedings of the 2015 IEEE International Parallel and Distributed Processing Symposium. 2015, 623–632
https://doi.org/10.1109/IPDPS.2015.55
14 A Kalia, M Kaminsky, D G Andersen. FaSST: fast, scalable and simple distributed transactions with two-sided (RDMA) datagram RPCs. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation. 2016, 185–201
15 Z Duan, H Liu, X Liao, H Jin. HME: a lightweight emulator for hybrid memory. In: Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition. 2018, 1375–1380
https://doi.org/10.23919/DATE.2018.8342227
16 A Dragojević, D Narayanan, M Castro, O Hodson. FaRM: fast remote memory. In: Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation. 2014, 401–414
17 Y Lu, J Shu, Y Chen, T Li. Octopus: an RDMA-enabled distributed persistent memory file system. In: Proceedings of the 2017 USENIX Annual Technical Conference. 2017, 773–785
18 P Stuedi, A Trivedi, J Pfefferle, A Klimovic, A Schüpbach, B Metzler. Unification of temporary storage in the nodekernel architecture. In: Proceedings of the 2019 USENIX Annual Technical Conference. 2019, 767–782
19 J Nelson, B Holt, B Myers, P Briggs, L Ceze, S Kahan, M Oskin. Latencytolerant software distributed shared memory. In: Proceedings of the 2015 USENIX Annual Technical Conference. 2015, 291–305
20 Y Shan, S Y Tsai, Y Zhang. Distributed shared persistent memory. In: Proceedings of the 2017 Symposium on Cloud Computing. 2017, 323–337
https://doi.org/10.1145/3127479.3128610
21 V K Vavilapalli, A C Murthy, C Douglas, S Agarwal, M Konar, R Evans, T Graves, J Lowe, H Shah, S Seth, B Saha, C Curino, O O’Malley, S Radia, B Reed, E Baldeschwieler. Apache hadoop YARN: yet another resource negotiator. In: Proceedings of the 4th ACM Symposium on Cloud Computing. 2013, 1–16
https://doi.org/10.1145/2523616.2523633
22 S Ghemawat, H Gobioff, S T Leung. The google file system. In: Proceedings of the 19th ACM Symposium on Operating Systems Principles. 2003, 29–43
https://doi.org/10.1145/1165389.945450
23 M Munetomo, Y Takai, Y Sato. A genetic approach to dynamic load balancing in a distributed computing system. In: Proceedings of the 1st IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence. 1994, 419–421
24 X Peng, X Liu, H Cao, Z Zhang. An efficient energy aware virtual network migration based on genetic algorithm. Frontiers of Computer Science, 2019, 13(2): 440–442
https://doi.org/10.1007/s11704-019-8084-6
25 X Chen, J Lin, Y Ma, B Lin, H Wang, G Huang. Self-adaptive resource allocation for cloud-based software services based on progressive QoS prediction model. Science China Information Sciences, 2019, 62(11): 1–3
https://doi.org/10.1007/s11432-018-9750-2
26 Y Xia, R Ren, H Cai, A V Vasilakos, Z Lv. Daphne: a flexible and hybrid scheduling framework in multi-tenant clusters. IEEE Transactions on Network and Service Management, 2018, 15(1): 330–343
https://doi.org/10.1109/TNSM.2017.2777885
27 B Hindman, A Konwinski, M Zaharia, A Ghodsi, A D Joseph, R H Katz, S Shenker, I Stoica. Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation. 2011, 295–308
28 A Ghodsi, M Zaharia, B Hindman, A Konwinski, S Shenker, I Stoica. Dominant resource fairness: fair allocation of multiple resource types. In: Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation. 2011, 323–336
29 V A B Sanchez, W Kim, Y Eom, K Jin, M Nam, D Hwang, J Kim, B Nam. EclipseMR: distributed and parallel task processing with consistent hashing. In: Proceedings of the 2017 IEEE International Conference on Cluster Computing. 2017, 322–332
https://doi.org/10.1109/CLUSTER.2017.12
30 L Shen, J Wu, Y Wang, L Liu. Towards load balancing for LSH-based distributed similarity indexing in high-dimensional space. In: Proceedings of the 20th IEEE International Conference on High Performance Computing and Communications; the 16th IEEE International Conference on Smart City; the 4th IEEE International Conference on Data Science and Systems. 2018, 384–391
https://doi.org/10.1109/HPCC/SmartCity/DSS.2018.00082
31 C Kulkarni, A Kesavan, T Zhang, R Ricci, R Stutsman. Rocksteady: fast migration for low-latency in-memory storage. In: Proceedings of the 26th Symposium on Operating Systems Principles. 2017, 390–405
https://doi.org/10.1145/3132747.3132784
32 K Psychas, J Ghaderi. Randomized algorithms for scheduling multiresource jobs in the cloud. IEEE/ACM Transactions on Networking, 2018, 26(5): 2202–2215
https://doi.org/10.1109/TNET.2018.2863647
33 N S Islam, M Wasi-ur-Rahman, X Lu, D K Panda. Efficient data access strategies for hadoop and spark on HPC cluster with heterogeneous storage. In: Proceedings of the 2016 IEEE International Conference on Big Data. 2016, 223–232
https://doi.org/10.1109/BigData.2016.7840608
34 P Zhou, J Huang, X Qin, C Xie. PaRS: a popularity-aware redundancy scheme for in-memory stores. IEEE Transactions on Computers, 2019, 68(4): 556–569
https://doi.org/10.1109/TC.2018.2876827
35 N Zhao, A Anwar, Y Cheng, M Salman, D Li, J Wan, C Xie, X He, F Wang, A R Butt. Chameleon: an adaptive wear balancer for flash clusters. In: Proceedings of the 2018 IEEE International Parallel and Distributed Processing Symposium. 2018, 1163–1172
https://doi.org/10.1109/IPDPS.2018.00125
36 B Atikoglu, Y Xu, E Frachtenberg, S Jiang, M Paleczny. Workload analysis of a large-scale key-value store. In: Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems. 2012, 53–64
https://doi.org/10.1145/2318857.2254766
37 W Zhang, J Hwang, T Wood, K K Ramakrishnan, H H Huang. Load balancing of heterogeneous workloads in memcached clusters. In: Proceedings of the 9th International Workshop on Feedback Computing. 2014, 1–8
38 R Salkhordeh, H Asadi. An operating system level data migration scheme in hybrid DRAM-NVMmemory architecture. In: Proceedings of the 2016 Design, Automation & Test in Europe Conference & Exhibition. 2016, 936–941
https://doi.org/10.3850/9783981537079_0605
39 Y Zhou, R Alagappan, A Memaripour, A Badam, D Wentzlaff. HNVM: hybrid NVM enabled datacenter design and optimization. Microsoft Research Technical Report, 2017
40 R N Calheiros, R Ranjan, A Beloglazov, C A F D Rose, R Buyya. Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software: Practice and Experience, 2011, 41(1): 23–50
https://doi.org/10.1002/spe.995
41 J Izraelevitz, J Yang, L Zhang, J Kim, X Liu, A Memaripour, Y J Soh, Z Wang, Y Xu, S R Dulloor, J Zhao, S Swanson. Basic performance measurements of the intel optane DC persistent memory module. 2019, arXiv preprint arXiv:1903.05714
42 Y J Hong, M Thottethodi. Understanding and mitigating the impact of load imbalance in the memory caching tier. In: Proceedings of the ACM Symposium on Cloud Computing. 2013, 1–17
https://doi.org/10.1145/2523616.2525970
43 J Ou, J Shu, Y Lu, L Yi, W Wang. EDM: an endurance-aware data migration scheme for load balancing in SSD storage clusters. In: Proceedings of the 28th IEEE International Parallel and Distributed Processing Symposium. 2014, 787–796
https://doi.org/10.1109/IPDPS.2014.86
44 B F Cooper, A Silberstein, E Tam, R Ramakrishnan, R Sears. Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 143–154
https://doi.org/10.1145/1807128.1807152
[1] Highlights Download
[1] Huiqun WANG, Di HUANG, Yunhong WANG. GridNet: efficiently learning deep hierarchical representation for 3D point cloud understanding[J]. Front. Comput. Sci., 2022, 16(1): 161301-.
[2] Najme MANSOURI, Mohammad Masoud JAVIDI, Behnam Mohammad Hasani ZADE. Hierarchical data replication strategy to improve performance in cloud computing[J]. Front. Comput. Sci., 2021, 15(2): 152501-.
[3] Shuai ZHANG, Xinjun MAO, Fu HOU, Peini LIU. A field-based service management and discovery method in multiple clouds context[J]. Front. Comput. Sci., 2019, 13(5): 976-995.
[4] Najme MANSOURI. Adaptive data replication strategy in cloud computing for performance improvement[J]. Front. Comput. Sci., 2016, 10(5): 925-935.
[5] Mingqiang GUO,Ying HUANG,Zhong XIE. A balanced decomposition approach to real-time visualization of large vector maps in CyberGIS[J]. Front. Comput. Sci., 2015, 9(3): 442-455.
[6] Guozhi SONG, Liying YANG, Jigang WU, John SCHORMANS. Performance comparisons between cellular-only and cellular/WLAN integrated systems based on analytical models[J]. Front Comput Sci, 2013, 7(4): 486-495.
[7] Chunjie LUO, Jianfeng ZHAN, Zhen JIA, Lei WANG, Gang LU, Lixin ZHANG, Cheng-Zhong XU, Ninghui SUN. CloudRank-D: benchmarking and ranking cloud computing systems for data processing applications[J]. Front Comput Sci, 2012, 6(4): 347-362.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed