Please wait a minute...
Shopping Cart Email List Chinese
Advanced Search Fig/Tab
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Featured Articles  |  Most Downloaded  |  Most Reading
Online Submission Subscribe to Our RSS Content Feed
Front. Comput. Sci.    2014, Vol. 8 Issue (3) : 367-377    https://doi.org/10.1007/s11704-014-3499-6
RESEARCH ARTICLE
Hybrid hierarchy storage system in MilkyWay-2 supercomputer
Weixia XU1,2,Yutong LU1,2,*(),Qiong LI2,Enqiang ZHOU1,2,Zhenlong SONG2,Yong DONG1,2,Wei ZHANG1,2,Dengping WEI2,Xiaoming ZHANG2,Haitao CHEN1,2,Jianying XING2,Yuan YUAN2
1. State Key Laboratory of High Performance Computing, Changsha 410073, China
2. College of Computer, National University of Defense Technology, Changsha 410073, China
 Download: PDF(679 KB)  
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabytes. To reduce this gap, large-scale storage systems need to be designed and implemented with high performance and scalability. MilkyWay-2 (TH-2) supercomputer system with peak performance 54.9 Pflops, definitely has this kind of requirement for storage system. This paper mainly introduces the storage system in MilkyWay-2 supercomputer, including the hardware architecture and the parallel file system. The storage system in MilkyWay-2 supercomputer exploits a novel hybrid hierarchy storage architecture to enable high scalability of I/O clients, I/O bandwidth and storage capacity. To fit this architecture, a user level virtualized file system, named H2FS, is designed and implemented which can cooperate local storage and shared storage together into a dynamic single namespace to optimize I/O performance in IO-intensive applications. The evaluation results show that the storage system in MilkyWay-2 supercomputer can satisfy the critical requirements in large scale supercomputer, such as performance and scalability.
Keywords supercomputer      storage system      file system      MilkyWay-2      hybrid      hierarchy     
Corresponding Author(s): Yutong LU   
Issue Date: 24 June 2014
 Cite this article:   
Weixia XU,Yutong LU,Qiong LI, et al. Hybrid hierarchy storage system in MilkyWay-2 supercomputer[J]. Front. Comput. Sci., 2014, 8(3): 367-377.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-014-3499-6
https://academic.hep.com.cn/fcs/EN/Y2014/V8/I3/367
1 FranksB. Taming the Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics. www.wiley.com, 2012
2 VertaO, MastroianniC, TaliaD. A super-peer model for resource discovery services in large-scale grids. Future Generation Computer Systems, 2005, 21(8): 1235-1248
doi: 10.1016/j.future.2005.06.001
3 BentJ, GriderG, KetteringBr, ManzanaresA, McClellandM, TorresA, TorrezA. Storage challenges at Los Alamos National Lab. In: Proceedings of the 2012 Symposium on Massive Storage Systems and Technologies. 2012: 1-5
4 WatsonR W, CoyneR A. The parallel I/O architecture of the highperformance storage system. In: Proceedings of the 14th IEEE Symposium on Mass Storage Systems. 1995, 27-44
doi: 10.1109/MASS.1995.528214
5 LofsteadJ, ZhengF, LiuQ, KlaskyS, OldfieldR, KordenbrockT, SchwanK, WolfM. Managing variability in the IO performance of petascale storage system. IEEE Computer Society, 2010: 1-12
6 ZhugeH. The Knowledge Grid. Singapore: World Scientific, 2004
doi: 10.1142/5738
7 OldfieldR A, MaccabeA B, ArunagiriS, KordenbrockT, RiesenR, WardL, WidenerP. Lightweight I/O for scientific applications. Technical Report of Sandia National Laboratories, 2006, 1-11
8 LiuN, CopeJ, CarnsP H, CarothersC D, RossR B, GriderG, CrumeA, MaltzahnC. On the role of burst buffers in leadership-class storage systems. In: Proceedings of the 2012 Symposium on Massive Storage Systems and Technologies. 2012: 1-11
9 ZhouE Q, LuY T, ZhangW, DongY. H2FS: a hybrid hierarchy filesystem for scalable data-intensive computing for HPC systems. Poster paper in International Supercomputing Conference. 2013
10 Lustre: A scalable, high-performance file system. Cluster File Systems Inc. Whitepaper, Version 1.0, November2002. http://www.lustre.org/docs/white paper.pdf
11 XieM, LuY T, LiuL, CaoH J, YangX J. Implementation and evaluation of network interface and message passing services for TianHe-1A supercomputer. In: Proceedings of the 19th Annual IEEE Symposium on High Performance Interconnects. 2011, 78-86
12 WelchB, UnangstM, AbbasiZ, GibsonG, MuellerB, SmallJ, ZelenkaJ, ZhouB. Scalable performance of the panasas parallel file system. FAST, 2008, 8: 1-17
13 Top500 Lists, http://www.top500.org/lists/
14 RyuK D, InglettT A, BellofattoR, Blocksome,M. A, GoodingT, KumarS, MamidalaA R, Megerian,M G, MillerS, NelsonM T, RosenburgB, SmithB, VanO J, WangA, WisniewskiR W. IBM Blue Gene/Q system software stack. IBM Journal of Research and Development, 2013, 57(1/2): 1-12
doi: 10.1147/JRD.2012.2227557
15 RogersJ. Power efflciency and performance with ORNL’s cray XK7 Titan. Star Craft Companion, 2012: 1040-1050
16 YuW, VetterJ S, OralH S. Performance characterization and optimization of parallel i/o on the cray XT. In: Proceedings of the IEEE International Symposium on Parallel and Distributed Processing. 2008, 1-11
17 YuW, OralS, VetterJ, BarrettR. Efflciency evaluation of Ccray XT parallel IO stack. Cray User Group Meeting, 2007, 1-9
18 MiyazakiH, KusanoY, ShinjouN, . Overview of the K computer system. Fujitsu Scientific and Technical Journal, 2012, 48
19 XingJ, XiongJ, SunN, JieM. Adaptive and scalable metadata management to support a trillion files. In: Proceedings of the ACM Conference on High Performance Computing Networking, Storage and Analysis. 2009, 26: 1-11
doi: 10.1145/1654059.1654086
20 SurendraB, ChouJ, RübelO, Prabhat, KarimabadiH, DaughtonW S, RoytershteynV, BethelE W, HowisonM, HsuK J, LinK W, ShoshaniA, UseltonA, WuK. Parallel I/O, analysis, and visualization of a trillion particle simulation. Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society Press, 2012, 59: 1-12
21 PayneM, WidenerP, WolfM, AbbasiH, McManusS, BridgesP G, SchwanK. Exploiting latent I/O asynchrony in petascale science applications. In: Proceedings of the 4th IEEE International Conference on EScience. 2008: 410-411
22 AliN, CarnsP, IskraK, KimpeD, LangS, LathamR, RossR, WardL, SadayappanP. Scalable I/O forwarding framework for highperformance computing systems. In: Proceedings of the 2009 IEEE International Conference on Cluster Computing and Workshops. 2009, 1-10
doi: 10.1109/CLUSTR.2009.5289188
23 LuY Y, ShuJ W, LiS, YiL T. Accelerating distributed updates with asynchronous ordered writes in a parallel file system. In: Proceedings of the 2012 IEEE International Conference on Cluster Computing. 2012, 302-310
doi: 10.1109/CLUSTER.2012.38
24 SheplerS, CallaghanB, RobinsonD, ThurlowR, Sun Microsystems Inc., BeameC, Hummingbird Ltd., EislerM, DoveckD, Network Appliance Inc. Network file system version 4 protocol. Network, 2003, 3530
25 GoodsonG, WelchB, HalevyB, BlackD, AdamsonA. NFSv4 pNFS extensions. Technical Report, 2005
26 CarnsP H, SettlemyerB W, LigonW B III. Using server-to-server communication in parallel file systems to simplify consistency and improve performance. In: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing. 2008, 6
27 DevulapalliA, OhioP W. File creation strategies in a distributed metadata file system. In: Proceedings of the 2007 IEEE International on Parallel and Distributed Processing Symposium. 2007, 1-10
doi: 10.1109/IPDPS.2007.370295
28 CarnsP, LangS, RossR, VilayannurM, KunkelJ, LudwigT. Smallfile access in parallel file systems. In: Proceedings of the 2009 IEEE International Symposium on Parallel and Distributed Processing. 2009, 1-11
doi: 10.1109/IPDPS.2009.5161029
29 SakaiK, SumimotoS, KurokawaM. High-performance and highly reliable file system for the K computer. FUJITSU Science Technology, 2012, 48(3): 302-209
30 LiuQ, KlaskyS, PodhorszkiN, LofsteadH, AbbasiC S, ChangJ, CummingsD, DinakarC, DocanS, EthierR, GroutT, KordenbrockZ, LinX, MaR, OldfieldM, ParasharA, RomosanN, SamatovaK, SchwanA, ShoshaniY, TianM, WolfW, YuF, ZhangF, ZhengF. ADIOS: powering I/O to extreme scale computing. 1-6
31 LofsteadJ, ZhengF, KlaskyS, SchwanK. Adaptable, metadata rich IO methods for portable high performance IO. In: Proceedings of the 2009 International Parallel and Distributed Processing. 2009, 1-10
32 LiJ, LiaoW, ChoudharyA, RossR, ThakurR, GroppW, LathamR, SiegelA, GallagherB, ZingaleM. Parallel netCDF: a highperformance scientific I/O interface. In: Proceedings of the 2003 ACM/IEEE Conference on Supercomputing. 2003, 39
doi: 10.1145/1048935.1050189
[1] Huichao DUAN, Huiqi HU, Weining QIAN, Aoying ZHOU. Incremental join view maintenance on distributed log-structured storage[J]. Front. Comput. Sci., 2021, 15(4): 154607-.
[2] Bing WEI, Limin XIAO, Bingyu ZHOU, Guangjun QIN, Baicheng YAN, Zhisheng HUO. Fine-grained management of I/O optimizations based on workload characteristics[J]. Front. Comput. Sci., 2021, 15(3): 153102-.
[3] Tingting CHEN, Haikun LIU, Xiaofei LIAO, Hai JIN. Resource abstraction and data placement for distributed hybrid memory pool[J]. Front. Comput. Sci., 2021, 15(3): 153103-.
[4] Shahid AKBAR, Maqsood HAYAT, Muhammad IQBAL, Muhammad TAHIR. iRNA-PseTNC: identification of RNA 5-methylcytosine sites using hybrid vector space of pseudo nucleotide composition[J]. Front. Comput. Sci., 2020, 14(2): 451-460.
[5] Jia HE, Fuzhen ZHUANG, Yanchi LIU, Qing HE, Fen LIN. Bayesian dual neural networks for recommendation[J]. Front. Comput. Sci., 2019, 13(6): 1255-1265.
[6] Juan CHEN, Wenhao ZHOU, Yong DONG, Zhiyuan WANG, Chen CUI, Feihao WU, Enqiang ZHOU, Yuhua TANG. Analyzing time-dimension communication characterizations for representative scientific applications on supercomputer systems[J]. Front. Comput. Sci., 2019, 13(6): 1228-1242.
[7] Xiaodong MENG, Chentao WU, Minyi GUO, Long ZHENG, Jingyu ZHANG. PAM: an efficient power-aware multilevel cache policy to reduce energy consumption of storage systems[J]. Front. Comput. Sci., 2019, 13(4): 850-863.
[8] Zhenxue HE, Limin XIAO, Fei GU, Tongsheng XIA, Shubin SU, Zhisheng HUO, Rong ZHANG, Longbing ZHANG, Li RUAN, Xiang WANG. An efficient and fast polarity optimization approach for mixed polarity Reed-Muller logic circuits[J]. Front. Comput. Sci., 2017, 11(4): 728-742.
[9] Shaha AL-OTAIBI, Mourad YKHLEF. Hybrid immunizing solution for job recommender system[J]. Front. Comput. Sci., 2017, 11(3): 511-527.
[10] Jian LIU,Huanqing DONG,Junwei ZHANG,Zhenjun LIU,Lu XU. HWM: a hybrid workload migration mechanism of metadata server cluster in data center[J]. Front. Comput. Sci., 2017, 11(1): 75-87.
[11] Wen ZHOU,Dan FENG,Yu HUA,Jingning LIU,Fangting HUANG,Yu CHEN,Shuangwu ZHANG. Prober: exploiting sequential characteristics in buffer for improving SSDs write performance[J]. Front. Comput. Sci., 2016, 10(5): 951-964.
[12] Xingbo WU,Xiang LONG,Lei WANG. FlexPoll: adaptive event polling for network-intensive applications[J]. Front. Comput. Sci., 2016, 10(3): 532-542.
[13] Han XUE,Bing QIN,Ting LIU,Shen LIU. Topic hierarchy construction from heterogeneous evidence[J]. Front. Comput. Sci., 2016, 10(1): 136-146.
[14] Quanqing XU,Rajesh Vellore ARUMUGAM,Khai Leong YONG,Yonggang WEN,Yew-Soon ONG,Weiya XI. Adaptive and scalable load balancing for metadata server cluster in cloud-scale file systems[J]. Front. Comput. Sci., 2015, 9(6): 904-918.
[15] Lailong LUO,Deke GUO,Wenxin LI,Tian ZHANG,Junjie XIE,Xiaolei ZHOU. Compound graph based hybrid data center topologies[J]. Front. Comput. Sci., 2015, 9(6): 860-874.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed