Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front Comput Sci    2012, Vol. 6 Issue (6) : 668-676    https://doi.org/10.1007/s11704-012-1080-8
RESEARCH ARTICLE
MiNT-OLAP cluster: minimizing network transmission cost in OLAP cluster for main memory analytical database
Min JIAO1,3(), Yansong ZHANG2, Zhanwei WANG1,3, Shan WANG1,3
1. DEKE Lab, Renmin University of China, Beijing 100872, China; 2. National Survey Research Center, Renmin University of China, Beijing 100872, China; 3. School of Information, Renmin University of China, Beijing 100872, China
 Download: PDF(573 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Powerful storage, high performance and scalability are the most important issues for analytical databases. These three factors interact with each other, for example, powerful storage needs less scalability but higher performance, high performance means less consumption of indexes and other materializations for storage and fewer processing nodes, larger scale relieves stress on powerful storage and the high performance processing engine. Some analytical databases (ParAccel, Teradata) bind their performance with advanced hardware supports, some (Asterdata, Greenplum) rely on the high scalability framework of MapReduce, some (MonetDB, Sybase IQ, Vertica) highlight performance on processing engine and storage engine. All these approaches can be integrated into an storage-performance-scalability (SP- S) model, and future large scale analytical processing can be built on moderate clusters to minimize expensive hardware dependency. The most important thing is a simple software framework is fundamental to maintain pace with the development of hardware technologies. In this paper, we propose a schema-aware on-line analytical processing (OLAP) model with deep optimization from native features of the star or snowflake schema. The OLAP model divides the whole process into several stages, each stage pipes its output to the next stage, we minimize the size of output data in each stage, whether in central processing or clustered processing. We extend this mechanism to cluster processing using two major techniques, one is using NetMemory as a broadcasting protocol based dimension mirror synchronizing buffer, the other is Received June 24, 2011; accepted August 16, 2012 E-mail: shingle@ruc.edu.cn predicate-vector based DDTA-OLAP cluster model which can minimize the data dependency of star-join using bitmap vectors. Our OLAP model aims to minimize network transmission cost (MiNT in short) for OLAP clusters and support a scalable but simple distributed storagemodel for large scale clustering processing. Finally, the experimental results show the speedup and scalability performance.

Keywords OLAP cluster      MiNT      NetMemory      schemaaware OLAP     
Corresponding Author(s): JIAO Min,Email:shingle@ruc.edu.cn   
Issue Date: 01 December 2012
 Cite this article:   
Min JIAO,Yansong ZHANG,Zhanwei WANG, et al. MiNT-OLAP cluster: minimizing network transmission cost in OLAP cluster for main memory analytical database[J]. Front Comput Sci, 2012, 6(6): 668-676.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-012-1080-8
https://academic.hep.com.cn/fcs/EN/Y2012/V6/I6/668
1 MacNicol R, French B. Sybase IQ multiplex-designed for analyticals. In: Proceedings of VLDB . 2004
2 Stonebraker M, Abadi D J, Batkin A, Chen X D, Cherniack M, Ferreira M, Lau E, Lin A, Madden S, O’Neil E J, O’Neil P E, Rasin A, Tran N, Zdonik S B. C-store: a column-oriented DBMS. In: Proceedings of VLDB . 2005, 553-564
3 Boncz P A, Mangegold S, Kersten M L. Database architecture optimized for the new bottleneck: memory access. In: Proceedings of VLDB . 1999, 266-277
4 Abadi D J. Tradeoffs between parallel database systems, hadoop, and hadoopDB as platforms for petabyte-scale analysis. In: Proceedings of SSDBM . 2010, 1-3
5 Abouzeid A, Bajda-Pawlikowski K, Abadi D J, Rasin A, Silberschatz A. HadoopDB: an architectural hybrid of MapReduce and DBMS technologies for analytical workloads. Proceedings of the VLDB Endowment , 2009, 2(1): 922-933
6 Zhang Y S, Hu W, Wang S. MOSS-DB: a hardware-aware OLAP database. In: Proceedings of WAIM . 2010, 582-594
7 O’Neil P, O’Neil B, Chen X D. The star schema benchmark (SSB). http://www.cs.umb.edu/~poneil/StarSchemaB.PDF
8 Li J Z, Srivastava J, Rotem D. CMD: a multidimensional declustering method for parallel data systems. In: Proceedings of VLDB . 1992, 3-14
9 Lima A A B, Furtado C, Valduriez P, Mattoso M. Parallel OLAP query processing in database clusters with data replication. Distributed and Parallel Databases , 2005, 25: 97-123
doi: 10.1007/s10619-009-7037-8
10 Furtado P. Model and procedure for performance and availability-wise parallel warehouses. Distributed and Parallel Databases , 2009, 25(1): 71-96
doi: 10.1007/s10619-009-7038-7
12 Yang C, Yen C, Tan C, Madden S. Osprey: implementing MapReducestyle fault tolerance in a shared-nothing distributed database. In: Proceedings of ICDE . 2010, 657-668
13 Chen S. Cheetah: a high performance, custom data warehouse on top of MapReduce. Proceedings of the VLDB Endowment , 2010, 3(2): 1459-1468
14 Winter Corporation White Paper. SAP NetWeaver: a complete platform for large-scale business intelligence. 2005
15 DeWitt D J, Gerber R H, Graefe G, Heytens M L, Kumar K B, Muralikrishna M. GAMMA-A high performance dataflow database machine. In: Proceedings of VLDB . 1986, 228-237
16 Fushimi S, Kitsuregawa M, Tanaka H. An overview of the system software of a parallel relational database machine. In: Proceedings of VLDB . 1986, 209-219
17 DeWitt D J, Gerber R H. Multiprocessor hash-based join algorithms. In: Proceedings of VLDB . 1985, 151-164
18 Candea G, Polyzotis N, Vingralek R. A scalable, predictable join operator for highly concurrent data warehouse. Proceedings of the VLDB Endowment , 2009, 2(1): 277-288
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed