Benchmarking on intensive transaction processing

doi:10.1007/s11704-019-8438-0

Front. Comput. Sci.

2020, Vol. 14

Issue (5) : 145204 https://doi.org/10.1007/s11704-019-8438-0

RESEARCH ARTICLE

Benchmarking on intensive transaction processing

Chunxi ZHANG, Yuming LI, Rong ZHANG(

), Weining QIAN, Aoying ZHOU

1School of Data Science and Engineering, East China Normal University, Shanghai 200062, China

Download: PDF(1280 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Benchmarks play a crucial role in database performance evaluation, and have been effectively promoting the development of database management systems.With critical transaction processing requirements of new applications, we see an explosion of innovative database technologies for dealing with highly intensive transaction workloads (OLTP) with the obvious characteristics of sharp dynamics, terrific skewness, high contention, or high concurrency (abbr. DSC²), which can not be well described or evaluated by current standard benchmarks. In this paper, based on the representative SecKill applications, we define a pacakge of workloads simulating intensive transactional processing requirements. And we create a general and flexible benchmark framework PeakBench for evaluating intensive OLTP workloads on databases. We are the first work to have full control on simulating DSC2, especially for the fine granularity control for contention generation. With a comprehensive set of experiments conducted on popular open sourced DBMSs compared with the other representative OLTP benchmarks, we completely demonstrate the usefulness of PeakBench.

Keywords benchmark transaction processing intensiveworkloads evaluation

Corresponding Author(s): Rong ZHANG

Just Accepted Date: 08 July 2019 Issue Date: 10 March 2020

Cite this article:

Chunxi ZHANG,Yuming LI,Rong ZHANG, et al. Benchmarking on intensive transaction processing[J]. Front. Comput. Sci., 2020, 14(5): 145204.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-019-8438-0
https://academic.hep.com.cn/fcs/EN/Y2020/V14/I5/145204

1	D E Difallah, A Pavlo, C Curino, P CudreMaurouxet. OLTP-bench: an extensible testbed for benchmarking relational databases. Proceedings of the VLDB Endowment, 2013, 7(4): 277–288 https://doi.org/10.14778/2732240.2732246
2	K Ren, J M Faleiro, i D J Abad. Design principles for scaling multi-core OLTP under high contention. In: Proceedings of the 2016 International Conference on Management of Data. 2016, 1583–1598 https://doi.org/10.1145/2882903.2882958
3	N Zhou, X Zhou, X Zhang, X Du, S Wang. Reordering transaction execution to boost high-frequency trading applications. Data Science and Engineering, 2017, 2(4): 301–315 https://doi.org/10.1007/s41019-017-0054-0
4	V Persico, A Pescapé, A Picariello, G Sperli. Benchmarking big data architectures for social networks data processing using public cloud platforms. Future Generation Computer Systems, 2018, 89: 98–109 https://doi.org/10.1016/j.future.2018.05.068
5	R Harding, D Van Aken, A Pavlo, M Stonebraker. An evaluation of distributed concurrency control. Proceedings of the VLDB Endowment, 2017, 10(5): 553–564 https://doi.org/10.14778/3055540.3055548
6	S Q L My. For traditional databases, details are published on the MySql home page
7	S Q L Postgre. Database details are published on the PostgreSQL home page
8	D B Volt , LLC. Voltdb technical overview. Whitepaper, 2010
9	D Bitton, D J Dewitt, C Turbyfill. Benchmarking database systems a systematic approach. In: Proceedings of the 9th International Conference on Very Large Data Bases. 1983, 8–19
10	J Gray. The benchmark handbook for database and transaction systems. In: Proceedings of Sigmod Conference on Gray Hardavellas. 1993
11	Y C Tay. Data generation for application-specific benchmarking. Proceedings of the VLDB Endowment, 2011, 4(12): 1–4
12	S Chen, A Ailamaki, M Athanassoulis, P B Gibbons, R Johnson, I Pandis, R Stoica. TPC-E vs. TPC-C: characterizing the new TPC-E benchmark via an I/O comparison study. ACM Sigmod Record, 2011, 39(3): 5–10 https://doi.org/10.1145/1942776.1942778
13	R Cole, F Funke, L Giakoumakis, W Guy, A Kemper, S Krompass, H Kuno, R Nambiar, T Neumann, M Poess, K Sattler, M Seibold, E Simon, F Waas. The mixed workload CH-benCHmark. In: Proceedings of the 4th International Workshop on Testing Database Systems. 2011 https://doi.org/10.1145/1988842.1988850
14	Council, Transaction Processing Performance. TPC-H benchmark specification. Tcp.org Website, 2008
15	M J Cahill, U Röhm, A D Fekete. Serializable isolation for snapshot databases. ACM Transactions on Database Systems, 2009, 34(4): 20 https://doi.org/10.1145/1620585.1620587
16	A Wolski. TATP Benchmark Description (Version 1.0). 2009
17	M Stonebraker. A measure of transaction processing power. In: Hellerstein J M, Stonebraker M, Readings in Database Systems, 2nd ed. Morgan Kaufmann Publishers Inc., 1994, 442–454
18	B F Cooper, A Silberstein, E Tam, R Ramakrishnan, R Sears. Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 143 https://doi.org/10.1145/1807128.1807152
19	S Patil, M Polte, R Kai, W Tantisiriroj, X Lin, J López, G Gibson, A Fuchs, B Rinaldi. YCSB++: benchmarking and performance debugging advanced features in scalable table stores. In: Proceedings of the 2nd ACM Symposium on Cloud Computing. 2011 https://doi.org/10.1145/2038916.2038925
20	J E Gonzalez, R S Xin, A Dave, D Crankshaw, I Stoica. Graphx: graph processing in a distributed dataflow framework. In: Proceedings of the 11th Usenix Conference on Operating Systems Design & Implementation. 2014

[1]

Article highlights

Download

[1]	Zhumin CHEN, Xueqi CHENG, Shoubin DONG, Zhicheng DOU, Jiafeng GUO, Xuanjing HUANG, Yanyan LAN, Chenliang LI, Ru LI, Tie-Yan LIU, Yiqun LIU, Jun MA, Bing QIN, Mingwen WANG, Jirong WEN, Jun XU, Min ZHANG, Peng ZHANG, Qi ZHANG. Information retrieval: a view from the Chinese IR community[J]. Front. Comput. Sci., 2021, 15(1): 151601-.
[2]	Zhenghui HU, Wenjun WU, Jie LUO, Xin WANG, Boshu LI. Quality assessment in competition-based software crowdsourcing[J]. Front. Comput. Sci., 2020, 14(6): 146207-.
[3]	Huan ZHOU, Jinwei GUO, Huiqi HU, Weining QIAN, Xuan ZHOU, Aoying ZHOU. Plover: parallel logging for replication systems[J]. Front. Comput. Sci., 2020, 14(4): 144606-.
[4]	Samuel IRVING, Bin LI, Shaoming CHEN, Lu PENG, Weihua ZHANG, Lide DUAN. Computer comparisons in the presence of performance variation[J]. Front. Comput. Sci., 2020, 14(1): 21-41.
[5]	Peng PENG, Lei ZOU, Zhenqin DU, Dongyan ZHAO. Using partial evaluation in holistic subgraph search[J]. Front. Comput. Sci., 2018, 12(5): 966-983.
[6]	Houkui ZHOU, Huimin YU, Roland HU. Topic evolution based on the probabilistic topic model: a review[J]. Front. Comput. Sci., 2017, 11(5): 786-802.
[7]	Cheqing JIN,Yangxin KONG,Qiangqiang KANG,Weining QIAN,Aoying ZHOU. Benchmarking in-memory database[J]. Front. Comput. Sci., 2016, 10(6): 1067-1081.
[8]	Xingbo WU,Xiang LONG,Lei WANG. FlexPoll: adaptive event polling for network-intensive applications[J]. Front. Comput. Sci., 2016, 10(3): 532-542.
[9]	Xiangke LIAO,Liquan XIAO,Canqun YANG,Yutong LU. MilkyWay-2 supercomputer: system and application[J]. Front. Comput. Sci., 2014, 8(3): 345-356.
[10]	Guozhi SONG, Liying YANG, Jigang WU, John SCHORMANS. Performance comparisons between cellular-only and cellular/WLAN integrated systems based on analytical models[J]. Front Comput Sci, 2013, 7(4): 486-495.
[11]	Chunjie LUO, Jianfeng ZHAN, Zhen JIA, Lei WANG, Gang LU, Lixin ZHANG, Cheng-Zhong XU, Ninghui SUN. CloudRank-D: benchmarking and ranking cloud computing systems for data processing applications[J]. Front Comput Sci, 2012, 6(4): 347-362.
[12]	Xiujuan ZHAO, Jianmin SHI. Evaluation of mutual funds using multi-dimensional information[J]. Front Comput Sci Chin, 2010, 4(2): 237-253.
[13]	LIU Qun, LIU Hong, TANG Sheng, LIN Shouxun, QIAN Yueliang, WANG Xiangdong, XIONG Deyi, HOU Hongxu, SUN Le, LV Yuanhua, LI Wenbo. HTRDP evaluations on Chinese information processing and intelligent human-machine interface[J]. Front. Comput. Sci., 2007, 1(1): 58-93.

Viewed

Full text

Abstract

Cited

Shared

Discussed