JAPO: learning join and pushdown order for cloud-native join optimization

doi:10.1007/s11704-024-3937-z

Front. Comput. Sci.

2024, Vol. 18

Issue (6) : 186614 https://doi.org/10.1007/s11704-024-3937-z

Information Systems

JAPO: learning join and pushdown order for cloud-native join optimization

Yuchen YUAN, Xiaoyue FENG, Bo ZHANG, Pengyi ZHANG, Jie SONG(

)

Software College, Northeastern University, Shenyang 110819, China

Download: PDF(3084 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Corresponding Author(s): Jie SONG

Just Accepted Date: 21 May 2024 Issue Date: 21 June 2024

Cite this article:

Yuchen YUAN,Xiaoyue FENG,Bo ZHANG, et al. JAPO: learning join and pushdown order for cloud-native join optimization[J]. Front. Comput. Sci., 2024, 18(6): 186614.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-024-3937-z
https://academic.hep.com.cn/fcs/EN/Y2024/V18/I6/186614

Fig.1 Same query with different pushdown and order selections. The yellow arrows are the transmission of join results. The gray arrows are the transmission of original tables. The third plan has the minimum join cost

Fig.2 JAPO framework

Fig.3 Costs with different optimal order. (a) Best join order without pushdown (RTOS, JAPO); (b) best join order with fully pushdown (RTOS, JAPO); (c) normal join order with fully pushdown (JAPO); (d) normal join order with partial pushdown (JAPO)

Fig.4 Query latency on different templates

Fig.5 Latency on TPC-H

Fig.6 Performance on different dataset size. (a) Small size of data; (b) medium size of data; (c) large size of data

1	X, Yu G, Li C, Chai N Tang . Reinforcement learning with tree-LSTM for join order selection. In: Proceedings of the 36th IEEE International Conference on Data Engineering (ICDE). 2020, 1297−1308
2	W, Cao Y, Liu Z, Cheng N, Zheng W, Li W, Wu L, Ouyang P, Wang Y, Wang R, Kuan Z, Liu F, Zhu T Zhang . POLARDB meets computational storage: efficiently support analytical workloads in cloud-native relational database. In: Proceedings of the 18th USENIX Conference on File and Storage Technologies. 2020, 29−42
3	D, Huang Q, Liu Q, Cui Z, Fang X, Ma F, Xu L, Shen L, Tang Y, Zhou M, Huang W, Wei C, Liu J, Zhang J, Li X, Wu L, Song R, Sun S, Yu L, Zhao N, Cameron L, Pei X Tang . TiDB: a raft-based HTAP database. Proceedings of the VLDB Endowment, 2020, 13( 12): 3072–3084
4	R, Marcus O Papaemmanouil . Deep reinforcement learning for join order enumeration. In: Proceedings of the 1st International Workshop on Exploiting Artificial Intelligence Techniques for Data Management. 2018, 3
5	V, Leis A, Gubichev A, Mirchev P, Boncz A, Kemper T Neumann . How good are query optimizers, really?. Proceedings of the VLDB Endowment, 2015, 9( 3): 204–215

Viewed

Full text

Abstract

Cited

Shared

Discussed