Accelerating the cryo-EM structure determination in RELION on GPU cluster

doi:10.1007/s11704-020-0169-8

Frontiers of Computer Science

2022, Vol. 16

Issue (3): 163102 https://doi.org/10.1007/s11704-020-0169-8

本期目录

Accelerating the cryo-EM structure determination in RELION on GPU cluster

Xin YOU¹, Hailong YANG^1,²(

), Zhongzhi LUAN¹, Depei QIAN¹

¹. Sino-German Joint Software Institute, School of Computer Science and Engineering, Beihang University, Beijing 100191, China
². State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi 214125, China

全文: PDF(20979 KB) HTML

Abstract：

The cryo-electron microscopy (cryo-EM) is one of the most powerful technologies available today for structural biology. The RELION (Regularized Likelihood Optimization) implements a Bayesian algorithm for cryo-EM structure determination, which is one of the most widely used software in this field. Many researchers have devoted effort to improve the performance of RELION to satisfy the analysis for the ever-increasing volume of datasets. In this paper, we focus on performance analysis of the most time-consuming computation steps in RELION and identify their performance bottlenecks for specific optimizations. We propose several performance optimization strategies to improve the overall performance of RELION, including optimization of expectation step, parallelization of maximization step, accelerating the computation of symmetries, and memory affinity optimization. The experiment results show that our proposed optimizations achieve significant speedups of RELION across representative datasets. In addition, we perform roofline model analysis to understand the effectiveness of our optimizations.

Key words： cryo-EM structure determination performance optimization GPU acceleration RELION

收稿日期: 2020-04-25 出版日期: 2021-10-18

Corresponding Author(s): Hailong YANG

引用本文:

. [J]. Frontiers of Computer Science, 2022, 16(3): 163102.
Xin YOU, Hailong YANG, Zhongzhi LUAN, Depei QIAN. Accelerating the cryo-EM structure determination in RELION on GPU cluster. Front. Comput. Sci., 2022, 16(3): 163102.

链接本文:

https://academic.hep.com.cn/fcs/CN/10.1007/s11704-020-0169-8
https://academic.hep.com.cn/fcs/CN/Y2022/V16/I3/163102

Fig.1

Tab.1

Tab.2

Fig.2

Tab.3

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

	Flops	Bytes	Operational Intensity
Select	$N a ? (36 + N R ? 42) + N e f f e c t ? 11$	$N a ? (6 + N R ? 15) + N e f f e c t ? 3$	$N a ? (36 + N R ? 42) + N e f f e c t ? 11 N a ? (6 + N R ? 15) + N e f f e c t ? 3$
Select:opt	$16 + N a ? (22 + N R ? 42) + N e f f e c t ? 8$	$N a ? (6 + N R ? 15) + N e f f e c t ? 3$	$16 + N a ? (22 + N R ? 42) + N e f f e c t ? 8 N a ? (6 + N R ? 15) + N e f f e c t ? 3$
doGridding	$N x ? N y ? N z ? (2 ? l o g 2 (N x ? N y ? N z) + 11)$	$N x ? N y ? N z ? 7$	$2 ? l o g 2 (N x ? N y ? N z) + 11 7$
doGridding div & buff	$N x ? N y ? N z ? (2 ? l o g 2 (N x ? N y ? N z) + 11)$	$N x ? N y ? N z ? 13$	$2 ? l o g 2 (N x ? N y ? N z) + 11 13$
Symmetry	$N x ? N y ? N z ? 98$	$42 ? N x ? N y ? N z + 32 ? N s y m s$	$≈ 2.333$
Symmetry:opt	$N x ? N y ? N z ? 98$	$39 ? N x ? N y ? N z + 9 ? N s y m s$	$≈ 2.513$

Tab.4

Fig.18

Fig.19

Fig.20

Tab.5

Tab.6

Fig.21

1	J Frank , B Shimkin , H Dowse . Spider—a modular software system for electron image processing. Ultramicroscopy, 1981, 6( 4): 343– 357
2	N Grigorieff . Frealign: high-resolution refinement of single particle structures. Journal of Structural Biology, 2007, 157( 1): 117– 125
3	G Tang , L Peng , P R Baldwin , D S Mann , W Jiang , I Rees , S J Ludtke . Eman2: an extensible image processing suite for electron microscopy. Journal of Structural Biology, 2007, 157( 1): 38– 46
4	D Elmlund , H Elmlund . Simple: software for ab initio reconstruction of heterogeneous single-particles. Journal of Structural Biology, 2012, 180( 3): 420– 427
5	S HW Schrers . Relion: implementation of a bayesian approach to cryo-em structure determination. Journal of Structural Biology, 2012, 180( 3): 519– 530
6	A Punjani , J L Rubinstein , D J Fleet , M A Brubaker . Cryosparc: algorithms for rapid unsupervised cryo-EM structure determination. Nature Methods, 2017, 14( 3): 290–
7	M Hu , H Yu , K Gu , Z Wang , H Ruan . A particle-filter framework for robust cryo-em 3d reconstruction. Nature Methods, 2018, 15( 12): 1083–
8	M Khoshouer , M Radjainia , W Baumeister , R Danev . Cryo-em structure of haemoglobin at 3.2 å determined with the volta phase plate. Nature Communications, 2017, 8 : 16099–
9	C Paulino , V Kalienkova , A KM Lam , Y Neldner , R Dutzler . Activation mechanism of the calciumactivated chloride channel tmem16a revealed by cryo-EM. Nature, 2017, 552( 7685): 421–
10	X Bai , C Yan , G Yang , P Lu , D Ma . An atomic structure of human γ-secretase. Nature, 2015, 525( 7568): 212–
11	R Fernandez-Leiro , S HW Scheres . A pipeline approach to single-particle processing in relion. Acta Crystallographica Section D: Structural Biology, 2017, 73( 6): 496– 502
12	Su H, Wen W, Du X, Lu X, Liao M, et al. Gerelion: Gpu-enhanced parallel implementation of single particle cryo-EM image processing. bioRxiv, 2016, 075887
13	D Kimanius , B O Forsberg , S HW Scheres , E Lindahl . Accelerated cryo-EM structure determination with parallelisation using GPUs in relion-2. Elife, 2016, 5 : e18722–
14	You X, Yang H, Luan Z, Qian D. Performance analysis and optimization of cyro-em structure determination in relion-2. In: Proceedings of Conference on Advanced Computer Architecture. 2018: 195–209
15	Relion version 2.1 stable, 2017.
16	X Li , N Grigorieff , Y Cheng . GPU-enabled frealign: accelerating single particle 3d reconstruction and refinement in fourier space on graphics processors. Journal of Structural Biology, 2010, 172( 3): 407– 412
17	Wang K, Xu S, Yu H, Fu H, Yang G. GPU-based 3d cryo-EM reconstruction with key-value streams: poster. In: Proceedings of the 24th Symposium on Principles and Practice of Parallel Programming. 2019: 421–422
18	Wang W, Duan B, Tang W, Zhang C, Tang G, Zhang P, Sun N. A coarse-grained stream architecture for cryo-electron microscopy images 3d reconstruction. In: Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays. 2012: 143–152
19	J Zivanov , T Nakane , B O Forsberg , D Kimanius , W JH Hagen , E Lindahle , S HW Scheres . New tools for automated high-resolution cryo-EM structure determination in relion-3. Elife, 2018, 7 : e42166–
20	J G Pipe , P Menon . Sampling density compensation in mri: rationale and an iterative numerical solution. Magnetic Resonance in Medicine, 1999, 41( 1): 179– 186
21	Reinders J. VTune (TM) Performance Analyzer Essentials: Measurement and Tuning Techniques for Software Developers. 1st ed. California: Intel Press, 2004.
22	E Wang, Q Zhang, B Shen, G Zhang, X Lu, Q Wu, Y Wang. High-Performance Computing on the Intel® Xeon PhiTM. 1st ed . New York: Springer, 2014: 167– 188.
23	CUDA Nvidia. Cufft library, 2010.
24	M Frigo , S G Johnson . Fftw user’s manual. Massachusetts Institute of Technology, 1999,
25	Hursey J, Mallove E, Squyres J M, Lumsdaine A. An extensible framework for distributed testing of mpi implementations. In: Proceedings of Euro PVM/MPI. 2007.
26	S Williams , A Waterman , D Patterson . Roofline: an insightful visual performance model for multicore architectures. Communications of the ACM, 2009, 52( 4): 65– 76
27	NVIDIA. Nvidia tesla v100 performance, 2019.
28	Intel. Intel® xeon® gold 6148 processor, 2019.
29	Sodani A. Knights landing (knl): 2nd generation intel® xeon phi processor. In: Proceedings of 2015 IEEE Hot Chips 27 Symposium (HCS). 2015: 1–24
30	David H, Gorbatoy E, Hanebutte U R, Khanna R, Le C. Rapl: memory power estimation and capping. In: Proceedings of 2010 ACM/IEEE International Symposium on Low-Power Electronics and Design (ISLPED). 2010: 189–194

[1]

Highlights

Download

Viewed

Full text

Abstract

Cited

Shared

Discussed