Bioinformatics and biomedical informatics with ChatGPT: Year one review
Jinge Wang1, Zien Cheng1, Qiuming Yao2, Li Liu3,4, Dong Xu5, Gangqing Hu1()
. Department of Microbiology, Immunology & Cell Biology, West Virginia University, Morgantown, West Virginia, USA . School of Computing, University of Nebraska‐Lincoln, Lincoln, Nebraska, USA . College of Health Solutions, Arizona State University, Phoenix, Arizona, USA . Biodesign Institute, Arizona State University, Tempe, Arizona, USA . Department of Electrical Engineer and Computer Science, Christopher S. Bond Life Sciences Center, University of Missouri, Columbia, Missouri, USA
The year 2023 marked a significant surge in the exploration of applying large language model chatbots, notably Chat Generative Pre‐trained Transformer (ChatGPT), across various disciplines. We surveyed the application of ChatGPT in bioinformatics and biomedical informatics throughout the year, covering omics, genetics, biomedical text mining, drug discovery, biomedical image understanding, bioinformatics programming, and bioinformatics education. Our survey delineates the current strengths and limitations of this chatbot in bioinformatics and offers insights into potential avenues for future developments.
H Wang, T Fu, Y Du, W Gao, K Huang, Z Liu, et al. Scientific discovery in the age of artificial intelligence. Nature. 2023; 620 (7972): 47- 60.
2
Y Xu, X Liu, X Cao, C Huang, E Liu, S Qian, et al. Artificial intelligence: a powerful paradigm for scientific research. Innovation. 2021; 2 (4): 100179.
3
R Van Noorden, JM Perkel. AI and science: what 1,600 researchers think. Nature. 2023; 621 (7980): 672- 5.
4
S Milano, JA McGrane, S Leonelli. Large language models challenge the future of higher education. Nat Mach Intell. 2023; 5 (4): 333- 4.
5
EAM van Dis, J Bollen, W Zuidema, R van Rooij, CL Bockting. ChatGPT: five priorities for research. Nature. 2023; 614 (7947): 224- 6.
6
P Lee, S Bubeck, J Petro. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. N Engl J Med. 2023; 388 (13): 1233- 9.
7
S Tian, Q Jin, L Yeganova, PT Lai, Q Zhu, X Chen, et al. Opportunities and challenges for ChatGPT and large language models in biomedicine and health. Briefings Bioinf. 2023; 25 (1): bbad493.
8
J Liu, M Yang, Y Yu, H Xu, K Li, and X Zhou. Large language models in bioinformatics: applications and perspectives. 2024. Preprint at arXiv:2401.04155.
9
D Xu, W Chen, W Peng, C Zhang, T Xu, X Zhao, et al. Large language models for generative information extraction: a survey. 2023. Preprint at arXiv:2312.17617.
10
E Shue L Liu, B Li, Z Feng, X Li, G Hu. Empowering beginners in bioinformatics with ChatGPT. Quantitative Biology. 2023; 11 (2): 105- 8.
11
SR Piccolo, P Denny, A Luxton-Reilly, SH Payne, PG Ridge. Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. PLoS Comput Biol. 2023; 19 (9): e1011511.
12
J Zhou, B Zhang, X Chen, H Li, X Xu, S Chen, et al. An AI agent for fully automated multi‐omic analyses. 2024. Preprint at bioRxiv:2023.09.08.556814.
13
G Hu, L Liu, D Xu. On the responsible use of chatbots in bioinformatics. Genom Proteom Bioinform. 2024; 22 (1): qzae002.
14
B Murdoch. Privacy and artificial intelligence: challenges for protecting health information in a new era. BMC Med Ethics. 2021; 22 (1): 122.
15
MR Karim, T Islam, M Shajalal, O Beyan, C Lange, M Cochez, et al. Explainable AI for bioinformatics: methods, tools and applications. Briefings Bioinf. 2023; 24 (5): bbad236.
16
W Hou, Z Ji. Assessing GPT-4 for cell type annotation in single-cell rna-seq analysis. Nat Methods. 2024; 21 (8): 1462- 5.
17
W Hou, Z Ji. Geneturing tests GPT models in genomics. 2023. Preprint at bioRxiv:2023.03.11.532238.
18
Q Jin, Y Yang, Q Chen, Z Lu. GeneGPT: augmenting large language models with domain tools for improved access to biomedical information. Bioinformatics. 2024; 40 (2).
19
P Ahimaz, AL Bergner, ME Florido, N Harkavy, S Bhattacharyya. Genetic counselors’ utilization of ChatGPT in professional practice: a cross‐sectional study. Am J Med Genet. 2024; 194 (4).
20
D Duong, BD Solomon. Analysis of large-language model versus human performance for genetics questions. Eur J Hum Genet. 2024; 32 (4): 466- 8.
21
IF Alkuraya. Is artificial intelligence getting too much credit in medical genetics? Am J Med Genet C. 2023; 193 (3): e32062.
22
F Emmert-Streib. Can ChatGPT understand genetics? Eur J Hum Genet. 2024; 32: 371- 2.
23
Q Chen, H Sun, H Liu, Y Jiang, T Ran, X Jin, et al. An extensive benchmark study on biomedical text generation and mining with ChatGPT. Bioinformatics. 2023; 39 (9): btad557.
24
Y Gu, R Tinn, H Cheng, M Lucas, N Usuyama, X Liu, et al. Domain‐specific language model pretraining for biomedical natural language processing. ACM Trans Comput Healthcare. 2021; 3 (Article 2): 1- 23.
25
Q Chen, J Du, Y Hu, V Kuttichi Keloth, X Peng, K Raja, et al. Large language models in biomedical natural language processing: benchmarks, baselines, and recommendations. 2023. Preprint at arXiv:2305.16326.
26
S Ateia, U Kruschwitz. Is ChatGPT a biomedical expert? ‐‐exploring the zero‐shot performance of current GPT models in biomedical tasks. 2023. Preprint at arXiv:2306.16108.
27
S Chen, Y Li, S Lu, H Van, H Aerts, GK Savova, et al. Evaluating the ChatGPT family of models for biomedical reasoning and classification. J Am Med Inf Assoc. 2024; 31 (4): 940- 8.
28
I Jahan, MTR Laskar, C Peng, JX Huang. A comprehensive evaluation of large language models on benchmark biomedical text processing tasks. Comput Biol Med. 2024; 171: 108189.
29
Y Hou, J Yeung, H Xu, C Su, F Wang, R Zhang. From answers to insights: unveiling the strengths and limitations of ChatGPT and biomedical knowledge graphs. Preprint at Res Sq. 2023: rs.3.rs- 3185632.
30
RF Rizvi, J Vasilakes, TJ Adam, GB Melton, JR Bishop, J Bian, et al. iDISK: the integrated dietary supplements knowledge base. J Am Med Inf Assoc. 2020; 27 (4): 539- 48.
31
Q Zhao, X Zhou, J Wu, J Cai, X Bao, L Tang, et al. Biotreasury: a community-based repository enabling indexing and rating of bioinformatics tools. Sci China Life Sci. 2024; 67 (2): 221- 9.
32
X Wu, Y Zeng, A Das, S Jo, T Zhang, P Patel, et al. ReguloGPT: harnessing GPT for knowledge graph construction of molecular regulatory pathways. 2024. Preprint at bioRxiv:2024.01.27.577521.
33
M Azam, Y Chen, M Arowolo, H Liu, M Popescu, D Xu. A comprehensive evaluation of large language models in mining gene interactions and pathway knowledge. Quantitative Biology. 2024. In press.
34
H Rehana, N Bengisu Çam, M Basmaci, J Zheng, C Jemiyo, Y He, , et al. Evaluation of GPT and BERT‐based models on identifying protein‐protein interactions in biomedical text. 2023. Preprint at arXiv:2303.17728.
35
K Tiwari, L Matthews, B May, V Shamovsky, M Orlic‐Milacic, K Rothfels, et al. ChatGPT usage in the Reactome curation process. 2023. Preprint at bioRxiv:2023.11.08.566195.
36
B Zhou, B Ji, C Shen, X Zhang, X Yu, P Huang, et al. Evlncrnas 3.0: an updated comprehensive database for manually curated functional long non‐coding RNAs validated by low‐throughput experiments. Nucleic Acids Res. 2024; 52 (D1): D98- 106.
37
X Chen, C Li, Z Wang, Y Zhou, M Chu. Computational screening of biomarkers and potential drugs for arthrofibrosis based on combination of sequencing and large nature language model. Journal of Orthopaedic Translation. 2024; 44: 102- 13.
38
K Fo, YS Chuah, H Foo, EE Davey, M Fullwood, G Thibault, et al. Plantconnectome: knowledge networks encompassing >100,000 plant article abstracts. 2023. Preprint at bioRxiv:2023.07.11.548541.
39
V Rawte, A Sheth, A Das. A survey of hallucination in large foundation models. 2023. Preprint at arXiv:2309.05922.
40
Y Zhang, Y Li, L Cui, D Cai, L Liu, T Fu, et al. Siren’s song in the AI ocean: a survey on hallucination in large language models. 2023. Preprint at arXiv:2309.01219.
41
Y Chen, J Gao, M Petruc, RD Hammer, M Popescu, D Xu. Iterative prompt refinement for mining gene relationships from ChatGPT. International Journal of Artificial Intelligence and Robotics Research. 2024. In press.
42
S Yao, D Yu, J Zhao, I Shafran, TL Griffiths, Y Cao, et al. Tree of thoughts: deliberate problem solving with large language models. In: Neural information processing systems 36 (NeurIPS 2023) 35. Curran Associates, Inc; 2023, p. 11809- 22.
43
N Savage. Drug discovery companies are customizing ChatGPT: here’s how. Nat Biotechnol. 2023; 41 (5): 585- 6.
44
C Chakraborty, M Bhattacharya, SS Lee. Artificial intelligence enabled ChatGPT and large language models in drug target discovery, drug discovery, and development. Mol Ther Nucleic Acids. 2023; 33: 866- 8.
45
A Zhao, Y Wu. Future implications of ChatGPT in pharmaceutical industry: drug discovery and development. Front Pharmacol. 2023; 14: 1194216.
46
G Hu, Z Xie. The artificial intelligence pharma era after “chat generative pre-trained transformer”. Medical Review. 2023; 3: 198- 9.
47
Z Gao, L Li, S Ma, Q Wang, L Hemphill, R Xu. Examining the potential of ChatGPT on biomedical information retrieval: fact-checking drug-disease associations. Ann Biomed Eng. 2023.
48
T Guo, K Guo, B Nan, Z Liang, Z Guo, NV Chawla, et al. What can large language models do in chemistry? A comprehensive benchmark on eight tasks. In: Advances in neural information processing systems, 36. Curran Associates, Inc; 2023. p. 59662- 88.
49
A Juhi, N Pipil, S Santra, S Mondal, JK Behera, H Mondal. The capability of ChatGPT in predicting and explaining common drug-drug interactions. Cureus. 2023; 15: e36272.
50
FY Al-Ashwal, M Zawiah, L Gharaibeh, R Abu-Farha, AN Bitar. Evaluating the sensitivity, specificity, and accuracy of ChatGPT-3.5, ChatGPT-4, Bing AI, and bard against conventional drug-drug interactions clinical tools. Drug Healthc Patient Saf. 2023; 15: 137- 47.
51
M Herrero-Zazo, I Segura-Bedmar, P Martinez, T Declerck. The DDI corpus: an annotated corpus with pharmacological substances and drug-drug interactions. J Biomed Inf. 2013; 46 (5): 914- 20.
52
YM Wang, HW Shen, TJ Chen. Performance of ChatGPT on the pharmacist licensing examination in Taiwan. J Chin Med Assoc. 2023; 86 (7): 653- 8.
53
Y Kunitsu. The potential of GPT-4 as a support tool for pharmacists: analytical study using the Japanese national examination for pharmacists. JMIR Medical Education. 2023; 9: e48452.
54
H Zong, J Li, E Wu, R Wu, J Lu, B Shen. Performance of ChatGPT on Chinese national medical licensing examinations: a five-year examination evaluation study for physicians, pharmacists and nurses. BMC Med Educ. 2024; 24 (1): 143.
55
X Huang, D Estau, X Liu, Y Yu, J Qin, Z Li. Evaluating the performance of ChatGPT in clinical pharmacy: a comparative study of ChatGPT and clinical pharmacists. Br J Clin Pharmacol. 2024; 90 (1): 232- 8.
56
R Wang, H Feng, GW Wei. ChatGPT in drug discovery: a case study on anticocaine addiction drug development with chatbots. J Chem Inf Model. 2023; 63 (22): 7189- 209.
57
S Liu, J Wang, Y Yang, C Wang, L Liu, H Guo, et al. Conversational drug editing using retrieval and domain feedback. In: The twelfth international conference on learning representations; 2024.
58
Y Liang, R Zhang, L Zhang, and P Xie. Drugchat: towards enabling ChatGPT‐like capabilities on drug molecule graphs. 2023. Preprint at arXiv:2309.03907.
59
G Ye, X Cai, H Lai, X Wang, J Huang, L Wang, et al. Drugassist: a large language model for molecule optimization. 2023. Preprint at arXiv:2401.10334.
60
Q Dong, L Li, D Dai, C Zheng, Z Wu, B Chang, et al. A survey on in‐context learning. 2022. Preprint at arXiv:2301.00234.
61
J Li, Y Liu, W Fan, X-Y Wei, H Liu, J Tang, et al. Empowering molecule discovery for molecule-caption translation with large language models: a ChatGPT perspective. IEEE transactions on knowledge and data engineering. 2023: 1- 13. In press.
62
KM Jablonka, P Schwaller, A Ortega-Guerrero, B Smit. Leveraging large language models for predictive chemistry. Nat Mach Intell. 2024; 6 (2): 161- 9.
63
X Cai, H Lai, X Wang, L Wang, W Liu, Y Wang, et al. Comprehensive evaluation of molecule property prediction with ChatGPT. Methods. 2024; 222: 133- 41.
64
M Caldas Ramos, SS Michtavy, MD Porosoff, AD White. Bayesian optimization of catalysts with in‐context learning. 2023. Preprint at arXiv:2304.05341.
65
Z Zeng, B Yin, S Wang, J Liu, C Yang, H Yao, et al. Interactive molecular discovery with natural language. 2023. Preprint at arXiv:2306.11976.
66
C Raffel, N Shazeer, A Roberts, K Lee, S Narang, M Matena, et al. Exploring the limits of transfer learning with a unified text‐to‐text transformer. J Mach Learn Res. 2020; 21.
67
H Hu, AJ Yang, S Deng, D Wang, M Song, S Shen. A generative drug-drug interaction triplets extraction framework based on large language models. Proceedings of the Association for Information Science and Technology. 2023; 60 (1): 980- 2.
68
J Wei, M Bosma, VY Zhao, K Guu, AW Yu, B Lester, et al. Fine‐tuned language models are zero‐shot learners. 2021. Preprint at arXiv:2109.01652.
69
Y Fang, X Liang, N Zhang, K Liu, R Huang, Z Chen, et al. Mol‐instructions: a large‐scale biomolecular instruction dataset for large language models. 2023. Preprint at arXiv:2306.08018.
70
Z Zhao, D Ma, L Chen, L Sun, Z Li, H Xu, et al. Chemdfm: dialogue foundation model for chemistry. 2024. Preprint at arXiv:2401.14818.
71
H Cao, Z Liu, X Lu, Y Yao, Y Li. Instructmol: multi‐modal integration for building a versatile and reliable molecular assistant in drug discovery. 2023. Preprint at arXiv:2311.16208.
72
W Zhang, Q Wang, X Kong, J Xiong, S Ni, D Cao, et al. Fine‐tuning large language models for chemical text mining. Chem Sci. 2024. In press.
73
JN Acosta, GJ Falcone, P Rajpurkar, EJ Topol. Multimodal biomedical AI. Nat Med. 2022; 28 (9): 1773- 84.
74
D Truhn, CD Weber, BJ Braun, K Bressem, JN Kather, C Kuhl, et al. A pilot study on the efficacy of GPT-4 in providing orthopedic treatment recommendations from MRI reports. Sci Rep-Uk. 2023; 13 (1): 20159.
75
Z Liu, H Jiang, T Zhong, Z Wu, C Ma, Y Li, et al. Holistic evaluation of GPT‐4v for biomedical imaging. 2023. Preprint at arXiv:2312.05256.
76
C Wu, J Lei, Q Zheng, W Zhao, W Lin, X Zhang, et al. Can GPT‐4v(ision) serve medical applications? Case studies on GPT‐4v for multimodal medical diagnosis. 2023. Preprint at arXiv:2310.09909.
77
Z Yan, K Zhang, R Zhou, L He, X Li, L Sun. Multimodal ChatGPT for medical applications: an experimental study of GPT‐4v. 2023. Preprint at arXiv:2310.19061.
78
T Buckley, JA Diao, A Rodman, AK Manrai. Accuracy of a vision‐language model on challenging medical cases. 2023. Preprint at arXiv:2311.05591.
79
Z Yang, Z Yao, M Tasmin, P Vashisht, WS Jang, F Ouyang, et al. Performance of multimodal GPT‐4v on usmle with image: potential for imaging diagnostic support with explanations. 2023. Preprint at medRxiv:2023.10.26.23297629.
80
Y Li, Y Liu, Z Wang, X Liang, L Liu, L Wang, et al. A comprehensive study of GPT‐4V’s multimodal capabilities in medical imaging. 2023. Preprint at medRxiv:2023.11.03.23298067.
81
W Hou, Z Ji. GPT‐4V exhibits human‐like performance in biomedical image classification. 2024. Preprint at bioRxiv:2023.12.31.573796.
82
J Wang, Q Ye, L Liu, NL Guo, G Hu. Scientific figures interpreted by ChatGPT: strengths in plot recognition and limits in color perception. NPJ Precis Oncol. 2024; 8 (1): 84.
83
OpenAI. GPT‐4V(ision) system card. (2023) 1- 18.
84
RS Nickerson. Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol. 1998; 2: 175- 220.
85
Q Jin, F Chen, Y Zhou, Z Xu, JM Cheung, R Chen, et al. Hidden flaws behind expert‐level accuracy of GPT‐4 vision in medicine. 2024. Preprint at arXiv:2401.08396.
86
J Wang, Z Liu, L Zhao, Z Wu, C Ma, S Yu, et al. Review of large vision models and visual prompt engineering. Meta-Radiology. 2023; 1 (3): 100047.
87
Z Yang, L Li, K Lin, J Wang, C Lin, Z Liu, et al. The dawn of lmms: preliminary explorations with GPT‐4v(ision). 2023. Preprint at arXiv:2309.17421.
88
Z Li, C Wang, C Liu, P Ma, D Wu, S Wang, et al. Vrptest: evaluating visual referring prompting in large multimodal models. 2023. Preprint at arXiv:2312.04087.
89
J Li, H Chen, Y Wang, MM Chen, H Liang. Next-generation analytics for omics data. Cancer Cell. 2021; 39 (1): 3- 6.
90
M Xie, L Yang, G Chen, Y Wang, Z Xie, H Wang. Ribochat: a chat‐style web interface for analysis and annotation of ribosome profiling data. Briefings Bioinf. 2022; 23 (2).
91
C Merow, JM Serra-Diaz, BJ Enquist, AM Wilson. AI chatbots can boost scientific coding. Nat Ecol Evol. 2023; 7: 960- 2.
92
JA Jansen, A Manukyan, NA Khoury, A Akalin. Leveraging large language models for data analysis automation. 2023. Preprint at bioRxiv:2023.12.11.571140.
93
Z Dong, V Zhong, YY Lu. Biomania: simplifying bioinformatics data analysis through conversation. 2023. Preprint at bioRxiv:2023.10.29.564479.
94
A Liu, X Hu, L Wen, PS Yu. A comprehensive evaluation of ChatGPT’s zero‐shot text‐to‐SQL capability. 2023. Preprint at arXiv:2303.13547.
95
A‐C Sima, TM de Farias. On the potential of artificial intelligence chatbots for data exploration of federated bioinformatics knowledge graphs. In: SeWebMeDa’23: 6th workshop on semantic web solutions for large‐scale biomedical data analytics, 3466. CEUR‐WS.org; 2023.
96
JC Rangel, TM de Farias, AC Sima, N Kobayashi. Sparql generation: an analysis on fine‐tuning openllama for question answering over a life science knowledge graph. In: WAT4HCLS 2024: 15th international semantic web applications and tools for health care and life Sciences conference; 2024. In press.
97
C Chen, T Stadler. Genspectrum chat: data exploration in public health using large language models. 2023. Preprint at arXiv: 2305.13821.
98
L Wang, XJ Ge, L Liu, GQ Hu. Code interpreter for bioinformatics: are we there yet? Ann Biomed Eng. 2024; 52 (4): 754- 6.
99
X Tang, B Qian, R Gao, J Chen, X Chen, M Gerstein. Biocoder: a benchmark for bioinformatics code generation with large language models. Bioinformatics. 2023. In press.
100
V Sarwal, V Munteanu, T Suhodolschi, D Ciorba, E Eskin, W Wang, et al. Biollmbench: a comprehensive benchmarking of large language models in bioinformatics. 2023. Preprint at bioRxiv:2023.12.19.572483.
101
T Lubiana, R Lopes, P Medeiros, JC Silva, ANA Goncalves, V Maracaja-Coutinho, et al. Ten quick tips for harnessing the power of ChatGPT in computational biology. PLoS Comput Biol. 2023; 19 (8): e1011319.
102
M Chen, J Tworek, H Jun, Q Yuan, H Ponde de Oliveira Pinto, J Kaplan, et al. Evaluating large language models trained on code. 2021. Preprint at arXiv:2107.03374.
103
T Lehtinen, L Haaranen, J Leinonen. Automated questionnaires about students’ Javascript programs: towards gauging novice programming processes. In: Ace’ 23: proceedings of the 25th australasian computing education conference. Association for Computing Machiner; 2023. p. 49- 58.
104
ZC Lin. How to write effective prompts for large language models. Nat Human Behav. 2024; 8 (4): 611- 5.
105
P Denny, J Leinonen, J Prather, A Luxton‐Reilly, T Amarouche, BA Becker, et al. Promptly: using prompt problems to teach learners how to effectively utilize Ai code generators. 2023. Preprint at arXiv:2307.16364.
106
K Soman, PW Rose, JH Morris, RE Akbas, B Smith, B Peetoom, et al. Biomedical knowledge graph‐enhanced prompt generation for large language models. 2023. Preprint at arXiv:2311.17330.
107
L Chen, M Zaharia, J Zou. How is ChatGPT’s behavior changing over time?. 2023. Preprint at arXiv:2307.09009.
108
G Wang, G Yang, Z Du, L Fan, X Li. ClinicalGPT: large language models fine‐tuned with diverse medical data and comprehensive evaluation. 2023. Preprint at arXiv:2306.09968.
109
C Peng, X Yang, A Chen, KE Smith, N PourNejatian, AB Costa, et al. A study of generative large language model for medical research and healthcare. NPJ Digit Med. 2023; 6 (1): 210.
110
T Lai, Y Shi, Z Du, J Wu, , K Fu, , Y Dou, et al. Psy‐llm: scaling up global mental health psychological services with AI‐based large language models. 2023. Preprint at arXiv:2307.11991.
111
JM Liu, D Li, H Cao, T Ren, Z Liao, J Wu. Chatcounselor: a large language models for mental health support. 2023. Preprint at arXiv:2309.15461.
112
T Han, LC Adams, J‐M Papaioannou, P Grundmann, T Oberhauser, A Löser, et al. Medalpaca ‐‐ an open‐source collection of medical conversational AI models and training data. 2023. Preprint at arXiv:2304.08247.
113
S Zhang, R Fan, Y Liu, S Chen, Q Liu, W Zeng. Applications of transformer-based language models in bioinformatics: a survey. Bioinform Adv. 2023; 3 (1): vbad001.
114
JN Qiu, L Li, JK Sun, JC Peng, PL Shi, RY Zhang, et al. Large AI models in health informatics: applications, challenges, and the future. IEEE J Biomed Health. 2023; 27 (12): 6074- 87.
