Please wait a minute...
Shopping Cart Email List Chinese
Advanced Search Fig/Tab
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Featured Articles  |  Most Downloaded  |  Most Reading
Online Submission Subscribe to Our RSS Content Feed
Front. Comput. Sci.    2024, Vol. 18 Issue (5) : 185350    https://doi.org/10.1007/s11704-024-31018-5
Artificial Intelligence
RA-CFGPT: Chinese financial assistant with retrieval-augmented large language model
Jiangtong LI1, Yang LEI1, Yuxuan BIAN1, Dawei CHENG1,2, Zhijun DING1,2, Changjun JIANG1,2()
1. Department of Computer Science and Technology, Tongji University, Shanghai 201804, China
2. Shanghai Artificial Intelligence Laboratory, Shanghai 200030, China
 Download: PDF(1489 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Corresponding Author(s): Changjun JIANG   
Just Accepted Date: 12 April 2024   Issue Date: 31 May 2024
 Cite this article:   
Jiangtong LI,Yang LEI,Yuxuan BIAN, et al. RA-CFGPT: Chinese financial assistant with retrieval-augmented large language model[J]. Front. Comput. Sci., 2024, 18(5): 185350.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-024-31018-5
https://academic.hep.com.cn/fcs/EN/Y2024/V18/I5/185350
Fig.1  The component of hybrid financial knowledge base
Fig.2  The overall framework of RA-CFGPT system
Model Size Company Product R.Avg Sector Event Sentiment C.Avg Summary Risk Suggestion A.Avg Avg
Human ? 0.931 0.744 0.838 0.975 0.939 0.912 0.942 1.000 1.000 1.000 1.000 0.927
ChatGPT 175B 0.797 0.198 0.498 0.453 0.458 0.425 0.455 0.593 0.541 0.771 0.635 0.529
ERNIE-Bot-4 ? 0.819 0.417 0.618 0.418 0.358 0.375 0.384 0.721 0.629 0.718 0.689 0.564
Qwen-Chat-7B 7B 0.763 0.360 0.562 0.400 0.367 0.265 0.344 0.548 0.307 0.379 0.411 0.439
ChatGLM2-6B 6B 0.747 0.313 0.530 0.285 0.300 0.357 0.314 0.657 0.454 0.671 0.594 0.479
Baichuan2-7B-Chat 7B 0.757 0.402 0.579 0.425 0.475 0.323 0.408 0.725 0.648 0.732 0.702 0.563
DISC-FinLLM 13B 0.801 0.357 0.579 0.481 0.512 0.482 0.492 0.728 0.611 0.702 0.680 0.583
CFGPT-stf-LoRA 7B 0.820 0.414 0.617 0.569 0.729 0.769 0.689 0.745 0.584 0.609 0.646 0.650
CFGPT-sft-Full 7B 0.836 0.476 0.656 0.700 0.808 0.829 0.779 0.798 0.669 0.808 0.758 0.731
RA-CFGPT-LoRA 7B 0.828 0.421 0.624 0.602 0.763 0.801 0.722 0.762 0.608 0.693 0.688 0.678
RA-CFGPT-Full 7B 0.853 0.492 0.672 0.731 0.841 0.851 0.808 0.821 0.692 0.829 0.781 0.754
RA-CFGPT-Full+Sys 7B ? ? ? ? ? ? ? 0.838 0.721 0.841 0.800 ?
Tab.1  The results on CFBenchmark. The experiment results of baseline methods are from CFBenchmark [10]. R.avg, C.avg, and A.avg are the average results of entity recognition, text classification, and content summarize task. Besides, “Company”, “Product”, “Sector”, “Event”, “Sentiment”, “Summary”, “Risk”, and “Sugguestion” indicate the evaluation accuracy on company recognition, product recognition, sector classification, event detection, sentiment analysis, content summary, investment suggestion, and risk alert, respectively. RA-CFGPT-Full + Sys indicates that we equipped the RA-CFGPT-Full model with three checking modules. Best results are highlighted in boldface
1 K, Lee M W, Chang K Toutanova . Latent retrieval for weakly supervised open domain question answering. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, 6086−6096
2 G, Izacard P, Lewis M, Lomeli L, Hosseini F, Petroni T, Schick J, Dwivedi-Yu A, Joulin S, Riedel E Grave . Atlas: few-shot learningwith retrieval augmented language models. Journal of MachineLearning Research, 2023, 24(251): 1−43
3 H, Yang X Y, Liu C D Wang . FinGPT: open-source financial large language models. 2023, arXiv preprint arXiv: 2306.06031
4 X, Zhang Q Yang . XuanYuan 2.0: a large Chinese financial chat model with hundreds of billions parameters. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023, 4435−4439
5 W, Chen Q, Wang Z, Long X, Zhang Z, Lu B, Li S, Wang J, Xu X, Bai X, Huang Z Wei . DISC-FinLLM: a Chinese financial large language model based on multiple experts fine-tuning. 2023, arXiv preprint arXiv: 2310.15205
6 Xiao S, Liu Z, Zhang P, Muennighoff N. C-pack: packaged resources to advance general Chinese embedding. 2023, arXiv preprint arXiv: 2309.07597
7 Johnson J, Douze M, Jégou H. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 2021, 7(3): 535−547
8 J, Li Y, Bian G, Wang Y, Lei D, Cheng Z, Ding C Jiang . CFGPT: Chinese financial assistant with large language model. 2023, arXiv preprint arXiv: 2309.10654
9 T, Sun X, Zhang Z, He P, Li Q, Cheng H, Yan X, Liu Y, Shao Q, Tang X, Zhao K, Chen Y, Zheng Z, Zhou R, Li J, Zhan Y, Zhou L, Li X, Yang L, Wu Z, Yin X, Huang X Qiu . Moss: training conversational language models from synthetic data. 2023, arXiv preprint arXiv: 2307.15020
10 Y, Lei J, Li M, Jiang J, Hu D, Cheng Z, Ding C Jiang . CFBenchmark: Chinese financial assistant benchmark for large language model. 2023, arXiv preprint arXiv: 2311.05812
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed