Similar articles

Abstract: The development of large language models (LLMs) has created transformative opportunities for the financial industry, especially in the area of financial trading. However, how to integrate LLMs with trading systems has become a challenge. To address this problem, we propose an intelligent trade order recognition pipeline that enables the conversion of trade orders into a standard format for trade execution. The system improves the ability of human traders to interact with trading platforms while addressing the problem of misinformation acquisition in trade execution. In addition, we create a trade order dataset of 500 pieces of data to simulate the real-world trading scenarios. Moreover, we design several metrics to provide a comprehensive assessment of dataset reliability and the generative power of big models in finance by using five state-of-the-art LLMs on our dataset. The results show that most models generate syntactically valid JavaScript object notation (JSON) at high rates (about 80%–99%) and initiate clarifying questions in nearly all incomplete cases (about 90%–100%). However, end-to-end accuracy remains low (about 6%–14%), and missing information is substantial (about 12%–66%). Models also tend to over-interrogate—roughly 70%–80% of follow-ups are unnecessary—raising interaction costs and potential information-exposure risk. The research also demonstrates the feasibility of integrating our pipeline with the real-world trading systems, paving the way for practical deployment of LLM-based trade automation solutions.

大语言模型能有效处理和执行金融交易指令吗？

[1]Aghion P, Bergeaud A, Boppart T, et al., 2018. Firm dynamics and growth measurement in France. J Eur Econ Assoc, 16(4):933-956.

[2]Agrawal A, Kedia N, Panwar A, et al., 2024. Taming throughput-latency tradeoff in LLM inference with Sarathi-serve. Proc 18^th USENIX Symp on Operating Systems Design and Implementation, p.117-134.

[3]Bollen J, Mao HN, Zeng XJ, 2011. Twitter mood predicts the stock market. J Comput Sci, 2(1):1-8.

[4]Cavalcante RC, Brasileiro RC, Souza VLF, et al., 2016. Computational intelligence and financial markets: a survey and future directions. Expert Syst Appl, 55:194-211.

[5]Chen YS, Cheng CH, Tsai WL, 2014. Modeling fitting-function-based fuzzy time series patterns for evolving stock index forecasting. Appl Intell, 41(2):327-347.

[6]Day MY, Lee CC, 2016. Deep learning for financial sentiment analysis on finance news providers. Proc IEEE/ACM Int Conf on Advances in Social Networks Analysis and Mining, p.1127-1134.

[7]DeepSeek-AI, 2025. DeepSeek-V3 technical report. https://arxiv.org/abs/2412.19437

[8]di Persio L, Honchar O, 2017. Recurrent neural networks approach to the financial forecast of Google assets. Int J Math Comput Simul, 11:7-13.

[9]Dolphin R, Dursun J, Chow J, et al., 2024. Extracting structured insights from financial news: an augmented LLM driven approach. https://arxiv.org/abs/2407.15788

[10]Fischer T, Krauss C, 2018. Deep learning with long short-term memory networks for financial market predictions. Eur J Oper Res, 270(2):654-669.

[11]Fons E, Dawson P, Zeng XJ, et al., 2020. Evaluating data augmentation for financial time series classification. https://arxiv.org/abs/2010.15111

[12]Garimella A, Chiticariu L, Li YY, 2021. Domain-aware dependency parsing for questions. Proc Findings of the Association for Computational Linguistics, p.4562-4568.

[13]Han JG, Barman U, Hayes J, et al., 2018. NextGen AML: distributed deep learning based language technologies to augment anti money laundering investigation. Proc ACL System Demonstrations, p.37-42.

[14]Huang JM, Xiao MX, Li D, et al., 2025. Open-FinLLMs: open multimodal large language models for financial applications. https://arxiv.org/abs/2408.11878

[15]Kim GI, Chung K, 2024. Extraction of features for time series classification using noise injection. Sensors, 24(19):6402.

[16]Kim M, Yoo S, Park S, 2025. A rule-based stock trading recommendation system using sentiment analysis and technical indicators. Electronics, 14(4):773.

[17]Kognole S, 2024. Fix protocol: the backbone of financial trading. Int J Comput Sci Inf Technol, 16(4):97-114.

[18]Li BL, Jiang YK, Gadepally V, et al., 2024. LLM inference serving: survey of recent advances and opportunities. Proc IEEE High Performance Extreme Computing Conf, p.1-8.

[19]Li CL, Shi Y, Luo YY, et al., 2025. Will LLMs be professional at fund investment? DeepFund: a live arena perspective. https://arxiv.org/abs/2503.18313

[20]Li JT, Bian YX, Wang GX, et al., 2023. CFGPT: Chinese financial assistant with large language model. https://arxiv.org/abs/2309.10654

[21]Li YH, Wang SF, Ding H, et al., 2023. Large language models in finance: a survey. Proc 4^th ACM Int Conf on AI in Finance, p.374-382.

[22]Li ZN, Huang YP, Li ZY, et al., 2023. Neuro-symbolic learning yielding logical constraints. Proc 37^th Int Conf on Neural Information Processing Systems, Article 946.

[23]Long L, Wang R, Xiao RX, et al., 2024. On LLMs-driven synthetic data generation, curation, and evaluation: a survey. Proc Findings of the Association for Computational Linguistics, p.11065-11082.

[24]Malo P, Sinha A, Korhonen P, et al., 2014. Good debt or bad debt: detecting semantic orientations in economic texts. J Assoc Inform Sci Technol, 65(4):782-796.

[25]Min BH, Borch C, 2022. Systemic failures and organizational risk management in algorithmic trading: normal accidents and high reliability in financial markets. Soc Stud Sci, 52(2):277-302.

[26]Nourbakhsh A, Bang G, 2019. A framework for anomaly detection using language modeling, and its applications to finance. https://arxiv.org/abs/1908.09156

[27]OpenAI, 2023. GPT-4 technical report. https://arxiv.org/abs/2303.08774

[28]OpenAI, 2024a. GPT-4o mini: Advancing Cost-Effective Intelligence. https://openi.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence [Accessed on Sept. 1, 2025>].

[29]OpenAI, 2024b. Hello GPT-4o. https://openai.com/index/hello-gpt-4o [Accessed on Sept. 1, 2025].

[30]Team Qwen, 2024. Introducing Qwen1.5. https://qwenlm.github.io/blog/qwen1.5 [Accessed on Sept. 1, 2025].

[31]Sanh V, Webson A, Raffel C, et al., 2022. Multitask prompted training enables zero-shot task generalization. Proc Int Conf on Learning Representations.

[32]Sawhney R, Wadhwa A, Agarwal S, et al., 2021. Quantitative day trading from natural language using reinforcement learning. Proc Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p.4018-4030. https://aclanthology.org/2021.naacl-main.316

[33]Schuster M, Paliwal KK, 1997. Bidirectional recurrent neural networks. IEEE Trans Signal Process, 45(11):2673-2681.

[34]Sinha A, Khandait T, 2021. Impact of news on the commodity market: dataset and results. Proc Future of Information and Communication Conf on Advances in Information and Communication, p.589-601.

[35]Sinha A, Kedas S, Kumar R, et al., 2022. SEntFiN 1.0: entity-aware sentiment analysis for financial news. J Assoc Inform Sci Technol, 73(9):1314-1335.

[36]Sinha A, Agarwal C, Malo P, 2025. FinBloom: knowledge grounding large language model with real-time financial data. https://arxiv.org/abs/2502.18471

[37]Sohangir S, Wang DD, Pomeranets A, et al., 2018. Big data: deep learning for financial sentiment analysis. J Big Data, 5(1):3.

[38]Tatarinov N, Sukhani S, Shah A, et al., 2025. Language modeling for the future of finance: a survey into metrics, tasks, and data opportunities.

[39]Vaswani A, Shazeer N, Parmar N, et al., 2017. Attention is all you need. Proc 31^st Int Conf on Neural Information Processing Systems, p.6000-6010.

[40]Wang K, Zhu JH, Ren MJ, et al., 2024. A survey on data synthesis and augmentation for large language models.

[41]Wei L, Wang ZY, Xu J, et al., 2023. A lightweight sentiment analysis framework for a micro-intelligent terminal. Sensors, 23(2):741.

[42]Xiao YJ, Sun E, Luo D, et al., 2024. TradingAgents: multi-agents LLM financial trading framework.

[43]Xie Q, Han W, Chen Z, et al., 2024. FinBen: a holistic financial benchmark for large language models.

[44]Yang Y, Uy MCS, Huang A, 2020. FinBERT: a pretrained language model for financial communications. https://arxiv.org/abs/2006.08097

[45]Yao MH, 2025. Robustness of big language modeling in finance. ITM Web Conf, 70:02003.

[46]Young A, Chen B, Li C, et al., 2025. Yi: open foundation models by 01.AI. https://arxiv.org/abs/2403.04652

[47]Yu W, Hamam H, 2024. Architecting an enterprise financial management model: leveraging multi-head attention mechanism-Transformer for user information transformation. Peer J Comput Sci, 10:e1928.

[48]Zhang Y, Jin R, Zhou ZH, 2010. Understanding bag-of-words model: a statistical framework. Int J Mach Learn Cyber, 1(1):43-52.

[49]Zhou ZY, Mehra R, 2025. An end-to-end LLM enhanced trading system.