Vol. 5 No. 1 (2025): Issue 5
Articles

Deep Reinforcement Learning Stock Trading Strategy Optimization Framework Based on TimesNet and Self-Attention Mechanism

PDF
  • Huitao Zhang,
  • Kaixian Xu,
  • Yunxiang Gan,
  • Shuguang Xiong
Huitao Zhang
Northern Arizona University, Flagstaff, AZ 86011, United States
Kaixian Xu
Corresponding, Risk & Quant Analytics, BlackRock, Jersey City, NJ 07311, United States
Yunxiang Gan
Moloco, CA 94025, United States
Shuguang Xiong
Microsoft Inc., Beijing 100080, China
DOI: https://doi.org/10.71070/oaml.v5i1.70

Published 2025-02-15

Keywords

  • Stock trading strategy,
  • Deep reinforcement learning,
  • TimesNet,
  • Self-attention mechanism,
  • Time series data,
  • Market dynamics,
  • Intelligent decision-making
    • ...More
    Less

How to Cite

Zhang, H., Xu, K., Gan, Y., & Xiong, S. (2025). Deep Reinforcement Learning Stock Trading Strategy Optimization Framework Based on TimesNet and Self-Attention Mechanism. Optimizations in Applied Machine Learning, 5(1). https://doi.org/10.71070/oaml.v5i1.70
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Abstract

In the financial market, the design and optimization of stock trading strategies have become a key focus for investors. With the globalization and digitization of markets, traditional trading strategies often struggle to cope with complex market dynamics, especially in environments characterized by high-frequency volatility and multiple influencing factors. Deep Reinforcement Learning (DRL), as an emerging intelligent algorithm, has shown potential in complex nonlinear markets by learning and optimizing strategies through interactions with the market environment. However, existing DRL models still face challenges in handling long-term dependencies in time series data and market noise. Therefore, this study proposes a deep reinforcement learning framework based on TimesNet and self-attention mechanisms, aiming to overcome the limitations of traditional methods in time series modeling, complex data feature capturing, and strategy optimization. By integrating the multi-scale feature extraction capability of TimesNet with the global dependency capturing advantages of self-attention mechanisms, this research seeks to enhance the intelligence level and trading effectiveness of stock trading strategies, thereby providing investors with more adaptive decision support.

 

PDF

References

  1. N. J. P. D. N. P. James, "Evolutionary correlation, regime switching, spectral dynamics and optimal trading strategies for cryptocurrencies and equities," vol. 434, p. 133262, 2022.
  2. M. Patel, K. Jariwala, and C. J. A. a. S. Chattopadhyay, "An Explainable Deep Relational Approach Towards Indian Stock Market Movement Prediction."
  3. L. J. A. C. S. Cao, "Ai in finance: challenges, techniques, and opportunities," vol. 55, no. 3, pp. 1-38, 2022.
  4. S. Xiong, H. Zhang, M. Wang, and N. Zhou, "Distributed Data Parallel Acceleration-Based Generative Adversarial Network for Fingerprint Generation," Innovations in Applied Engineering and Technology, pp. 1-12, 2022.
  5. T. Kabbani and E. J. I. A. Duman, "Deep reinforcement learning approach for trading automation in the stock market," vol. 10, pp. 93564-93574, 2022.
  6. P. Ren, Z. Zhao, and Q. Yang, "Exploring the Path of Transformation and Development for Study Abroad Consultancy Firms in China," Journal of Computational Methods in Engineering Applications, pp. 1-12, 2023.
  7. Z. R. ZHAO, Ping; TANG, Min, "Analyzing the Impact of Anti-Globalization on the Evolution of Higher Education Internationalization in China.," Journal of Linguistics and Education Research, vol. [S.l.], v. 5, n. 2, p. 15-31,dec. 2022., 2022, doi: https://doi.org/10.30564/jler.v5i2.5552.
  8. N. J. J. o. T. I. Jarunde, "Trading Financial markets-Fundamental Vs Technical analysis: Pros and Cons for differentinvestment styles," vol. 4, no. 3, 2023.
  9. X. Yu, W. Wu, X. Liao, and Y. J. A. I. Han, "Dynamic stock-decision ensemble strategy based on deep reinforcement learning," vol. 53, no. 2, pp. 2452-2470, 2023.
  10. Y. Li, P. Liu, and Z. J. S. P. Wang, "Stock trading strategies based on deep reinforcement learning," vol. 2022, no. 1, p. 4698656, 2022.
  11. S. Xiong, H. Zhang, and M. Wang, "Ensemble Model of Attention Mechanism-Based DCGAN and Autoencoder for Noised OCR Classification," Journal of Electronic & Information Systems, vol. 4, no. 1, pp. 33-41, 2022.
  12. X. Chen and H. Zhang, "Performance Enhancement of AlGaN-based Deep Ultraviolet Light-emitting Diodes with AlxGa1-xN Linear Descending Layers," Innovations in Applied Engineering and Technology, pp. 1-10, 2023.
  13. A. Koshiyama, S. B. Blumberg, N. Firoozye, P. Treleaven, and S. J. Q. F. Flennerhag, "QuantNet: transferring learning across trading strategies," vol. 22, no. 6, pp. 1071-1090, 2022.
  14. S. Xiong, X. Chen, and H. Zhang, "Deep Learning-Based Multifunctional End-to-End Model for Optical Character Classification and Denoising," Journal of Computational Methods in Engineering Applications, pp. 1-13, 2023.
  15. D. Zhu, Y. Gan, and X. Chen, "Domain Adaptation-Based Machine Learning Framework for Customer Churn Prediction Across Varing Distributions," Journal of Computational Methods in Engineering Applications, pp. 1-14, 2021.
  16. C. Li and Y. Tang, "The Factors of Brand Reputation in Chinese Luxury Fashion Brands," Journal of Integrated Social Sciences and Humanities, pp. 1-14, 2023.
  17. K. Alkhatib, H. Khazaleh, H. A. Alkhazaleh, A. R. Alsoud, L. J. J. o. O. I. T. Abualigah, Market,, and Complexity, "A new stock price forecasting method using active deep learning approach," vol. 8, no. 2, p. 96, 2022.
  18. Z. Zhou, J. Wu, Z. Cao, Z. She, J. Ma, and X. Zu, "On-Demand Trajectory Prediction Based on Adaptive Interaction Car Following Model with Decreasing Tolerance," in 2021 International Conference on Computers and Automation (CompAuto), 2021: IEEE, pp. 67-72.
  19. Y. Tang and C. Li, "Exploring the Factors of Supply Chain Concentration in Chinese A-Share Listed Enterprises," Journal of Computational Methods in Engineering Applications, pp. 1-17, 2023.
  20. C. L. Yingda Tang, "Examining the Factors of Corporate Frauds in Chinese A-share Listed Enterprises.," OAJRC Social Science, vol. 2023, 4(3), 63-77., 2023, doi: 10.26855/oajrcss.2023.06.001.
  21. S. Kalva and N. Satuluri, "Stock Market Investment Strategy Using Deep-Q-Learning Network," in International Conference on Multi-disciplinary Trends in Artificial Intelligence, 2023: Springer, pp. 484-495.
  22. D. J. Chiumera, "Deep Reinforcement Learning for Quantitative Finance: Time Series Forecasting using Proximal Policy Optimization," Carleton University, 2022.
  23. S. Demir, B. Stappers, K. Kok, and N. G. J. A. E. Paterakis, "Statistical arbitrage trading on the intraday market using the asynchronous advantage actor–critic method," vol. 314, p. 118912, 2022.
  24. C.-H. Cheng, M.-C. Tsai, and C. J. S. Chang, "A time series model based on deep learning and integrated indicator selection method for forecasting stock prices and evaluating trading profits," vol. 10, no. 6, p. 243, 2022.
  25. D. Xiao and J. J. S. P. Su, "Research on stock price time series prediction based on deep learning and autoregressive integrated moving average," vol. 2022, no. 1, p. 4758698, 2022.
  26. A. Lawi, H. Mesra, and S. J. J. o. B. D. Amir, "Implementation of Long Short-Term Memory and Gated Recurrent Units on grouped time-series data to predict stock prices accurately," vol. 9, no. 1, p. 89, 2022.
  27. V. Dogra, F. S. Alharithi, R. M. Álvarez, A. Singh, and A. M. J. S. Qahtani, "NLP-Based application for analyzing private and public banks stocks reaction to news events in the Indian stock exchange," vol. 10, no. 6, p. 233, 2022.
  28. C. Wang, Y. Chen, S. Zhang, and Q. J. E. S. w. A. Zhang, "Stock market index prediction using deep Transformer model," vol. 208, p. 118128, 2022.
  29. H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. J. a. p. a. Long, "Timesnet: Temporal 2d-variation modeling for general time series analysis," 2022.
  30. C. Zuo, J. Wang, M. Liu, S. Deng, and Q. J. E. Wang, "An ensemble framework for short-term load forecasting based on timesnet and tcn," vol. 16, no. 14, p. 5330, 2023.
  31. H. J. A. a. S. Gobato Souto, "TimesNet for Realized Volatility Prediction," 2023.
  32. Y. Liu et al., "Revolutionising Financial Portfolio Management: The Non-Stationary Transformer’s Fusion of Macroeconomic Indicators and Sentiment Analysis in a Deep Reinforcement Learning Framework," vol. 14, no. 1, p. 274, 2023.
  33. Z. Zhang, Y. Han, B. Ma, M. Liu, Z. J. I. T. o. I. Geng, and Measurement, "Temporal chain network with intuitive attention mechanism for long-term series forecasting," 2023.
  34. Z. Zhang and Q. Sang, "Enhancing Bitcoin Log-return Prediction through TimesNet and Sentiment Analysis Integration," in Proceedings of the 2023 4th International Conference on Machine Learning and Computer Application, 2023, pp. 82-91.
  35. Q. Zhang, C. Qin, Y. Zhang, F. Bao, C. Zhang, and P. J. E. S. w. A. Liu, "Transformer-based attention network for stock movement prediction," vol. 202, p. 117239, 2022.
  36. H. Yang, X.-Y. Liu, S. Zhong, and A. Walid, "Deep reinforcement learning for automated stock trading: An ensemble strategy," in Proceedings of the first ACM international conference on AI in finance, 2020, pp. 1-8.
  37. J. Abramson et al., "Imitating interactive intelligence," 2020.
  38. J. E. Boritz and W. G. J. J. o. I. S. No, "How significant are the differences in financial data provided by key data sources? A comparison of XBRL, Compustat, Yahoo! Finance, and Google Finance," vol. 34, no. 3, pp. 47-75, 2020.
  39. M. M. Akhtar, A. S. Zamani, S. Khan, A. S. A. Shatat, S. Dilshad, and F. J. J. o. K. S. U.-S. Samdani, "Stock market prediction based on statistical data using machine learning algorithms," vol. 34, no. 4, p. 101940, 2022.
  40. A. Z. J. T. U. o. C. B. L. R. Robertson, "The (mis) uses of the S&P 500," vol. 2, no. 1, p. 3, 2023.
  41. B. Fazlija and P. J. M. Harder, "Using financial news sentiment for stock price direction prediction," vol. 10, no. 13, p. 2156, 2022.
  42. L. Wang, F. Ma, J. Liu, and L. J. I. J. o. F. Yang, "Forecasting stock price volatility: New evidence from the GARCH-MIDAS model," vol. 36, no. 2, pp. 684-694, 2020.
  43. Y. Li, Y. J. I. J. o. D. S. Pan, and Analytics, "A novel ensemble deep learning model for stock prediction based on stock prices and news," vol. 13, no. 2, pp. 139-149, 2022.
  44. H. N. Bhandari, B. Rimal, N. R. Pokhrel, R. Rimal, K. R. Dahal, and R. K. J. M. L. w. A. Khatri, "Predicting stock market index using LSTM," vol. 9, p. 100320, 2022.
  45. J. P. Verma, S. Tanwar, S. Garg, I. Gandhi, and N. H. Bachani, "Evaluation of pattern based customized approach for stock market trend prediction with big data and machine learning techniques," in Research Anthology on Machine Learning Techniques, Methods, and Applications: IGI Global, 2022, pp. 1255-1270.
  46. M. Jiang, L. Jia, Z. Chen, and W. J. A. o. O. R. Chen, "The two-stage machine learning ensemble models for stock price prediction by combining mode decomposition, extreme learning machine and improved harmony search algorithm," pp. 1-33, 2022.
  47. D. K. Sharma, H. Hota, K. Brown, R. J. I. J. o. S. A. E. Handa, and Management, "Integration of genetic algorithm with artificial neural network for stock market forecasting," vol. 13, no. Suppl 2, pp. 828-841, 2022.