{rfName}
A

Indexed in

License and use

Icono OpenAccess

Citations

3

Altmetrics

Article has an altmetric score of 10

Analysis of institutional authors

Falces Marin JCorresponding Author

Share

Publications
>
Article

A reinforcement learning approach to improve the performance of the Avellaneda-Stoikov market-making algorithm

Publicated to:Plos One. 17 (12): e0277042- - 2022-01-01 17(12), DOI: 10.1371/journal.pone.0277042

Authors: Marin, JF; de Vera, DDP; Gonzalo, EL

Affiliations

Escuela Técnica Superior de Ingenieros de Telecomunicació n, SSR, Universidad Politécnica de Madrid, Madrid, Spain - Author
Escuela Técnica Superior de Ingenieros de Telecomunicación, SSR, Universidad Politécnica de MadridMadrid, Spain - Author
Univ Politecn Madrid, SSR, Escuela Tecn Super Ingenieros Telecomunicac, Madrid, Spain - Author

Abstract

Market making is a high-frequency trading problem for which solutions based on reinforcement learning (RL) are being explored increasingly. This paper presents an approach to market making using deep reinforcement learning, with the novelty that, rather than to set the bid and ask prices directly, the neural network output is used to tweak the risk aversion parameter and the output of the Avellaneda-Stoikov procedure to obtain bid and ask prices that minimise inventory risk. Two further contributions are, first, that the initial parameters for the Avellaneda-Stoikov equations are optimised with a genetic algorithm, which parameters are also used to create a baseline Avellaneda-Stoikov agent (Gen-AS); and second, that state-defining features forming the RL agent's neural network input are selected based on their relative importance by means of a random forest. Two variants of the deep RL model (Alpha-AS-1 and Alpha-AS-2) were backtested on real data (L2 tick data from 30 days of bitcoin-dollar pair trading) alongside the Gen-AS model and two other baselines. The performance of the five models was recorded through four indicators (the Sharpe, Sortino and P&L-to-MAP ratios, and the maximum drawdown). Gen-AS outperformed the two other baseline models on all indicators, and in turn the two Alpha-AS models substantially outperformed Gen-AS on Sharpe, Sortino and P&L-to-MAP. Localised excessive risk-taking by the Alpha-AS models, as reflected in a few heavy dropdowns, is a source of concern for which possible solutions are discussed. Copyright: © 2022 Falces Marin et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Keywords

gameAlgorithmAlgorithmsArticleArtificial neural networkControlled studyDeepGenetic algorithmHigh risk behaviorLearningNeural networks, computerNonhumanPriceRandom forestReinforcement learning (machine learning)Reinforcement, psychologyRisk aversionTick

Quality index

Bibliometric impact. Analysis of the contribution and dissemination channel

The work has been published in the journal Plos One due to its progression and the good impact it has achieved in recent years, according to the agency Scopus (SJR), it has become a reference in its field. In the year of publication of the work, 2022, it was in position , thus managing to position itself as a Q1 (Primer Cuartil), in the category Multidisciplinary.

Independientemente del impacto esperado determinado por el canal de difusión, es importante destacar el impacto real observado de la propia aportación.

Según las diferentes agencias de indexación, el número de citas acumuladas por esta publicación hasta la fecha 2025-06-09:

  • Scopus: 3

Impact and social visibility

From the perspective of influence or social adoption, and based on metrics associated with mentions and interactions provided by agencies specializing in calculating the so-called "Alternative or Social Metrics," we can highlight as of 2025-06-09:

  • The use, from an academic perspective evidenced by the Altmetric agency indicator referring to aggregations made by the personal bibliographic manager Mendeley, gives us a total of: 14.
  • The use of this contribution in bookmarks, code forks, additions to favorite lists for recurrent reading, as well as general views, indicates that someone is using the publication as a basis for their current work. This may be a notable indicator of future more formal and academic citations. This claim is supported by the result of the "Capture" indicator, which yields a total of: 14 (PlumX).

With a more dissemination-oriented intent and targeting more general audiences, we can observe other more global scores such as:

  • The Total Score from Altmetric: 9.65.
  • The number of mentions on the social network X (formerly Twitter): 14 (Altmetric).

It is essential to present evidence supporting full alignment with institutional principles and guidelines on Open Science and the Conservation and Dissemination of Intellectual Heritage. A clear example of this is:

  • The work has been submitted to a journal whose editorial policy allows open Open Access publication.

Leadership analysis of institutional authors

There is a significant leadership presence as some of the institution’s authors appear as the first or last signer, detailed as follows: First Author (FALCES MARIN, FRANCISCO JAVIER) and Last Author (López Gonzalo E).

the author responsible for correspondence tasks has been FALCES MARIN, FRANCISCO JAVIER.