Parametric models such as the one presented in the paper are better suited to this task, but once developed, they can also aid the training of deep learning engines. “It can provide a realistic benchmark which you can simulate a sufficient number of times to train the neural network before engaging in a model-free journey,” says Barzykin. Random forest is an efficient and accurate classification model, which makes decisions by aggregating a set of trees, either by voting or by averaging class posterior probability estimates.

The new genetic values form the chromosome of the offspring model. The Q-value iteration algorithm assumes that both the transition probability matrix and the reward matrix are known. Hasselt, Guez and Silver developed an algorithm they called double DQN.

The Alpha-AS agent records the new market tick information by modifying the appropriate market features it keeps as part of its state representation. The agent also places one bid and one ask order in response to every tick. Once every 5 seconds, the agent records the asymmetric dampened P&L it has obtained as its reward for placing these WAVES avellaneda stoikov market making bid and ask orders during the latest 5-second time step.

What is the optimal spread?

Based on the estimates of historical VaR and returns for successful/failed actions, we provide a theoretical closed-form solution for the optimal investment proportion. Finally, we demonstrate the significance of this novel system in multiple experiments. What is common to all the above approaches is their reliance on learning agents to place buy and sell orders directly. That is, these agents decide the bid and ask prices of their orderbook quotes at each execution step. The main contribution we present in this paper resides in delegating the quoting to the mathematically optimal Avellaneda-Stoikov procedure.

It works the same as the pure market making strategy’s inventory_skew feature in order to achieve this target. The role of a dealer in securities markets is to provide liquidity on the exchange by quoting bid and ask prices at which he is willing to buy and sell a specific quantity of assets. The performance results for the 30 days of testing of the two Alpha-AS models against the three baseline models are shown in Tables 2–5. All ratios are computed from Close P&L returns (Section 4.1.6), except P&L-to-MAP, for which the open P&L is used.

Avellaneda-Stoikov HFT model implementation

To maximize trade profitability, spreads should be enlarged such that the expected future value of the account is maximized. But for now, it is essential to know that using a significant κ value, you are assuming that the order book is denser, and your optimal spread will have to be smaller since there is more competition on the market. The basic strategy for market making is to create symmetrical bid and ask orders around the market mid-price.

By our numerical results, we deduce that the jump effects and comparative statistics metrics provide us with the information for the traders to gain expected profits. For instance, the model given by has a considerable Sharpe ratio and inventory management with a lower standard deviation comparing to the symmetric strategy. Besides, we further quantify the effects of a variety of parameters in models on the bid and ask spreads and observe that the trader follows different strategies on positive and negative inventory levels, separately. The strategy derived by the model , for instance, illustrates that when time is approaching to the terminal horizon, the optimal spreads converge to a fixed, constant value. Furthermore, in case of the jumps in volatility, it is observed that a higher profit can be obtained but with a larger standard deviation.

The model we will explore is based on a stock price that is generated by Poisson processes with various intensities representing the different jump amounts to employ the adverse selection effects. Reinforcement learning algorithms have been shown to be well-suited for use in high frequency trading contexts [16, 24–26, 37, 45, 46], which require low latency in placing orders together with a dynamic logic that is able to adapt to a rapidly changing environment. In the literature, reinforcement learning approaches to market making typically employ models that act directly on the agent’s order prices, without taking advantage of knowledge we may have of market behaviour or indeed findings in market-making theory. These models, therefore, must learn everything about the problem at hand, and the learning curve is steeper and slower to surmount than if relevant available knowledge were to be leveraged to guide them.

Market making models: from Avellaneda-Stoikov to Gue´ant- Lehalle, and beyond

Market making is a high-frequency trading problem for which solutions based on reinforcement learning are being explored increasingly. Two variants of the deep RL model (Alpha-AS-1 and Alpha-AS-2) were backtested on real data (L2 tick data from 30 days of bitcoin–dollar pair trading) alongside the Gen-AS model and two other baselines. The performance of the five models was recorded through four indicators (the Sharpe, Sortino and P&L-to-MAP ratios, and the maximum drawdown). Gen-AS outperformed the two other baseline models on all indicators, and in turn the two Alpha-AS models substantially outperformed Gen-AS on Sharpe, Sortino and P&L-to-MAP. Localised excessive risk-taking by the Alpha-AS models, as reflected in a few heavy dropdowns, is a source of concern for which possible solutions are discussed. In most of the many applications of RL to trading, the purpose is to create or to clear an asset inventory.

For instance, even after comments about reference formatting, some references have missing publications, years, issues, or even author names . Also, there seems to be a large number of arxiv or SSRN preprints listed for references which are actually published, either as working papers by some institutions or even in peer reviewed journals . Some of these will most likely be handled by the editorial team, but the extent of the errors is too large, evidently due to the revisions made by authors being mostly superficial. In general, the legibility of the paper is hardly improved, and the revisions in this regards were mostly superficial. The reviewer can point in the directions and give some examples but it is simply impossible to list all of the specific details, and it should be on the authors to check the manuscript in detail. The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception .

As we shall see in Section 4.2, the parameters for the direct Avellaneda-Stoikov model to which we compare the Alpha-AS model are fixed at a parameter tuning step once every 5 days of trading data. The two most important features for all three methods are the latest bid and ask quantities in the orderbook , followed closely by the bid and ask quantities immediately prior to the latest orderbook update and the latest best ask and bid prices . There is a general predominance of features corresponding to the latest orderbook movements (i.e., those denominated with low numerals, primarily 0 and 1). This may be a consequence of the markedly stochastic nature of market behaviour, which tends to limit the predictive power of any feature to proximate market movements. Nevertheless, the prices 4 and 8 orderbook movements prior the action setting instant also make fairly a strong appearance in the importance indicator lists , suggesting the existence of slightly longer-term predictive component that may be tapped into profitably. These successes with games have attracted attention from other areas, including finance and algorithmic trading.

• For a single tick, the computation time required for the main procedures is recorded in Table 8.
• A closed-form solution for options with stochastic volatility with applications to bond and currency options.
• We have designed a market making agent that relies on the Avellaneda-Stoikov procedure to minimize inventory risk.
• Therefore, by choosing a Skew value the Alpha-AS agent can shift the output price upwards or downwards by up to 10%.

Topics in stochastic control with applications to algorithmic trading. PhD Thesis, The London School of Economics and Political Sciences. Using the exponential utility function and the results are provided for the following models. In order to recall the models easier, we call the model studied in in Case 1 in Sect. 3 with stock price dynamics as “Model 1” and the model with the dynamics “Model 2”. Increment means that more buy market orders arrived and are filled by sell orders which causes larger spreads.

Optimal high-frequency trading with limit and market orders

Similar to the proof of Proposition2, the optimal spreads can be found by the first order optimality conditions. Is the set of the admissible strategies, F and G are the instantaneous and terminal reward functions, respectively. Are the related depths at which the market maker posts the limit orders.

Therefore the strategy may take longer than 200 seconds to start placing orders. Both the start_time and the end_time parameters are defined to be in the local time of the computer on which the client is running. For the infinite timeframe these two parameters have no effect. Since cryptocurrency markets are open 24/7, there is no “closing time”, and the strategy should also be able run indefinitely, based on an infinite timeframe. Parameter min_spread has a different meaning, parameter risk_factor is being used differently in the calculations and therefore attains a different range of values.

• More advanced models have been developed with adverse selection effects and stronger market order dynamics, see for example the paper of Cartea et al. .
• From the negative values in the Max DD columns, we see that Alpha-AS-1 had a larger Max DD (i.e., performed worse) than Gen-AS on 16 of the 30 test days.
• This helps to keep the models simple and shorten the training time of the neural network in order to test the idea of combining the Avellaneda-Stoikov procedure with reinforcement learning.
• The greater inventory risk taken by the Alpha-AS models during such intervals can be punished with greater losses.
• The optimal bid and ask quotes are obtained from a set of formulas built around these parameters.
• Balancing exploration and exploitation advantageously is a central challenge in RL.

The instant_volatility estimator defines volatility as a deviation of prices from one tick to another in regards to a zero-change price action. Table11 which is obtained from all simulations depicts the results of these two strategies. We can see that when the jumps occur in volatility, it causes not only larger profits but also larger standard deviation of the profit and loss function. This is a small inventory-risk aversion value but is enough to force the inventory process to revert to zero at the end of the trading. With the same assumptions and quadratic utility function as in Case 1 in Sect.

That is because volatility value depends on the market price movement, and it isn’t a factor defined by the market maker. If the market volatility increases, the distance between reservation price and market mid-price will also increase. But this kind of approach, depending on the market situation, might lead to the market maker inventory skewing in one direction, putting the trader in a wrong position as the asset value moves against him. In addition to the programming code, the web site provides tick data samples on selected instruments, well suited for testing the algorithms and for developing new trading models. An innovative feature of the model is the segmentation of clients into tiers, which allows it to capture their response to prices changes more accurately. “In this paper, we offer a way to separate clients into tiers in a purely quantitative way, by analysing their trading flow.

Meanwhile, interprehttps://www.beaxy.com/ results show that IIFI can effectively distinguish between important and redundant features via rating corresponding scores to each feature. As a byproduct of our interpretable methods, the scores over features can be used to further optimize the investment strategy. In this paper, we investigated the high-frequency trading strategies for a market maker using a mean-reverting stochastic volatility models that involve the influence of both arrival and filled market orders of the underlying asset. First, we design a model with variable utilities where the effects of the jumps corresponding to the orders are introduced in returns of the asset and generate optimal bid and ask prices for trading.

We call avellaneda stoikov market making cycles the interval of time where spreads start the widest possible and end up the smallest. Once the cycle is reset, spreads will start again, being the widest possible. This parameter, denoted by the letter gamma, is related to the aggressiveness when setting the spreads to achieve the inventory target.

The large amount of data available in these fields makes it possible to run reliable environment simulations with which to train DRL algorithms. DRL is widely used in the algorithmic trading world, primarily to determine the best action to take in trading by candles, by predicting what the market is going to do. For instance, Lee and Jangmin used Q-learning with two pairs of agents cooperating to predict market trends (through two “signal” agents, one on the buy side and one on the sell side) and determine a trading strategy (through a buy “order” agent and a sell “order” agent). RL has also been used to dose buying and selling optimally, in order to reduce the market impact of high-volume trades which would damage the trader’s returns . The minimum_spread parameter is optional, it has no effect on the calculated reservation price and the optimal spread.

Thus, the Alpha-AS models came 1st and 2nd on 20 out of the 30 test days (67%). The btc-usd data for 7th December 2020 was used to obtain the feature importance values with the MDI, MDA and SFI metrics, to select the most important features to use as input to the Alpha-AS neural network model. The data for the first use of the genetic algorithm was the full day of trading on 8th December 2020. Our algorithm works through 10 generations of instances of the AS model, which we will refer to as individuals, each with a different chromosomal makeup . In the first generation, 45 individuals were created by assigning to each of the four genes random values within the defined ranges.