The difference between backtesting performance and performance on a real account is a major problem in algorithmic trading. All automated trading systems are concerned without exception. In this post, I will try to give you some explanations that could help you to understand why sometimes there are differences between a backtest and a real account.
Performance differences root causes
Three main causes could explain why a backtest result is always different from the real trading results. These causes are spread, tick-by-tick mode, and slippage. But the problem may come from the latency of your algorithm or some parameters.
The three causes explaining the differences between a backtest and a real account
The three main causes explaining why the backtest behavior is different than a trading strategy running on a real account are the spread, the tick-by-tick mode, and the slippage. I will explain to you how it is possible more precisely in the following paragraphs.
1. The spread
Spread variation by hour
Before you launch a trading strategy from the Probacktest interface, you can choose the spread applied to each trade realized by your automated trading system. This spread will be flat during your backtest, whereas the spread is varying on the real market.
For example, on the DAX, the spread had a value of 2 points from 08h00 to 09h00, 1.2 points from 09h00 to 17h30, 2 points until 22h00, and 5 points all night.
Spread and fees
The following table gives an overview of the IG Market spread :
|Indexes||Value of one contract by standard point / Mini||Trading hours||Spread|
|FTSE 100 24 hours||10£ / 2£||02.00-08.00 |
|Wall Street 24 hours||10$ / 2$||9.00-15.30 |
All other times
|US 500 24 heures||250$ / 50$ /1€||15.30-22.00 |
All other times
|German 30 24 hours||25€ / 5€||01.15-08.00 |
|France 40 24 hours||10€ / 1€||01.15-08.00 |
|Hong Kong HS50 24 hours||50HKD / 10HKD||02.15-05.00 |
All other times
|Japan 225 24 hours||5$ / 1$||23.00-00.30 |
All other times
Events causing increasing spread
You need to know that the spread could significantly increase in case an unexpected event did happen. Consequences are the performance of your automated trading system could suffer a negative impact.
Overnight taxes and fees
If you keep your opening positions at night, you absolutely must integrate overnight fees in your backtest result.
2. The tick by tick mode
Before launching a backtest, you are able to activate the tick-by-tick mode.
This option allows your backtest to integrate each ticker inside each candle. When the tick-by-tick mode is active, the result of your backtest will reflect a better reality.
The problem with tick by tick mode is that is not possible to get more than 1 million ticks in the history of Prorealtime. Cause available space and computing time constraints.
Consequently, if you wish to run a backtest on several trimesters or several years, you will have to inactive tick by tick mode. In this case, there is a risk to obtain a backtest result wish would be significantly different than reality result.
In practical terms, if the tick-by-tick mode is inactivated, the automated trading system that you test will see only the fours follow data describing a candle, which means the opening price, the closing price, the highest price, and the lowest price.
The problem is the following: if inside a candle the STOP or the TARGET could have been touched, it’s impossible to know if the TARGET would have been touched before the STOP or the reverse. In this case, Probacktest will arbitrarily choose to execute the TARGET first.
You should keep in mind that this problem exists for all technical indicators like RSI and others.
3. The slippage
Slippage is a deviation between the order price and execution price of this order.
There are several reasons which explain why slippage occurs :
- Slippage may originate from a too long time of execution. That provokes a price gap because markets are every-time changing.
- Slippage could also be a lack of liquidity consequence at a given moment.
Let us the following example
- DAX index actual price is 11705.5 points
- There are two contracts at 11705.5 and one contract at 11705.6
If a trader buys three contracts at the market then DAX is at 11705.5, he will obtain two contracts at 11705.5 and one at 11705.6. Thus it will be a 0.1 point slippage on one contract.
There are often many slippages when markets are very volatile when unexpected news happens. But sometimes markets can have an episode of volatility caused only by technical reasons.
Can slippage occur with a LIMIT order ?
Yes, slippage can occur with a LIMIT order because when the market touches the price LIMIT, this order transformed into MARKER order just before execution.
Other causes explaining why a Prorealtime backtest is different than the reality
Latency time of your trading algorithm
The impact of the latency
If your algorithm is too complex, you take the risk of increasing the time of execution of your automated trading system. I often noted that the latency of strategy was between 1 and 4 seconds. It is maybe possible that the technical problems explain latency as an overloaded network. However, technical incidents are infrequent on the Prorealtime platform.
The live cycle of the variables
In a Prorealtime program, the code-lines are run from the head of the source code until the tail. In addition, this reading of each code-line repeats itself for each candle. If the needed execution time of the program is too important, the next candle may begin before the end of the algorithm. In this case, the value of some variables could be modifier while the algorithm execution and distort the calculation. This problem risks appearing mainly on the day-trading systems because they often run on a one-or two-minute time frame.
The DEFPARAM behavior
Another explanation that could generates differences between a backtest and the execution of a trading algorithm on a real account would be caused by “DEFPARAM” instructions.
The first problem could come from the “PRELOADBARS” parameter. This parameter will load candles before the date of the automated trading system running. It is possible that more candles were loaded during a backtest than while a real execution of the strategy.
In this case, it would be possible that some entries are not found by the system. To fix this problem, you can try to increase the number of candles that your system need to “pre-load”:
DEFPARAM PRELOADBARS = 5000
I recently noted that sometimes, a variation of the PRELOADBARS value change the result of the backtest by emerging and disappearing some entries. I can not explain why today.
FlatBefore and FlatAfter
The second problem could come from “FlatBefore” and “FlatAfter” parameters. These two parameters will cancel a trading order if it is sent before or after the defined time. The FlatBefore parameter will cancel an order sent before the defined time, and the FlatAfter parameter will cancel an order sent after the defined time.
You need to know that the backtest environment and the real running environment are two different and separated environments. Their respective architectures are also different. The backtesting environment will try to simulate a real environment, but that can not be exactly identical. Some parts of the real system are simulated on backtest environment thanks to predefined instructions.
In the case of the “FlatBefore” and “FlatAfter”, in a real environment, if an order is sent, it will be canceled after being sent. Then on a backtesting environment, this mechanism is simulated and, this order is never sent thus never canceled.
This is important because if you count and limit the maximum number of entries by day in your source code, a canceled entry by FlatBefore or FlatAfter parameter does not count. In contrast, a backtest will be counted in a real environment. This difference could provoke a different behavior between a backtest and real execution of your automated trading system.
To solve this problem, you can try to define a time condition to avoid your system sending an order before the FlatAfter time and after the FlatAfter time while a backtest.
The value changes while the life of a candle
It may be possible that the difference of the result between a backtest and a real account comes from the value change during the life of a candle. Sometimes, the value of a condition can be in an ambiguous state. That is to say, in the same candle, a condition can be true, after false, after true, etc.
For example, if you integrate a linear regression slope in your algorithm, sooner or later, the value of the slope will be near 0. In the same candle, this value could become a few above 0 and a few below 0. If you use the linear regression slope to determine the veracity of a decision-making condition, the value of this condition will change in the same candle.
The above picture shows that the polynomial regression slope becomes negative and positive on the same last candle. It could be possible that in a backtest environment, when a condition becomes true while the life of a candle, this value stays true. At this time, your automated trading system will open an entry during the backtest but not on your real account.
Missing data in the quotation historical
There is another explanation that could be a source of the difference between a backtest and a real account. It is the ticks loading problem in the backtesting environment. The fact is that some ticks could miss while a backtest in the historical data. This problem is known in algorithmic trading, and it is not specific at the Prorealtime platform. The tick-by-tick data represents a considerable quantity of information, and it frequently happens that some ticks miss in the historical data. It is possible there was a network failure while the building of data or while the loading in the backtesting environment.
The quality of the historical data is a major problem in the algorithmic trading industry, and some companies are specialized in the collecting of these data. The price of high-quality market data is often expensive.
The number of loaded candles
I recently noted that the number of loaded candles while a backtest could generate some mistakes in the result. When I launch a backtest using more than 100 000 candles, differences between the backtest result and my real trading account result. To fix this problem, you can reload your backtest using less than 100 000 candles.
100 000 candles backtest
200 000 candles backtest
You can see that the behavior of these two backtests are different between 100k candles and 200k candles then I ran exactly the same strategy.
The rejected orders
It is also possible that the difference between a backtest and a real account comes from rejected order by your broker. For example, the IG broker does not allow that your system places a stop-loss less than 5 points. If your automated trading system tries to place a stop-loss 4 points under the price of the last closed candle, this order will be rejected by the broker.
This behavior does not appears while a backtest. If your trading system automatically calculates the stop-loss without controlling this minimal level, this order will be rejected until the minimal condition of the stop-loss placement would be completed. That is why sometimes, your automated trading system opens an entry with latency compared to the backtest.
On the Prorealtime platform, you can see all the orders using “Ctrl o”:
On the belong screenshot, you can see two rejected orders in the automated trading list. The broker rejects these orders because my trading system has tried to place a stop order less than 5 points under the actual price of the candle.
IG Market spread and fees : https://www.ig.com/en/help-and-support/cfds/fees-and-charges/what-are-igs-indices-cfd-product-details
Download for free the “SEVEN PILLARS TO BUILD A WINNER PROREALTIME BOT”
The “SEVEN PILLARS TO BUILD A WINNER PROREALTIME BOT” is a PDF containing 63 pages allowing you to improve the success of your automated trading system.
If you want to learn more about automatic trading, please visit our automated trading learning section.
If you are any questions, please ask me in a comment and if this article pleased you I would be grateful to see you share it.