Why a Prorealtime Backtest gives a different result than a real account?

The difference between backtesting performance and performance on a real account is a significant problem in algorithmic trading. All automated trading systems are concerned without exception. In this post, I will try to give you some explanations that could help you understand why there are sometimes differences between a backtest and a real account.

Performance difference’s root causes

Three leading causes could explain why a backtest result always differs from the real trading results. These causes are spread tick-by-tick mode and slippage. However the problem may come from the latency of your algorithm or some parameters.

The three causes explaining the differences between a backtest and a real account

The three main causes explaining why the backtest behavior differs from a trading strategy running on a real account are the spread, the tick-by-tick mode, and the slippage. I will explain more precisely how it is possible in the following paragraphs.

1. The spread

Spread variation by hour

Spread and fees

The following table gives an overview of the IG Market spread:

Indexes	Value of one contract by standard point / Mini	Trading hours	Spread
FTSE 100 24 hours	10£ / 2£	02.00-08.00 08.00-09.00 9.00-17.30 17.30-22.00 22.00-02.00	3 2 1 2 4
Wall Street 24 hours	10$ / 2$	9.00-15.30 15.30-22.00 22.15-22.30 23.00-00.00 All other times	3,6 2,4 9,8 9,8 4,8
US 500 24 heures	250$ / 50$ /1€	15.30-22.00 22.15-22.30 23.00-00.00 All other times	0,4 1,5 1,5 0,6
German 30 24 hours	25€ / 5€	01.15-08.00 08.00-09.00 09.00-17.30 17.30-22.00 22.00-01.15	4 2 1,2 2 5
France 40 24 hours	10€ / 1€	01.15-08.00 08.00-09.00 09.00-17.30 17.30-22.00 22.00-01.15	3 2 1 2 4
Hong Kong HS50 24 hours	50HKD / 10HKD	02.15-05.00 06.00-09.30 10.15-19.59 All other times	5 5 5 30
Japan 225 24 hours	5$ / 1$	23.00-00.30 00.30-21.15 All other times	15 7 30

Events causing the increasing spread

You must know that the spread could significantly increase if an unexpected event happens. Consequences are the performance of your automated trading system could suffer a negative impact.

Overnight taxes and fees

If you keep your opening positions at night, you must integrate overnight fees into your backtest result.

2. The tick-by-tick mode

The problem with the tick-by-tick mode is that it is impossible to get more than 1 million ticks in the history of Prorealtime because of available space and computing time constraints.

Consequently, if you wish to run a backtest on several trimesters or years, you must inactivate the tick-by-tick mode. In this case, there is a risk of obtaining a backtest result significantly different from the real result.

In practical terms, if the tick-by-tick mode is inactivated, the automated trading system that you test will see only the following data describing a candle: the opening price, the closing price, the highest price, and the lowest price.

The problem is that if the STOP or the TARGET could have been touched inside a candle, it’s impossible to know if the TARGET would have been touched before the STOP or the reverse. In this case, Probacktest will arbitrarily choose to execute the TARGET first.

Remember that this problem exists for all technical indicators like RSI and others.

3. The slippage

Slippage is a deviation between the order price and the execution price of this order.

There are several reasons which explain why slippage occurs :

• Slippage may originate from a too-long time of execution. That provokes a price gap because markets change all the time.
• Slippage could also result from a lack of liquidity at a given moment.

Let us see the following example

• DAX index’s actual price is 11705.5 points
• There are two contracts at 11705.5 and one contract at 11705.6

If a trader buys three contracts at the market, then DAX is at 11705.5. He will obtain two contracts at 11705.5 and one at 11705.6. Thus, it will be a 0.1-point slippage on one contract.

There are often many slippages when markets are very volatile when unexpected news happens. But sometimes, markets can have an episode of volatility caused only by technical reasons.

Can slippage occur with a LIMIT order?

Yes, slippage can occur with a LIMIT order because when the market touches the price LIMIT, this order transforms into a MARKER order just before execution.

Other causes explaining why a Prorealtime backtest is different than the reality

Latency time of your trading algorithm

The impact of the latency

If your algorithm is too complex, you take the risk of increasing the time of execution of your automated trading system. I often noted that the latency of the strategy was between 1 and 4 seconds. It is possible that the technical problems explain latency as an overloaded network. However, technical incidents are infrequent on the Prorealtime platform.

The live cycle of the variables

In a Prorealtime program, the code lines are run from the head of the source code until the tail. In addition, this reading of each code line repeats itself for each candle. If the needed execution time of the program is too important, the next candle may begin before the end of the algorithm. In this case, the value of some variables could be modified while the algorithm execution and distort the calculation. This problem risks appearing mainly on the day-trading systems because they often run on a one-or two-minute time frame.

The DEFPARAM behavior

Another explanation that could generate differences between a backtest and the execution of a trading algorithm on a real account would be caused by “DEFPARAM” instructions.

PRELOADBARS

The first problem could come from the “PRELOADBARS” parameter. This parameter will load candles before the date of the automated trading system running. It is possible that more candles were loaded during a backtest than during a real strategy execution.

In this case, it is possible that the system will not find some entries. To fix this problem, you can try to increase the number of candles that your system needs to “pre-load”:

DEFPARAM PRELOADBARS = 5000

I recently noted that sometimes, a variation of the PRELOADBARS value changes the result of the backtest by causing some entries to emerge and disappear. I can not explain why today.

FlatBefore and FlatAfter

The second problem could come from “FlatBefore” and “FlatAfter” parameters. These two parameters cancel a trading order if sent before or after the defined time. The FlatBefore parameter will cancel an order forwarded before the defined time, and the FlatAfter parameter will cancel an order sent after the defined time.

You need to know that the backtest environment and the real running environment are separate. Their respective architectures are also different. The backtesting environment will try to simulate a real environment, but that can not be precisely identical. Some parts of the real system are simulated on a backtest environment thanks to predefined instructions.

In the case of the “FlatBefore” and “FlatAfter”, in a real environment, if an order is sent, it will be canceled after being sent. Then in a backtesting environment, this mechanism is simulated and, this order is never sent thus never canceled.

This is important because if you count and limit the maximum number of entries by day in your source code, a canceled entry by FlatBefore or FlatAfter parameter does not count. In contrast, a backtest will be counted in a real environment. This difference could provoke a different behavior between a backtest and the real execution of your automated trading system.

To solve this problem, you can try to define a time condition to avoid your system sending an order before the FlatAfter time and after the FlatAfter time while a backtest.

The value changes during the life of a candle

It may be possible that the difference in the result between a backtest and a real account comes from the value change during the life of a candle. Sometimes, the value of a condition can be in an ambiguous state. That is to say, in the same candle, a condition can be true, after false, after true, etc.

For example, if you integrate a linear regression slope in your algorithm, sooner or later, the value of the slope will be near 0. This value could become a few above 0 and a few below 0 in the same candle. If you use the linear regression slope to determine the veracity of a decision-making condition, the value of this condition will change in the same candle.

The above picture shows that the polynomial regression slope becomes negative and positive on the same last candle. It could be possible that in a backtest environment, when a condition becomes true while the life of a candle, this value stays true. Your automated trading system will open an entry during the backtest but not on your real account.

Missing data in the quotation historical

There is another explanation that could be a source of the difference between a backtest and a real account. It is the ticks loading problem in the backtesting environment. The fact is that some ticks could be missed while a backtest in the historical data. This problem is known in algorithmic trading and is not specific to the Prorealtime platform. The tick-by-tick data represents a considerable quantity of information, and it frequently happens that some ticks miss in the historical data. A network failure may occur while building data or loading in the backtesting environment.

The quality of historical data is a significant problem in the algorithmic trading industry, and some companies specialize in collecting this data. High-quality market data is often expensive.

The number of loaded candles

I recently noted that the number of loaded candles, while a backtest, could generate some mistakes in the result. When I launched a backtest using more than 100 000 candles, there were differences between the backtest result and my real trading account result. To fix this problem, you can reload your backtest using less than 100 000 candles.

100 000 candles backtest

200 000 candles backtest

You can see that the behavior of these two backtests is different between 100k candles and 200k candles, so I ran exactly the same strategy.

The rejected orders

It is also possible that the difference between a backtest and a real account comes from a rejected order by your broker. For example, the IG broker does not allow your system to place a stop-loss of less than 5 points. If your automated trading system tries to place a stop-loss 4 points under the price of the last closed candle, the broker will reject this order.

This behavior does not appear during a backtest. If your trading system automatically calculates the stop-loss without controlling this minimal level, this order will be rejected until the minimal condition of the stop-loss placement is completed. That is why sometimes, your automated trading system opens an entry with latency compared to the backtest.

On the Prorealtime platform, you can see all the orders using “Ctrl o”:

In the screenshot, you can see two rejected orders in the automated trading list. The broker rejects these orders because my trading system has tried to place a stop order less than 5 points under the actual price of the candle.

Sources

IG Market spread and fees: https://www.ig.com/en/help-and-support/cfds/fees-and-charges/what-are-igs-indices-cfd-product-details

Slippage: https://www.investopedia.com/terms/s/slippage.asp

---

Please visit our automated trading learning section to learn more about automatic trading.

If you have any questions, please ask me in a comment, and if this article pleases you, I would be grateful to see you share it.

---