Why a Prorealtime Backtest gives a different result than a real account?

The difference between backtesting performance and performance on a real account is a significant problem in algorithmic trading. All automated trading systems are concerned without exception. In this post, I will try to give you some explanations that could help you understand why there are sometimes differences between a backtest and a real account.

Performance difference’s root causes

Three leading causes could explain why a backtest result always differs from the real trading results. These causes are spread tick-by-tick mode and slippage. However, the problem may be caused by the latency of your algorithm or some parameters.

The three causes explaining the differences between a backtest and a real account

The three main causes explaining why the backtest behavior differs from a trading strategy running on a real account are the spread, the tick-by-tick mode, and the slippage. I will explain more precisely how it is possible in the following paragraphs.

1. The spread

Spread variation by hour

Before you launch a trading strategy from the Probacktest interface, you can choose the spread applied to each trade your automated trading system realizes. This spread will be flat during your backtest, whereas the spread varies on the real market.

For example, on the DAX, the spread is 2 points from 08h00 to 09h00, 1.2 points from 09h00 to 17h30, 2 points until 22h00, and 5 points all night.

Spread and fees

The following table gives an overview of the IG Market spread:

Events causing the increasing spread

You must know that the spread could significantly increase if an unexpected event happens. The consequence is that the performance of your automated trading system could suffer a negative impact.

Overnight taxes and fees

If you keep your opening positions at night, you must integrate overnight fees into your backtest result.

2. The tick-by-tick mode

Before launching a backtest, you can activate the tick-by-tick mode.

This option allows your backtest to integrate each ticker inside each candle. When the tick-by-tick mode is active, your backtest’s result will reflect a better reality.

The problem with the tick-by-tick mode is that it is impossible to get more than 1 million ticks in the history of Prorealtime because of available space and computing time constraints.

Consequently, if you wish to run a backtest on several trimesters or years, you must inactivate the tick-by-tick mode. In this case, there is a risk of obtaining a backtest result significantly different from the real result.

In practical terms, if the tick-by-tick mode is inactivated, the automated trading system that you test will see only the following data describing a candle: the opening price, the closing price, the highest price, and the lowest price.

The problem is that if the STOP or the TARGET could have been touched inside a candle, it’s impossible to know if the TARGET would have been touched before the STOP or the reverse. In this case, Probacktest will arbitrarily choose to execute the TARGET first.

Remember that this problem exists for all technical indicators like RSI and others.

3. The slippage

Slippage is a deviation between the order price and the execution price of this order.

There are several reasons which explain why slippage occurs :

Slippage could also result from a lack of liquidity at a given moment.

Slippage may originate from a too-long time of execution. That provokes a price gap because markets change all the time.

Let me show the following example

• DAX index’s actual price is 11705.5 points
• There are two contracts at 11705.5 and one contract at 11705.6

If a trader buys three contracts at the market, then DAX is at 11705.5. He will obtain two contracts at 11705.5 and one at 11705.6. Thus, it will be a 0.1-point slippage on one contract.

When markets are very volatile, there are often many slippages when unexpected news happens. But sometimes, markets can have an episode of volatility caused only by technical reasons.

Can slippage occur with a LIMIT order?

Yes, slippage can occur with a LIMIT order because when the market touches the price LIMIT, this order transforms into a MARKER order just before execution.

Other causes explaining why a Prorealtime backtest is different than the reality

Latency time of your trading algorithm

The impact of the latency

If your algorithm is too complex, you risk increasing the time of execution of your automated trading system. I often noted that the strategy’s latency was between 1 and 4 seconds. Technical problems may explain latency as an overloaded network. However, technical incidents are infrequent on the Prorealtime platform.

The live cycle of the variables

In a Prorealtime program, the code lines are run from the head of the source code until the tail. In addition, this reading of each code line repeats itself for each candle. If the program’s execution time is too important, the next candle may begin before the end of the algorithm. In this case, the value of some variables could be modified while the algorithm is being executed and the calculation is being distorted. This problem risks appearing mainly on the day-trading systems because they often run on a one or two-minute time frame.

The DEFPARAM behavior

Another explanation that could generate differences between a backtest and the execution of a trading algorithm on a real account would be caused by “DEFPARAM” instructions.

PRELOADBARS

The first problem could come from the “PRELOADBARS” parameter. This parameter will load candles before the date of the automated trading system running. More candles may have been loaded during a backtest than during a real strategy execution.

In this case, it is possible that the system will not find some entries. To fix this problem, you can try to increase the number of candles that your system needs to “pre-load”:

DEFPARAM PRELOADBARS = 5000

I recently noted that sometimes, a variation of the PRELOADBARS value changes the result of the backtest by causing some entries to emerge and disappear. I can not explain why today.

FlatBefore and FlatAfter

The second problem could come from “FlatBefore” and “FlatAfter” parameters. These two parameters cancel a trading order if sent before or after the defined time. The FlatBefore parameter will cancel an order forwarded before the defined time, and the FlatAfter parameter will cancel an order sent after the defined time.

You need to know that the backtest environment and the real running environment are separate. Their respective architectures are also different. The backtesting environment will try to simulate a real environment, but that can not be precisely identical. Some parts of the real system are simulated on a backtest environment thanks to predefined instructions.

In the case of the “FlatBefore” and “FlatAfter”, in a real environment, if an order is sent, it will be canceled after being sent. Then in a backtesting environment, this mechanism is simulated and, this order is never sent thus never canceled.

This is important because if you count and limit the maximum number of entries by day in your source code, a canceled entry by FlatBefore or FlatAfter parameter does not count. In contrast, a backtest will be counted in a real environment. This difference could provoke a different behavior between a backtest and the real execution of your automated trading system.

To solve this problem, you can try to define a time condition so that your system does not send an order before the FlatBefore time and after the FlatAfter time during a backtest.

The value changes during the life of a candle

It may be possible that the difference in the result between a backtest and a real account comes from the value change during the life of a candle. Sometimes, the value of a condition can be ambiguous. That is to say, in the same candle, a condition can be true, false, true, etc.

For example, if you integrate a linear regression slope in your algorithm, sooner or later, the slope’s value will be near 0. In the same candle, this value could become a few above 0 and a few below 0. If you use the linear regression slope to determine the veracity of a decision-making condition, the value of this condition will change in the same candle.

The above picture shows that the polynomial regression slope becomes negative and positive on the same last candle. It could be possible that in a backtesting environment when a condition becomes true while the life of a candle, this value stays true. Your automated trading system will open an entry during the backtest but not on your real account.

Missing data in the quotation historical

There is another explanation that could be a source of the difference between a backtest and a real account. It is the ticks loading problem in the backtesting environment. The fact is that some ticks could be missed while a backtest in the historical data. This problem is known in algorithmic trading and is not specific to the Prorealtime platform. The tick-by-tick data represents a considerable quantity of information, and it frequently happens that some ticks miss in the historical data. A network failure may occur while building data or loading in the backtesting environment.

The quality of historical data is a significant problem in the algorithmic trading industry, and some companies specialize in collecting this data. High-quality market data is often expensive.

The number of loaded candles

I recently noted that the number of loaded candles, while a backtest, could generate some mistakes in the result. When I launched a backtest using more than 100,000 candles, there were differences between the backtest result and my real trading account result. To fix this problem, you can reload your backtest using less than 100,000 candles.

100 000 candles backtest

200 000 candles backtest

You can see that the behavior of these two backtests is different between 100k candles and 200k candles, so I ran exactly the same strategy.

The rejected orders

It is also possible that the difference between a backtest and a real account comes from a rejected order by your broker. For example, the IG broker does not allow your system to place a stop-loss of less than 5 points. If your automated trading system tries to place a stop-loss 4 points under the price of the last closed candle, the broker will reject this order.

This behavior does not appear during a backtest. If your trading system automatically calculates the stop-loss without controlling this minimal level, this order will be rejected until the minimal condition of the stop-loss placement is completed. That is why sometimes, your automated trading system opens an entry with latency compared to the backtest.

On the Prorealtime platform, you can see all the orders using “Ctrl o”:

In the screenshot, you can see two rejected orders in the automated trading list. The broker rejected these orders because my trading system tried to place a stop order that was less than 5 points under the actual price of the candle.

Sources

IG Market spread and fees: https://www.ig.com/en/help-and-support/cfds/fees-and-charges/what-are-igs-indices-cfd-product-details

Slippage: https://www.investopedia.com/terms/s/slippage.asp

Seizing the best trading opportunities

The Range Breaker will help you seize the best trading opportunities! It detects range breakouts, which are often followed by strong price movements. Thanks to this indicator, you will never miss a breakout again.

Download for FREE

Please visit our automated trading learning section to learn more about automatic trading.

If you have any questions, please ask me in a comment. If this article pleases you, I would be grateful if you shared it.