A backtest shows how a strategy would have done on past prices. Software can now write and test hundreds of strategies in seconds and show you the one that did best. The trouble is that the best of hundreds is the one most likely to have been lucky.
Two numbers decide how much a backtest’s best result says: how many strategies were tried, and how many years they were tested on. Enter them below with the result claimed, as a Sharpe ratio (the yearly return above cash divided by how much it swings in a year).
Check a backtest against the search behind it
Arithmetic · your numbers
Strategies tried, including the ones dropped
Best of these by luck
1.34
Claimed, above luck
+0.16
A no-edge search does this well
16%
Luck matches it at
1,490 tries
Over 5 years, the best of 450 strategies with no edge at all would show a Sharpe ratio of about 1.34 on average by luck alone; the claimed 1.50 is 0.16 above that. A search that size turns up a result this good or better 16% of the time with no edge anywhere in it, against under 1% for one strategy tried once. Luck's average would match the claim at about 1,490 tries.
The benchmark is the one Bailey and López de Prado (2014, the deflated Sharpe ratio) set a claim against: the best result expected from that many strategies with no edge. The chance shown is exact for independent strategies with roughly normal returns. The strategies are taken as independent and their daily returns as roughly normal; near-copies of one strategy mean luck explains less than shown, while fat tails, trading costs and look-ahead are left out and each makes a claim weaker. The Sharpe ratio is the yearly return above cash divided by how much it swings in a year; a backtest that does not print it, or the number of strategies tried, has left out the two numbers this check needs.
Five questions to ask of any backtest
1. How many strategies were tried, including the ones dropped?
The best of many tries is the one most likely to be lucky. A result means far less when it is the best of hundreds than when it is the only one.
2. Over what years, and did they include a falling market?
A test that only covers a long rise can make almost anything that holds stocks look skilful.
3. What trading costs and fills were assumed?
A strategy that trades often can lose most of its edge to costs, and a backtest can fill every order at a price no one could actually have traded at.
4. Was any of it tested on years the strategy never saw while it was being built?
Rules tuned and judged on the same history will fit that history. Only data held back, or data that arrives later, can show whether they work.
5. Is there a forward record: calls written down before the outcome, misses included?
A record kept from the day a rule was fixed cannot be improved by hindsight. Without one, every figure shown is about the past.
This page describes arithmetic and asks questions; it does not judge any product or tell anyone what to trade. The full lesson behind it, with the reasoning step by step, is in the academy.
IntellaZone is an educational and informational platform. Nothing on it constitutes investment, financial or trading advice, or a recommendation to buy or sell any security. All scores and plans are descriptions of data.
Past performance — including the forward-tested record — does not guarantee future results. Investing involves risk, including the risk of loss; short selling carries losses that can exceed the amount invested.