Test how much to trust a backtest with the Robustness tab

Before you start

• The strategy needs a completed backtest. A simulation stored before the tab existed has to be run again first — see Common problems.
• The tab and both of its checks depend on your plan: if the tab carries bars beside its name — the level of the plan that includes it — it opens to say which plan includes it. Backtest reliability and the stress test each name the plan that includes them.

Steps

1. Open the strategy, choose Analysis, then Robustness. The tab has two panels, Backtest reliability and Stress test; choose one.
2. For Backtest reliability: under Method choose Blocks of days or Single days, under Resamples choose 100 or 1,000, then press Calculate. While it runs the button reads, for example, "Calculating 1,000 resamples…".
3. For Stress test: pick one of the Ready-made scenarios — "A −20% year", "Double volatility", "A flat year" or "Crash: −40% and volatility ×2" — or set your own with the two sliders, Assumed annual return (from −50% to +50%, switched on with "Use an assumed annual return") and Volatility multiplier (from 0.5× to 3×; "Off = 1×"). At least one of the two must differ from normal. Then press Apply scenario.

Each result stays until you change a control on its panel, or one of the Simulation settings switches; then calculate again.

What you should see

Backtest reliability opens on a verdict under the question "Could it be luck?" — "Probably not", "Uncertain" or "It could be luck" — with the number behind it and one sentence saying what that number means. Below it come Years in profit, a band showing how much the average annual return, the Sharpe and the maximum drawdown could have varied, and a Year by year table. The figures are explained on backtest reliability.

Stress test shows two gauges — How far we bent history and Days that still count — and a table, On the strategy's historical days, that sets the Base figures beside With the scenario: average annual return, annual volatility, and the daily VaR and CVaR at 95%. The gauges and the table are explained on stress test.

How is the Robustness tab different from the Projection tab?

The two tabs point in different directions. Projection looks forward: it builds possible futures out of the strategy's past days and shows their spread. Robustness looks back at the backtest itself: reliability asks how much of the result the same days could have produced by chance, and the stress test asks how the result depends on the kind of days the history happened to hold. Neither tab is a forecast.

Why is the return here not the one on Performance Metrics?

Because both panels report the arithmetic average annual return — the average daily return scaled to a year — while Performance Metrics reports the compound growth rate, CAGR. The two answer different questions and differ most when returns swing widely; the app says so under the reliability band ("The return is arithmetic, so it does not match the one on the Metrics page."). In the stress test, the Base column is also computed differently from the Metrics page, which the table states beside it — compare Base with With the scenario, not with Performance Metrics.

Common problems

A panel says it isn't available for this simulation

Reliability reads "Backtest reliability isn't available for this simulation. If the backtest ran before the update, run it again to enable it." Both checks draw on a record of daily returns that older simulations do not store. Run the backtest again, then calculate.

The stress test says it couldn't calculate the scenario

The message reads "We couldn't calculate this scenario." A scenario too far from anything in the strategy's history cannot be reached by reweighting its days. Try a less extreme scenario — the panel offers a button, "Soften to …", that brings the more extreme of the two settings halfway back to normal — or run the backtest again if the simulation predates the update.