---
title: "Cherry-picking bias"
description: "Cherry-picking bias is quoting only the periods, instruments, or runs that flattered a strategy and omitting the rest — every figure true, the picture not."
canonical_url: "https://fincanva.com/docs/investing-theory/cherry-picking-bias"
last_updated: "2026-07-25"
md_url: "https://fincanva.com/docs/investing-theory/cherry-picking-bias.md"
---

# Cherry-picking bias

Cherry-picking bias is the practice of quoting only the periods, instruments, or runs that flattered a strategy while leaving the rest out, so every number reported is true and the picture they add up to is not. Nothing has to be falsified — a good year really was a good year — but the reader is shown a slice of the evidence that was chosen *because* it was favorable, and has no way to see what was left out. Cherry-picking bias is an error of **reporting**: the underlying test can be entirely correct, and the distortion enters only when its results are described.

**Also seen as:** Cherry picking, selective reporting

## What gets cherry-picked in a backtest?

Four things are usually picked, and each is picked the same way — after the results are already known:

- **The window.** A start and end date chosen because the stretch between them went well. This is the most common form, because a backtest makes any window one setting away.
- **The instruments.** The names in the report are the ones that contributed; the ones that dragged get described as "not really part of the idea".
- **The metric.** The return is quoted and the worst [drawdown](/docs/analysis/max-drawdown) is not, or the [gross](/docs/analysis/gross-vs-net) curve is shown while the net-of-costs one — the number a real account would have kept — is not (see [cost-ignoring bias](/docs/investing-theory/cost-ignoring-bias)).
- **The run.** One configuration out of many tried is presented as *the* strategy, with no mention of the others.

One test catches all four: could someone reproduce the claim without knowing which slice you chose? If the claim only holds on your slice, the slice is doing the work, not the strategy.

## Worked example: reporting 2019 and omitting 2022

A strategy is backtested over the ten calendar years 2015–2024. Two of those years stand out: **2019 returned +31%** and **2022 returned −29%**.

A report built around 2019 — the year label, the rising curve, the +31% — states a fact. Here is what the same run also says:

| Figure from the same run | Value |
|---|---|
| 2019 calendar year | +31% |
| 2022 calendar year | −29% |
| Full period, total return | +72% |
| Full period, annualized ([CAGR](/docs/analysis/cagr)) | 5.6% a year |
| Worst peak-to-trough fall in the period | −34% |

Quoting 2019 alone invites the reader to treat +31% as what the strategy does in a year; the run's own annualized figure is about a fifth of that. Put the omitted year back and the pair alone leaves the strategy below where it started: 1.31 × 0.71 = 0.93, or −7% across the two years together. None of those five numbers contradicts the others — they all come from one run. The distortion is entirely in which of them got quoted.

{/* VISUAL: chart — the same equity curve shown twice, once cropped to the flattering window and once over the full period, with the omitted years shaded — tracked in VISUAL_BACKLOG */}

## How is cherry-picking bias different from selection bias and data-snooping bias?

The three differ by **which step goes wrong**: cherry-picking bias is about what you *report*, [selection bias](/docs/investing-theory/selection-bias) is about what you *tested*, and [data-snooping bias](/docs/investing-theory/data-snooping-bias) is about *how many* things you tested before something looked good.

They also stack, in that order. A researcher who tries fifty variants (data-snooping), keeps the one with the most flattering instrument list (selection), and then presents its best three years (cherry-picking) has committed all three, and the final report shows no trace of the first two.

## What does Fincanva do about cherry-picking bias?

A Fincanva backtest reports the whole period it ran rather than a chosen stretch of it, and reports the falls alongside the gains: the same run that produces the return also produces its worst peak-to-trough [drawdown](/docs/analysis/max-drawdown), so the bad part of the history arrives attached to the good part. Because the run is a [walk-forward replay](/docs/investing-theory/walk-forward-replay) of the full span, there is no version of the result that covers only the years that worked.

[Start-date sensitivity](/docs/analysis/start-date-sensitivity) is the direct answer to the window form of cherry-picking: it re-runs the same strategy across many entry dates and several holding windows and reports the *range* of outcomes — the best start against the worst start, and the share of start dates that ended positive. A claim that survives only one entry month shows up immediately as a wide range.

What the product cannot do is decide what you tell other people, or yourself. Choosing one favorable screenshot out of a run that offers the full picture is still available to anyone, which is why cherry-picking is usually discussed next to [confirmation bias](/docs/investing-theory/confirmation-bias) — the same selection applied inwards. See [the nine biases Fincanva helps you avoid](/docs/getting-started/the-nine-biases-fincanva-helps-you-avoid) for the product behavior in context, and [data-quality bias](/docs/investing-theory/data-quality-bias) for the case where the numbers themselves, not their selection, are the problem.

*Backtests show what would have happened — not what will. Fincanva provides no financial advice — see [Is this financial advice?](/docs/investing-theory/is-this-financial-advice).*