# Sample size and uncertainty

> How many trades you need before a win rate or expectancy means something, with a simple error estimate.

Source: https://quantraider.com/learn/backtesting/sample-size

Course: Backtesting properly (lesson 2 of 6) · Level: Advanced · 5 min

A win rate of 60% from ten trades and a win rate of 60% from a thousand trades are very different pieces of evidence. This lesson gives you a simple formula for how uncertain a win rate or an expectancy is, so you can judge whether a sample is large enough to mean anything.

## Why small samples mislead

Trade results are noisy. Even a coin that is perfectly fair will produce 7 heads in 10 flips fairly often. A strategy tested on a few dozen trades can look excellent or terrible purely by chance, and the fewer the trades, the wider that range of chance outcomes.

The **standard error** measures how much a statistic such as a win rate would typically move from one sample to another. A smaller standard error means a more trustworthy estimate.

## Standard error of a win rate

```
SE = √( p × (1 − p) / n )
approximate 95% range ≈ p ± 1.96 × SE
```

p is the observed win rate as a fraction and n is the number of trades.

The 1.96 comes from the normal distribution and is a rule of thumb; it is less accurate for very small samples or win rates close to 0% or 100%. The table applies the formula to a win rate of p = 50%, where the error is largest.

**Standard error and approximate 95% range of a 50% win rate (rounded)**

| Trades (n) | Standard error | ± 1.96 × SE | Approximate 95% range |
| --- | --- | --- | --- |
| 30 | 9.1% | 17.9% | 32% to 68% |
| 100 | 5.0% | 9.8% | 40% to 60% |
| 300 | 2.9% | 5.7% | 44% to 56% |
| 1000 | 1.6% | 3.1% | 47% to 53% |

Notice that to halve the error you need four times as many trades, because the error shrinks with the square root of n. A 50% win rate measured over 30 trades is statistically compatible with anything from about 32% to 68%.

## Standard error of the expectancy

The same idea applies to the expectancy, which is what you ultimately care about. Its standard error is roughly the standard deviation of the R results per trade divided by the square root of the number of trades.

```
SE(expectancy) ≈ σ_R / √n
```

σ_R is the standard deviation of the R result of single trades.

**Worked example: How many trades to tell 0.2R from zero**

1. A strategy wins 2R in 40% of trades and loses 1R in 60%. Its expectancy is 0.4 × 2 − 0.6 × 1 = +0.20R.
2. The average squared result is 0.4 × 4 + 0.6 × 1 = 2.2, so the variance is 2.2 − 0.2² = 2.16 and σ_R = √2.16 = 1.47R.
3. With n = 100 trades, SE = 1.47 / 10 = 0.147R, so the 95% range is 0.20 ± 0.288, about −0.09R to +0.49R.
4. With n = 400 trades, SE = 1.47 / 20 = 0.0735R, so the 95% range is 0.20 ± 0.144, about +0.06R to +0.34R.

At 100 trades the range still includes zero; at 400 trades it no longer does, so a few hundred trades are needed to separate even a decent edge from noise.

## Putting it to use

The formulas give you a way to decide, before testing, how much data you need. If the edge you hope to detect is small, the sample has to be large; if the strategy trades only a few times a year, you may need many years of data, and then the older data may no longer resemble current markets. That trade-off is a real limit of backtesting, not something clever analysis removes.

**Worked example: Is 60% from 40 trades convincing?**

1. A backtest shows 24 wins in 40 trades, so the observed win rate is p = 24 / 40 = 60%.
2. The standard error is √(0.6 × 0.4 / 40) = √0.006 = 0.0775, about 7.7%.
3. The approximate 95% range is 60% ± 1.96 × 7.7% = 60% ± 15.2%, which is about 45% to 75%.
4. A coin-flip win rate of 50% lies inside that range.

Forty trades cannot distinguish a 60% win rate from a 50% one, so the result is interesting but not convincing on its own.

Remember that a win rate alone does not say whether a strategy is profitable; that depends on the reward per win and the loss per loss, which is why the expectancy and its standard error are the better numbers to track.

> **The trades must be independent-ish**
>
> The formulas assume each trade is a separate draw. Trades that overlap, or that all react to the same market event, carry less independent information than their count suggests, so the real uncertainty is larger.

## Key takeaways

- The standard error of a win rate is √(p(1 − p)/n), and an approximate 95% range is ±1.96 times that.
- A 50% win rate has an error of about 9.1% at 30 trades, 5.0% at 100, 2.9% at 300 and 1.6% at 1000.
- The expectancy's standard error is about σ_R / √n, so halving the error needs four times the trades.
- A few hundred trades are often needed before a modest edge stands out from noise.

> Educational content, not financial advice. Trading involves risk of loss.
