Sample size and uncertainty

How many trades you need before a win rate or expectancy means something, with a simple error estimate.

All courses · Lesson 2 of 6 · Advanced · 5 min

A win rate of 60% from ten trades and a win rate of 60% from a thousand trades are very different pieces of evidence. This lesson gives you a simple formula for how uncertain a win rate or an expectancy is, so you can judge whether a sample is large enough to mean anything.

Why small samples mislead

Trade results are noisy. Even a coin that is perfectly fair will produce 7 heads in 10 flips fairly often. A strategy tested on a few dozen trades can look excellent or terrible purely by chance, and the fewer the trades, the wider that range of chance outcomes.

The standard error measures how much a statistic such as a win rate would typically move from one sample to another. A smaller standard error means a more trustworthy estimate.

Standard error of a win rate

SE = √( p × (1 − p) / n )
approximate 95% range ≈ p ± 1.96 × SE
p is the observed win rate as a fraction and n is the number of trades.

The 1.96 comes from the normal distribution and is a rule of thumb; it is less accurate for very small samples or win rates close to 0% or 100%. The table applies the formula to a win rate of p = 50%, where the error is largest.

Standard error and approximate 95% range of a 50% win rate (rounded)
Trades (n)Standard error± 1.96 × SEApproximate 95% range
309.1%17.9%32% to 68%
1005.0%9.8%40% to 60%
3002.9%5.7%44% to 56%
10001.6%3.1%47% to 53%

Notice that to halve the error you need four times as many trades, because the error shrinks with the square root of n. A 50% win rate measured over 30 trades is statistically compatible with anything from about 32% to 68%.

Standard error of the expectancy

The same idea applies to the expectancy, which is what you ultimately care about. Its standard error is roughly the standard deviation of the R results per trade divided by the square root of the number of trades.

SE(expectancy) ≈ σ_R / √n
σ_R is the standard deviation of the R result of single trades.

Worked example

How many trades to tell 0.2R from zero

  1. A strategy wins 2R in 40% of trades and loses 1R in 60%. Its expectancy is 0.4 × 2 − 0.6 × 1 = +0.20R.
  2. The average squared result is 0.4 × 4 + 0.6 × 1 = 2.2, so the variance is 2.2 − 0.2² = 2.16 and σ_R = √2.16 = 1.47R.
  3. With n = 100 trades, SE = 1.47 / 10 = 0.147R, so the 95% range is 0.20 ± 0.288, about −0.09R to +0.49R.
  4. With n = 400 trades, SE = 1.47 / 20 = 0.0735R, so the 95% range is 0.20 ± 0.144, about +0.06R to +0.34R.

At 100 trades the range still includes zero; at 400 trades it no longer does, so a few hundred trades are needed to separate even a decent edge from noise.

Putting it to use

The formulas give you a way to decide, before testing, how much data you need. If the edge you hope to detect is small, the sample has to be large; if the strategy trades only a few times a year, you may need many years of data, and then the older data may no longer resemble current markets. That trade-off is a real limit of backtesting, not something clever analysis removes.

Worked example

Is 60% from 40 trades convincing?

  1. A backtest shows 24 wins in 40 trades, so the observed win rate is p = 24 / 40 = 60%.
  2. The standard error is √(0.6 × 0.4 / 40) = √0.006 = 0.0775, about 7.7%.
  3. The approximate 95% range is 60% ± 1.96 × 7.7% = 60% ± 15.2%, which is about 45% to 75%.
  4. A coin-flip win rate of 50% lies inside that range.

Forty trades cannot distinguish a 60% win rate from a 50% one, so the result is interesting but not convincing on its own.

Remember that a win rate alone does not say whether a strategy is profitable; that depends on the reward per win and the loss per loss, which is why the expectancy and its standard error are the better numbers to track.

Key takeaways

  • The standard error of a win rate is √(p(1 − p)/n), and an approximate 95% range is ±1.96 times that.
  • A 50% win rate has an error of about 9.1% at 30 trades, 5.0% at 100, 2.9% at 300 and 1.6% at 1000.
  • The expectancy's standard error is about σ_R / √n, so halving the error needs four times the trades.
  • A few hundred trades are often needed before a modest edge stands out from noise.

Sign in to save your progress and quiz results.