# Overfitting and optimisation

> Why the best parameters on past data are usually too good, and how to optimise without curve fitting.

Source: https://quantraider.com/learn/backtesting/overfitting

Course: Backtesting properly (lesson 4 of 6) · Level: Advanced · 4 min

Given enough settings to adjust, you can make almost any rule set look profitable on past data. This lesson explains overfitting, why the best result of an optimisation is usually too good, and how to optimise in a way that holds up.

## Fitting the noise

**Overfitting**, or curve fitting, happens when settings are tuned so closely to the past that they capture random quirks of that sample instead of a repeatable pattern. The more parameters you tune and the smaller the sample, the more room there is to fit noise.

Think of picking the best of many random results. Even if every setting has zero true edge, some will score well by luck, and the optimiser will pick exactly those.

**Worked example: The best of 500 useless settings**

1. You test 10 stop distances × 10 take-profit distances × 5 moving-average lengths, which is 10 × 10 × 5 = 500 combinations.
2. Suppose every combination has a true expectancy of exactly 0R, and each is measured on 100 trades with σ_R = 1R.
3. The standard error of each measurement is 1 / √100 = 0.1R.
4. The best of 500 random measurements typically lies about 3 standard errors above the mean (the expected maximum of 500 standard normal values is about 3.0).
5. So the winning combination shows roughly 3.0 × 0.1 = +0.30R per trade.

A setting with no edge at all can show about +0.30R in-sample, and on fresh data it would be expected to fall back towards 0R.

## Optimisers are in-sample by construction

The analysis page includes an optimiser, "Find the best stop loss and take profit", which searches for the setting with the highest adjusted expectancy per trade on the trades it is given. Its answer is therefore an in-sample result and is optimistic by construction. It is a useful way to explore, not a promise of what the setting will earn.

The page also reports "adjusted" expectancy and win rate values next to the observed ones. They exist to temper small-sample optimism, so when the two differ a lot, take that as a sign that the sample is thin.

## Optimising more safely

- Prefer a plateau: a broad region of neighbouring settings that all perform acceptably is more trustworthy than a single sharp peak surrounded by poor results.
- Keep the number of free parameters small, and have a reason for each rule.
- Re-check the chosen setting on out-of-sample data you did not use while optimising.
- Expect the live result to be lower than the optimised one, and plan the risk accordingly.

## Signs that a result is overfitted

Overfitting leaves traces you can look for. Results that are excellent in one period and poor in the next, a strategy with many special-case conditions, and rules whose thresholds have oddly specific values such as an RSI of 37.4 are all typical. The more rules were added after looking at the losing trades, the more likely the strategy has been shaped to the past.

A useful comparison is the number of trades per free parameter. If you tune five settings on 60 trades, you have only about 12 trades per setting, and the estimates for each are very noisy. Many practitioners want far more than that, although there is no exact threshold.

**Strengths**

- A robust strategy keeps a similar result when parameters move slightly.
- Few, well-reasoned rules leave less room to fit noise.
- Out-of-sample results of similar sign and size support the idea.

**Weaknesses**

- An overfitted strategy often collapses when parameters are nudged.
- Rules added after studying losers tend to explain the past, not the future.
- Beautiful in-sample equity curves are cheap to produce and easy to trust.

> **A sharp peak is a red flag**
>
> If moving a stop by a small amount turns a strong result into a weak one, the strategy depends on a coincidence in the data, not on a robust behaviour.

> **Try the optimiser**: Open the Analysis tab of a strategy and look at how results change across nearby stop-loss and take-profit settings before trusting the top one. [Open your strategies](https://quantraider.com/strategy)

## Key takeaways

- Overfitting means tuning to noise; it grows with the number of parameters and shrinks with sample size.
- The best of many tested settings is biased upwards, even when none of them has a real edge.
- The analysis optimiser reports an in-sample result, so treat it as optimistic and verify on out-of-sample data.
- Prefer plateaus of good settings over a single peak, and keep the rule set simple.

> Educational content, not financial advice. Trading involves risk of loss.
