Backtesting.

Every trading strategy sounds plausible when someone tells it to you. "Buy when the price bounces off the 0.618 Fibonacci level." "Enter when RSI drops below 30." "Follow the smart money." The problem is that a plausible story and a money-making strategy are two different things. Backtesting is the bridge — or the chasm — between them.

This article explains what backtesting is, why it matters for retail investors on the Indonesia Stock Exchange (IDX), and — the part most often overlooked — how backtests can lie, and how to read them honestly.

The big picture: a backtest is an experiment, not a prophecy

A backtest, at its simplest, means taking a buy-sell rule, running it against historical price data, and calculating what would have happened had that rule been followed exactly. The result is a number — return, maximum drawdown, win rate — not an opinion.

This is what separates an evidence-based approach from market pseudoscience. A pattern that "always fits after the fact" can never be honestly tested, because there's always a new explanation for every outcome. A proper backtest forces you to commit to a rule before seeing the results, and then accept whatever comes out. It can prove your favorite strategy is actually useless — and that's a feature, not a bug.

But there's one uncomfortable truth to hold onto from the start:

A good backtest does not guarantee a good future.

Markets are non-stationary and reflexive — the rules of the game change, and the more people who trade on the same signal, the thinner its edge becomes. Even the most honest backtest only gives you a distribution of possibilities, not a certainty.

Why IDX retail investors should care

Without backtesting, you're testing a strategy with real money in real time — the most expensive way to learn there is. Backtesting gives you three things:

• A fast filter. Many strategies that sound great fail the moment they're tested against data. Better to know now than after a loss.
• Realistic expectations. A backtest shows how deep and how long losses can run, so you don't panic at the worst possible moment.
• Discipline. A tested rule is easier to stick to than a hunch, especially when the market is pressing on your emotions.

Reading backtest results honestly

A headline "300% profit" number means nothing without context. Here's what actually matters:

• Return vs. benchmark. Compare it against simply buying and holding the IHSG (the Jakarta Composite Index). Beating the market is far harder than merely making money.
• Maximum drawdown. The deepest decline from peak to trough. A strategy that returns 40% but once fell 60% along the way will make most people quit before the gains arrive.
• Sample size (number of trades). A strategy with only 8 trades over 5 years doesn't have enough evidence behind it. Luck needs very little data; statistics need a lot.
• Consistency across periods. Is the profit spread across many years, or did it all come from one lucky break in 2020?
• Risk-adjusted return (e.g., Sharpe ratio). Return per unit of risk (Sharpe, 1994). Two strategies with the same return but different volatility are not equivalent.

Seven traps that make a backtest lie

This is where most backtests — including the ones used to sell "miracle systems" — fall apart.

1. Overfitting (a forced-fit curve). If you tweak parameters until the results look perfect in the past, you haven't found a strategy — you've memorized history. A strategy with 10 parameters that all happen to line up almost always fails on new data. Bailey et al. (2014) even showed that impressive simulated performance is easy to achieve just by trying a small number of configurations — and the more configurations you try, the greater the chance the result is illusory.

2. Look-ahead bias. Using information that wasn't actually available at the moment the decision was made. A classic example: using that day's closing price to "decide" to buy that same morning. In the real world, you don't yet know the closing price.

3. Survivorship bias. Testing only on stocks that still exist today, ignoring the ones that were delisted or went bankrupt. Brown, Goetzmann, Ibbotson & Ross (1992) showed that survivorship bias can meaningfully inflate historical performance figures.

4. Ignoring real costs. Brokerage fees, sales tax, bid-ask spread, and slippage (the gap between the expected price and the actual execution price) can turn an apparently profitable strategy into a losing one — especially for frequent trading or illiquid IDX stocks.

5. Small sample size. A handful of trades cannot statistically distinguish skill from luck.

6. Data snooping / selection bias. Testing hundreds of parameter combinations, then reporting only the best one, without correcting for how many attempts were made. Harvey, Liu & Zhu (2016) argue that most "discovered" factors in finance research fail to hold up once this multiple-testing problem is properly accounted for.

7. Regime change. A rule that worked during a bull market, a low-interest-rate era, or a particular commodity cycle may fail completely once conditions shift. The past regime is not guaranteed to return.

IDX-specific factors that make backtests lie in particular ways

• ARA/ARB (auto rejection). IDX enforces daily price limits — Auto Rejection Atas/Bawah (upper/lower auto-rejection) — that cap how far a stock's price can move in a single day. A backtest that assumes you could buy or sell at any price ignores the real possibility that a stock is locked at its limit, with no one able to trade at that price at all.
• Thin liquidity in second- and third-tier stocks. A rule that looks great on paper may be unexecutable in practice if the actual daily trading volume is too small to fill your order without moving the price yourself.
• Lot size (1 lot = 100 shares). Position sizing and rebalancing on IDX must respect the 100-share lot minimum, which a naive backtest in continuous units may ignore.
• Withholding tax on capital gains and dividends. Indonesian tax rules differ from other markets and materially affect net return; a backtest that ignores them overstates real-world profit.

How to read a backtest without fooling yourself

• Split your data. Test the rule on one period (in-sample), then verify it on a period it has never seen (out-of-sample). A rule that only works in-sample is memorization, not an edge.
• Include realistic costs from the start — fees, tax, and slippage — not as an afterthought.
• Demand a reasonable sample size. Treat backtests with very few trades with real skepticism.
• Think in ranges, not a single number. A backtest gives you a distribution of plausible outcomes (think P10/P50/P90), not a guarantee.
• Be suspicious of anyone who only shows the winning examples. A strategy that's been properly tested shows its failures too.

Closing

Backtesting is neither a crystal ball nor a scam — it's a discipline. Used honestly, it tells you whether a strategy would have survived real market conditions, at what cost, and with what risk. Used dishonestly (tuned after the fact, stripped of costs, cherry-picked periods), it tells you nothing except what you wanted to hear.

This is the standard we hold ourselves to at Sobat Investor: state the methodology plainly, disclose the traps, and report uncertainty as it is — not as we wish it were.

References & Further Reading

This article is educational and does not constitute investment advice. Investment decisions and their risks are entirely your own responsibility. Past performance does not guarantee future results.