A ghost story circulating in quant circles.
A, a geek who knew some Python but half-understood finance, got his "quant dream" lit by AI last year. He used a low-code platform, dragged a few factors, let AI auto-run parameter optimization. When the backtest came out, he nearly spat coffee at the screen: on an A-share tech leader, 2019-2024 annualized return 78%, max drawdown only 11%, Sharpe absurdly high. The curve was impossibly smooth-printing-press smooth.
A couldn't resist and poured his 500k year-end bonus in. Month one, slight profit-he felt chosen. Month two, style rotated, the strategy started bleeding slowly. Month three, a violent sector rotation broke his psyche, floating loss over 35%. He manually liquidated the night before a rebound, losing more than a small-apartment down payment in a tier-2 city.
The prettier the backtest curve, the harder the live-trading slap. A's story isn't unique. The answer depends on whether you can see through these three fatal illusions.
Illusion 1: AI Peeked at the Answers-Look-Ahead Bias as a Cheater
First-timers are fooled by "AI can compute anything." Truth is, many bad backtest frameworks or code inadvertently let AI see the future.
Classic example: using close-price signals to trade at that day's close. A simple breakout rule "buy when price breaks the 20-day high." In backtest, if you trade at the close on the signal day, you've committed look-ahead bias. In reality, at the moment of intraday breakout you can't confirm it'll be the close, and by close confirmation you can no longer trade at that price. AI's pretty curve from "prophet" data is blatant cheating.
Key point: any backtest code, first check whether "signal time" and "trade time" have a strict gap. When using AI to generate a backtest script, add this constraint to the prompt: "No future data allowed; all signals based on confirmed info at time T; trade at T+1 open/mid price; include 0.1% slippage cost." Without this, AI likely gives you an "answer-peeked" perfect curve.
Illusion 2: The Overfit Crystal Ball, a Fragile Holy Grail Tailored for You
Even without look-ahead, the second trap is deeper: overfitting. AI excels at finding a curve that perfectly fits historical price moves, even carving noise and randomness into "rules."
BigQuant's docs mention parameter optimization and genetic algorithm tools to "help investors find optimal parameter combos." But the risk: the harder you optimize, the more godly in-sample, the more likely out-of-sample failure-like filing a key to perfectly fit one lock, useless on another.
The AI forex backtesting agent guide nails it: "Manual optimizers often curve-fit historical data without realizing it, creating strategies that fail in live markets." Modern AI agents can auto-generate 30 variant strategies by risk-reward, time window, stop-loss, then rank by cross-market stability. The right direction: pick the most stable across environments, not the best historically.
Truly viable strategies often look unimpressive in backtest-even mediocre-but stable out-of-sample with simple, crude logic. A crystal ball that catches every wave in every market is overfit incarnate.
Illusion 3: Ignored Friction Costs-Slippage and Fees as Silent Killers
The third illusion is most overlooked by beginners yet most fatal. Many AI backtest platforms, especially "one-click" low-barrier tools, default to idealized or zero friction. Reality differs.
In live trading, every trade pays commissions, stamp duty, transfer fees, plus bears bid-ask spread and impact cost. For high-frequency, it's life or death. A 2026 quant software review noted BigQuant's "96.5% backtest accuracy"-upper-mid tier-but the remaining 3.5% gap comes mainly from slippage simulation and matching realism. If AI gives you a 100% annualized minute-level high-freq strategy, after live slippage and fees it likely becomes -20% annualized.
Do the math: a strategy with 0.2% expected return per trade looks nice. But pay 0.1% fixed commission + 0.05% slippage + 0.05% impact per trade, and net return goes to zero. Add overnight interest and you steadily lose. AI backtests that don't define these costs are drawing a pie you can't eat.
The Right Hardcore SOP for AI Backtests: From Pie to Bulletproof Vest
After seeing the three illusions, should we ditch AI? No. AI backtesting is a superb risk-lab and strategy magnifier-used correctly. Here's a copyable SOP.
1. Mindset Init: AI Is a Microscope, Not Autopilot
AI turns ideas into code and data fast, but won't think strategy logic for you. You're the architect; AI is the construction crew. Never All-in on a strategy because the curve is pretty-you're exploring a possibility, not validating a truth.
2. Three Hands-on Steps: From Data to Closed Loop
Step 1: Generate backtest code with AI, but lock the rules
Classic golden-cross example:
-
Get data: download historical daily bars (CSV with OHLCV) from free sources like Investing.com or Tushare.
-
Write the prompt: open Perplexity or ChatGPT, input something like:
"Write a Python backtest script for a CSV (OHLCV) implementing a 50-day-MA-crosses-above-200-day-MA buy, cross-below sell strategy. Requirements:
- Compute MAs on close, but confirm signal at day's close, execute at next day's open.
- Include 0.1% slippage and 0.03% fixed commission (both sides).
- Initial capital 1M, fixed 100k per trade.
- Output annualized return, max drawdown, Sharpe, win rate, profit-loss ratio, vs buy-and-hold.
- No future data; comment key logic."
-
Review code: after AI generates, focus on signal-vs-execution time alignment. Don't be lazy-it's your money.
Step 2: Stress test and multi-dimensional validation, not optimal params
Don't rush to returns. Test extreme scenarios:
- Cross-market: run the same strategy on CSI 300, S&P 500, Bitcoin-check universality.
- Cross-time: split history into bull/bear cycles (2008 crisis, 2015 A-share crash, 2020 circuit breaker), run each.
- Parameter perturbation: don't fix MAs at 50/200. Batch-test (40,160), (50,200), (60,240). If returns cliff-dive on slight param changes, it's an overfit dud.
Good strategies barely survive everywhere, not shine in one historical slice.
Step 3: Build backtest-sim-small-capital closed loop
The last line of defense. No matter how pretty the backtest, going straight to live is gambling. Shadow Account and sim are mandatory.
- Sim: run on a broker or third-party (Juejin Quant, BigQuant sim) for 1-2 months, experiencing real latency, liquidity shock, data flow.
- Small-capital live: if sim doesn't diverge from backtest by an order of magnitude, put in tiny capital (e.g., 5% of total). You're testing not just the strategy but your execution discipline and psychology.
Tool Selection: Low-Code Platform vs AI Self-Coding
| Dimension | Low-code (BigQuant / Jellyfish) | AI-assisted coding (ChatGPT / Perplexity + Python) |
|---|---|---|
| Learning curve | ★☆☆☆☆ drag-generate, no coding | ★★☆☆☆ must read and modify AI code |
| Backtest transparency | Mid, some black-box (matching logic) | Fully transparent, code in hand |
| Anti-overfit | Offers optimization, but induces over-tuning | Fully your design, free OOS tests |
| Friction simulation | Basic settings, impact cost crude | Arbitrary slippage and fee models |
| Flexibility | Limited to platform factors/templates | Unlimited, any logic |
| For whom | Strategy-validation beginners, fast ideas | Geeks wanting full control |
My advice: beginners use BigQuant to feel the flow, but don't make live decisions on its backtests. Once you have a credible logic, switch to AI-assisted coding, generate fully transparent Python, and devil-test it yourself. That's the path that actually controls risk.
Stay Alive and Sober
AI is driving backtest cost toward zero-one person, one computer, one AI window can finish in ten minutes what took a team weeks. But investing's essence is unchanged: cognition of risk and management of your own human nature.
Those telling you "AI guarantees profit" are either fools or scammers. Real alchemy isn't using AI to find a perfect wealth-freedom curve, but using AI as a mirror to expose your greed, fear, and blind spots. Only then can you design a strategy that stands neither arrogant nor groveling before the market, knowing when to attack and when to kneel.
Remember: a strategy that loses in backtest almost certainly loses live; one that backtests huge gains is even more likely to lose big live. Surviving beats beating the index.
References