r/algorithmictrading

Please don’t backtest thousands of strategies

edit: as people are downvoting this I would like to clarify something. I am a full time trader, have been trading for 16 years and currently profitably run 7 algorithmic strategies. i am a big supporter of backtesting and this is post is not saying that backtesting is bad. It’s just explaining why blindly backtesting thousands of strategies will cost you money.

I recently saw multiple people on Reddit following this approach of automatically testing thousands or even tens of thousands of strategies to then afterwards live trade whatever survives the backtesting and demo trading so I just want to create this short post and explain why this won’t work and why it will cost you a lot of money.

Basically the idea when creating a strategy and when then testing this strategy on out of sample data during a backtest as well as during demo trading is that afterwards you can ask yourself “what’s the probability that a non-profitable strategy would perform this well?”. Then usually if you only did this with one strategy the answer would be that it’s highly unlikely that an unprofitable strategy would perform well on out of sample data as well as demo trading and you could confidently conclude that your strategy does indeed have an edge and that it will most likely continue working when live trading.

However when backtesting for example 10.000 strategies that changes the whole question. in that case if one of your strategies performs well the relevant question wouldn’t be what’s the chance that this specific strategy performed so well just by chance but instead “what’s the probability that at least 1 of 10.000 strategies would perform that well just by pure chance?”. And the answer is basically always that it’s very very high and that most likely your strategy won’t continue working when live trading.

There are also ways to simulate this exact probability which I can share in case this intuition isn’t enough but I just felt like sharing this because this approach seems to become more and more common and it will cost people a lot of money.

reddit.com
u/Lanky-Ad4790 — 6 days ago

Outgrowing TradingView’s limitations. Need a high-performance alternative for custom signal scanning (2,000+ symbols).

Hi everyone,

I’ve hit a wall with TradingView. I’ve been using Pine Script to run custom signal logic across multiple watchlists, but their recent decision to cap lists at 500 symbols has effectively broken my workflow. I need to scan at least 2,000+ symbols simultaneously, and TV’s cloud limitations are no longer cutting it.

I’m looking for a platform that offers a TradingView-like experience (clean charts, easy scripting, alerts) but with much more horsepower under the hood for large-scale scanning.

My specific needs:

Heavy Custom Logic: I’m not looking for a basic price/volume screener. I need to run complex, multi-condition scripts (like I did with Pine) that process the entire universe on the fly.

Scale: Must handle 2,000+ tickers without lagging or hitting arbitrary "symbol caps."

Alerting/Dashboard: I need to be notified or see a real-time list when my script triggers a signal on any of those 2,000+ stocks.

What I'm considering:

QuantConnect: I know it's the "gold standard" for algo trading, but how is the UI/UX for someone used to TV’s visual environment? Is the Python/Lean learning curve worth it just for scanning?

TrendSpider: I’ve heard their "Market Scanner" is powerful, but can it handle the same level of script complexity as a dedicated coding environment?

Local/Python Frameworks: I’m open to running things locally if there’s a framework that handles the data pipe and provides a decent UI for visualization.

Has anyone else here "graduated" from Pine Script to something more robust for market-wide scanning? I love the ease of use of TV, but I can’t deal with the constant nerfing of capacity. What’s the best "middle ground" between a retail charting app and a full-blown institutional HFT setup?

Thanks for any suggestions!

reddit.com
u/Worried_Contact_1523 — 2 days ago

Unpopular opinion? To succeed in algo trading, you have to be a trader first and a coder second.

When I see the rate of failure in the algo trading community I wonder how many people are actual traders themselves? I see people complaining about not being able to build a single bot that can be profitable for more than "just a year or two" when there is simply no real trader in the world who trades the exact same way all the time! They all adapt their strategy to the market.

To "make it" in algo trading I believe you have to be a trader first with robust knowledge of the market you trade, and a developer/programmer second. Unfortunately I see too many people who are inexperienced with the markets who think their background in programming is enough to help them build profitable trading bots.

All profitable algo traders I know are seasoned manual traders first who transition to algo trading and then they usually only use scripts/algos to help them execute in a semi-automated fashion. They all like to keep some level of discretion, monitoring market sentiment, bond yields, news of the day i.e. stuff that's not so simple to code into a bot, then when it's time to execute they let the machine do its thing.

Algo trading isn't a magic wand that lets you bypass the hard work of acquiring real screen time and market knowledge. That has to be your foundation. It's like a weekend golfer buying a set of Tour-level clubs hoping it will magically fix his terrible swing. The clubs might be state-of-the-art, but if you don't understand the fundamental mechanics of the game you're still going to slice the ball into the woods on every drive!

reddit.com
u/Mysterious_Gear_4000 — 5 days ago

Python for quant-Guidance

I'm an 18M with a few months of free time before starting college. how and where should I learn Python? I'm not really sure which specific quant role I want to go for, but I've heard Python is pretty important. I know the basics of the language... just looking for a bit of guidance.
cheers!!

reddit.com
u/Minimum_Dragonfly_94 — 3 days ago

Backtesting using ai

Would I get accurate results backtesting with AI if I gave it candle data and a detailed strategy description? Would this also work for detecting events and distributions around it?

reddit.com
u/Maleficent-Win8567 — 1 day ago

EODdata dot com down today?

It looks like EODdata dot com is down right now, Friday morning, May 8, 2026, at 8:50 AM. Is he aware? Anybody know anything?

Know of any alternatives?

reddit.com
u/projecttoday — 5 days ago

For the last couple of months I have been tinkering with an ML model that predicts certain (relatively rare) events of BTC price movements. Recently, I got some results that are sometimes good and sometimes terrible. I have a few ideas on what experiments could improve performance, but I don't really understand the underlying cause of the problem. Hopefully someone had a similar experience once and can give me some tips.

More details:

I am using mostly 1-second granularity data of prices, trades, and some other metrics of BTC.

As a validation scheme, I am using rolling windows for now with a block of 500,000 rows as training and 86,400 rows as validation, mirroring an actual live use. Train size was chosen based on some small experiments with autocorrelation (nothing sophisticated).

Currently, I am evaluating my feature selection and model-building process as a whole, not a particular model or fixed feature set. For this I plan to use around 10 to 20 folds. In the following, I am showing 4 folds that illustrate what is going right and wrong. Dates (validation data ends at 23:59:59 on these dates) = 2026-04-28, 2026-02-28, 2025-11-28, 2025-07-28. The month offsets are a bit arbitrary but lean to more recent data: [0, 2, 5, 9].

Based on early experiments using other data (not the validation folds), I have found embedded feature selection using only train data to work well sometimes when combined with a large amount of candidate features. From my perspective, it seems that the selection process can find features with predictive power sometimes. Other times the model cannot beat 40% precision.

For now I am using XGB as a classifier with mostly basic parameters: I only quickly tuned the max_depth on some other data apart from the validation folds and set it to 10. The XGB predictions are also ensembled across 30 seeds to stabilize the PNL, as I found it was unstable using just one random seed.

The chosen feature sets, using only the recent training data, and models are evaluated on the validation fold using a set fee logic. The simulated trades don't use any position sizing yet, just a fixed amount per trade ($150). This is why there can be large negative results. When it works, the positions often get opened in quick succession (concurrency of up to 20 positions).

Here's a snapshot of using the prediction threshold 0.8 performance of the out of sample, unseen validation folds:

threshold n n_tp n_fp precision edge_per_trade total_net_pnl
f64 i64 i64 i64 f64 f64 f64
0.8 98 70 28 0.714286 22.779897 2232.42992
0.8 597 192 405 0.321608 -39.229474 -23419.995954
0.8 558 217 341 0.388889 -15.50954 -8654.323338
0.8 0 0 0 0.0 0.0 0.0

Using no feature engineering the PNL is negative on that first fold.

Per fold plots:

img

img

img

img

img

Some of my ideas of what I could do without knowing the core underlying problem:

- Regime or per-trade filter
- Use more data for training
- Use feature stability when selecting features

What should I consider doing next?

Thanks in advance.

u/Apprehensive_Fox8212 — 13 days ago
▲ 9 r/algorithmictrading+7 crossposts

Data ingestion and avoiding lookahead bias is a massive headache, so I built an open-source CLI agent to automate my backtesting setup.

It takes a plain-English strategy idea, generates validated Python using your own LLM key, and runs a historical backtest.

I just added Binance support today.

My biggest challenge right now is the automated safety checks—it currently scans the AST for lookahead flaws before executing.

The tool is free and open source locally at finnyai.tech, with an optional $10/mo tier for managed hosting.

If anyone here builds automated validation for strategy code, how do you handle edge cases and LLM data hallucinations?

u/Awkward_Weather5721 — 12 days ago

I've been building a trading bot using LLMs for the last year and running on Railway, currently in paper trading phase after finally finding profitable candidates at around 66% annual. most profitable setup from back testing and walk forward is the below

1H decides direction

ATE blocks bad conditions

ATE mode uses things like

trend_strength = 0.62

macd_slope = +0.0008

atr_expansion_ratio = 1.18

chop_probability = 0.47

Regime Router checks if setup is valid

Weak-pair filter removes bad combos

Bias favors stronger side

5m finds entry timing

Enter with 10% sizing

Exit when 1H state breaks (fast or confirmed)

Anyone built anything similar? I've been a QA engineer for the past 16 years so everything works, just difficult finding a decent strategy so any help is appreciated

reddit.com
u/bal1981 — 14 days ago

I’ve been iterating on a tactical ETF rotation strategy in QuantConnect and wanted to share the current backtest results for feedback.

The strategy is a concentrated dual-momentum / breadth model. It rotates monthly across a small ETF universe focused mostly on QQQ/growth/tech/semis, with defensive assets like BIL, SHY, IEF, and GLD. It uses trend, relative momentum, canary signals, volatility targeting, and leverage gates to decide when to hold risk-on ETFs versus defensive assets. Some leveraged ETFs are allowed, but only when trend/breadth conditions pass.

Current backtest window is 2016-01-01 to Now. Starting capital is $100k.

Headline results from the latest strong version:

  • Net profit: ~7,148%
  • CAGR: ~51.3%
  • Max drawdown: ~44.7%
  • Sharpe: ~1.05
  • Sortino: ~1.01
  • Total orders: ~458
  • Win rate: ~58%

The strongest periods were 2020-2021 and 2024-2025, mostly driven by correctly rotating into semis / leveraged tech / growth momentum. The weak periods were mostly during regime shifts and high-volatility drawdowns.

I’m not claiming this is production-ready. It is obviously aggressive, uses margin/leverage, and has significant drawdown. I’m mainly interested in feedback on robustness, overfitting risk, live-trading realism, and what failure modes people would look for before trusting something like this with real money.

Things I’m already thinking about:

  • Walk-forward / out-of-sample validation
  • Margin call risk and live buying-power behavior
  • Slippage and liquidity assumptions
  • Whether the 2024-2025 performance is too regime-specific
  • Whether drawdown is acceptable for the return profile
  • Better diagnostics around why the strategy enters/exits leveraged ETFs

Curious what people here would stress test first.

https://preview.redd.it/yfdwedznqmyg1.png?width=2098&format=png&auto=webp&s=34aad2d0a436995f7c713886612ed93de76b788c

https://preview.redd.it/bfo6ceznqmyg1.png?width=2088&format=png&auto=webp&s=6f3501c5db6638f13e2eddd571436f3252a6adf8

reddit.com
u/DiskProfessional1657 — 12 days ago