SPY Reddit Mentions | ChartExchange

So I've been iterating on a US equities swing strategy for about a year. Earlier this year I audited my backtest methodology end to end to address some lingering doubts about the backtest results.
I see survivor bias comes up in nearly every backtesting thread on here, but I rarely see anyone post actual before/after numbers, or show what fixing it looks like in practice. Most retail backtests I see are also single-ticker, not multi-ticker portfolios with a rotating, rules-based PIT universe. So this writeup is about my attempt, on a portfolio strategy with a PIT mechanical universe, end to end. Strategy specifics are deliberately left out for obvious reasons.
# My latest setup
I was using Alpaca's SIP daily-bar feed, which goes back to early 2016 (10 years). The strategy is a multi-ticker swing portfolio with a mechanical PIT universe and standard portfolio-level controls (position sizing, concurrency caps, per-day entry caps, etc.).
The universe construction is rules-based, not curated. At each quarterly rebalance (1st day of Jan/APril/July/Oct) the script takes all tickers with sufficient trailing dollar volume and price history, ranks them by trailing X-month return, takes the top N, and holds that list as eligible until the next rebalance. Sticky periods prevent edge-case tickers from jumping in and out. No hand-picking, no "names I liked," no curating at backtest time.
At runtime the strategy runs once a day on bar completion, evaluating each ticker in the current universe and emitting buy or sell signals. A portfolio coordinator sits on top: it takes the signals from across the universe, applies risk-per-trade sizing, enforces concurrency caps and per-day entry caps, and decides which signals actually get executed that day. A few other filters, ranking logic, etc. on top of that. The strategy doesn't size or rank, the coordinator does. This separation matters for the backtest because it means I can change one layer without invalidating the other.
A few execution-realism guards I built into the backtest, to keep it from cheating itself:
* **Fills happen on the next bar's open, not the signal bar's close.** The strategy can only act on info available at the signal-bar close; the price it actually gets is the next morning's open.
* **Position size capped at 0.5% of trailing 20-day average dollar volume.** If a "correct" risk-based size would exceed that, it gets truncated. If truncation pushes the resulting position below the minimum allocation, the trade is rejected entirely. This is based on the assumption that a real fill at >0.5% ADV of the stock would move the market, so don't do that.
* **No leverage.** Total deployed capital can't exceed equity, with a 2% buying-power buffer reserved to absorb the price gap between the signal-bar close (used for sizing) and the next-open fill (in case of gap ups)
* **No same-day recycling.** Positions exiting on day D don't free their slot or capital for new entries the same day
* **Per-side slippage** charged on every entry and exit (default 5bps per side, 10bps round-trip). Models the gap between intended fill price and actual fill.
* **Whole-share rounding.** No fractional shares.
Backtested on 2017-01-04 to 2026-06-07 (9.4 years), starting capital $100k:
|Metric|Strategy|SPY buy & hold|QQQ buy & hold|
|:-|:-|:-|:-|
|Final equity|$7.5M|$328k|$590k|
|CAGR|58.2%|13.4%|20.7%|
|Max DD|\-28.6%|\-34.2%|\-35.6%|
|Calmar|2.04|0.39|0.58|
|Sharpe|1.47|n/a|n/a|
439 trades, win rate 36.7%, 16 drawdown periods of -10% or worse over the window. The deepest was the COVID crash, -28.6% from 2020-02-19 to 2020-03-16, fully recovered 107 days later.
IS/OOS split:
|Window|Dates|CAGR|Max DD|
|:-|:-|:-|:-|
|OOS-early|2017-01-04 to 2018-12-31|\+13.6%|\-23.2%|
|IS|2019-01-01 to 2022-12-31|\+45.0%|\-28.6%|
|OOS-recent|2023-01-01 to today|\+112.5%|\-17.3%|
The methodology looked clean: multiple OOS windows, mechanical universe construction, no peeking, beating buy-and-hold by a wide margin.
# Survivor bias
The Alpaca data feed only carries currently-tradeable tickers. Anything that delisted, got acquired, went bankrupt, or merged out of existence between 2016 and today is not in the feed. My mechanical universe ranker was therefore pulling from a pool that had already been pre-filtered by the future. The filter was rules-based, but the input list wasn't. So I set out to figure out how much survivor bias actually impacted my returns.
I subscribed to **Norgate Data**, the gold standard for historical data. They have a watchlist for "Russell 3000 current and past constituents" as a single export: about 11,200 unique tickers (from 1990). A lot of them are not in the Alpaca data feed.
I then pointed my universe construction at the new feed and reran the same strategy code, same windows, same parameters. The mechanical universe ranker stayed exactly the same. Only the underlying pool changed.
[Same code, same parameters, same window. Only the pool changed.](https://preview.redd.it/ghv3y6i4b06h1.png?width=1589&format=png&auto=webp&s=1cbaa25c86678e8764cb7707bf3314848e3e5348)

Results:
|Metric|Original(survivors only)|Same code,full universe|Delta|
|:-|:-|:-|:-|
|CAGR (2017 to today)|\~57%|\~36%|\-21pp|
|Max drawdown|\-28%|\-29%|flat|
|Sharpe|1.4|1.1|\-0.3|
|Final equity from $100k|$7.5M|$1.85M|\-75%|
21pp of CAGR was coming from survivor bias. The drawdown stayed roughly the same. That $5.6M gap in final equity was the cost of picking from a list pre-filtered by the future.
# Extending the backtest window to 2000
Norgate's daily-bar coverage goes back to 1990. My original Alpaca backtest started in 2016 because that's where Alpaca's daily bars started. So I ran 2000 to today on the Norgate feed, a 26-year window that includes dotcom crash, GFC, 2018 Q4, COVID, and the 2022 bear.
[26 years of survivor-bias-free history. Regime-filter version $see next section$.](https://preview.redd.it/fv6446i4b06h1.png?width=1573&format=png&auto=webp&s=594e69607c408cf4484b59a7175f46b8b5a59850)
Results:
|Metric|2017 to today|2000 to today|
|:-|:-|:-|
|CAGR|\~36%|\~15%|
|Max drawdown|\-29%|\-67%|
|Calmar|1.27|0.22|
\-67% MaxDD in 2000-2002. None of my IS/OOS tests had touched this because the data wasn't there. The strategy had never been stress-tested against a regime where the benchmark itself was in a multi-year bear.
# Adding a regime filter
I'm leaving strategy specifics out, but here's the broad idea of the fix.
When I looked at the trades log, the -67% drawdown was concentrated. Almost all of it came from 2000-2002. The strategy's behavior there was: a short period of stop-outs as the universe collapsed, then a long period mostly in cash. The problem wasn't the active-trading damage; it was a tactical overlay that held the broad-market benchmark when the universe was empty. Holding the benchmark through a 75% bear was the actual mistake.
So I added a simple regime check: don't hold the benchmark when it's more than X% below its trailing 1-year high; re-enter when it recovers to within Y%.
I tuned the thresholds with a parameter sweep across about 70 combinations on the 2000-2010 window, picked the best by Calmar, then validated on the full 26-year window and on the modern window to make sure it didn't degrade good years.
[Same strategy, same data, same window. Only the regime filter changed.](https://preview.redd.it/jpq456i4b06h1.png?width=1573&format=png&auto=webp&s=eca38b36e9f9e275a3f7d7e75af046d518fc5b6c)
Results:
|Metric|Before|After|
|:-|:-|:-|
|CAGR|\~15%|\~16%|
|Max drawdown|\-67%|\-34%|
|Calmar|0.22|0.47|
https://preview.redd.it/ex6pu6i4b06h1.png?width=1503&format=png&auto=webp&s=6880dcc47d65189ef24f5fee4ea57829661f2ffd
Top 5 drawdowns. The dotcom one halves. Everything else moves by a few percent or stays put.
CAGR moved slightly up. MaxDD halved. The 2020 COVID drawdown also got a small improvement as a side effect.
In previous iterations, I actually tested a regime filter, but decided not to use it because it lowers CAGR. But this one clearly showed me that I need to protect myself against ruins like the dotcom crash.
# The IS/OOS framework after the changes
Going from 9 years to 26 years meant the IS/OOS split needed a redesign. I ended up with four non-contiguous windows:
* **OOS-deep** 2000-01-01 to 2015-12-31. 16 years that predate any strategy development. Used as a survival test, not an alpha test.
* **OOS-early** 2016-01-04 to 2018-12-31. The Alpaca-era window the retail feed could see but that predates my tuning work.
* **IS** 2019-01-01 to 2022-12-31. The four years I actually used for tuning.
* **OOS-recent** 2023-01-01 to today. Post-tuning live-ish performance.
[CAGR by window. Blue is in-sample $tuned$.](https://preview.redd.it/2uyei7i4b06h1.png?width=1503&format=png&auto=webp&s=5d78440a250377e8a16ed298fa8457c7d52363f5)
The OOS-deep window CAGR is much lower than the IS window. Some of that is real (the modern regime suits this strategy better than the dotcom decade did), and some is overfit (the IS bar is the noisiest estimate). Either way, the OOS-deep number is a sanity check I didn't have before. It rules out scenarios where the strategy quietly relies on a feature of the post-2015 regime that won't hold in the next bear.
# Asking for advice
Posting this partly because I'm hoping someone catches gaps I haven't seen yet. To be clear: I'm not asking for strategy feedback. The strategy details aren't in this post on purpose. I'm asking for methodology feedback.
Things I've been wondering about:
* **Other biases I might still have.** Norgate covers Russell 3000 current and past constituents, but the R3000 itself reconstitutes annually and has its own membership rules. Am I trading hidden bias from "the universe of names that ever made it into the R3000" against the more obvious survivor bias I just fixed?
* **Execution realism beyond what I already model.** I cap at 0.5% of 20-day average dollar volume, fill at next-bar open, charge per-side slippage, run with no leverage and a buying-power buffer (see "What I had" for the full list). But slippage is a flat per-side bps constant rather than something liquidity-aware, and I don't explicitly model the bid-ask spread per ticker. For names at the lower end of the universe, is constant-bps slippage realistic, or should I be modeling spread and market impact more carefully?
* **Validation beyond IS/OOS.** Four non-contiguous windows is better than one trailing OOS, but is there a stronger framework? Block bootstrap on the trade-level returns? Monte Carlo trade-sequence shuffling? Walk-forward with rolling re-tuning? I don't have an intuition for which of these actually adds confidence vs which is just more numbers.
* **Parameter sensitivity.** I haven't done a full systematic sweep over how each parameter affects the result. So my confidence in any specific point estimate is fuzzy. Is there a standard practice for parameter-sensitivity reporting in backtest writeups?
* **Norgate adjustment policy.** I'm using "Capital Reconstructions" (splits and certain corporate actions, no special distributions, to match my live Alpaca feed's split-only adjustment). Is that the right choice for this kind of multi-year portfolio backtest, or should I be using the dividend-adjusted series?
* **Universe construction.** My PIT universe is rules-based (dollar volume, trailing return, sticky window) but the parameters in those rules are themselves choices. Is there a way to validate that the universe construction isn't quietly overfit to the post-2015 regime?
If there's something obvious I'm not doing, please call it out. Pointers to papers, tools, or example writeups that have stronger backtest-validation methodology than what I've put together here are also welcome.
Thanks!

.dark-mode .Wg3xsT{ color: white; background-color: #333333; border: 2px solid white; }.light-mode .Wg3xsT{ color: #333333; background-color: white; border: 2px solid #333333; }SPYSPDR S&P 500 ETF Truststock NYSE ETF

SPY
SPDR S&P 500 ETF Trust
stock NYSE ETF