Strategies that crash and burn - How to avoid overfitting and selection bias.

Today, you’re going to learn the most core lessons of Algorithmic Finance.

How to avoid building strategies destined to fail.

For this lesson, I’d like to re-introduce you to a strategy.

I talked about this last week in my roundup edition, going over strategies that trade less frequently than daily. In writing that, I dug in to the logic a little bit and got intrigued with it. “Hows it do so well with so little logic, and one logic condition?”

Well, after a couple hours of free time and some experimentation, I figured it out.

It only picks stuff that goes on absolutely face-melting runs.

Well, yeah, that’s the whole goal of what we’re trying to do, right? Pick the right stuff, make money. Easy peasy.

The problems begin to arise when you take a basket of assets you know performed well in the past and then set yourself on a path where you need that massive outperformance to continue. Now, luckily for anyone running the exact strategy linked above, its up roughly 150% since November of 2022. Let me present to you an alternative situation though.

You see 150% returns, it gives you good feelings. This is the intoxicating power of selection bias. You need to test to make sure you’re not getting drunk off past performance.

You take the framework and swap out the 2-3 things providing gains, (NVDA, AMD, and SOXX) to things that should perform just as well (TSLA, SOXL, and TQQQ) under the same circumstances. You see returns nothing like what you had before. Something is off.

In hindsight, we know NVDA has been on an Blockchain and AI fueled rally for… a long time now. Its going to stop eventually. Personally, I consider this to fall under overfitting, the assets you trade are a parameter that can be over-tuned. While there is an argument to be made about “Diworseification” as Peter Lynch puts it, that isn’t an excuse to only pick 5 assets that you just think will grow forever, and hope they do. If you want a strategy that focuses entirely on the top performing microchip stock, by all means, continue with the original. I prefer to sleep easier at night.

Okay, so you’ve successfully shit on the strategy. What now?

This is an excellent example of why I usually recommend against single stocks entirely, but something like this framework is likely the safest place to use them. I really, really like the absolute simplicity of the framework so we’re going to keep the bones and the theme of “limited assets and logic.” Lets see what we can make work.

Diversify Intelligently.

I present to you Pareto’s Portfolio. Named after Vilfredo Pareto, the Pareto Distribution is a power law that states roughly that when looking at anything, productivity of workers, land ownership in a country, fundraising sources, that 80% of productivity comes from 20% of sources. I love taking it and applying to the market, because it seems to work incredibly well in a broad range of situations. This is how I got to my rotator setup of 10 or 15 assets with the top 2 or 3 being picked at a time. 20% of the available holdings are outperforming the other 80% at any given time. Poor market breadth is the name of the game when we’re talking about the major indexes, lets capitalize on that logically.

We’ve kept most of the simplicity of the original, only adding a risk on/risk off switch to determine what kind of assets to be holding, based off of what the short to mid-term trend in bonds is. I modified the black swan catcher to simply remove the strategy from highly volatile periods by jumping out to BIL, my cash equivalent of choice. While I’m a fan of highly dangerous and degenerate things on occasion, I want to demonstrate stability and continuity with this strategy. Going all-in UVXY is pretty antithetical to that!

We’ve got a risk-on basket consisting of the 3 major equity indexes SPY, QQQ, and DIA. Yes, there is overlap between them, but in the interest of not trying to balance or tune the indexes themselves, we accept that an move on. We take the top 3 from the basket of 15 stocks, based on the 20 Day Moving Average of Returns.

Risk off moves the strategy to the same basket system, but this time focused on broad-basket Commodity ETF’s and a collection of Bonds. When controlling for the covid crash of 2020, the risk-off basket with SPY as the risk on section is actually highly competitive with SPY on its own, but much more protected to the downside.

So there’s the Pros, now what’s the cons?

Minor cons, in my opinion.
You’ll have to follow along with the reconstitution dates of the major indexes once a quarter to make sure your baskets are up to date. However, the Commodity and Bond sections can be mostly ignored unless an ETF shuts down.

As an upgrade to this strategy, you may consider adding a way to control for inverted bond yield curves. Without getting too far in to the weeds, there are periods before recessions where the risk off section will be selected despite it not being the absolute best time to run it (we’re currently in that). The baskets mitigate this somewhat by having inverse bonds, a wide array of different bond classes, and precious metal ETF’s in them. I also didn’t want to overcomplicate the logic.

So there we go, 3 strategies, 2 of which I recommend against using.

I hope you found this illustration insightful. One of the biggest questions that gets asked is about overfitting, and it can be hard to demonstrate clear examples sometimes. I wanted to put out a clear example for folks to look back on whenever the topic comes up.

Let me know in the comments below, or over on discord what topics you’d like to see discussed in the future!

Looking for a deeper dive?

Check out Garen’s in-depth course, or 1 on 1 coaching!

How about curated strategies?

Check out The Hard Road Premium for access to strategies that have been tested live!