Transcript
0:00 In today's video, I'm going to show you why most back tests are lying to you and how you can actually use AI to properly back test your trading strategies, helping you become a more profitable trader. We're not just going to use simple TradingView back tests since as you'll see later in the video, they're basically useless. Instead, we'll be training our own AI model with complex math logic completely with Claude code. This way, we'll have a proper back testing engine that can actually adapt to unseen market data to help you become a better trader and ultimately use the same systems as quants and hedge funds.
0:32 So, make sure to stay until the end of the video so you can see how you can build this out and test it out for yourself and make sure to subscribe if you're interested in more AI trading content. Now, like always, I'm not a financial advisor and this is not financial advice. I'm not guaranteeing any profits in this video. I'm just simply showing you how you can use the power of AI to improve your back testing and ultimately your trading strategies as a whole. So, why do normal back tests fail so badly? Well, it's from a trap called the curve fitting. When you test a strategy on years of historical data all at once, your bot is basically memorizing the past. It's like giving a student the answer key the night before an exam. They're probably going to get 100% but they don't actually know how to do the math. When you see people flexing these massive TradingView back test returns online, it's generally because they worked backwards to find the perfect cherry-picked settings on a very specific time window. So, to fix this, we're going to be building a walk forward analysis engine with Claude code. This is going to be a huge step up from the fake 300% back test returns you would typically see on TradingView. So, with this new engine, rather than letting the bot cheat, the walk forward analysis forces the model to train on a very small window of data. Then, it will lock its settings, then train on the next few months of completely unseen data. The back testing algo will basically tell us if our strategy will survive the real world since it's not working with all of the historical data all at once. Now, let me go ahead and show you how this back testing engine works. So, here on the left-hand side, we can set all the settings. So, first is the ticker and then we can also choose the start and end dates for when we want the back testing window to start and end. Then, below this, we can set the actual walk forward windows, which is the training window as well as the blind test window. So, the training window here, which is for 12 months or a year, is where the bot is allowed to basically look up the answers. So, during this one-year period, it optimizes thousands of different parameters to find the exact combination that produces the highest profit or sharp ratio. So, this is basically just what a normal TradingView back test does. But, because we're building our own engine, we want to go a step deeper and that's where we use this blind test window. So, once we find the perfect settings from the training window, the settings are permanently locked and then the bot is pushed into this blind test window. So, you can think of it like for 12 months, we're finding the perfect settings based on those 12 months of data. Then, once we do, we're going to run that same exact setting on 3 months or the next 3 months of blind data. Now, because this 3-month chunk is data that's never been seen before, if the strategy was just faking it right from the past 12 months, then it'll lose money here. If the strategy actually does have an edge, then it would also apply in the next 3 months. And then once this 3 months of testing is over, the entire engine walks forward and then repeats everything again, hence the walk forward analysis. Now, if that's a little confusing, this right-hand side basically explains everything once again. So, here with the problem of traditional back testing, it's basically that it uses all the historical data at once. So, if we were to run this back test from 2018 to 2024, it would look at all this data for the past 7 years and then find the best settings, which again is something we don't want to do because the point is just because you get the highest return in the past period does not mean that it's going to be the best strategy moving forward. So, what we're doing here instead is the same process of finding the best parameters, so like traditional back testing, but only on a one-year window and then using those same best parameters on 3 months of unseen data. So, down here, you can visually see what's happening.
4:08 Basically, from 2018 to 2024, we're going to group all those chunks of data into folds. So, the first year is going to be from January to December 2018. This is going to be the entire training window. And then for the next 3 months, so January, February, March of the next year, 2019, is where we're going to apply the best settings from the initial training window. And then all we're going to do is repeat this once again.
4:31 So, for the next 3 months, we're going to use the past 12 months of training data, then look at the next 3 months completely blind, apply the same parameters to see the true performance, and then rinse and repeat until we reach all the way to the end. Now, the strategy here we're using is pretty simple for the back test. I'm sure you've seen it a million times before. All this basically does is say that when RSI crosses above the entry and the price is above the 20-period SMA, then long. In traditional back testing, we would look at the entire data set from 2018 to 2024 and then based on this entire historical period, we're going to find the best RSI numbers and entry lines. And then you obviously think in your head and like, "Wow, this was, you know, a great strategy. I was able to dodge the 2020 crash and then also capitalize on the 2021 bull run." And you would think that it predicted the market but again, we just gave it an answer sheet and it found the best possible combination that doesn't actually apply to present day. But instead, with a walk forward analysis, we're going to find the best RSI numbers for this one year, then apply it blindly to the next 3 months, see the true performance, and then finally, what I've added here is a simple transaction cost with like slippage and exchange fees. So now, if we go ahead and click this run walk forward analysis, we can see that it's actually downloading the data and then testing each fold individually. So, we have 19 total folds from 2018 to 2024. So, just to summarize on the timeline, we have 19 folds. So, this was the start from January 1st, 2018 and as you can see, we first do the in-sample training and then the blind test.
6:04 In-sample training, blind test. So, 3 months all the way down to present day or 2024 in this case. Below this here, we can also see the best RSI parameters being chosen in each fold. So, fold one had an RSI period of 21, 7, 10, 21, etc. But below this is where the real magic happens and what truly differentiates this back testing engine from just a very standard TradingView one. So, if you look at these results, it's pretty insane, right? The difference on the retail back test, it's showing basically almost a 3x return and it's basically telling you that this RSI strategy is a gold mine. You would have made a ton of money. you have a sharp ratio over one and barely a 5-6% drawdown. But then if we look at the walk forward analysis, this strategy just completely falls apart. So, when it trains on data that it's never seen before, it's only returning 5%, which is obviously very, very different from 199%.
7:00 Furthermore, if we if we scroll down, we can see the return degradation. It's basically saying that 75% of this total return it was just completely fake and created by curve fitting. So, to sum it up, in the traditional back test here, it was basically looking at the entire 5 years of data, crunching millions of combinations to find the optimal RSI parameters and applied it, you know, historically to get this massive return. But if we were to run it live, it would completely fall apart. In this walk forward here, what we did is apply a little fold. So, each fold we tested on ideal parameters and then blind tested.
7:34 We repeated this 19 times to get the true performance and how it would have, you know, behaved assuming that it didn't have all the data beforehand. And then finally, I know I've hammered this through like a billion times but just to show you visually one last time, this blue chart is what the retail back test looked like and this orange chart was the real performance. So again, massive, massive discrepancy. So, basically, all this tells you is that this RSI strategy is fundamentally weak. Even if you were to optimize the past 12 months with the perfect RSI parameters, you still wouldn't be making any real money because RSI is a lagging indicator. And this is obviously huge because we just saved ourselves from an extremely bad strategy. Now, let me go ahead and show you how you can build this yourself so you can begin testing out your own strategies and potentially even find new ones to trade on the market. Now that you see why traditional back testing is basically useless, let me go ahead and show you how you can build this walk forward engine from scratch. And before we continue, if you want a more detailed guide and templates on how to back test with AI as well as improve trading performance with AI tools like Claude code, make sure to click the link in the description for my community as we have complete guides on how you can actually use AI to automate your trades and improve your performance. So, starting off, you don't need to know how to write Python code at all. We're basically going to have Claude code do all the heavy lifting for us. So, all you want to do is open up Visual Studio Code or VS Code. Then, once you're in Visual Studio Code, go ahead and click open and all you want to do is open up a blank folder and this is where all the code for your back testing engine is going to live at. So, if you don't have a dedicated folder, just click create new folder. Now that we're inside our blank folder where we're going to build out the back testing engine, the first thing we need to do is add Claude code to VS Code. So, all you need to do is go over to plugins right here and then search for Claude code. If it's not already installed, you can just go ahead and click install here and it should pop up sooner or later on this top bar right here. So, once you have Claude code installed, just go ahead and click open.
9:38 So, after opening up Claude code, if this is your first time, just going to ask you some setup questions, settings, and then for you to log into your account. And if you don't want to log in, you can also just use Claude code via API as well. But now, assuming all goes well, what we have is a completely blank Claude code screen where we can now begin to prompt what we want to build and in our case a very robust backtesting engine. So next what I have here is the prompt that we're actually going to use to build out our backtesting engine. So what we want is to build the entire Streamlit dashboard.
10:10 We also want the sliding window walk forward engine. So basically all the things I showed you at the beginning of the video as well as just a very basic RSI strategy that I'll show you how to tweak later on. So going through this one by one, I'm just saying build a professional walk forward trading app in Python using Streamlit, Plotly, and Yahoo Finance. For the asset you can change this to any ticker you want, but we obviously have spy from January 2018 to today. Then again, like I mentioned, the strategy will be that same simple RSI crossover strategy which we will upgrade later on.
10:41 For the backtester, once again, we want to prevent curve fitting. That's why we have this to be trained on a 12-month rolling window and then tested on the 3-month blind window. And then to evaluate the performance, we only want to stitch the out of sample returns together to get a true measure of how the strategy did. For risk management, we want to make sure that we're actually factoring in the transaction costs. So we have 0.1% exchange fee and slippage. Then finally for the UI, we want to show the progress for each of the folds. This way we know all of data that's being chunked together and what is actually happening on the back end.
11:17 And then just like the beginning of the video, I've added some charts and this is two more so visualize how this backtester works versus a more traditional method. Once that all looks good to go, just go ahead and click run and let Cloud Code basically do its magic. It should first install any dependencies that you'll need for the application. So in some cases, you'll need to install Python. And after waiting a few minutes, we can see it's gone ahead and built the app. So we can just open up this URL to see the progress. But in terms of the strategy, it's the same simple RSI crossover strategy with costs. We have the walk forward engine that rolls a 12-month training period with a 3-month testing window. And then again, only the out of sample returns are stitched together.
11:58 Then finally we have the dashboard that basically shows all of the progress in what we're testing in terms of the strategy and the returns. Now again, this is just a previous app, right? We have the same failed returns 5.8% versus almost a 3x if you did a traditional backtesting route. But let's go ahead and update the prompt so we get something a bit more tangible and useful rather than just flying on this lagging indicator for RSI crossings. Now if you're on TradingView, we would basically keep tweaking the RSI numbers until we saw green. But since we want to behave just like institutions and quants, if the underlying math is weak, then what we should do is just take it out and upgrade it. So that's exactly what we're doing here inside this new prompt. All I wrote was that the walk forward engine proved the retail RSI strategy fails on unseen data. So what we want to do is just completely rip out the RSI and SMAs and we're going to replace it with a volatility scaled momentum indicator. And this is going to be based on average true range instead and then we'll smooth out the price action using a two-pole Butterworth filter. So here I'm just saying build a volatility scaled momentum indicator using ATRs. We want to filter the price noise with this Butterworth filter. Now one small nuance, if you do want to use a similar strategy, is that typically when you work with AI coding agents, they love using this filtfilt rather than lfilter, which is what we want. Now this filtfilt function here does make the chart look perfectly smooth, but one issue is that it actually uses future price data. So it's basically just using future prices to calculate today's indicator, which is a massive look-ahead bias. So by using lfilter here instead, we're basically guaranteeing strictly casual forward-only math and this is just to help stop the AI from hallucinating fake returns.
13:46 So once this looks good, let's go ahead and run it and then see our updated backtesting engine on a completely new strategy. So we can see in the to-dos, it's basically replacing the RSI SMA logic with volatility scaled momentums. Then it's going to update our dashboard with labels, sidebars, and tables for the new strategy and then finally actually run the walk forward backtest on our completely new approach. So here you can literally see all of the changes in the code that the AI coding agent is making for us to reflect this new strategy that we want to implement. Now we can see what was built. I'm not going to go ahead and open up the dashboard again cuz it's going to show the same exact type of chart. Instead, I just have a text summary here. So again, what we did was rip out all of the RSI SMA logic and replaced it with more institutional grade logic instead. So here we have a two-pole Butterworth low-pass filter, average true range, and volatility scaled momentum. And now looking at the results, this is honestly probably the most important lesson. Once again, just really showcasing what I talked about in the beginning of the video. So if you looked at the in-sample illusion backtest, this had over a 1,500% return, which again would be massive. So if someone basically showed this strategy, it would seem like a guaranteed money maker. And that's because we are using a bit more of a legitimate strategy that is actually adopted by a lot of institutions. But once we go back to our reality, which is the walk forward analysis, we can see once again that the true return is not 1,500% rather 7% if it was testing on blind data. And as you can see, the Sharpe ratio did basically collapse. And this basically proves that you can curve fit all you want advanced institutional math as easily as you can curve fit like a basic lagging RSI indicator. Adding complex math does not magically create a profitable edge. And if you do truly want to find an edge in the market, it's going to take a lot of backtesting and true backtesting especially with something like a walk forward analysis versus just believing these numbers that people, you know, typically show with their returns. So the point here is that the markets evolve constantly. So by using tools like Cloud Code, you can actually create a meaningful backtest that isn't used to show, you know, a beautiful chart that just goes straight up. The goal is to ultimately find out if the strategy that you have will actually make you money in the markets.
16:05 Now if you found this video helpful, make sure to like, comment, and subscribe since I'll be covering everything AI trading and investing related. So I'll be going over all the latest tools like Cloud Code and how you can actually apply them to either your trading performance or your investments.