Written lesson
Lesson transcript
one one Every event from an outside perspective is unique. The behavioral signatures beneath them are ancient. When you study human history with honest neutrality, you're studying the one thing that does not change across time. How human beings respond to uncertainty, opportunity, and fear. This study is what we know as backtesting. A precise example of this concept through real life event would be…Paul Tudor Jones in nineteen eighty seven. His research director, Peter Borish, overlaid the nineteen twenty nine pre crash market trajectory onto the nineteen eighty seven market…and found what Jones later called a spooky similarity. Both periods showed parabolic run ups driven by optimism over fundamentals. They positioned with put options on equity indexes. When Black Monday hit on October nineteen nineteen eighty seven, the Dow dropped twenty two point sixty one percent in a single day. Tudor investment tripled their money. Here is what matters about this story for every systematic trader. Jones did not predict the crash. He studied the historical pattern, recognized structural similarity, and prepared with guardrails. They were looking for structural rhyme across history, and they size their positions to survive in case they were proven wrong. That is backtesting… operating at its highest level. A defined observation about recurring human behavior…applied to a current market condition…measured against the historical record…and backed by position sizing that assumed the possibility of being incorrect… Here are the chapters. First, what backtesting is, why it works, and how it fails. Second, the mirror, one ancient astronomical quest and another in most recent times by Ray Dalio in current financial market. Third, the protocol, a structured process for honest testing. My name is Ashim Nandi. I build decision system for uncertain conditions… Think in odds. Act with discipline. Chapter one, what backtesting is, how it works, and how it fails. Backtesting is the practice of applying a defined set of trading rules to historical price data and observing what would have happened. You take a strategy with specific conditions, when to enter, when to exit, how to size the position. You apply those conditions to a historical data set as if you're living through it in real time with no knowledge of what comes next. You record every trade, every win, every loss, every drawdown. The output is a measurement. How did this set of rules interact with this set of market conditions? From our expected value video, we established the formula. Win rate times average win minus loss rate times average loss. A backtest is where you generate those numbers. It is where the theoretical becomes empirical. The backtest does not tell you the future. It tells you whether your rules have demonstrated under rigorous conditions that they capture something truly genuine. There is a deeper question beneath the mechanics. A question most backtesting education never… reaches. Why would patterns from fifty years ago or five hundred years ago carry any information about markets today? The instruments are different. The technology is different. The speed of information flow is incomparably different. The outer architecture has transformed completely across generations. The inner architecture remains. Fear still operates the same way it operated in the cotton pits of the eighteen hundreds. Euphoria still overextends in the same sequence it followed during the tulip mania of sixteen thirty seven. Collective human behavior follows structural patterns across time because the consciousness driving it operates on the same principles. It always has. A backtest works because the record beneath the price data is a record of human response, and human response at the collective level holds constant across centuries even as every surface condition transforms. The outer events are unique. The behavioral signature beneath them are ancient. That is why historical data still carries information. There's a concept in systematic trading called overfitting. Some traders call it curve fitting. It is a single most important thing to understand about how bad testing fails because it is the failure… that feels like success. Here is what it looks like from the inside. You design a strategy. You test it on historical data. The results are disappointing. So you add a filter. The results improve. You adjust a parameter, further improvement. You add another condition, you tighten the stop, you keep adjusting until the equity curve is smooth, the drawdowns are small, and the returns are extraordinary… What has happened is that the strategy has memorized the specific noise of the particular dataset…instead of learning the behavioral pattern beneath it. Every data set contains two things, signal and noise. Signal is a recurring behavioral pattern. The structural tendency that persists across time because it is rooted in how consciousness responds to uncertainty. Noise is a random variation. The specific sequence of events that occurred once due to unique circumstances and will never repeat in the same configuration. Overfitting means the strategy has been shaped around the noise. In the backtest, this looks like exceptional performance. In live trading, it collapses because the noise it learned is gone. The specific random sequence it memorized does not exist in the future. Research has demonstrated that a strategy can show a sharp ratio of one point two in backtesting and drop to negative zero point two on data it has never seen. A strategy that appeared to generate consistent risk adjusted returns revealed itself as performing worse than random…when the specific noise it had been fitted to was removed. There are three versions of this problem you really need to understand. The first is obvious curve fitting. This happens when you run an optimization across hundreds or thousands of parameter combinations and then pick the one that performed the best. On the surface, it feels rigorous. It feels scientific. But the more knobs you tune, the more likely it is that you have simply tuned the system to historical noise instead of a real signal. When too many variables are optimized, you're often fitting the past… instead of preparing for the future… The second one is more subtle, implicit fitting. This one does not show up in your code. It shows up in your decisions. You choose momentum instead of mean reversion…because you already know momentum did well in that period. You select certain instruments because you have seen how they have behaved historically. Every time a design decision is influenced by information that would not have been available at the moment of the trade, you are quietly leaking future knowledge into the past. Researchers call this data snooping bias. One described it as the time machine problem. Ask yourself a simple question. Could this decision have been made without knowing what happened next? If the honest answer is no, then the backtest contains information leakage. The third is selection bias. You test ten variations of strategy. Nine performed poorly. One looks exceptional. So you discard the nine and trade the one. The problem is mathematical, not philosophical. If you test enough variations, one will eventually look extraordinary, purely by chance. That is probability at work. It is not h. The honest question every bat test must answer is simple. Are these results showing a real behavioral pattern in the record, or are they just the noise of a specific dataset… shaped by a process that was quietly looking for confirmation? This question has already been answered, and it was answered with extraordinary clarity…by two observers…separated by twenty seven centuries. One studied the sky, the other studied death. Both discovered the same thing. Chapter two, the mirror. Twenty seven hundred years ago, temple scribes in Babylon began recording the position of the moon every night on clay tablet. They documented eclipses…planetary movements, every visible celestial body, night after night, year after year for over seven hundred years. It is considered the longest continuous research program in recorded history. From that record, they extracted a rule. Every two hundred and twenty three lunar months, about eighteen years, eclipses reoccur. The moon returns to the same position relative to the sun and the earth. The pattern repeats. They called it the Saros Cycle. Then they did something that matters deeply for every systematic trader today. They applied it backward across centuries of observation and forward into dates that had not yet arrived. They predicted when eclipses should occur, then they watched the sky and compared prediction to reality. When the prediction was accurate, the rule stood. When it failed, the record overruled the theory. The rule was refined through contact with evidence. One astronomer was even arrested for an incorrect eclipse prediction that triggered an expensive ritual. Accuracy mattered. The record enforced accountability. Think about what this actually is. A rule extracted from historical data, applied systematically… measured against what occurred, refined when wrong, trusted when repeatedly right. That is a complete backtest…twenty seven centuries ago. Now let's move forward twenty seven centuries. The same architecture produced one of the most important investment outcomes of our lifetime. Ray Dalio, founder of Bridgewater, studied debt crisis across centuries. Forty eight major crises, multiple continents, multiple currencies, multiple political systems. What he found was structural. Debt crisis follow a recognizable behavioral sequence. Healthy growth becomes extrapolation. Extrapolation becomes leverage. Leverage becomes speculation…Speculation becomes a bubble, then tightening, then contraction, sometimes depression. The outer details changed every time. The inner behavioral sequence did not. Dalio turned that into a template, a framework derived from the historical record so that at any moment they could identify where they were in the cycle. When two thousand and eight arrived, Bridgewater was positioned accordingly. The template held. Now let's look at the common thread. The Babylonians studied seven hundred years of celestial movement and extracted a rule that predicted eclipses… Dalio studied centuries of death cycles and extracted a template that anticipated crisis. Both accumulated an honest historical record. Both observed recurring patterns. Both extracted rule. Both tested those rules against reality. Both held themselves accountable to the outcome. The surface environment transforms across time. The behavioral architecture beneath it persists. Chapter three, the protocol. Honest backtesting is structured. It requires discipline at every stage, and it requires active resistance to our natural desire for confirmation. The protocol has five components. First, define before you test. Write your complete strategy rules before you're looking at any data. Entry, exit, position sizing parameters. Everything defined in advance. This addresses overfitting at the root. If rules are shaped after seeing outcomes, the time machine has already been used. Measurement must exist before observation. Second, separate your data. Divide it into in sample and out of sample. Develop on one. Validate on the other. If performance collapses out of sample, the strategy learned the noise of the training period rather than the underlying pattern. Some practitioners reserve an additional holdout segment. Data untouched until the very end. A final gate before life capital. Third, walk forward through time. Optimize on past data. Test on the next unseen segment, record results, shift forward, repeat. This simulates reality. Parameters come from the past. Testing happens in the unknown. One strong backtest can be noise. Repeated out of sample consistency is signal. Fourth, stress the results. Run Monte Carlo simulations. Randomize straight order. See whether performance depends on sequence. Perform sensitivity analysis. Adjust parameters slightly. Robust systems tolerate variation. Fragile ones collapse. Test across markets. Behavioral edges tend to generalize. Data specific artifacts do not. Fifth, measure what matters. Expected value per trade. After costs, after slippage, after realistic execution, maximum drawdown, and whether your capital protocol survives it. Geometric growth rate, Expected return minus half the variance. Arithmetic profitability is not the same as compounding. Sample size matters. Thirty trades reveal very little. Three hundred begin to establish structure. So here's a summary for you. Define before testing, separate data, walk forward, stress results, measure expected value, drawdown, geometric growth, and statistical significance. This is honest testing. This is System R AI, and you are watching the first principles of trading series. We are building a complete systematic framework. If you want the full architecture, go through the playlist in order. Think… in odds. Act with discipline. See you in the next