The first step is to define buy and sell rules for your initial testing. The actual code for defining these rules depends on your specific system‐development package. Therefore, trading rules are described as generally as possible. The rules for an initial test may look like this:
- Buy at tomorrow’s opening price when today’s price crosses and closes above the 50‐day exponential moving average (EMA).
- Sell at tomorrow’s opening price when today’s price crosses and closes below the 50‐day EMA.
You can try to avoid these losing trades, and possibly improve your overall trading results, by filtering out trading‐range situations. One way to accomplish that goal is by changing the buy rule to read as follows: Buy at tomorrow’s open when the following conditions are true:
- Today’s closing price is above the 50‐day EMA.
- The stock crossed above the 50‐day EMA sometime during the last 5 days.
- Today’s 50‐day EMA is greater than the 50‐day EMA from 5 days ago.
You can find out whether other changes that you can make in your simple system can actually improve profitability. You may, for example, test different types of moving averages. Try, for example, a simple moving average (SMA) instead of an exponential moving average (EMA). Or you may want to try using different time frames for your moving average, such as 9‐day, 25‐day, or 100‐day moving averages.
Identifying system‐optimization pitfalls
Most system‐development and testing software comes equipped with a provision for system optimization, which allows you to fine‐tune the technical analysis tools used in your trading system. You can, for example, tell the system to find the time frame of the moving average that produces the highest profit for one stock and then ask it to do the same thing for a different stock. Some systems enable you to test this factor simultaneously for many stocks.Although this approach is alluring, using it is likely to cause you trouble. If you find, for example, that a 22‐day moving average works best for one stock, a 37‐day moving average works best for the next stock, and another stock performs best using a 74‐day moving average, you’re going to run into problems. The set of circumstances leading to these optimized results won’t likely repeat in precisely the same way again in the future. It’s almost guaranteed that whatever optimized parameters you may find for these moving averages won’t be the optimal choices when trading real capital.
This is a simple example of a problem that’s well known to scientists and economists who build mathematic models to forecast future events. It’s called curve fitting because you’re molding your model to fit the historical data. You can expend quite a bit of effort fine‐tuning a system to identify all the major trends and turning points in historical data for a particular stock, but that effort isn’t likely to result in future trading profits. In that case, your optimized system is more likely to cause a long string of losses rather than profits.Testing a long moving average and comparing the results to a short moving average is fine, and so is testing a few points in between a long moving average and a short moving average. As long as you use this exercise to understand why short moving averages work best for short‐term trades and why longer moving averages work better for traders with longer trading horizons, you’ll be fine. Otherwise, you’re probably moving into the realm of curve fitting and becoming frustrated with your actual trading results.
Testing with blind simulation
Blind simulation is a method for setting aside enough historical data so you can test your system‐optimization results and avoid the problem of curve fitting. For example, you may test data from 1990 through 1999 and thus exclude data from 2000 through the present. After you’ve developed a system that looks good enough for you to base your trades on, you can then test your system against the data that was excluded.If the system performs as well with the excluded data as it did with the original test data, you may have a system worth trading. If it fails, you obviously need to rethink your system.
Another approach is choosing your historical data with extreme care. You can expect trend‐following systems like a moving‐average system to perform well during long, powerful trends. If your stock had a strong run up during the long‐lasting 1990s bull market, that kind of price data can skew your results, magically making any trend‐following system appear profitable. Whether that success actually can be duplicated during a subsequent bull market, however, must first be thoroughly tested.
If the majority of your profits come from a single trade or only a small number of trades, the system probably won’t perform well when you begin trading real money. You may want to address this problem by excluding periods from your test data when your stock was doing exceptionally well or when the results of any trades were significantly more profitable than the average trade.
This technique is a valid approach to eliminating the extraordinary results arising from extraordinary situations in your historical data. Using it should give you a better idea of your system’s potential for generating real profits in the future.