#curvefitting

One of the advantages of expert advisors is the ability to evaluate their historical performance by “back testing” the strategies based on historical price data. Computers and back adjusted data make it possible to see what a system “would have performed” going back years in history.

The problem, of course, is that the system has been designed on this very same data. Because systems are designed on past data, they are often the victims of what we call “curve fitting”, making the ability to back test results one of the biggest disadvantages of trading systems also. Because the future may look nothing like the past in a particular market – the “fitting” of parameters onto the past “curve” of data may cause big problems on the future data curve, causing the system to be out of phase and potentially causing investors losses.

The easiest way to understand “curve fitting” is through a simple example. Imagine a system that buys or sells eur.usd on a breakout above or below the market high or low for the past X number of days. When testing the system on the past data, the testing may show $1,000 in profits when using a 10 day high/low, $2,000 in profits when using a 20 day high/low, and $5,000 when using a 30 day high/low.

If you were the developer, which value would you use in designing the system, 10, 20, or 30? Most people would use the 30 value, as it gives the highest profit. Now a developer will look at more than just profit, and test for lowest drawdown or most winning months, for example; but whatever your goal for the system, it is human nature to design a system whose parameters produce results as close as possible to those desired. The problem is, just because one parameter worked on the past data does not mean it will work on the future, unknown data. So how would a developer attempt to avoid such a problem?