Don't over-optimize

[richard]

Aloha,

Great tool.  Thanks for sharing.

I built a bot about ten years ago to trade futures.  It did really well backtesting.  Not so good in the real market.  Too hard to get good fills.  Best I managed to do was break even.

Your Backtesting result report is a bit optimistic (and misleading.)  You really don't want to be chasing accumulated profit.  That will surely have you matching your strategy to your historical data.  There's lots more data you probably want to include in that report, but accumulated profit will lead you astray.

Thanks!

Richard

[askmike]

You are very right, I am building a new interface and I want to have a more heavy focus on more useful (risk focussed) metrics that look at stable & profitable returns. Note that this is more of an education problem than a technical one: We need to build something that people understand, and most people only really understand (and care about) raw profit.

[tommiehansen]

There's some really simple values that could be calculated but does offer a bit more to go on e.g. win%, avg. win/avg. lose etc:



Note how that backtest beat the market by ~430% but still has some quite obvious problems if one check the other statistics.

[richard]

Thanks, Tommie,

The items I like best are these:

Trades
Losing Trades
Winning Percentage Trades
Average Gain
Average Loss
Worst Loss
Trades Per Day

I think best trades should probably be subtracted and removed from the stats.  Better to assume they won't happen again for back testing purposes.  Not sure how many to delete.  All the overly big ones.

And strings of losing trades is good to show.  Largest number of losing trades in a row.  That's a possible indication of strategy failure if the number is too large

I also like to divide things into chunks.  If you back test a year then how did each week do?  Did you have a really bad or really good week (or month.)  If you had a stellar week then you might want to remove that from the stats too.  Things like that tend to add to wishful thinking, and poorer performing strats.

Mostly just thinking out loud.  Been a while since I've done this stuff.  I wrote up a bunch of stuff in C, including full charts.  Can't find where I put them.  :-(  Rats.

Good trading!