12-15-2018, 02:03 PM
The contest is over, here is a report of what happened: https://medium.com/@gekkoplus/results-of...0dd5e34f68
But definitely also watch the video of @crypto49er as it goes even more in depth.
@crypto49er
Woow awesome video man, I will share this in the Gekko telegram as well
I love your deep dive with additional ways of looking at the performance of a strategy!
Controversy: you are 100% right regarding alpha and profit. I realized this a few days into the contest. When I planned out the contest the markets were flat and nothing was happening, I thought at the time that using profit or alpha as a metric would both be fine. And profit is something everybody understands (and desires). As such I decided (in November) to use the profit metric to determine the winner. In the end I made the decision to NOT change the rules midway through the contest by switching the deciding metric to alpha, because that would be unfair as well.
I see this contest as a great experiment, I think in a lot of ways it was a great success but there are also some thing we can improve next time (such as not relying on profit).
About the potential miscalculations
My first guess would be: the strategies in the contest started out with 1 asset (in the case of "IWannaBeRich-Private" 1 BTC). Meaning that the actual loss of the first roundtrip is NOT sell price minus buy price. But sell price minus price at start of contest (which was higher, eg. more was lost than is shown in the roundtrip list). Since the first buy of the first roundtrip didn't actually buy anything since the bot already had 1 BTC and 0 asset (unless there was a sell before, but there appears to not be one here).
But I haven't done an extensive analysis here, some more context so we can all figure this out: Just before the contest I had to change some minor things in the Gekko backtest engine that was needed by Gekko Plus and how it is running these strategies. ALL changes are merged into Gekko on github (most changes are in this PR: https://github.com/askmike/gekko/pull/2667) and are now part of both the Gekko develop branch and the latest stable release.
All strategies found in the contest are run by the same Gekko you can find on Github. The output of the backtests can be found as JSON files here: https://app.gekkoplus.com/data/bots/IWan...ivate.json (or replace "IWannaBeRich-Private" with another bot you want to see exact numbers of). Those files are raw outputs of a Gekko backtest, that means that if some numbers don't add up correctly this is a bug in Gekko as well.
I will go through these files and numbers myself as well very soon! While I can't give you access to the strategies submitted by other users, you should be able to reproduce the same strange behaviour by running a backtest of any strategy with stock Gekko.
But definitely also watch the video of @crypto49er as it goes even more in depth.
@crypto49er
Woow awesome video man, I will share this in the Gekko telegram as well
I love your deep dive with additional ways of looking at the performance of a strategy!
Controversy: you are 100% right regarding alpha and profit. I realized this a few days into the contest. When I planned out the contest the markets were flat and nothing was happening, I thought at the time that using profit or alpha as a metric would both be fine. And profit is something everybody understands (and desires). As such I decided (in November) to use the profit metric to determine the winner. In the end I made the decision to NOT change the rules midway through the contest by switching the deciding metric to alpha, because that would be unfair as well.
I see this contest as a great experiment, I think in a lot of ways it was a great success but there are also some thing we can improve next time (such as not relying on profit).
About the potential miscalculations
My first guess would be: the strategies in the contest started out with 1 asset (in the case of "IWannaBeRich-Private" 1 BTC). Meaning that the actual loss of the first roundtrip is NOT sell price minus buy price. But sell price minus price at start of contest (which was higher, eg. more was lost than is shown in the roundtrip list). Since the first buy of the first roundtrip didn't actually buy anything since the bot already had 1 BTC and 0 asset (unless there was a sell before, but there appears to not be one here).
But I haven't done an extensive analysis here, some more context so we can all figure this out: Just before the contest I had to change some minor things in the Gekko backtest engine that was needed by Gekko Plus and how it is running these strategies. ALL changes are merged into Gekko on github (most changes are in this PR: https://github.com/askmike/gekko/pull/2667) and are now part of both the Gekko develop branch and the latest stable release.
All strategies found in the contest are run by the same Gekko you can find on Github. The output of the backtests can be found as JSON files here: https://app.gekkoplus.com/data/bots/IWan...ivate.json (or replace "IWannaBeRich-Private" with another bot you want to see exact numbers of). Those files are raw outputs of a Gekko backtest, that means that if some numbers don't add up correctly this is a bug in Gekko as well.
I will go through these files and numbers myself as well very soon! While I can't give you access to the strategies submitted by other users, you should be able to reproduce the same strange behaviour by running a backtest of any strategy with stock Gekko.