[SHARE] GAB - Gekko Automated Backtests

[donkykong017]

1. Make sure you run a recent PHP version. A problem could be that you're running something old; goto your terminal and write php -v. Don't run versions prior to PHP 7.
2. "pretty recent" doesn't mean much, if you want to know your max check your cpu-usage when running 1, 2, 3.. threads.
3. The errors you're seeing is because it could not add the trades to the $trades array. Thus you also see 'division by zero'. I don't know how to solve that since it makes no logical sense since there should be values there. This could be because of old php version that does things differently.
4. No appending results to a date range would be terrible since different date ranges always yield different results and thus SET A shouldn't be compared to B since they would live in completely different universes backtesting-wise.

Btw -- if my answers seem a bit short it's simply because i got a lot of stuff to do.

thx again.

1. should not be the problem:

php -v
PHP 7.2.3-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Mar  6 2018 11:18:25) ( NTS )
Copyright © 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright © 1998-2018 Zend Technologies
    with Zend OPcache v7.2.3-1+ubuntu16.04.1+deb.sury.org+1, Copyright © 1999-2018, by Zend Technologies

2. i know its not a powerhouse. its a dell xps 13 i7 8550U CPU @ 1.80GHz x 8 16Gb Ram. it should cope with most of what i throw at it. have around 80% CPU with 4-5 backtests at the same time
3. ok no clue there lol
4. ok get it. seems legit if you say so 

dont worry man. you have been very helpful. thanks for that.

will try to test some further. 
btw do you know if resuming tests on the same exact set will append the results with tests with never before used settings. do you know what i mean. sry i cant seem to explain that in english 
cheers

edit1:

so one new developement. now i also got error output in gekko commandline.
i have not seen it before:

 xxx POST /api/backtest 500 242ms -

  Error: non-error thrown: Child process has died.
      at Object.onerror (/home/david/gekko/node_modules/koa/lib/context.js:105:40)
      at <anonymous>
      at process._tickCallback (internal/process/next_tick.js:188:7)

[tommiehansen]

1. Make sure you run a recent PHP version. A problem could be that you're running something old; goto your terminal and write php -v. Don't run versions prior to PHP 7.
2. "pretty recent" doesn't mean much, if you want to know your max check your cpu-usage when running 1, 2, 3.. threads.
3. The errors you're seeing is because it could not add the trades to the $trades array. Thus you also see 'division by zero'. I don't know how to solve that since it makes no logical sense since there should be values there. This could be because of old php version that does things differently.
4. No appending results to a date range would be terrible since different date ranges always yield different results and thus SET A shouldn't be compared to B since they would live in completely different universes backtesting-wise.

Btw -- if my answers seem a bit short it's simply because i got a lot of stuff to do.

thx again.

1. should not be the problem:

php -v
PHP 7.2.3-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Mar  6 2018 11:18:25) ( NTS )
Copyright © 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright © 1998-2018 Zend Technologies
    with Zend OPcache v7.2.3-1+ubuntu16.04.1+deb.sury.org+1, Copyright © 1999-2018, by Zend Technologies

2. i know its not a powerhouse. its a dell xps 13 i7 8550U CPU @ 1.80GHz x 8 16Gb Ram. it should cope with most of what i throw at it. have around 80% CPU with 4-5 backtests at the same time
3. ok no clue there lol
4. ok get it. seems legit if you say so 

dont worry man. you have been very helpful. thanks for that.

will try to test some further. 
btw do you know if resuming tests on the same exact set will append the results with tests with never before used settings. do you know what i mean. sry i cant seem to explain that in english 
cheers

edit1:

so one new developement. now i also got error output in gekko commandline.
i have not seen it before:

 xxx POST /api/backtest 500 242ms -

  Error: non-error thrown: Child process has died.
      at Object.onerror (/home/david/gekko/node_modules/koa/lib/context.js:105:40)
      at <anonymous>
      at process._tickCallback (internal/process/next_tick.js:188:7)

1. No, not a problem at all. You're basically on the last version which is great.
2. I would really not run over 3 threads with an i7 8550u (or any laptop) though. This could lead to what i was saying about it being a que created and do note that max timeout for runner.php (the thing that runs stuff) is 900 by default (which equals 15 minutes). If you start queing up backtests that take longer then that time you'll start getting errors.
3. -
4. -

Resuming
Yes, resuming will always append if the daterange has not changed (one has not imported new data etc). This is sort of a core feature due to the fact that one may not want to run all the runs one would want in a single run (because that could literally take over 30 hours). So just stop anything and run it at a later time if you need to to free up CPU-resources etc.

Gekko error
I don't know about this. Be sure that you have your UIconfig.js @ /gekko/web/vue/UIconfig.js set to a higher then default value though (default is like 2 minutes).


This is my UIconfig.js:

Code:

// This config is used in both the
// frontend as well as the web server.

// see https://github.com/askmike/gekko/blob/stable/docs/installing_gekko_on_a_server.md

const CONFIG = {
 headless: false,
 api: {
   host: '127.0.0.1',
   port: 3000,
   timeout: 7200000 // 120 minutes
 },
 ui: {
   ssl: false,
   host: 'localhost',
   port: 3000,
   path: '/'
 },
 adapter: 'sqlite'
}

if(typeof window === 'undefined')
 module.exports = CONFIG;
else
 window.CONFIG = CONFIG;

[donkykong017]

1. Make sure you run a recent PHP version. A problem could be that you're running something old; goto your terminal and write php -v. Don't run versions prior to PHP 7.
2. "pretty recent" doesn't mean much, if you want to know your max check your cpu-usage when running 1, 2, 3.. threads.
3. The errors you're seeing is because it could not add the trades to the $trades array. Thus you also see 'division by zero'. I don't know how to solve that since it makes no logical sense since there should be values there. This could be because of old php version that does things differently.
4. No appending results to a date range would be terrible since different date ranges always yield different results and thus SET A shouldn't be compared to B since they would live in completely different universes backtesting-wise.

Btw -- if my answers seem a bit short it's simply because i got a lot of stuff to do.

thx again.

1. should not be the problem:

php -v
PHP 7.2.3-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Mar  6 2018 11:18:25) ( NTS )
Copyright © 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright © 1998-2018 Zend Technologies
    with Zend OPcache v7.2.3-1+ubuntu16.04.1+deb.sury.org+1, Copyright © 1999-2018, by Zend Technologies

2. i know its not a powerhouse. its a dell xps 13 i7 8550U CPU @ 1.80GHz x 8 16Gb Ram. it should cope with most of what i throw at it. have around 80% CPU with 4-5 backtests at the same time
3. ok no clue there lol
4. ok get it. seems legit if you say so 

dont worry man. you have been very helpful. thanks for that.

will try to test some further. 
btw do you know if resuming tests on the same exact set will append the results with tests with never before used settings. do you know what i mean. sry i cant seem to explain that in english 
cheers

edit1:

so one new developement. now i also got error output in gekko commandline.
i have not seen it before:

 xxx POST /api/backtest 500 242ms -

  Error: non-error thrown: Child process has died.
      at Object.onerror (/home/david/gekko/node_modules/koa/lib/context.js:105:40)
      at <anonymous>
      at process._tickCallback (internal/process/next_tick.js:188:7)

1. No, not a problem at all. You're basically on the last version which is great.
2. I would really not run over 3 threads with an i7 8550u (or any laptop) though. This could lead to what i was saying about it being a que created and do note that max timeout for runner.php (the thing that runs stuff) is 900 by default (which equals 15 minutes). If you start queing up backtests that take longer then that time you'll start getting errors.
3. -
4. -

Resuming
Yes, resuming will always append if the daterange has not changed (one has not imported new data etc). This is sort of a core feature due to the fact that one may not want to run all the runs one would want in a single run (because that could literally take over 30 hours). So just stop anything and run it at a later time if you need to to free up CPU-resources etc.

Gekko error
I don't know about this. Be sure that you have your UIconfig.js @ /gekko/web/vue/UIconfig.js set to a higher then default value though (default is like 2 minutes).


This is my UIconfig.js:

Code:

// This config is used in both the
// frontend as well as the web server.

// see https://github.com/askmike/gekko/blob/stable/docs/installing_gekko_on_a_server.md

const CONFIG = {
 headless: false,
 api: {
   host: '127.0.0.1',
   port: 3000,
   timeout: 7200000 // 120 minutes
 },
 ui: {
   ssl: false,
   host: 'localhost',
   port: 3000,
   path: '/'
 },
 adapter: 'sqlite'
}

if(typeof window === 'undefined')
 module.exports = CONFIG;
else
 window.CONFIG = CONFIG;

thx again

1. done
2. ok well thats a bummer. will try with up to 3. it makes no sense to change the timeout, because if its queued it will not be faster anyways. on the other hand this should not be a problem with gekko itself as i can run 8 backtests at a time in parallel with other tools.
3.
4.

Resuming:
well thats awesome. just as i would have done it. 

Error. 
i found an issue on github that tries to solve that.
https://github.com/askmike/gekko/issues/2040
https://github.com/askmike/gekko/issues/1879

this could have something to do with the strat using tulib/talind. im using your "old" strat. will try updating to the new version with built in indicators.

Edit:
ok sry. it was obviously that. runs butter smooth now and 10 times faster than before. now i am just shaking my head over me.


btw i thought a little bit about reading the results of GAB and just to use the Strats with the best test results is not always the best way. as one should consider e.g sharpe ratio and other things.
wouldn't it be a good idea to somehow mix the listing of the best results that had good results (profit) and good sharpe ratio not just one of the two?


and i think it could also be a good idea to put in the slippage value as that allows you to get more realistic results to real trading.

Edit2:
to get the average from the best results i can only choose the best 99 tests. wouldn't it be good to have even more when we have more than 1000 or even 10000 tests done. what would you recommentd for average?


thanks
cheers 
DK

[tommiehansen]

1. done
2. ok well thats a bummer. will try with up to 3. it makes no sense to change the timeout, because if its queued it will not be faster anyways. on the other hand this should not be a problem with gekko itself as i can run 8 backtests at a time in parallel with other tools.
3.
4.

Resuming:
well thats awesome. just as i would have done it. 

Error. 
i found an issue on github that tries to solve that.
https://github.com/askmike/gekko/issues/2040
https://github.com/askmike/gekko/issues/1879

this could have something to do with the strat using tulib/talind. im using your "old" strat. will try updating to the new version with built in indicators.

Edit:
ok sry. it was obviously that. runs butter smooth now and 10 times faster than before. now i am just shaking my head over me.


btw i thought a little bit about reading the results of GAB and just to use the Strats with the best test results is not always the best way. as one should consider e.g sharpe ratio and other things.
wouldn't it be a good idea to somehow mix the listing of the best results that had good results (profit) and good sharpe ratio not just one of the two?


and i think it could also be a good idea to put in the slippage value as that allows you to get more realistic results to real trading.

Edit2:
to get the average from the best results i can only choose the best 99 tests. wouldn't it be good to have even more when we have more than 1000 or even 10000 tests done. what would you recommentd for average?


thanks
cheers 
DK

1. Ok
2. If you can run 8x with other tools you can run 8x with this, all the tools uses the same thing (the localhost:3000/api/ endpoint). GAB doesn't really use any resources in itself and when you run the only thing taking resources is the AJAX stuff... but that's basically 0.
I still would watch the que though. My older laptop using a 4720HQ starts queing stuff up after around 3 threads and your newer is only about +14% faster.
3. Yes, the new version is faster. Don't create new versions for the fun of it.   (except the last unpublished version that is more of a cleanup) 
-  
4. The 'best profit' is only one way to sort it. You got multiple ways of sorting the data like sorting it by Sharpe, Win% etc. Use the tool?


5. Slippage: Yes, more options would be good but i personally think the defaults for slippage is a good "worst case scenario" since the slippage and loss for each trade is usually qute a bit higher then at the less expensive exchanges.

Also adding slippade etc would make RUNS A incompatible with RUNS B since if you run with the same strategy, same params and same daterange but have different slippage these two will not be comparable any longer so each such change would create a new dataset.

6. Max 99: Yes, problem is that with many runs there will be a lot of junk runs; runs that simply wasn't that great anyway. I changed that now, new limit is 10,000. Do note that looping trough 10,000 of datapoints take longer time though since there will be more data to process etc.
How many results in the average depends, i find that 10 is usually a good value. It totally depends on how many successful runs one got in total.

[0mathew0]

Awesome work Tommie

EDIT: one little feature wish from my side: limit date/time-range for the chosen dataset (sometimes I may not want to have the dec bullrun in my backtest).

[tommiehansen]

Awesome work Tommie

EDIT: one little feature wish from my side: limit date/time-range for the chosen dataset (sometimes I may not want to have the dec bullrun in my backtest).

Yes, it's on the list! First things first though; need to fix all the things that currently are wrong or just plainly suck.

Do note however that setting it will cause GAB to create a new set of runs for that specific time period though since Date A <> Date B is not the same as Date C <> Date D and backtests results for e.g. 2+ years of data for a exchange + pair will differ wildly from using that same data and limiting it to only 2-3 months.

[tommiehansen]

A major update just went through. Do note that it's a breaking change which means your old result sets will get invalid.
But you can easily just save your old gab folder to somewhere else by copy-pasting it first and renaming it.

Read the info about it at https://github.com/tommiehansen/gab/

[askmike]

I am doing some updates to the backtest API (in 0.6), see this thread for more info: https://forum.gekko.wizb.it/thread-56603.html

[donkykong017]

1. done
2. ok well thats a bummer. will try with up to 3. it makes no sense to change the timeout, because if its queued it will not be faster anyways. on the other hand this should not be a problem with gekko itself as i can run 8 backtests at a time in parallel with other tools.
3.
4.

Resuming:
well thats awesome. just as i would have done it. 

Error. 
i found an issue on github that tries to solve that.
https://github.com/askmike/gekko/issues/2040
https://github.com/askmike/gekko/issues/1879

this could have something to do with the strat using tulib/talind. im using your "old" strat. will try updating to the new version with built in indicators.

Edit:
ok sry. it was obviously that. runs butter smooth now and 10 times faster than before. now i am just shaking my head over me.


btw i thought a little bit about reading the results of GAB and just to use the Strats with the best test results is not always the best way. as one should consider e.g sharpe ratio and other things.
wouldn't it be a good idea to somehow mix the listing of the best results that had good results (profit) and good sharpe ratio not just one of the two?


and i think it could also be a good idea to put in the slippage value as that allows you to get more realistic results to real trading.

Edit2:
to get the average from the best results i can only choose the best 99 tests. wouldn't it be good to have even more when we have more than 1000 or even 10000 tests done. what would you recommentd for average?


thanks
cheers 
DK

1. Ok
2. If you can run 8x with other tools you can run 8x with this, all the tools uses the same thing (the localhost:3000/api/ endpoint). GAB doesn't really use any resources in itself and when you run the only thing taking resources is the AJAX stuff... but that's basically 0.
I still would watch the que though. My older laptop using a 4720HQ starts queing stuff up after around 3 threads and your newer is only about +14% faster.
3. Yes, the new version is faster. Don't create new versions for the fun of it.   (except the last unpublished version that is more of a cleanup) 
-  
4. The 'best profit' is only one way to sort it. You got multiple ways of sorting the data like sorting it by Sharpe, Win% etc. Use the tool?


5. Slippage: Yes, more options would be good but i personally think the defaults for slippage is a good "worst case scenario" since the slippage and loss for each trade is usually qute a bit higher then at the less expensive exchanges.

Also adding slippade etc would make RUNS A incompatible with RUNS B since if you run with the same strategy, same params and same daterange but have different slippage these two will not be comparable any longer so each such change would create a new dataset.

6. Max 99: Yes, problem is that with many runs there will be a lot of junk runs; runs that simply wasn't that great anyway. I changed that now, new limit is 10,000. Do note that looping trough 10,000 of datapoints take longer time though since there will be more data to process etc.
How many results in the average depends, i find that 10 is usually a good value. It totally depends on how many successful runs one got in total.

 
2./3. yeah that seems to work. tried different strats and they all work. this definitely had something to do with the tulib talind indicators. but seems to be working great now.
4./6. yeah i got the sorting. i had some special cases where i tested other strats where i would like so take 2/3 values into account when sorting. but this should be solved when i can use more than 99 results. when i have 10000+ runs there should be lots more in consideration that could be very useful. instead with 99 there could be lots of results that never show e.g. when sorting with Strat profit there could be false positives (results with billions and trillions of profit) that i would like to avoid. a combination of Strat profit and sharpe could be a good idea.

5. ok got that. so is the default with backtesting the default from the UI (0.05)?

7. one thing that came to my mind with testing Strats that use Parameters in the 0.001 range: the average Strat parameters in the results view only shows whole numbers. more detailed views would be awesome there.

i will have to have a look at the new update though and will test there. 

thx for the update 
cheers 
DK

[tommiehansen]

I am doing some updates to the backtest API (in 0.6), see this thread for more info: https://forum.gekko.wizb.it/thread-56603.html

Thanks for the heads up!