Backtesting crash

[Valleyman]

Hello,

When backtesting in the UI it sometimes occurs that it finishes without displaying results but when I look at the CLI the test is clearly still running. 
This used to happen mainly when working with very short candles (1-2-5 minutes) but it's been happening more recently. 

Can somebody give me some pointers on how to examine this further? 

PS: Thanks for the effort Mike!

Cheers!

[askmike]

If you can upload the output of the the terminal (or commandline) window and post the link here that would help

[Valleyman]

I have uploaded a txt file with Gekko's output on the following link:
https://nofile.io/f/ufEbAsfzgrK/gekkolog.txt

The strange thing is, while running it with output going straight to the logfile the following error did appear in the shell:

what(): std::bad_alloc

Error: non-error thrown: Child process has died.
at Object.onerror (/home/geoffreyd/gekko/node_modules/koa/lib/context.js:105:40)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7)

[askmike]

Can you share some information about both the computer you are using (how much memory and so on) as well as the backtest you were running (what strategy, what candle size and what dataset)?

[Valleyman]

It's running on a Debian VM on Xenserver, the VM has 16GB of RAM, 80GB of swap (thought it was a memory issue at first, so went overboard...) and it has 16 cpu cores.
I just made it happen with backtesting on Poloniex USDT/BTC data for the year 2016.
Candle size: 2 Minutes
Warmup period: 100 candles
PPO Strategy
short = 400
long = 800
signal = 9

[thresholds]
down = -0.025
up = 0.025
persistence = 9

If I increase the candle size to 20 minutes it completes like it should and I get my backtest result on the GUI.

[askmike]

The backtester has been quite stable for quite a while. I have a feeling this has to do with the fact that all debug output of the backtest is pumped via IPC over stdout of the server (the `gekko --ui` process).

Can I ask you to try editing this file: https://github.com/askmike/gekko/blob/de...log.js#L14

Can you try changing line 14 and 15 to the following:

Code:

var debug = false;
var silent = true;

and try again?

Thanks!

[tommiehansen]

I also have this problem with longer backtest periods (and have for some time). This seems to occur on both Windows (WSL bash) and Linux (Docker container).
None of the purposed optimizations work (but they do make a difference, not just on larger datasets).

System 1: Windows x64 + WSL (bash), 32 GB ram etc
System 2, server: Debian + Docker container, 128 GB ram etc

Dataset: ETH-USDT (Poloniex)
Candle Size: 15 min
Data from: 2015-08-08 06:15
Data to: 2017-11-07 14:37
Duration: 2 years, 3 months, 12 hours, 52 minutes

Other settings @ init() function:
// performance
config.backtest.batchSize = 1000; // increase performance
config.silent = true;
config.debug = false;

Other:
running gekko ui with:
node -max-old-space-size=14000 server.js
@ /gekko/web/

All other debugging set to off, so 0 messages except something that occurs on end().

Edit: 20 minutes works and takes 1.7350 minutes to complete, but as soon as i go lower...

---

Edit 2: Screen


Note how i get the 'Finish' in the terminal = the end() function for the strategy has been ran but for some reason finish() has not been triggered and thus no output @ UI?
Another thing that happens is that Gekko tries to re-run the backtest for no apparent reason, it's basically like it's stalling and then retries.
After 2-4 tries it just dies.

---

SOLVED!

Someone thought it was a good idea to set the timeout to 2 minutes @:
/gekko/web/vue/UIconfig.js

Why @askmike?

Solution
1. open /gekko/web/vue/UIconfig.js
2. change timeout 120000 (2 minutes!) to 21600000 (6 hours) + save
3. Restart Gekko UI

UIconfig.js

Code:

// This config is used in both the
// frontend as well as the web server.

// see https://github.com/askmike/gekko/blob/stable/docs/installing_gekko_on_a_server.md

const CONFIG = {
 headless: false,
 api: {
   host: '127.0.0.1',
   port: 3000,
   timeout: 7200000 // 120 minutes
 },
 ui: {
   ssl: false,
   host: 'localhost',
   port: 3000,
   path: '/'
 },
 adapter: 'sqlite'
}

if(typeof window === 'undefined')
 module.exports = CONFIG;
else
 window.CONFIG = CONFIG;

[Valleyman]

Thanks Mike! That solution worked for me :-)

[izzy0323]

Same here, no results after finishing on some backtesting. Changing the timeout to 6 hrs resolved it for me!!!!
Thanks.

[chico]

I changed timeout to 6 hours first and 2 hours, restarted the UI but i have the same problem.

I'm on Macbook Pro with OS X

Anything i can do to solve this problem?

Thank you