[gekko 0.6] API change for running a backtest

[tommiehansen]

Could you validate what you're saying? The "..is fastest" without any sort of reasoning behind it is quite odd.


As i previously said i don't see how the simplest of SELECT queries would go faster since those do not really take any time even for datasets that nearly has 3 years worth of candles. As it is now the db's in themselves are not slow, it's the parsing and processing that is slow OR the drivers used by Gekko.

[thegamecat]

a 60 minute candle is a high / low of 60 minutes, not the 60th candle, or am I misunderstanding something?

[thegamecat]

To go back to the api change, I’ve gone back to my bulktester and added this:

"backtestResultExporter": {
"enabled": true,
"writeToDisk": false,
"data": {
"stratUpdates": false,
"roundtrips": true,
"stratCandles": true,
"stratCandleProps": [
"open"
],
"trades": true
}
},

Now the api request is failing. Are there any other differences in the gekkoConfig object from 0.5x?

[askmike]

0.5 also has a few changes a long the way. Can you turn on debug and post logs?

[thegamecat]

request:
Request {
domain: null,
_events: [Object],
_eventsCount: 5,
_maxListeners: undefined,
body: '{"gekkoConfig":{"debug":true,"strategyrefname":"zuki_all_v1-1531658375958.js","watch":{"exchange":"binance","currency":"USDT","asset":"ETH"},"backtest":{"daterange":{"from":"2018-06-18 18:17:00","to":"2018-06-26 20:45:00"},"batchSize":50},"zuki_all_v1":{"ichimoku":{"conversionPeriod":22,"basePeriod":17,"spanPeriod":66,"displacement":34,"keh":10,"dt":"0.009"},"logging":false,"macd":{"short":14,"long":23,"signal":6,"emalength":32},"stoploss":{"zuki_enable_stop_loss":true,"zuki_stoploss_above_buy_price":0.8759058679531262,"zuki_stoploss_below_buy_price":0.9138038983058749},"controller":{"ichimoku_short_use":1,"ichimoku_long_use":1,"ichimoku2_short_use":1,"ichimoku2_long_use":2,"macd_short_use":2,"macd_long_use":1}},"filewriter":{"enabled":false,"logpath":"/mnt/d/bash/backtester/strategy_logs/"},"paperTrader":{"slippage":0.05,"feeTaker":0.25,"feeMaker":0.15,"feeUsing":"maker","simulationBalance":{"asset":1,"currency":100},"reportRoundtrips":true,"enabled":true},"backtestResultExporter":{"enabled":true,"writeToDisk":false,"data":{"stratUpdates":false,"roundtrips":true,"stratCandles":false,"stratCandleProps":["open"],"trades":false}},"writer":{"enabled":false,"logpath":""},"tradingAdvisor":{"enabled":true,"method":"zuki_all_v1","candleSize":30,"historySize":50},"performanceAnalyzer":{"riskFreeReturn":2,"enabled":true},"valid":true}}',
headers: [Object],
timeout: 1200000,
method: 'POST',
readable: true,
writable: true,
explicitMethod: true,
_qs: [Object],
_auth: [Object],
_oauth: [Object],
_multipart: [Object],
_redirect: [Object],
_tunnel: [Object],
_rp_resolve: [Function],
_rp_reject: [Function],
_rp_promise: [Object],
_rp_callbackOrig: undefined,
callback: [Function],
_rp_options: [Object],
setHeader: [Function],
hasHeader: [Function],
getHeader: [Function],
removeHeader: [Function],
localAddress: undefined,
pool: {},
dests: [],
__isRequestRequest: true,
_callback: [Function: RP$callback],
uri: [Object],
proxy: null,
tunnel: false,
setHost: true,
originalCookieHeader: undefined,
_disableCookies: true,
_jar: undefined,
port: '3001',
host: 'localhost',
path: '/api/backtest',
_json: true,
httpModule: [Object],
agentClass: [Object],
agent: [Object],
_started: true,
href: 'http://localhost:3001/api/backtest',
req: [Object],
ntick: true,
timeoutTimer: null,
response: [Circular],
originalHost: 'localhost:3001',
originalHostHeaderName: 'host',
responseContent: [Circular],
_destdata: true,
_ended: true,
_callbackCalled: true },
toJSON: [Function: responseToJSON],
caseless: Caseless { dict: [Object] },
read: [Function],
body: 'Internal Server Error' } }
(node:2707) UnhandledPromiseRejectionWarning: StatusCodeError: 500 - "Internal Server Error"
at new StatusCodeError (/mnt/d/bash/backtester/node_modules/request-promise-core/lib/errors.js:32:15)
at Request.plumbing.callback (/mnt/d/bash/backtester/node_modules/request-promise-core/lib/plumbing.js:104:33)
at Request.RP$callback [as _callback] (/mnt/d/bash/backtester/node_modules/request-promise-core/lib/plumbing.js:46:31)
at Request.self.callback (/mnt/d/bash/backtester/node_modules/request/request.js:186:22)
at emitTwo (events.js:126:13)
at Request.emit (events.js:214:7)
at Request.<anonymous> (/mnt/d/bash/backtester/node_modules/request/request.js:1163:10)
at emitOne (events.js:116:13)
at Request.emit (events.js:211:7)
at IncomingMessage.<anonymous> (/mnt/d/bash/backtester/node_modules/request/request.js:1085:12)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9)
(node:2707) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:2707) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

[thegamecat]

I'm checking what's in "this" in Koa context.js (where the error is) - line 105 and I see this on the server side which seems a bit strange to me:

xxx POST /api/backtest 500 195ms -
{ request:
  { method: 'POST',
    url: '/api/backtest',
    header:
     { 'content-type': 'application/json',
       host: '127.0.0.1:3001',
       accept: 'application/json',
       'content-length': '1349',
       connection: 'close' } },
 response:
  { status: 404,
    message: 'Not Found',
    header:
     { 'access-control-allow-origin': '*',
       'access-control-allow-methods': 'GET,HEAD,PUT,POST,DELETE' } },
 app: { subdomainOffset: 2, proxy: false, env: 'development' },
 originalUrl: '/api/backtest',
 req: '<original node req>',
 res: '<original node res>',
 socket: '<original node socket>' }

 Error: non-error thrown: Child process has died.
     at Object.onerror (/mnt/d/bash/gekko/node_modules/koa/lib/context.js:106:40)
     at <anonymous>
     at process._tickCallback (internal/process/next_tick.js:188:7)

[thegamecat]

I get the same errors even if I copy paste your example from the docs:

{
"watch": {
"exchange": "poloniex",
"currency": "USDT",
"asset": "ETH"
},
"paperTrader": {
"feeMaker": 0.25,
"feeTaker": 0.25,
"feeUsing": "maker",
"slippage": 0.05,
"simulationBalance": {
"asset": 1,
"currency": 100
},
"reportRoundtrips": true,
"enabled": true
},
"tradingAdvisor": {
"enabled": true,
"method": "MACD",
"candleSize": 60,
"historySize": 10
},
"MACD": {
"short": 10,
"long": 21,
"signal": 9,
"thresholds": {
"down": -0.025,
"up": 0.025,
"persistence": 1
}
},
"backtest": {
"daterange": {
"from": "2016-06-01T11:57:00Z",
"to": "2016-11-13T14:57:00Z"
}
},
"backtestResultExporter": {
"enabled": true,
"writeToDisk": false,
"data": {
"stratUpdates": false,
"roundtrips": true,
"stratCandles": true,
"stratCandleProps": [
"open"
],
"trades": true
}
},
"performanceAnalyzer": {
"riskFreeReturn": 2,
"enabled": true
}
}

[askmike]

@thegamecat can you please put your code inside a code block? Pasting it messes up the formatting.

From your example config I can see that your paperTrader plugin does not have "enabled: true", causing Gekko to not enable it. If that doesn't fix it, please run Gekko over the CLI with debug enabled and post the output here.

Note that this would have also been in issue in 0.5.

[thegamecat]

Papertrader is set to true:

"paperTrader": {
"feeMaker": 0.25,
"feeTaker": 0.25,
"feeUsing": "maker",
"slippage": 0.05,
"simulationBalance": {
"asset": 1,
"currency": 100
},
"reportRoundtrips": true,
"enabled": true
},

[thegamecat]

It's enabled true. Have I missed something?

"paperTrader":{"slippage":0.05,"feeTaker":0.25,"feeMaker":0.15,"feeUsing":"maker","simulationBalance":{"asset":1,"currency":100},"reportRoundtrips":true,"enabled":true}