02-06-2018, 09:46 AM
(This post was last modified: 02-06-2018, 10:01 AM by tommiehansen.)
- logging: Yes, disabling logging increases performance but not that much. From ~9s to ~7.8s using the same data. I use a simple this.debug true/false for my own stuff to enable or disable debugging messages since one doesn't seem to be able to control the default debugging state within strategies.
-io: By default Gekko also seem to use journal mode FILE (instead of RAM), syncronous FULL etc which isn't the best settings for faster READ, see e.g:
https://blog.devart.com/increasing-sqlit...mance.html
WRITE is another thing, the setting(s) should be dynamic depending on write or read.
Also -- just getting all the data for over 1 years of data only takes around ~200ms (or ~450ms if order by id desc) when querying the db manually (with bad settings). Just doing a large READ doesn't take that long even if number of rows is over 1 million. One would also ofc get all the data in a single query and not query the db all the time. With that in mind i really don't see how SQLite could be a culprint during a backtest since we only got (or should have) a single and simple SELECT?
simple perf check:
var strat = {
init: function(){
this.startTime = new Date();
},
end: function(){
let seconds = ((new Date()- this.startTime)/1000),
minutes = seconds/60,
str;
minutes < 1 ? str = seconds + ' seconds' : str = minutes + ' minutes';
log.debug('Finished in ' + str);
}
}
-io: By default Gekko also seem to use journal mode FILE (instead of RAM), syncronous FULL etc which isn't the best settings for faster READ, see e.g:
https://blog.devart.com/increasing-sqlit...mance.html
WRITE is another thing, the setting(s) should be dynamic depending on write or read.
Also -- just getting all the data for over 1 years of data only takes around ~200ms (or ~450ms if order by id desc) when querying the db manually (with bad settings). Just doing a large READ doesn't take that long even if number of rows is over 1 million. One would also ofc get all the data in a single query and not query the db all the time. With that in mind i really don't see how SQLite could be a culprint during a backtest since we only got (or should have) a single and simple SELECT?
simple perf check:
var strat = {
init: function(){
this.startTime = new Date();
},
end: function(){
let seconds = ((new Date()- this.startTime)/1000),
minutes = seconds/60,
str;
minutes < 1 ? str = seconds + ' seconds' : str = minutes + ' minutes';
log.debug('Finished in ' + str);
}
}