+- Gekko Forum (https://forum.gekko.wizb.it)
+-- Forum: Gekko (https://forum.gekko.wizb.it/forum-13.html)
+--- Forum: Third Party Software (https://forum.gekko.wizb.it/forum-18.html)
+--- Thread: BacktestTool - batch backtest, import and strategy parameters optimalization (/thread-1670.html)
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - whoodat - 05-18-2018
When I import data using the CLI where does it save the incoming imported data?
**Disregard**
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - Kris191 - 05-18-2018
(05-18-2018, 06:26 AM)xFFFFF Wrote: I was noticed by susitronix with this issue. I will try on my Windows fix it. Temporary You can disable printing analysis and TOP * tables in config file. Brute force work now? I forgot reply on Your message.
I created develop branch on my Github repo. So new feature will be available faster in this branch but can be unstable. Today I added import via CryptoCompare API so bittrex , zaif and others.
Thank you seems to have fixes it from crashing i had to leave it running and will feedback later once i get home to check if it completed.
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - xFFFFF - 05-18-2018
(05-18-2018, 02:21 PM)whoodat Wrote: When I import data using the CLI where does it save the incoming imported data?To SQLite DB, folder: history, version: 0.1. But if You dont use --from last paramater You can change SQLite to mongoDB or postgre in backtest.pl. Fully support for postgreSQL soon.
**Disregard**
@Kris I will recompile binaries soon. Open source version should work fine now.
In Windows 7 I need change cmd.exe window size. If someone dont know can do it here:
![[Image: ximage30.png.pagespeed.gp+jp+jw+pj+ws+js...dTph9N.png]](https://www.howtogeek.com/wp-content/uploads/2008/11/ximage30.png.pagespeed.gp+jp+jw+pj+ws+js+rj+rp+rw+ri+cp+md.ic.l1AwdTph9N.png)
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - xFFFFF - 05-19-2018
Today I finally add fixed binaries for Windows: https://github.com/xFFFFF/Gekko-BacktestTool/releases/download/v0.7/Gekko-BacktestTool-v0.7-Win7-amd64-fixed.zip
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - whoodat - 05-20-2018
Hello xFFFFF or anyone who can help,
I am behind the power curve finally got all of gekko running properly on a headless server and just now trying to get the backtest tool to work. I tried to follow the steps as closely as possible and now when I got to import I am receiving the following error, any help is greatly appreciated.
perl backtest.pl -i -p gdax:USD:ETH -f last -t now
Can't locate DBD/CSV.pm in @INC (you may need to install the DBD::CSV module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at backtest.pl line 21.
BEGIN failed--compilation aborted at backtest.pl line 21.
** I am following the "you may need to install for each missing piece) not sure I must have messed up the cpan install or something**
Update: When I run it now I get this error: Can't locate LWP/Protocol/https.pm in @INC (you may need to install the LWP:
rotocol::https module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at backtest.pl line 28.BEGIN failed--compilation aborted at backtest.pl line 28.
--When I try to do a sudo cpan install LWP:
rotocol::https I see a ton of this error t/startssl.t ...................... Can't locate Net/SSLeay.pm in @INC (you may need to install the Net::SSLeay module) (@INC contains: /home/whoodat/.cpan/build/IO-Socket-SSL-2.056-2/blib/lib /home/whoodat/.cpan/build/IO-Socket-SSL-2.056-2/blib/arch /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at t/startssl.t line 5.
BEGIN failed--compilation aborted at t/startssl.t line 5.
t/startssl.t ...................... Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
t/sysread_write.t ................. Can't locate Net/SSLeay.pm in @INC (you may need to install the Net::SSLeay module) (@INC contains: /home/whoodat/.cpan/build/IO-Socket-SSL-2.056-2/blib/lib /home/whoodat/.cpan/build/IO-Socket-SSL-2.056-2/blib/arch /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at t/sysread_write.t line 11.
BEGIN failed--compilation aborted at t/sysread_write.t line 11.
t/sysread_write.t ................. Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
t/verify_fingerprint.t ............ Can't locate Net/SSLeay.pm in @INC (you may need to install the Net::SSLeay module) (@INC contains: /home/whoodat/.cpan/build/IO-Socket-SSL-2.056-2/blib/lib /home/whoodat/.cpan/build/IO-Socket-SSL-2.056-2/blib/arch /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /home/whoodat/.cpan/build/IO-Socket-SSL-2.056-2/blib/lib/IO/Socket/SSL.pm line 19
I try to install Net::SSLeay module it appears to install but every time I run the perl script I continue to get the same error of you may need to install the LWP:
rotocol::https module. At this point I am lost. Any help is appreciated.The smiley faces should be LWP : : Protocol
Not sure if this will be asked but I'll put it out there, I am running:
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04 LTS
Release: 18.04
Codename: bionic
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - whoodat - 05-20-2018
Sorry another question, didn't want to kep tacking on to the last. Are these 2 separate steps?
Binaries: Easiest install way
Download latest version from repository's releases.
Extract downloaded zip.
Copy all extrated files to main Gekko's directory.
Open Source: Debian, Ubuntu, Linux Mint
Clone git https://github.com/xFFFFF/Gekko-BacktestTool
Copy files to Gekko's main directory
Install dependies:
$ sudo cpan install Parallel::ForkManager Time::Elapsed Getopt::Long List::MoreUtils File::chdir Statistics::Basic DBI DBD::SQLite JSON::XS TOML File::Basename File::Find::Wanted Template LWP::UserAgent LWP:
rotocol::https Set::CrossProduct DBD::CSV Text::Table File::CopyIf so, how do I get the binaries onto a server, or does this happen when I do the clone git in the second step. thanks
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - whoodat - 05-20-2018
Sweet I got more to work, slowly but surely. All I know is the base instructions are missing a good bit imo but I'm also very green. Now I am encountering this error again any help is appreciated.
Import 1 pairs: gdax:USD:ETH.
USD:ETH (2018-01-17 15:06:00 - 2018-05-20 23:23:56) is started at 2018-05-20 23:23:56...
events.js:183
throw er; // Unhandled 'error' event
^
Error: SQLITE_BUSY: database is locked
Importing gdax USD-ETH is failed:
Had 1 errors on 1 jobs. Elapsed time: zero seconds
[2018-05-20 23:23:56] Collect data from coinmarketcap.com
[2018-05-20 23:23:57] Use TOML files for strategies configuration
[2018-05-20 23:23:57] Save output to csvfile: database.csv
[2018-05-20 23:23:57] Starting Backtest Machine (5 threads)...
============================================================== I N P U T D A T A ===============================================================
Pairs
-----
gdax:USD:ETH
Total: 1
===================================================================================================================================================
Strategies
----------
RSI_BULL_BEAR_RSI_ADX
Total: 1
===================================================================================================================================================
Candles
-------
5:295 5:300 10:295 10:300
Total: 4
===================================================================================================================================================
[2018-05-20 23:23:57] 4 backtests remain
============================================================= L A S T R E S U L T S ============================================================
Pair Strategy Profit Profit-market Trades/day Win trades Best win Worst loss Candle Days Time
---- -------- ------ ------------- ---------- ---------- -------- ---------- ------ ---- ----
===================================================================================================================================================
[2018-05-20 23:23:57] 100% complete. Elapsed: 00:00:00, average backtest duration: 00:00
[2018-05-20 23:23:57] Creating ALL RESULTS table (sorted profit/day)...
================================================== A L L R E S U L T S ===================================================
Curr. Asset Strat Profit Profit-market Profit/day Trades/day Best win Worst loss Avg.Hodl Avg.price Ov.trades/day Volume/day
---- ----- ----- ------ ------------- ------------ ---------- -------- ---------- -------- --------- ------------- ---------
==============================================================================================================================
[2018-05-20 23:23:57] Creating TOP STRATEGY table...
================================================ T O P S T R A T E G Y =================================================
Strat Best % % profit > Best Worst Sum % Avg % % Avg win Best Avg trade Worst Avg trade Avg Avg
profitable market % P/L % P/L profit profit trades trade profit trade loss trades/day HODL min
----- ---- ---------- ---------- ----- ----- ----- ------ --------- ----- --------- ----- --------- ---------- ---------
==========================================================================================================================
[2018-05-20 23:23:57] Creating TOP DATASET table...
=========================================================== T O P D A T A S E T ============================================================
Pair Best % % profit > % Market Best Worst Sum % Avg % % Avg win Avg Avg Price CMC Current Current Days
profitable market change % P/L % P/L profit profit trades trades/day HODL min volatility Rank marketcap CMC volume
----- ---- ---------- ---------- -------- ----- ----- ----- ------ --------- ---------- --------- ---------- ---- --------- ---------- ----
===============================================================================================================================================
[2018-05-20 23:23:57] All jobs are done. Elapsed time: zero seconds
whoodat@whoodatserver:~/gekko$ USD-ETH Backtest is failed.
USD-ETH Backtest is failed.
The storable module was unable to store the child's data structure to the temp file "/tmp/UVxuY_pfgQ/Parallel-ForkManager-26979-26991.txt": can't create /tmp/UVxuY_pfgQ/Parallel-ForkManager-26979-26991.txt: No such file or directory at /usr/local/share/perl/5.26.1/Parallel/ForkManager.pm line 84.
The storable module was unable to store the child's data structure to the temp file "/tmp/UVxuY_pfgQ/Parallel-ForkManager-26979-26990.txt": can't create /tmp/UVxuY_pfgQ/Parallel-ForkManager-26979-26990.txt: No such file or directory at /usr/local/share/perl/5.26.1/Parallel/ForkManager.pm line 84.
USD-ETH Backtest is failed.
The storable module was unable to store the child's data structure to the temp file "/tmp/UVxuY_pfgQ/Parallel-ForkManager-26979-26993.txt": can't create /tmp/UVxuY_pfgQ/Parallel-ForkManager-26979-26993.txt: No such file or directory at /usr/local/share/perl/5.26.1/Parallel/ForkManager.pm line 84.
USD-ETH Backtest is failed.
The storable module was unable to store the child's data structure to the temp file "/tmp/UVxuY_pfgQ/Parallel-ForkManager-26979-26992.txt": can't create /tmp/UVxuY_pfgQ/Parallel-ForkManager-26979-26992.txt: No such file or directory at /usr/local/share/perl/5.26.1/Parallel/ForkManager.pm line 84.
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - whoodat - 05-20-2018
Ok Ok!!!! I know I've been talking to myself on this thread lol..but I think I have it working....if I'm importing is this all I will see until it finishes?
perl backtest.pl -i -p gdax:USD:ETH -f "2017-01-01" -t now
█████ ████
█████ ████
██ ██ █████ ████
██ █████████████ ██ █████ ████
█████████████████████ █████████████ █████████ █████ █████ ████ █████ █████████
█████████████████████ ███████████████ ██████████████ █████ █████ ████ ██████ ██████████████
█████████████████████ █████ █████ █████ █████ █████ █████ ████ ██████ █████ █████
█████████████████████████ █████ █████ █████ ████ █████ █████ ██████████ █████ █████
█████████████████████████ ████ █████ ██████████████████ █████████ ████████ ████ █████
██████ ██████ █████ ████ █████ ██████████████████ ███████████ ██████████ ████ █████
████ ███████ ████ █████ █████ █████ █████ █████ ████ ██████ █████ █████
███████████████████ ████████████████ ██████ █████ ██████ ████ █████ █████ █████
█████████████████ ██████████████ ██████████████ █████ ██████ ████ █████ ██████████████
█████████████ █████ ███████████ █████ █████ ████ █████ ████████
██████ ████
███████████████
█████████████
████████ ███ ███ ██ █████████ █████
███ ███ ████ ███ ███ ██ ██ ███ ██
████████ █ ██ ███████ ███ ███ ████████ ███████ ██████ █████████ ███ ███████ ███████ ██
████████ ███████ ██ ██████ ███ █████████ █████ ██ ███ ██ ███ ██ ███ ██
███ ███ ███ ███ ██ ███ ███ ███ ███ ███ ██ ███ ██ ███ ██ ███ ██
█████████ █████████ ██████ ███ ███ █████ ████████ ████████ ██████ ███ ███████ ███████ ██████
Gekko BacktestTool v0.7
https://github.com/xFFFFF/Gekko-BacktestTool
Import 1 pairs: gdax:USD:ETH.
USD:ETH (2017-01-01 - 2018-05-20 23:36:27) is started at 2018-05-20 23:36:27...
I'm used to seeing the Gekko bringing things in that is why I'm asking...thanks. If this is downloading, will it download the data into the history folder under gekko/history?
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - whoodat - 05-21-2018
When I run the actual backtest I am still receiving these errors:
2018-05-21 02:44:29 (WARN): TALIB indicators could not be loaded, they will be unavailable.
2018-05-21 02:44:29 (WARN): TULIP indicators could not be loaded, they will be unavailable.
2018-05-21 02:44:29 (ERROR): Failed to load indicator
2018-05-21 02:44:29 (ERROR): Failed to load indicator README
2018-05-21 02:44:29 (ERROR): Failed to load indicator STC
events.js:183
throw er; // Unhandled 'error' event
^
Error: SQLITE_BUSY: database is locked
RE: BacktestTool - batch backtest, import and strategy parameters optimalization - whoodat - 05-21-2018
I got it to this point and this is my new error...it keeps looping over and over with this error:
Deprecation warning: value provided is not in a recognized RFC2822 or ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions. Non RFC2822/ISO date formats are discouraged and will be removed in an upcoming major release. Please refer to http://momentjs.com/guides/£/warnings/js-date/ for more info.
Arguments:
[0] _isAMomentObject: true, _isUTC: true, _useUTC: true, _l: undefined, _i: now, _f: undefined, _strict: undefined, _locale: [object Object]
Error
at Function.createFromInputFallback (/home/whoodat/gekko/node_modules/moment/moment.js:320:98)
at configFromString (/home/whoodat/gekko/node_modules/moment/moment.js:2368:15)
at configFromInput (/home/whoodat/gekko/node_modules/moment/moment.js:2594:13)
at prepareConfig (/home/whoodat/gekko/node_modules/moment/moment.js:2577:13)
at createFromConfig (/home/whoodat/gekko/node_modules/moment/moment.js:2544:44)
at createLocalOrUTC (/home/whoodat/gekko/node_modules/moment/moment.js:2631:16)
at Function.createUTC [as utc] (/home/whoodat/gekko/node_modules/moment/moment.js:88:16)
at Object.<anonymous> (/home/whoodat/gekko/core/markets/backtest.js:12:17)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
** I was reading some of the prior thread is it correct in saying you can't import and run a backtest at the same time? **