The Average Strategy - Settings is just an average of the top 10 settings, right? If we backtest it does not correspond to the profit mentioned at Average Strategy - Meta (That probably is an average of the top 10 profits). So, whats is the method to not have an overfitted strategy?
Does this Profit calculation already contemplates the Market Performance? Profit = GrossProfit + MarketPerformance or Profit = GrossProfit?
(04-06-2018, 03:20 AM)Ualas Wrote: The Average Strategy - Settings is just an average of the top 10 settings, right? If we backtest it does not correspond to the profit mentioned at Average Strategy - Meta (That probably is an average of the top 10 profits). So, whats is the method to not have an overfitted strategy?
Does this Profit calculation already contemplates the Market Performance? Profit = GrossProfit + MarketPerformance or Profit = GrossProfit?
as far as i understand no. the profit of the results is without market performance. this is Alpha. profit-market.
only in the runlog it shows profit-market.
@tommiehansen
thx for the new update. now everything works just perfect. there where just a few bugs though that came up for me.
1. the average parameters don't show 1000s characters when the number goes into the 1000s
2. the sorting in the Top runs is screwed up sometimes. that happened to me with all different columns sometimes. it also seems to be something with special characters or 1000s numbers.
(see pix)
04-07-2018, 03:21 PM (This post was last modified: 04-07-2018, 04:21 PM by tommiehansen.)
(04-07-2018, 07:48 AM)donkykong017 Wrote:
(04-06-2018, 03:20 AM)Ualas Wrote: The Average Strategy - Settings is just an average of the top 10 settings, right? If we backtest it does not correspond to the profit mentioned at Average Strategy - Meta (That probably is an average of the top 10 profits). So, whats is the method to not have an overfitted strategy?
Does this Profit calculation already contemplates the Market Performance? Profit = GrossProfit + MarketPerformance or Profit = GrossProfit?
as far as i understand no. the profit of the results is without market performance. this is Alpha. profit-market.
only in the runlog it shows profit-market.
@tommiehansen
thx for the new update. now everything works just perfect. there where just a few bugs though that came up for me.
1. the average parameters don't show 1000s characters when the number goes into the 1000s
2. the sorting in the Top runs is screwed up sometimes. that happened to me with all different columns sometimes. it also seems to be something with special characters or 1000s numbers.
(see pix)
No, the 'profit' in the box is the total profit, not alpha.
Ualas: Yes.
No, the average strategy settings is also just an average. It isn't like you can run 'average strategy' and get 'average profit' out of it.
These two basically live in two different universes but both are averages of something (in this case Top 10 most profitable runs since this is the default sorting).
I get the confusion though.
See 'Generated settings' more as a guide to what ranges usually works better. If it e.g. say 'SMA = 200' after 5000+ runs it stands to reason that using 'SMA = 1000' instead of 'SMA = 100', 'SMA = 200' or 'SMA = 300' will most likely be a lot worse since we now got a good statistical average that tells us this.
There's also a reason one can generate new settings out of the average settings (and a reason that it say one should modify those settings slighly since often it scews up some values).
The method not to overfit is in the hands of the user. Being able to see things such as averages could be useful though. Other values such as win% is also very useful. How you specifically use or abuse the tool is up to you though.
There is some use of the word "best" in the 'Viewer' that needs to change though since the word 'Best' isn't really that suited since 'best profit' really does not mean that it will be the best strategy params to run with live since the 'best profit' could have some other values that simply suck such as a win% under 50% and stuff like that. That's also the reason you can globally sort the entire list by things such as Win% and Sharpe instead of % Profit.
-
donkykong017:
Thank you for submitting bugs, it is very helpful and appreciated.
But -- if you don't mind, could you in the future add new issues at Github instead?
It makes it easier to track and it's a much more suited environment for discussing specific issues.
I added both these issues now though: https://github.com/tommiehansen/gab/issues
1. Yes, this is a bug
2. Yes, this is a known bug and the lib used will get thrown out.
There just haven been enough time to work on that part since most libraries used for this doesn't really fit (i could explain why, but time...)
(04-06-2018, 03:20 AM)Ualas Wrote: The Average Strategy - Settings is just an average of the top 10 settings, right? If we backtest it does not correspond to the profit mentioned at Average Strategy - Meta (That probably is an average of the top 10 profits). So, whats is the method to not have an overfitted strategy?
Does this Profit calculation already contemplates the Market Performance? Profit = GrossProfit + MarketPerformance or Profit = GrossProfit?
as far as i understand no. the profit of the results is without market performance. this is Alpha. profit-market.
only in the runlog it shows profit-market.
@tommiehansen
thx for the new update. now everything works just perfect. there where just a few bugs though that came up for me.
1. the average parameters don't show 1000s characters when the number goes into the 1000s
2. the sorting in the Top runs is screwed up sometimes. that happened to me with all different columns sometimes. it also seems to be something with special characters or 1000s numbers.
(see pix)
No, the 'profit' in the box is the total profit, not alpha.
Ualas: Yes.
No, the average strategy settings is also just an average. It isn't like you can run 'average strategy' and get 'average profit' out of it.
These two basically live in two different universes but both are averages of something (in this case Top 10 most profitable runs since this is the default sorting).
I get the confusion though.
See 'Generated settings' more as a guide to what ranges usually works better. If it e.g. say 'SMA = 200' after 5000+ runs it stands to reason that using 'SMA = 1000' instead of 'SMA = 100', 'SMA = 200' or 'SMA = 300' will most likely be a lot worse since we now got a good statistical average that tells us this.
There's also a reason one can generate new settings out of the average settings (and a reason that it say one should modify those settings slighly since often it scews up some values).
The method not to overfit is in the hands of the user. Being able to see things such as averages could be useful though. Other values such as win% is also very useful. How you specifically use or abuse the tool is up to you though.
There is some use of the word "best" in the 'Viewer' that needs to change though since the word 'Best' isn't really that suited since 'best profit' really does not mean that it will be the best strategy params to run with live since the 'best profit' could have some other values that simply suck such as a win% under 50% and stuff like that. That's also the reason you can globally sort the entire list by things such as Win% and Sharpe instead of % Profit.
-
donkykong017:
Thank you for submitting bugs, it is very helpful and appreciated.
But -- if you don't mind, could you in the future add new issues at Github instead?
It makes it easier to track and it's a much more suited environment for discussing specific issues.
I added both these issues now though: https://github.com/tommiehansen/gab/issues
1. Yes, this is a bug
2. Yes, this is a known bug and the lib used will get thrown out.
There just haven been enough time to work on that part since most libraries used for this doesn't really fit (i could explain why, but time...)
yeah sure man of course. thx for the tool that is very helpful.
everything works like a charm now.
thx for fixing it and i will submit bugs on github the next time.
I got it successfully installed. However, after I ran a test, I got an exception error shown in the log:
INFO Exhausted all possible combinations -- stopping automatically. 19:00:50
Fatal error: Uncaught Exception: Unknown primative for `'no'` on line 2. in /var/www/html/gab/system/class.toml.php:176
Stack trace:
#0 /var/www/html/gab/system/class.toml.php(121): Toml\Parser->parseValue(''no'')
#1 /var/www/html/gab/system/class.toml.php(78): Toml\Parser->processLine('buyImmediately ...')
#2 /var/www/html/gab/system/class.toml.php(24): Toml\Parser->parse()
#3 /var/www/html/gab/system/class.gab.php(178): Toml\Parser::fromString('buyImmediately ...')
#4 /var/www/html/gab/system/class.gab.php(67): GAB\core->parse_toml('buyImmediately ...')
#5 /var/www/html/gab/system/runner.php(39): GAB\core->get_strategies()
#6 {main}
thrown in /var/www/html/gab/system/class.toml.php on line 176
I am using php 7.2.4 and running the test from UI. Do you know what could be the root cause? Thanks!
Hi,
The root cause may be because of the mixed input that causes the thing to not really know what type of parsing to do.
Basically there are two modes: multi-dimensional array and plain. There should be a line-per-line check for this, but it may not be a 100%
since i haven't had time to really test all variations of everything yet.
Could you try running something that isn't structured in such a way and see if you get the same errors?
Basically something that does not mix [HEADER] with plain type(s) as per your example.
Another thing you can do is run in DEBUG-mode to see if the params changes the way you have set it up to do.
That will also clearly say if it's the POST-function that fucks up or if there is something else.
(04-06-2018, 03:20 AM)Ualas Wrote: The Average Strategy - Settings is just an average of the top 10 settings, right? If we backtest it does not correspond to the profit mentioned at Average Strategy - Meta (That probably is an average of the top 10 profits). So, whats is the method to not have an overfitted strategy?
Does this Profit calculation already contemplates the Market Performance? Profit = GrossProfit + MarketPerformance or Profit = GrossProfit?
as far as i understand no. the profit of the results is without market performance. this is Alpha. profit-market.
only in the runlog it shows profit-market.
@tommiehansen
thx for the new update. now everything works just perfect. there where just a few bugs though that came up for me.
1. the average parameters don't show 1000s characters when the number goes into the 1000s
2. the sorting in the Top runs is screwed up sometimes. that happened to me with all different columns sometimes. it also seems to be something with special characters or 1000s numbers.
(see pix)
No, the 'profit' in the box is the total profit, not alpha.
Ualas: Yes.
No, the average strategy settings is also just an average. It isn't like you can run 'average strategy' and get 'average profit' out of it.
These two basically live in two different universes but both are averages of something (in this case Top 10 most profitable runs since this is the default sorting).
I get the confusion though.
See 'Generated settings' more as a guide to what ranges usually works better. If it e.g. say 'SMA = 200' after 5000+ runs it stands to reason that using 'SMA = 1000' instead of 'SMA = 100', 'SMA = 200' or 'SMA = 300' will most likely be a lot worse since we now got a good statistical average that tells us this.
There's also a reason one can generate new settings out of the average settings (and a reason that it say one should modify those settings slighly since often it scews up some values).
The method not to overfit is in the hands of the user. Being able to see things such as averages could be useful though. Other values such as win% is also very useful. How you specifically use or abuse the tool is up to you though.
There is some use of the word "best" in the 'Viewer' that needs to change though since the word 'Best' isn't really that suited since 'best profit' really does not mean that it will be the best strategy params to run with live since the 'best profit' could have some other values that simply suck such as a win% under 50% and stuff like that. That's also the reason you can globally sort the entire list by things such as Win% and Sharpe instead of % Profit.
-
donkykong017:
Thank you for submitting bugs, it is very helpful and appreciated.
But -- if you don't mind, could you in the future add new issues at Github instead?
It makes it easier to track and it's a much more suited environment for discussing specific issues.
I added both these issues now though: https://github.com/tommiehansen/gab/issues
1. Yes, this is a bug
2. Yes, this is a known bug and the lib used will get thrown out.
There just haven been enough time to work on that part since most libraries used for this doesn't really fit (i could explain why, but time...)
yeah sure man of course. thx for the tool that is very helpful.
everything works like a charm now.
thx for fixing it and i will submit bugs on github the next time.
looking forward to the next update.
cheers
The stuff you wrote about is already fixed in the last commit, just git pull it.
(04-06-2018, 03:20 AM)Ualas Wrote: The Average Strategy - Settings is just an average of the top 10 settings, right? If we backtest it does not correspond to the profit mentioned at Average Strategy - Meta (That probably is an average of the top 10 profits). So, whats is the method to not have an overfitted strategy?
Does this Profit calculation already contemplates the Market Performance? Profit = GrossProfit + MarketPerformance or Profit = GrossProfit?
as far as i understand no. the profit of the results is without market performance. this is Alpha. profit-market.
only in the runlog it shows profit-market.
@tommiehansen
thx for the new update. now everything works just perfect. there where just a few bugs though that came up for me.
1. the average parameters don't show 1000s characters when the number goes into the 1000s
2. the sorting in the Top runs is screwed up sometimes. that happened to me with all different columns sometimes. it also seems to be something with special characters or 1000s numbers.
(see pix)
No, the 'profit' in the box is the total profit, not alpha.
Ualas: Yes.
No, the average strategy settings is also just an average. It isn't like you can run 'average strategy' and get 'average profit' out of it.
These two basically live in two different universes but both are averages of something (in this case Top 10 most profitable runs since this is the default sorting).
I get the confusion though.
See 'Generated settings' more as a guide to what ranges usually works better. If it e.g. say 'SMA = 200' after 5000+ runs it stands to reason that using 'SMA = 1000' instead of 'SMA = 100', 'SMA = 200' or 'SMA = 300' will most likely be a lot worse since we now got a good statistical average that tells us this.
There's also a reason one can generate new settings out of the average settings (and a reason that it say one should modify those settings slighly since often it scews up some values).
The method not to overfit is in the hands of the user. Being able to see things such as averages could be useful though. Other values such as win% is also very useful. How you specifically use or abuse the tool is up to you though.
There is some use of the word "best" in the 'Viewer' that needs to change though since the word 'Best' isn't really that suited since 'best profit' really does not mean that it will be the best strategy params to run with live since the 'best profit' could have some other values that simply suck such as a win% under 50% and stuff like that. That's also the reason you can globally sort the entire list by things such as Win% and Sharpe instead of % Profit.
-
donkykong017:
Thank you for submitting bugs, it is very helpful and appreciated.
But -- if you don't mind, could you in the future add new issues at Github instead?
It makes it easier to track and it's a much more suited environment for discussing specific issues.
I added both these issues now though: https://github.com/tommiehansen/gab/issues
1. Yes, this is a bug
2. Yes, this is a known bug and the lib used will get thrown out.
There just haven been enough time to work on that part since most libraries used for this doesn't really fit (i could explain why, but time...)
yeah sure man of course. thx for the tool that is very helpful.
everything works like a charm now.
thx for fixing it and i will submit bugs on github the next time.
looking forward to the next update.
cheers
The stuff you wrote about is already fixed in the last commit, just git pull it.
yes thx tommie. pulled and running successfully just now
Screen Shot 2018-04-07 at 7.51.36 PM.png (Size: 474.75 KB / Downloads: 15)
