01-14-2019, 11:05 PM
(This post was last modified: 02-06-2019, 11:10 PM by crypto49er.)
I had seen this issue for sometime but I finally came up with a really rough fix. I would really appreciate if someone can write the code to do this asynchronously.
Issue: (Known to occur in Coinbase Pro, but can occur for any exchange that lets you use limit orders and a post flag that prevent executing market orders) When Gekko issues a sell order, in the seconds that it receives the current price from the exchange and before it sends the sell order to the exchange, the price went up. Normally, that's a good thing. This shows there's so much volume that your order shouldn't have any problem getting filled. The problem is, when the order goes to the exchange, the exchange rejects it because Gekko is now trying to sell below the ask price. Once the order is rejected, Gekko doesn't retry the order. Technically, even if Gekko did, it would get a message back from the exchange "HTTP 400 Error: order not found" and it will continue to get this message until the retry attempts are exhausted. So that's the issue. If your strategy issued a sell order to get out of the market in anticipation of further drops, you're now screwed!!! Gekko will be holding the bag for you while the crypto goes down in price from 2% to 5% to 10% (on a bad day) or more if you are trading an extremely volatile crypto.
To Test This: Modify sticky.js inside the exchange/orders folder. (You need to check your trade pair in the Coinbase Pro UI to see the spread first. See note directly below). In line 129, replace this:
with:
(for Coinbase Pro)
If you want to see rejected buy trades, you can modify line 111 from this:
with:
(for Coinbase Pro)
***Note: I previously modified it from 0.03 to 10% but in my recent test, I was getting insufficient funds error instead of the rejected trade error. I now changed it to 1 but it really comes down to the trade pair. The optimum number is a number that is slightly larger than the spread.***
This essentially tricks Gekko into selling below ask or buying above bid prices. For rejected buy trades, they don't cost you anything except for potential gain. But that could be a lot! Anyway, the next time your strategy issues a buy/sell order, you will see the rejected trade error in the console. It looks something like this.
Solution: I am not an advanced coder in Javascript, so this isn't the cleanest solution, but it works. The idea is to have your strategy issue another buy order after getting the "rejected" message from the exchange. One thing I noticed while coming up with this is if you try to issue another buy order, Gekko completely ignores it and doesn't even output a message to console. I'm guessing this is because older strategies send multiple buy orders and Gekko has to ignore them. So the only way to get Gekko to accept a new buy order is to issue a sell order first.
The files I modified are:
config file
gdax.js
strategy file
In the config file, we are going to store the state of "rejected" so the other files can read/write to them. I just added the following before the last line. You can probably add them anywhere.
In gdax.js, we are going to give it the ability to write to the rejected variable in the config file, so we need to add this at the top where the other required files are declared.
Now, in the checkOrder function, in the if (status == 'pending') section, I changed it from:
to:
It essentially updates the "rejected" variable in the config file to true if the trade was rejected. It will also store the side so we know if it is a buy or sell order. If the trade is successfully placed, it will change the "rejected" variable in the config file to false.
In the strategy, I again have to give it the ability to access the config file by adding this at the top, where all the other required files are declared:
The line is slightly different than the one for gdax.js because of the strategy file is stored in a different location than gdax.js.
I then added a variable to check how often to check if the "rejected" variable is still true. This is a global variable at the top where other variables are placed.
This is where asynchronous code would make this much cleaner. Instead of checking after x minutes, you can have this run only when status changes from either rejected or completed/settled in gdax.js. But I couldn't figure out how to implement that.
So instead this is the code in the check function of the strategy:
With this code, it will create a new buy/sell order if "rejected" is set to true in the config file. As mentioned previously, it has to issue a sell first if a buy order was rejected or buy first if a sell order was rejected. It will then wait ~10 minutes (the check order function doesn't always run every minute, assuming 1 minute candles). If the "rejected" variable is still valid, it will repeat this until the order is no longer rejected. Again, I wish it doesn't have to do this every x minutes but this is the only way to do it that I know of synchronously.
I tested this and confirmed that it works. Don't forget to remove the code in sticky.js otherwise every order will be rejected. If you're interested to simulating a scenario where every other trade starting with the first is a rejected trade, you can modify sticky.js to read from the "rejected" variable from the config file. I will include the code in a reply post if anyone is interested.
I now have to implement this in all of the strategies that I use. I can't wait for the official fix from AskMike. I know it is on his to-do list.
Issue: (Known to occur in Coinbase Pro, but can occur for any exchange that lets you use limit orders and a post flag that prevent executing market orders) When Gekko issues a sell order, in the seconds that it receives the current price from the exchange and before it sends the sell order to the exchange, the price went up. Normally, that's a good thing. This shows there's so much volume that your order shouldn't have any problem getting filled. The problem is, when the order goes to the exchange, the exchange rejects it because Gekko is now trying to sell below the ask price. Once the order is rejected, Gekko doesn't retry the order. Technically, even if Gekko did, it would get a message back from the exchange "HTTP 400 Error: order not found" and it will continue to get this message until the retry attempts are exhausted. So that's the issue. If your strategy issued a sell order to get out of the market in anticipation of further drops, you're now screwed!!! Gekko will be holding the bag for you while the crypto goes down in price from 2% to 5% to 10% (on a bad day) or more if you are trading an extremely volatile crypto.
To Test This: Modify sticky.js inside the exchange/orders folder. (You need to check your trade pair in the Coinbase Pro UI to see the spread first. See note directly below). In line 129, replace this:
Code:
return r(ticker.ask);with:
(for Coinbase Pro)
Code:
return r(ticker.ask - 1);If you want to see rejected buy trades, you can modify line 111 from this:
Code:
return r(ticker.bid);with:
(for Coinbase Pro)
Code:
return r(ticker.bid + 1);***Note: I previously modified it from 0.03 to 10% but in my recent test, I was getting insufficient funds error instead of the rejected trade error. I now changed it to 1 but it really comes down to the trade pair. The optimum number is a number that is slightly larger than the spread.***
This essentially tricks Gekko into selling below ask or buying above bid prices. For rejected buy trades, they don't cost you anything except for potential gain. But that could be a lot! Anyway, the next time your strategy issues a buy/sell order, you will see the rejected trade error in the console. It looks something like this.
Quote:2019-01-14 16:07:18 (INFO): Trader Received advice to go long. Buying ETCI know ETC was 51% attacked recently. This is just a test and ETC is the cheapest crypto I have access to on Coinbase Pro. But do note the erroneous 1969 date listed in console. It usually is 12/31/1969 or 1/1/1970. If you ever seen this in the past, your trade failed to execute.
2019-01-14 16:07:18 (DEBUG): Creating order to buy 2.174301930195067 ETC
Mon Jan 14 2019 16:07:23 GMT-0500 (EST) {}
sticky create
buy
2019-01-14 16:07:23 (DEBUG): [ORDER] statusChange: SUBMITTED
2019-01-14 16:07:23 (DEBUG): [ORDER] statusChange: OPEN
2019-01-14 16:07:24 (DEBUG): [ORDER] statusChange: REJECTED
2019-01-14 16:07:24 (INFO): [ORDER] summary: { price: 0,
amount: 0,
date: moment("1969-12-31T19:00:00.000"),
side: 'buy',
orders: 1 }
2019-01-14 16:07:24 (DEBUG): syncing private data
Solution: I am not an advanced coder in Javascript, so this isn't the cleanest solution, but it works. The idea is to have your strategy issue another buy order after getting the "rejected" message from the exchange. One thing I noticed while coming up with this is if you try to issue another buy order, Gekko completely ignores it and doesn't even output a message to console. I'm guessing this is because older strategies send multiple buy orders and Gekko has to ignore them. So the only way to get Gekko to accept a new buy order is to issue a sell order first.
The files I modified are:
config file
gdax.js
strategy file
In the config file, we are going to store the state of "rejected" so the other files can read/write to them. I just added the following before the last line. You can probably add them anywhere.
Code:
config.IssueState = {
rejected: false,
side: 'sell',
}In gdax.js, we are going to give it the ability to write to the rejected variable in the config file, so we need to add this at the top where the other required files are declared.
Code:
var config = require('../../core/util.js').getConfig();Now, in the checkOrder function, in the if (status == 'pending') section, I changed it from:
Code:
if(status == 'pending') {
// technically not open yet, but will be soon
return callback(undefined, { executed: false, open: true, filledAmount: 0 });
} if (status === 'done' || status === 'settled') {
return callback(undefined, { executed: true, open: false });
} else if (status === 'rejected') {
return callback(undefined, { executed: false, open: false });
} else if(status === 'open' || status === 'active') {
return callback(undefined, { executed: false, open: true, filledAmount: parseFloat(data.filled_size) });
}to:
Code:
if(status == 'pending') {
// technically not open yet, but will be soon
return callback(undefined, { executed: false, open: true, filledAmount: 0 });
} if (status === 'done' || status === 'settled') {
config.IssueState.rejected = false;
return callback(undefined, { executed: true, open: false });
} else if (status === 'rejected') {
if (data.reject_reason == 'post only' ) {
config.IssueState.rejected = true;
config.IssueState.side = data.side;
}
return callback(undefined, { executed: false, open: false });
} else if(status === 'open' || status === 'active') {
return callback(undefined, { executed: false, open: true, filledAmount: parseFloat(data.filled_size) });
}In the strategy, I again have to give it the ability to access the config file by adding this at the top, where all the other required files are declared:
Code:
var config = require ('../core/util.js').getConfig();I then added a variable to check how often to check if the "rejected" variable is still true. This is a global variable at the top where other variables are placed.
Code:
var waitForRejectedRetry = 0;So instead this is the code in the check function of the strategy:
Code:
if (config.IssueState.rejected){
if (waitForRejectedRetry == 0){
if (config.IssueState.side == 'buy'){
this.advice('short'); // To reset previous buy order by issuing a sell order
this.advice('long');
} else {
this.advice('long'); // To reset previous sell order by issuing a buy order
this.advice('short');
}
waitForRejectedRetry = 11;
}
if (waitForRejectedRetry > 0) {
waitForRejectedRetry--;
}
}I tested this and confirmed that it works. Don't forget to remove the code in sticky.js otherwise every order will be rejected. If you're interested to simulating a scenario where every other trade starting with the first is a rejected trade, you can modify sticky.js to read from the "rejected" variable from the config file. I will include the code in a reply post if anyone is interested.
I now have to implement this in all of the strategies that I use. I can't wait for the official fix from AskMike. I know it is on his to-do list.
If it isn't crypto, it isn't worth mining, it isn't worth speculating.
https://www.youtube.com/c/crypto49er
https://www.youtube.com/c/crypto49er
