[ARBITRAGE] Agent Smith: Arbing the great bull run of 2017

[askmike]

I decided early on that I wasn't going to get into the ultra low latency / co-location game as I was confident that wouldn't be a battle I could not win given the exchanges can (and likely do) run such strategies themselves directly with far lower latency access and without fees.

I know this is commonly believed and it does make sense that they are doing it themselves. But everything I have seen so far points against this (on a few exchanges anyway). What I've seen for exchange infrastructures so far is that different matching engines (for different markets) run on different servers (or threads, probably servers on big exchanges). Meaning that if they want to do this themselves they need to monitor their other orderbooks async and maybe even do some kind of locking. I am confident the risk of slowing down their matching engine is much bigger than the money they would make here.

In perspective: I've made some profit, but I paid at least 10 times that amount in fees. So on exchanges with decent fees they'll make a ton of money anyway. Why risk a lagging metching engine for an extra 10% (max during crazy bull runs)?

Measuring the 'ping' time is obviously a useful starting point but I really wanted to understand how 'old' a given message is when I processed it given the various buffers and queues it will go through both within my code and OS network stack and outside on the network and even on the exchange itself. Luckily HitBTC provide a fairly accurate 'timestamp' on the ticker messages which indicates when the message was sent and so I was able to measure how old each message was when I processed it. Surprisingly (or perhaps not) not every exchange has a way to do this and in fact Poloniex does not tat I could see (at the time I wrote it at least, I haven't looked recently). I would be interested to knowhow you approached this problem?

Though if the API does not offer such an API call, pinging the host of the API does not work in my experience (as you are simply pinging cloudflare/their CDN/their loadbalancers). I usually measure different API calls over a period of time as well as other metrics (in the case of AGENT SMITH: success metrics: how often was I able to take crossing orders from the book before anyone else).

Eventually I'd developed all this into a fairly solid async-only exchange-agnostic API (operations not available via websocket are run as async GET/POST on a thread to keep everything event driven) which could be plugged into any bot 'strategy' such that the strategy code didn't have to care about any of these exchange specific things and could focus on the algorithm itself. All fairly standard engineering type stuff. I may open source this pat of the code at some point as it is genuinely quite useful.

Please do! I would be very interested in this. I am actually doing something similar myself with Gekko Broker, see here: https://github.com/askmike/gekko/tree/ge...kko-broker

The (now gapingly obvious to me) flaw here is of course that there may not be sufficient liquidity at the best bid/offer price to execute all 3 pairs for the notional amount. To solve this I moved away from trying to roundtrip a fixed notional amount and instead worked out the 'highest common liquidity' between the 3 ticker pairs involved in the three-way transaction.

Yes, you always want to want the orderbook (at least a few levels down) instead of the ticker for this exact reason. The other big reason is that if you can (quite complex to do) aggregate a few levels down (say take both the top ask and the second ask) to take more volume. Agent Smith was doing this for a while, but it becomes harder to track whether you were fast enough:

If these are the top asks:

10 @ 100
1000@ 101

And you want to arb 20 for an average price of 100.5 (by taking the first ask and 10 of the second).

You can do this in a single order (preferable), however figuring out whether you were fast enough becomes harder since if you posted a limit order to buy 20 at 101 it would get filled even if the top ask is gone. (for a price where than you projected).

However in the real world the exchanges have minimum order quantities and minimum order quantity increments (sometimes published, sometimes not and discovered by trial and error). After some analysis it turned out the on HitBTC the minimum order sizes, in $USD terms vary dramatically between pairs (magnitude different). Most notably the minimum order size and minimum increment for BTC based pairs is 0.01 BTC which is a coarse grained number. Essentially it means if the 'highest common liquidity' was 0.015 BTC then my choice is either to trade 0.01 BTC or 0.02 BTC.

Yes this is a big problem, I didn't go into detail into the article but it all becomes pretty ugly very soon. Luckily on poloniex specifically this is not really a problem since their minimums are tiny: eventually I settled on a simple filter that would not arb opportunities smaller than something like 15 bucks. Not just for the reason above, also because you don't want to arb 10 dollars if you could wait one tick to arb 1000 dollars over the same market instead.