How To Build A Crypto Arbitrage Trading Bot
May 13, 2022
The code for this trading bot is open source. You can access it here: https://github.com/Macrometacorp/tutorial-cryptotrading
If you're going to try this project or customize it for your own strategy, you're going to need a Macrometa account. Take a minute to sign up for a free account by clicking on the button below.
Sign up for a free dev account
I've been fortunate to have had a professional life that has exposed me to many experiences. I've not just tried my hand at software architecture and engineering, but also have been a full time professional quantitative trader for many years. The majority of my trading was with Index and Currency futures on Chicago Mercantile Exchange (CME), as well as trading US equities.
The general excitement surrounding cryptocurrencies piqued my interest. There are ~267 cryptocurrency trading exchanges spread around the world with probably around 8-10 exchanges with enough liquidity and reputation that one might consider working with them. Given the number of exchanges and volatility of cryptocurrencies, I wanted to see if it is feasible to build a trading bot that can trade on various exchanges doing something like exchange arbitrage.
What is exchange arbitrage?
An exchange arbitrage is basically a trading strategy based on the differences between the price of the cryptocurrency at different exchanges. Generally these opportunities open up if there is a price discrepancy and the discrepancy accumulates over time to finally become a significant amount like 1–3% at the cryptocurrency market. These arbitrage opportunities could exist for merely seconds, so an arbitrage trader would have to search for the best opportunities constantly and then implement them whenever possible.
Implementing arbitrage strategies in the crypto market is not that easy though, and involves a number of technical complexities. Besides having deposits in multiple major exchanges, on the technical side it involves the ability to monitor arbitrage opportunities simultaneously, making trades locally at each exchange with minimum delay and timely visibility & interactions between the trading agents distributed globally. The challenge requires big dollars and a team with the necessary technical expertise to build this type of distributed infrastructure.
Real-time crypto price data with Macrometa
Macrometa is a Global Data Network that offers a decentralized, distributed database, stream processing engine and compute platform that runs across 175 global edge regions. Apps, APIs and web services written on Macrometa are automatically deployed globally and handle requests from the closest edge location relative to the request with local read-write latencies (i.e. very low latencies).
Macrometa offers a range of ready-made capabilities that enable an application like a crypto trading bot to be built quickly and easily. We will use the following features of Macrometa in this tutorial:
- Macrometa's serverless, decentralized and geo distributed developer platform that provides a database, a stream engine and stream processor and a function/container runtime.
- We will use the global document database for storing trade data and sharing all trade related metadata and state between the decentralized, stateless bots and components
- We will use geo distributed streams for publishing real-time price data from each exchange. Each bot subscribes to the stream and makes a decision on whether to buy or sell
- Finally we will use real-time updates from the global database to automatically notify the clients/bots/subscribers of changes to the database so they can modify their trading strategy
So with the features listed above, I set out to see if Macrometa can be used to build a crypto trading bot to trade locally in multiple exchanges while providing global visibility & communication.
Some disclaimers before we proceed further - the trading strategy is an example of the capabilities of the Macrometa platform. Also, at present I do not trade cryptocurrencies. Trading involves significant risks, so always perform your due diligence. Please remember this post is about how to use the Macrometa platform and not about how or what to trade. This post is not to be used as trading advice or financial guidance.
Crypto trading strategy
For this tutorial, lets use a simple trend trading strategy:
- Buy → When current price crosses above 10 bar simple moving average
- Sell → When current price crosses below 10 bar simple moving average
The trading bot will run this strategy with:
- BTC/USD pair on Coinbase Pro exchange
- BTC/EUR pair on Bitstamp exchange
- BTC/JPY pair on BitFlyer exchange
Please note that the trading strategy is a hypothetical example and does not take into account slippage, spreads, position sizing, account balances etc. Feel free to enhance the strategy and let me know what the results look like. The code for this trading bot is open source.
The Crypto Arbitrage Trading Bot in Action
You can access the GUI for the single page app here - Live Demo
Clicking on above link should show something like what you see below.
For now go with defaults and click Confirm.
Once you click Confirm, the next screen will ask you to select the region for the dashboard to connect to. Currently this demo is running in 4 regions. Pick a region that is closest to you geographically.
Once you click Confirm, you will see a realtime dashboard like what is seen below.
Each of the charts in above picture represent the cryptocurrency pair quotes, its moving average and the exchange the quotes are from. This is served from the geo-replicated streams I mentioned before. The bot uses 1 geo-replicated stream per currency pair and exchange.
The bottom panel shows the trades made by the bot at each of the exchanges in that region. The panel is updated in realtime as the trades are made. You can use the filter box to filter the trades.
Now that you have an idea of what the dashboard looks like, it is time to dive into the nitty-gritty of the trading bot and dashboard code.
30,000 ft View
The trading bot uses geo-replicated streams in Macrometa's platform to publish local cryptocurrency quotes & moving averages in each region, and subscribe to the quotes and moving averages from other regions. Geo-replicated streams enable you to publish in any region and subscribe in any other region.
Similarly it uses the real-time database features of Macrometa to record trades locally in each region which are then automatically geo-replicated globally and visible to all regions in real-time. Most databases are single region and pull based in nature. Macrometa's database is geo-distributed and enables both pull and push based updates. The idea is each trading bot subscribe for updates on the trades collection (table) in each region to get updates in real-time.
Finally, the GUI is a small single page app that you can connect locally in any region and subscribe to the geo-replicated streams and trades collection to get a global view in realtime.
Crypto bot source code
Building the global trading bot and dashboard wasn't as difficult as you'd think.The platform abstracts many of the complexities associated with distributed systems. As a developer, you can focus more on writing apps talking to a local system (for example, the trading bot & dashboard).
The source code is available on GitHub. You can access it here: https://github.com/Macrometacorp/tutorial-cryptotrading
A few more notes:
- Originally the trading bot was developed to run in 3 geo-locations (i.e., 3 instances) where each instance connects to one exchange locally, does the trades and publishes on respective geo-replicated streams and database. To make demos easier, we subsequently changed the code so that a single instance of the bot connects to all 3 exchanges and does the trades, while still publishing on respective geo-replicated streams.
- The trading bot was developed in python using pyC8 driver (Macrometa's python driver). To make demos easier we changed the code to javascript using jsC8 (Macrometa's javascript driver).
Compiling & Deployment
You can read in the tutorial README.md the details as to how to compile and run the trading bot and dashboard locally as well as via S3.
Crypto bot code structure and details
This Crypto currency trading demo has two parts:
- A node application
- A UI dashboard application written in ReactJS
Node Application (aka Trading Bot)
The node application has four main files:
- config.js - Contains the Macrometa credentials to use for the demo.
- index.js - The initialization work for jsc8 and streams is in this file.
- producer.js - This file gets the latest values from different exchanges and publishes them to their respective geo-replicated streams. It uses CCXT library to connect to various exchanges.
- consumer.js - This file subscribes to the streams, calculates the moving average to respective geo-replicated streams. Also, it triggers Buy and Sell trades and records them in the trades collection.
The demo creates two geo-replicated streams for each cryptocurrency pair it trades:
- BTC/USD — cryto-trader-quotes-USD and crypto-tader-quotes-avg-USD streams
- BTC/EUR — cryto-trader-quotes-EUR and crypto-tader-quotes-avg-EUR streams
- BTC/JPY — cryto-trader-quotes-JPY and crypto-tader-quotes-avg-JPY streams
Below are the main code segments in index.js.
// BEGIN GLOBAL CONSTANTS
const QUOTECURR_EXCHANGE_MAP = {
"USD": {
"region": "USA",
"exchange": "gdax", //This is the id of the exchange in ccxt
quoteStream: null,
maStream: null
},
"EUR": {
"region": "Europe",
"exchange": "bitstamp", //This is the id of the exchange in ccxt
quoteStream: null,
maStream: null
},
"JPY": {
"region": "Asia-Pacific",
"exchange": "bitflyer", //This is the id of the exchange in ccxt
quoteStream: null,
maStream: null
},
}
// C8Streams
const QUOTES_TOPIC_PREFIX = "crypto-trader-quotes-";
const AVGQUOTES_TOPIC_PREFIX = "crypto-trader-quotes-avg-";
async function init() {
// Connect to Macrometa data platform
fabric = new Fabric(`https://${regionUrl}`);
await fabric.login(tenantName, userName, password);
fabric.useTenant(tenantName);
fabric.useFabric(fabricName);
// Get crypto exchange symbols
const keys = Object.keys(QUOTECURR_EXCHANGE_MAP);
for (let key of keys) {
const obj = QUOTECURR_EXCHANGE_MAP[key];
// Create geo-replicated stream for quotes
const quote_topic = `${QUOTES_TOPIC_PREFIX}${key}`;
obj.quoteStream = fabric.stream(quote_topic, false);
await obj.quoteStream.createStream();
// Create geo-replicated stream for moving average
const ma_topic = `${AVGQUOTES_TOPIC_PREFIX}${key}`;
obj.maStream = fabric.stream(ma_topic, false);
await obj.maStream.createStream();
const onOpenCallback = () => {
produceData(key, obj, regionUrl);
}
await consumeData(obj, onOpenCallback, regionUrl, fabric);
}
};
Next are the main code fragments in the producer.js file.
// Code to connect to the exchange:
async function init_exchange(value) {
if (!value) throw "Quote object not passed";
const eid = value.exchange;
const exchange = new ccxt[eid]();
exchange.enableRateLimit = true;
exchange.rateLimit = RATE_LIMIT;
console.log("Loading markets for Cryptocurrency exchange: " + exchange.name);
await exchange.load_markets();
return exchange;
}
// Code to get ticker data from the exchange:
async function get_ticker(exchange, quote_currency, regionName) {
if (!exchange) throw "ERROR : exchange is null or empty!";
let symbol = `${base_currency}/${quote_currency}`;
let ticker = await exchange.fetch_ticker(symbol);
let close = ticker['close']
let ts = Math.floor(Date.now() / 1000);
let quote_dict = {}
quote_dict.region = regionName
quote_dict.exchange = exchange.name
quote_dict.symbol = symbol
quote_dict.timestamp = ts
quote_dict.close = close
return JSON.stringify(quote_dict)
}
// Code to publish the ticker data to respective geo-replicated streams:
async function produceData(key, value, regionUrl) {
const exchangeObj = await init_exchange(value);
const { quoteStream, region } = value;
setInterval(async () => {
let ticker = await get_ticker(exchangeObj, key, region);
quoteStream.producer(ticker, regionUrl);
}, delay);
}
Next are the main code fragments in the consumer.js file.
async function consumeData(obj, onOpenCallback, regionUrl, fabric) {
let collectionhandle = await fabric.collection('trades')
tradectr = await collectionhandle.count()
tradectr = tradectr.count
const close_history = [];
const ma_history = [];
const { quoteStream, region, exchange, maStream } = obj;
const subscriptionName = `${region}-${exchange}`;
quoteStream.consumer(subscriptionName, {
onopen: () => {
// start the producer for this stream
onOpenCallback();
},
onmessage: async (msg) => {
try {
let decode_msg_obj = JSON.parse(msg);
let buff = new Buffer(decode_msg_obj.payload, 'base64');
let dec = buff.toString('ascii');
dec = JSON.parse(dec);
// Parse message to extract buy and sell prices
var close = dec.close;
var timestamp = dec.timestamp;
var symbol = dec.symbol;
var exchange = dec.exchange;
var quoteregion = dec.region;
if (close && timestamp) {
close_history.push(close);
}
//Compute & Publish SMA
if (close_history.length >= ma_len) {
ma_history.push(nj.mean(close_history));
let sma_dict = {};
sma_dict['region'] = quoteregion;
sma_dict['exchange'] = exchange;
sma_dict['symbol'] = symbol;
sma_dict['ma'] = ma_history[ma_history.length - 1];
sma_dict['close'] = close;
sma_dict['timestamp'] = timestamp.toString();
let sma_dic_str = JSON.stringify(sma_dict);
maStream.producer(sma_dic_str, regionUrl);
}
//Do we need to BUY?
if (ma_history.length > 3 &&
close_history[close_history.length - 1] > ma_history[ma_history.length - 1] &&
close_history[close_history.length - 2] < ma_history[ma_history.length - 2]) {
let tradeobj = {};
tradeobj["_key"] = "BUY-" + (timestamp).toString();
tradeobj["exchange"] = exchange;
tradeobj["symbol"] = symbol;
tradeobj["quote_region"] = quoteregion;
tradeobj["trade_strategy"] = "MA Trading";
tradeobj["timestamp"] = timestamp;
tradeobj["trade_type"] = "BUY";
tradeobj["trade_price"] = close;
await insert_trade_into_c8db(regionUrl, tradeobj, fabric);
tradectr += 1 // Increment the trade count
console.log("Buy Trade: " + JSON.stringify(tradeobj));
}
// Do we need to SELL?
else if (ma_history.length > 3 &&
close_history[close_history.length - 1] < ma_history[ma_history.length - 1] &&
close_history[close_history.length - 2] > ma_history[ma_history.length - 2]) {
let tradeobj = {};
tradeobj["_key"] = "SELL-" + timestamp.toString();
tradeobj["exchange"] = exchange;
tradeobj["symbol"] = symbol;
tradeobj["quote_region"] = quoteregion;
tradeobj["trade_strategy"] = "MA Trading";
tradeobj["timestamp"] = timestamp;
tradeobj["trade_type"] = "SELL";
tradeobj["trade_price"] = close;
try {
await insert_trade_into_c8db(regionUrl, tradeobj, fabric);
tradectr += 1; // Increment the trade count
console.log(`Sell Trade: ${JSON.stringify(tradeobj)}`);
} catch (e) {
console.log("Error in inserting to collection", e);
}
}
}
}, regionUrl);
}
The below method:
- consumes data from the quote stream,
- computes and publishes moving average to geo-replicated moving average stream
- simulates buy & sell signal and
- inserts a record into the trades collection.
Next are method inserts for the trade into the database feature of Macrometa's data platform.
async function insert_trade_into_c8db(cluster, tradeobj, fabric) {
if (cluster === undefined || cluster === null) {
console.log("ERROR: cluster is null or empty!")
}
if (tradeobj === undefined || tradeobj === null) {
console.log(("ERROR: trade data object is null or empty!"))
}
let c8url = cluster
const collection = fabric.collection('trades')
let exists = await collection.exists()
if (exists === false) {
await collection.create()
}
//Insert the trade
let doc = {};
doc.exchange = tradeobj.exchange;
doc.symbol = tradeobj.symbol;
doc.quote_region = tradeobj.quote_region;
doc.trade_strategy = tradeobj.trade_strategy;
doc.timestamp = tradeobj.timestamp;
doc.trade_type = tradeobj.trade_type;
doc.trade_price = tradeobj.trade_price;
doc.trade_location = cluster;
collection.save(doc);
console.log("Saved trade info to C8DB at '" + c8url + "': " + (doc).toString());
}
GUI (crypto dashboard)
The single page application (i.e., dashboard) subscribes to all the geo-replicated streams as well as to the trades collection to display the charts and trades in the dashboard.
This post became a lot longer than anticipated, so we're skipping the code explanation for the dashboard. Savvy javascript developers should be able to understand by looking at the code directly. You can find code for this in the below mentioned directory.
Final thoughts
I hope you enjoyed the tutorial and can use it as a sample to write your own trading bots. Please feel free to let me know what you built and also any comments or feedback.
Finally, some unsolicited advice as a way of saying thanks for reading all the way. To me, a good trader is someone who gets checks from their broker regularly. Unfortunately for most people the checks go only in the other direction, i.e. from them to the broker. If you fall into this later category, ignore most of the stuff you read about trading. Instead, pick one simple strategy, stick to it, and paper trade until you are consistently profitable. That will be an education in patience, discipline, skill and trading mindset. Afterwards, trade with small money until you overcome the psychological challenges associated with real money on the line. That summarizes my two decades of trading experience part-time and full-time.
The code for this arbitrage trading bot is open source. You can access it here: https://github.com/Macrometacorp/tutorial-cryptotrading
