Automatic Trading - The journey begins...

For a long time I have been dreaming about building my own trading bot. I have many years of data science experience and always figured I should be able to make a bot that can be profitable.

Finally, I have decided to embark on this journey and wanted to document it throughout a series of blog posts. This first post will detail a rough plan on how to create such a trader bot and document all of the different decisions that need to be taken.

There's a number of components that are required for even the most basic trading bot.

A stock broker

In order to automatically buy and sell assets, the most essential component is obviously a stock broker account. This is straightforward to create at hundreds (if not thousands) of stock brokers or trading platforms.

Besides the basic account which would allow me to buy and sell assets manually, the stock broker needs to have an API that allows for buying and selling assets. For simplicity, I even set the criteria that the stock broker should officially support a python API, since that is the programming language I will be using.

This requirement is quite strict, since many stock brokers offer REST APIs which I could easily use as well, but this would require just a bit more programming effort to create the proper requests etc.. This decision is something I can easily change later on and is only to get started as quickly as possible to see whether there is any potential to this project at all.

From my research into the different stock brokers, Alpaca stands out because of their seemingly very well supported python API. For now I will start with Alpaca and can always change later on if required.

A strategy to buy and sell

In order to automatically trade stocks I will need some kind of 'strategy' to decide what and when to buy and what and when to sell. The most basic strategy out there is simply to divide your starting capital uniformly over all the assets you consider for this trading bot and buy as many stocks as you can with the share of the starting capital.

For example, assume we start with $ 1000 to invest with. Furthermore, assume that we are considering 10 different stocks to invest in. This strategy would simply allocate $ 100 to each stock and buy as many as possible with those $ 100. Throughout time no more trades are made, but the stocks are simply held.

Of course this is not much of an strategy, at least, I don't expect it to perform very well, but it is a strategy and it will 'automatically' choose what to buy and what to sell and when.

This strategy will be our baseline. Imagine you spent a week on some fancy strategy and it could not even beat this simplistic uniform strategy then you know there's a lot of work left to make it profitable.

Evaluation

I want to be able to evaluate how a strategy performs historically before actually starting to use the strategy with real money. In order to evaluate the performance of a strategy I will apply it on past data to pretend that I was using this strategy in the past. Since I also have the actual price data I can calculate exactly how much each strategy would generate in terms of profit and losses.

This so called evaluation is often referred to as backtesting. There are many python packages available that claim to do exactly this. I will instead write my own. First of all, this should not be a lot of effort and second of all I would prefer total freedom to define the strategy how I want without being forced to use a certain pattern.

Of course this means I will be reinventing the wheel a bit, I will check out some of the backtesting frameworks to decide whether there are certain features I want to implement. Once everything is a bit more mature, I might switch to a publicly available backtesting framework, but I imagine the code design will have matured and stabilised a bit by then as well.

Historical data

As the text above already implies I will need to store historical price data for all of the assets I consider for this trading bot. I did not mention it before, but another criteria for the stock broker is of course that they give access to historical data as well via their API. From my research it seems that all stock brokers with some form of an API also supply historical data, so this should be fine.

Now I could of course retrieve all the historical data from the stock broker API every time that I want to evaluate a strategy, but I decided that I should store all the historical data in my own database instead. That way I can experiment as much as I want without ever hitting some form of an API limit on retrieving data. In theory, this should also make the retrieval of the historical data faster, but that is not a concern at the moment.

This means I will need to select a database suitable for this project and host it myself. I have a home server at my disposal, so I should be able to host any database I want and have it centrally available. This allows me to run the evaluation and anything else requiring the historical data from any device I want instead of being tied to using local CSV files for example.

Database

There are again many alternatives available. Before diving into all of the options I want to decide on some criteria before hand to guide my search.

1. It should be possible to self-host
2. It should be easy to setup
3. It should be easy to use with Python
4. I want it to be free
5. It should be open source. I don't want to be locked behind a paywall after having used it for some time
6. In terms of data it should handle I expect it to be relatively small no matter what I do.
7. Latency or anything like that is also not an issue
8. User management or access patterns are of no concern as it will have a single user anyway (me / the bot).

The only requirement I am doubting about is whether it should be possible to host on a server or whether it is okay to use SQLite in combination with a backup to the central server. If I want to continue work on another device I can simply retrieve the latest backup and continue work.

For simplicity I will start with a SQLite database at first. If and when I run into issues or really need to host it centrally I can always switch database. I will create a separate project to fill the database with historical data though, so it should be simple to migrate to a different database.

Furthermore, I will use pandas which uses SQLAlchemy under the hood, so any operations should be able to stay the same and only requires a different engine to use a different database.

Design

With all of the components having been decided the design will look roughly like the following:

Each square block is a python script. The Database used is SQLite and the stock broker is Alpaca.

The next post will outline the two modules to retrieve and store all historical data from the Alpaca API, see you there!