We are starting with a fresh install of a Debian 10 buster on a dedicated server and we will use Docker to run our platform.
This is how it works:
Two images are started on the server:
A Postgresql image to backup trades & positions from our trading bot.
A Postgresql backup image to automatically backup data from Postgresql.
The trading bot is pushed as a Docker image on the server from your continuous integration.
We choose PostgreSQL as a database but you can choose the one you want, Just think to add the JDBC driver to your bot pom.xml
.
First, with the apt command, we install some useful packages:
sudo apt updatesudo apt -y install apt-transport-https ca-certificates curl gnupg2 pass software-properties-commonsudo apt -y upgrade
Now it's time to install docker and docker-compose:
sudo curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"sudo apt updatesudo apt-cache policy docker-cesudo apt -y install docker-ce docker-composesudo apt-mark hold docker-ce docker-composesudo chmod 666 /var/run/docker.sock
We ask Debian to never upgrade docker/docker-compose automatically with the apt-mark hold
command because an update could stop all the running images.
Our bot will be deployed to this server from another server (in our case, our continuous integration server), so we need to create a user that can connect with SSH:
sudo useradd -m -d /home/sma-trading-bot sma-trading-botsudo passwd sma-trading-botsudo gpasswd -a sma-trading-bot dockersudo usermod --shell /bin/bash sma-trading-bot
Download the docker-compose.yml file on your server, edit your preferences (password, timezone, backup settings...) and run it with the command:
sudo docker-compose up -d
You can download it directly with the command : curl -o docker-compose.yml https://raw.githubusercontent.com/cassandre-tech/cassandre-trading-bot/development/trading-bot-server/docker-compose.yml
networks:cassandre:name: cassandre
This part declares a network named cassandre
.
volumes:database:backup:
This part declares two volumes (space on disk) :
database
for the database.
backup
for the database backups.
postgresql:image: library/postgres:13-alpinerestart: alwaysnetworks:- cassandrevolumes:- database:/var/lib/postgresql/dataenvironment:- POSTGRES_DB=cassandre_trading_bot- POSTGRES_USER=cassandre_trading_bot- POSTGRES_PASSWORD=mypassword
This starts a Postgresql image where our trading bot will store its data (trades & positions).
postgresql-backup:image: prodrigestivill/postgres-backup-local:13-alpinedepends_on:- postgresqlrestart: alwaysnetworks:- cassandrevolumes:- backup:/backupsenvironment:- POSTGRES_HOST=postgresql- POSTGRES_DB=cassandre_trading_bot- POSTGRES_USER=cassandre_trading_bot- POSTGRES_PASSWORD=mypassword- POSTGRES_EXTRA_OPTS=--schema=public- SCHEDULE=@every 01h00m00s- BACKUP_KEEP_DAYS=7- BACKUP_KEEP_WEEKS=4- BACKUP_KEEP_MONTHS=0- HEALTHCHECK_PORT=8080
This starts an image that will connect to the Postgresql image and make backup according to the parameters: SCHEDULE
, BACKUP_KEEP_DAYS
, BACKUP_KEEP_WEEKS
and BACKUP_KEEP_MONTHS
.
There are several ways to do what we are trying to do, we choose this one during our tests:
Our trading bot source code is hosted in a private Github project.
On every push, our Github actions script does the following steps:
Creates the docker image for our trading bot.
Login to our docker hub repository.
Push the image to our docker hub repository.
Connect to our private server (ssh).
Stop the previous running image of our bot and run the new image.
The source of our script is here and this is what it does:
- name: Build with Maven and creates the docker imagerun: mvn spring-boot:build-image
- name: Push image to docker hubrun: |echo ${{ secrets.DOCKER_HUB_PASSWORD }} | docker login -u ${{ secrets.DOCKER_HUB_USERNAME }} --password-stdindocker push straumat/trading-bot:latest
The deployment is made in several steps:
Connect to our production server with SSH.
Login to our docker private account.
Stop & delete the image of the previous trading bot.
Retrieve the image from docker hub.
Run the image with all the parameters specified in github secrets.
- name: Deploy to production serveruses: appleboy/ssh-action@masterwith:host: ${{ secrets.SSH_HOST }}port: ${{ secrets.SSH_PORT }}username: ${{ secrets.SSH_USERNAME }}password: ${{ secrets.SSH_PASSWORD }}script: |echo ${{ secrets.DOCKER_HUB_PASSWORD }} | docker login -u ${{ secrets.DOCKER_HUB_USERNAME }} --password-stdindocker stop $(docker ps -aq --filter "label=trading-bot")docker rm -f $(docker ps -aq --filter "label=trading-bot")docker pull straumat/trading-bot:latestdocker run -d \--security-opt apparmor=unconfined \--network="cassandre" \-e TZ=Europe/Paris \-e CASSANDRE_TRADING_BOT_EXCHANGE_NAME='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_NAME }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_USERNAME='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_USERNAME }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_PASSPHRASE='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_PASSPHRASE }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_KEY='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_KEY }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_SECRET='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_SECRET }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_MODES_SANDBOX='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_MODES_SANDBOX }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_MODES_DRY='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_MODES_DRY }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_RATES_ACCOUNT='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_RATES_ACCOUNT }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_RATES_TICKER='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_RATES_TICKER }}' \-e CASSANDRE_TRADING_BOT_EXCHANGE_RATES_ORDER='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_RATES_ORDER }}' \-e CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_DRIVER-CLASS-NAME=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_DRIVER_CLASS_NAME }} \-e CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_URL=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_URL }} \-e CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_USERNAME=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_USERNAME }} \-e CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_PASSWORD=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_PASSWORD }} \-e CASSANDRE_TRADING_BOT_DATABASE_TABLE-PREFIX=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_TABLE_PREFIX }} \-l trading-bot \straumat/trading-bot:latest
These are the parameters for the Postgresql connection:
Parameter | Value |
DRIVER-CLASS-NAME | org.postgresql.Driver |
URL | jdbc:postgresql://postgresql/cassandre_trading_bot |
USERNAME | cassandre_trading_bot |
PASSWORD | mypassword |
On the server, thanks to the docker label, you can view the bot logs with the command : docker logs $(docker ps -aq --filter "label=trading-bot") --follow