Deploy & run

This page describes our favorite way to deploy & run our trading bots.

Server installation.

We are starting with a fresh install of a Debian 10 buster on a dedicated server and we will use Docker to run our platform.

This is how it works:

  • Two images are started on the server:

  • The trading bot is pushed as a Docker image on the server from your continuous integration.

We choose PostgreSQL as a database but you can choose the one you want, Just think to add the JDBC driver to your bot pom.xml.

Useful tools.

First, with the apt command, we install some useful packages:

sudo apt update
sudo apt -y install apt-transport-https ca-certificates curl gnupg2 pass software-properties-common
sudo apt -y upgrade

Docker installation.

Now it's time to install docker and docker-compose:

sudo curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
sudo apt update
sudo apt-cache policy docker-ce
sudo apt -y install docker-ce docker-compose
sudo apt-mark hold docker-ce docker-compose
sudo chmod 666 /var/run/docker.sock

We ask Debian to never upgrade docker/docker-compose automatically with the apt-mark hold command because an update could stop all the running images.

Add a user for the trading bot.

Our bot will be deployed to this server from another server (in our case, our continuous integration server), so we need to create a user that can connect with SSH:

sudo useradd -m -d /home/sma-trading-bot sma-trading-bot
sudo passwd sma-trading-bot
sudo gpasswd -a sma-trading-bot docker
sudo usermod --shell /bin/bash sma-trading-bot

Docker images on the server.

Download the docker-compose.yml file on your server, edit your preferences (password, timezone, backup settings...) and run it with the command:

sudo docker-compose up -d

You can download it directly with the command : curl -o docker-compose.yml https://raw.githubusercontent.com/cassandre-tech/cassandre-trading-bot/development/trading-bot-server/docker-compose.yml

Network.

networks:
cassandre:
name: cassandre

This part declares a network named cassandre.

Volumes.

volumes:
database:
backup:

This part declares two volumes (space on disk) :

  • database for the database.

  • backup for the database backups.

Postgresql.

postgresql:
image: library/postgres:13-alpine
restart: always
networks:
- cassandre
volumes:
- database:/var/lib/postgresql/data
environment:
- POSTGRES_DB=cassandre_trading_bot
- POSTGRES_USER=cassandre_trading_bot
- POSTGRES_PASSWORD=mypassword

This starts a Postgresql image where our trading bot will store its data (trades & positions).

Postgresql backup.

postgresql-backup:
image: prodrigestivill/postgres-backup-local:13-alpine
depends_on:
- postgresql
restart: always
networks:
- cassandre
volumes:
- backup:/backups
environment:
- POSTGRES_HOST=postgresql
- POSTGRES_DB=cassandre_trading_bot
- POSTGRES_USER=cassandre_trading_bot
- POSTGRES_PASSWORD=mypassword
- POSTGRES_EXTRA_OPTS=--schema=public
- SCHEDULE=@every 01h00m00s
- BACKUP_KEEP_DAYS=7
- BACKUP_KEEP_WEEKS=4
- BACKUP_KEEP_MONTHS=0
- HEALTHCHECK_PORT=8080

This starts an image that will connect to the Postgresql image and make backup according to the parameters: SCHEDULE, BACKUP_KEEP_DAYS, BACKUP_KEEP_WEEKS and BACKUP_KEEP_MONTHS.

Your bot.

There are several ways to do what we are trying to do, we choose this one during our tests:

  • Our trading bot source code is hosted in a private Github project.

  • On every push, our Github actions script does the following steps:

    • Creates the docker image for our trading bot.

    • Login to our docker hub repository.

    • Push the image to our docker hub repository.

    • Connect to our private server (ssh).

    • Stop the previous running image of our bot and run the new image.

The source of our script is here and this is what it does:

Build the docker image.

- name: Build with Maven and creates the docker image
run: mvn spring-boot:build-image

Push image to our private docker hub.

- name: Push image to docker hub
run: |
echo ${{ secrets.DOCKER_HUB_PASSWORD }} | docker login -u ${{ secrets.DOCKER_HUB_USERNAME }} --password-stdin
docker push straumat/trading-bot:latest

Deploy to the production server.

The deployment is made in several steps:

  • Connect to our production server with SSH.

  • Login to our docker private account.

  • Stop & delete the image of the previous trading bot.

  • Retrieve the image from docker hub.

  • Run the image with all the parameters specified in github secrets.

- name: Deploy to production server
uses: appleboy/ssh-action@master
with:
host: ${{ secrets.SSH_HOST }}
port: ${{ secrets.SSH_PORT }}
username: ${{ secrets.SSH_USERNAME }}
password: ${{ secrets.SSH_PASSWORD }}
script: |
echo ${{ secrets.DOCKER_HUB_PASSWORD }} | docker login -u ${{ secrets.DOCKER_HUB_USERNAME }} --password-stdin
docker stop $(docker ps -aq --filter "label=trading-bot")
docker rm -f $(docker ps -aq --filter "label=trading-bot")
docker pull straumat/trading-bot:latest
docker run -d \
--security-opt apparmor=unconfined \
--network="cassandre" \
-e TZ=Europe/Paris \
-e CASSANDRE_TRADING_BOT_EXCHANGE_NAME='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_NAME }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_USERNAME='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_USERNAME }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_PASSPHRASE='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_PASSPHRASE }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_KEY='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_KEY }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_SECRET='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_SECRET }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_MODES_SANDBOX='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_MODES_SANDBOX }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_MODES_DRY='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_MODES_DRY }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_RATES_ACCOUNT='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_RATES_ACCOUNT }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_RATES_TICKER='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_RATES_TICKER }}' \
-e CASSANDRE_TRADING_BOT_EXCHANGE_RATES_ORDER='${{ secrets.CASSANDRE_TRADING_BOT_EXCHANGE_RATES_ORDER }}' \
-e CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_DRIVER-CLASS-NAME=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_DRIVER_CLASS_NAME }} \
-e CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_URL=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_URL }} \
-e CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_USERNAME=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_USERNAME }} \
-e CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_PASSWORD=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_DATASOURCE_PASSWORD }} \
-e CASSANDRE_TRADING_BOT_DATABASE_TABLE-PREFIX=${{ secrets.CASSANDRE_TRADING_BOT_DATABASE_TABLE_PREFIX }} \
-l trading-bot \
straumat/trading-bot:latest

These are the parameters for the Postgresql connection:

Parameter

Value

DRIVER-CLASS-NAME

org.postgresql.Driver

URL

jdbc:postgresql://postgresql/cassandre_trading_bot

USERNAME

cassandre_trading_bot

PASSWORD

mypassword

On the server, thanks to the docker label, you can view the bot logs with the command : docker logs $(docker ps -aq --filter "label=trading-bot") --follow