CNLearn FastAPI Continuous Integration

Like I promised in the previous post, this post will be about adding a few things to our continuous integration pipeline. We will accomplish the following in this post:


Let’s start with pre-commit. is a nice and simple way of adding certain hooks to happen before committing: could be linting, reformatting, blank line checker, whatever you want really. In this case, we will run a few hooks they can provide. First, let’s install pre-commit and add it to our dev.txt (newly created) requirements file.

pip install pre-commit

Then, let’s add a .pre-commit-config.yaml config file.

  - repo:
    rev: v2.3.0
      - id: check-yaml
      - id: end-of-file-fixer
      - id: trailing-whitespace
      - id: flake8
  - repo:
    rev: 22.3.0
      - id: black
  - repo:
    rev: 5.10.1
      - id: isort
        name: isort (python)

Then, we run pre-commit install to install the hooks.

Finally, let’s do a run over all the files outside a commit -> this actually showed me a looooot of errors and inconsistencies which I have since fixed.

pre-commit run --all-files

It’s worth noting I also made some changes to the black config by settings line-length to 120 in pyproject.toml. I also made the same change and added some flake8 settings in tox.ini.

After running the command above and making the changes, let’s create a workflow file in the .github/workflows directory that will set up our Continuous Integration.

Github Actions Continuous Integration

For now, I will only run the tests on Python 3.9, but I am adding the action in such a way as to make it easy to add other versions later on. In terms of the workflow we have:

  • Set up Python 3.9
  • Install testing dependencies as well as flake8 and black
  • Run flake8 and black in case our pre-commit missed something
  • Set up PostgreSQL
  • Wait a few seconds for it to be ready
  • Check the running containers
  • Run the tests

Keep in mind that usernames/passwords were read by our settings from .dev.env. We must therefore add the necessary variables to the environment. We are doing that in the environment file as well.

name: Run flake8, black and pytest

on: [push, pull_request]

    runs-on: ubuntu-latest
        python-version: ["3.9"]

      - uses: actions/checkout@v3
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v4
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install wheel
          pip install flake8 black
          pip install -r requirements/testing.txt          
      - name: Lint with flake8
        run: |
          # stop the build if there are Python syntax errors or undefined names
          flake8 . --count --show-source --statistics          
      - name: Run black --check .
        run: black --check .
      - name: Setup PostgreSQL
        uses: Harmon758/postgresql-action@v1.0.0
          postgresql version: "14"
          postgresql db: "cnlearn_testing"
          postgresql user: "postgres"
          postgresql password: "postgres"
      - name: Wait / Sleep
        uses: jakejarvis/wait-action@v0.1.0
          time: "10s"
      - name: Collect Docker Logs
        uses: jwalton/gh-docker-logs@v1.0.0
      - name: Check running containers
        run: docker ps -a
      - name: Test with pytest
          APP_NAME: "CNLearn Testing"
          VERSION: 0.0.5
          SERVER_NAME: localhost
          POSTGRES_SERVER: localhost
          POSTGRES_PORT: 5432
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: cnlearn
          TESTING: 1
        run: |

Ok let’s add all the files we changed, push and hopefully the tests run :)

The commit for this post is here.