GitHub Action to Run Tests for a Common Lisp Library

This is a Github Action can be used to run tests for any Common Lisp supporting (asdf:test-system :my-system).

It should be used after the setup-lisp action.

What this action does for you?

A typical usage

Here is how a minimal GitHub Workflow might look like:

name: 'CI'

on:
  push:
    branches:
      - 'main'
      - 'master'
  pull_request:

jobs:
  tests:
    runs-on: ubuntu-latest
    
    strategy:
      matrix:
        lisp:
          - sbcl-bin
          - ccl-bin
          
    env:
      LISP: ${{ matrix.lisp }}

    steps:
      - uses: actions/checkout@v1
      - uses: 40ants/setup-lisp@v1
        with:
          asdf-system: cl-info
      - uses: 40ants/run-tests@v2
        with:
          asdf-system: cl-info

The part, corresponding to an action call is:

- uses: 40ants/run-tests@v2
  with:
    asdf-system: cl-info

Here we provided a system name cl-info, but action is smart enough to detect that really there is a separate cl-info-test ASDF system.

To guess a system name to run tests against, action will check ${asdf-system}-test, ${asdf-system}-tests, ${asdf-system}/test and ${asdf-system}/tests. And if none of them found - fall back to ${asdf-system}.

Please, note, that (asdf:test-system :your-system-name) should signal error in case if some tests were failed. Only in this case action will exit with error code.

Custom test runner

Sometimes you might want to use something special instead of (asdf:test-system :your-system-name). You can pass any lisp code to the action:

- uses: 40ants/run-tests@v2
  with:
    asdf-system: cl-info
    run-tests: |
      (ql:quickload :cl-info-test)

      (unless (rove:run :cl-info-test)
         (error "Tests failed"))

Publishing reports to Coveralls

This action automates coverage collection and reporting. To publish report to the Coveralls, pass your github token as a coveralls-token argument:

- uses: 40ants/run-tests@v2
  with:
    asdf-system: cl-info
    coveralls-token: ${{ secrets.github_token }}

If you are using "matrix", then it is good idea to collect coverage report only on one matrix combination. To do this, use a logical expression which will check some variables and returns a token only if all of them are true:

- uses: 40ants/run-tests@v2
  with:
    asdf-system: cl-info
    coveralls-token: |
      ${{ matrix.lisp == 'sbcl-bin' &&
          matrix.os == 'ubuntu-latest' &&
          matrix.quicklisp-dist == 'ultralisp' &&
          secrets.github_token }}

Here is an example how your report on Coveralls can look like:

https://coveralls.io/github/40ants/cl-info

Note, that coverage reporting currently works only on SBCL and CCL 1.4. You can contribute support for other implementations to cl-coveralls.

Roadmap