Workflows

knotless-run runs a repository’s workflows when its refs move. It reports each run as a dev.knotless.check record in its own account, and the repository’s owner chooses which runners’ checks readers show.

Workflow files is the full file format, and the command reference lists the runner’s flags. knotless-run explains how it decides what to run and keeps caches and secrets apart. deploy/README.md shows how to run one beside knotless-serve.

Start a runner

A runner acts as an account of its own. Give it that account’s handle and an app password, and name the repositories to watch:

KNOTLESS_IDENTIFIER=ci.example.com KNOTLESS_PASSWORD=<app password> \
  knotless-run --secrets secrets.json alice.example.com/my-project

Steps run in Docker containers by default. With --executor shell they run as the runner’s user instead, and can read whatever it can, the secrets file and session included. Use shell only for repositories whose writers you trust with the host.

Then, as the repository’s owner, name the runner’s account. Readers show the checks of the accounts named, and no others:

knotless runners knotless://alice.example.com/my-project add ci.example.com

A runner starts from what a repository holds when it first watches it, and runs only what is pushed after: the branches and pulls already there wait for their next push. To check one now, ask for it:

knotless rerun knotless://alice.example.com/my-project --ref refs/heads/main

Write a workflow

Workflows are YAML files in .knotless/workflows/. Each says which refs it runs on, and lists steps. Under Docker, the steps run one after another in one container of the workflow’s image:

# .knotless/workflows/ci.yml
on:
  push:
    refs: [refs/heads/main]
  pull:
    base: [refs/heads/main]
image: rust:1
cache: [target, /usr/local/cargo/registry]
steps:
  - name: test
    run: cargo test

Push it, and the runner runs it on that commit. pull triggers run only on a runner started with --pulls.

Each step can read KNOTLESS_RUN_REPO, KNOTLESS_RUN_REF, KNOTLESS_RUN_SHA, and, in a pull’s run, KNOTLESS_RUN_PULL and KNOTLESS_RUN_BASE, the commit the head is merged with. Images a step leaves in KNOTLESS_RUN_OUTPUTS become the check’s pictures.

Run only on some paths

A trigger can name paths. The workflow then runs only on a change that touches one of them:

on:
  push: {refs: [refs/heads/main]}
  pull:
    base: [refs/heads/main]
    paths: [web/**, harness/layers.mjs, .knotless/workflows/web.yml]

web/** matches everything under web. A workflow a change leaves out still files its check at once, as success with its steps skipped and a note saying why. Merges and deploys that wait on it go ahead, and the check list shows it as skipped. Workflow files has the glob rules.

Run a workflow only when asked

Some workflows cost too much to run on every push. Give one an asked trigger, and it runs only when the owner asks for it by name:

on:
  asked: {refs: [refs/heads/main]}
steps:
  - run: node harness/scenarios.mjs $TRACKS
    timeout: 240

knotless rerun knotless://alice.example.com/my-project --ref refs/heads/main \
  --workflow scenarios -e TRACKS="team solo"

The owner can ask the same way for a workflow that runs on pushes to that ref. Variables given with -e reach the steps.

A run asked for waits behind every other run. A step’s timeout, in minutes, can only shorten the runner’s --step-timeout, except in a run asked for, where it may go past it. When the runner stops, as for a deploy, it does not wait for a run asked for: it cancels it, and runs it again once it starts.

Read the results

knotless checks <url>                              # newest first
knotless checks <url> --pull <at-uri> --wait 600 --log
knotless checks <url> --commit <sha> --images ./shots

--wait waits for runs to finish, and fails if none has shown by then. --commit takes a short SHA too, as git log --oneline prints it. --log adds the end of each log, and --images saves each check’s pictures to a directory. The web pages show the same: beside a branch, how the newest run of each workflow went, and on a pull, its checks.

In a repository that names runners, knotless merge refuses a pull until its checks pass. See Collaborating.

To run a ref or a pull again, as the owner:

knotless rerun <url> --ref refs/heads/main
knotless rerun <url> --pull <at-uri>
knotless rerun <url> --pull <at-uri> --workflow shots

The last runs only one of the pull’s workflows, leaving the others’ checks as they are.

Give workflows secrets

A workflow lists the secrets it needs by name:

secrets: [DEPLOY_TOKEN]

As the repository’s owner, give the runners a secret with knotless secret, the value on stdin. It is sealed to each runner the repository names, so only they can read it, and --ref limits it to some refs’ pushes:

printf %s "$DEPLOY_TOKEN" | knotless secret knotless://alice.example.com/my-project set DEPLOY_TOKEN --ref refs/heads/main
knotless secret knotless://alice.example.com/my-project ls

Or add them on the repository’s Settings, under Actions, signed in as its owner. The page seals each value in the browser and lists the secrets by name, with their refs and the runners they are sealed to.

A runner named later gets each only once it is set again.

A repository that names more than one runner can keep a deploy and its secrets on one of them: say the hosted runner checks every push and pull, and a runner of your own deploys. runners in the workflow names the runners that run it, and --runner gives the secret to those alone:

on: {push: {refs: [refs/heads/main]}}
runners: [deploy.alice.example.com]
secrets: [DEPLOY_TOKEN]

printf %s "$DEPLOY_TOKEN" | knotless secret knotless://alice.example.com/my-project set DEPLOY_TOKEN --ref refs/heads/main --runner deploy.alice.example.com

Whoever runs a runner can also grant secrets in its --secrets file, per repository, which wins over one of the same name. A secret can be granted to the whole repository, or only to pushes to some refs. A branch can change the workflow, so limit a deploy key to the branch that deploys:

{
  "alice.example.com/my-project": {
    "API_KEY": "…",
    "DEPLOY_TOKEN": { "value": "…", "refs": ["refs/heads/main"] }
  }
}

Pulls get no secrets. The runner replaces each secret’s value with *** in the public log, but a step can still leak one transformed, encoded or a little at a time.

Mirror to another git host

Git is what knotless shares with every other forge, so a workflow can keep a copy of a repository on one, such as Tangled. Each push to the repository then pushes the same ref there. The step has a clone of the repository, its history included, at the commit pushed.

First make the repository on the other host, and give it a deploy key: an SSH key made for this alone, its public half added to your account there. Then give the private half to the runners, for the refs to mirror:

ssh-keygen -t ed25519 -N '' -C mirror -f mirror
knotless secret knotless://alice.example.com/my-project set MIRROR_KEY \
  --ref 'refs/heads/*' --ref 'refs/tags/*' < mirror

And add the workflow, with the SSH address the other host’s repository page shows:

# .knotless/workflows/mirror.yml
on:
  push: {refs: [refs/heads/*, refs/tags/*]}
image: debian:stable-slim
secrets: [MIRROR_KEY]
env: {MIRROR: git@knot.example.com:alice.example.com/my-project}
steps:
  - name: mirror
    run: |
      apt-get update -qq && apt-get install -qq -y git openssh-client >/dev/null
      printf '%s\n' "$MIRROR_KEY" > /tmp/key && chmod 600 /tmp/key
      git config --global --add safe.directory /work
      GIT_SSH_COMMAND='ssh -i /tmp/key -o StrictHostKeyChecking=accept-new' \
        git push "$MIRROR" "+$KNOTLESS_RUN_SHA:$KNOTLESS_RUN_REF"

The + makes the mirror follow the repository even where history was rewritten. A branch or tag deleted here stays on the mirror until deleted there. accept-new trusts the host’s key on first sight. To pin it, put the host’s line from ssh-keyscan in the workflow instead.

Only the code is mirrored. Pulls, issues and stars stay on the forge they were written on.

Run pulls and the merge queue

Start the runner with --pulls to run open pulls too. A pull runs at its head merged with its base, so it fails if merging it would break the base. Pulls whose author and head repository’s owner may both push to the repository run as they move. Anyone else’s runs only when the owner asks, with knotless rerun --pull.

Start it with --queue to land the repository’s merge queue. The runner’s account must be able to push to the branch: the owner, or a collaborator through the gateway.

Both take the Docker executor.