chadtmiller.com/knotless
git that lives in your Atmosphere account
7
# Workflows
`knotless-run` runs a repository's workflows when its refs move. It reports
each run as a `dev.knotless.check` record in its own account, and the
repository's owner chooses which runners' checks readers show.
[Workflow files](../reference/workflows.md) is the full file format, and
the [command reference](../reference/cli.md#knotless-run) lists the
runner's flags. [knotless-run](../design/runner.md) explains how it decides
what to run and keeps caches and secrets apart.
[deploy/README.md](../../deploy/README.md#knotless-run) shows how to run one
beside knotless-serve.
## Start a runner
A runner acts as an account of its own. Give it that account's handle and
an app password, and name the repositories to watch:
```sh
KNOTLESS_IDENTIFIER=ci.example.com KNOTLESS_PASSWORD=<app password> \
knotless-run --secrets secrets.json alice.example.com/my-project
```
Steps run in Docker containers by default. With `--executor shell` they run
as the runner's user instead, and can read whatever it can, the secrets file
and session included. Use `shell` only for repositories whose writers you
trust with the host.
Then, as the repository's owner, name the runner's account. Readers show
the checks of the accounts named, and no others:
```sh
knotless runners knotless://alice.example.com/my-project add ci.example.com
```
A runner starts from what a repository holds when it first watches it,
and runs only what is pushed after: the branches and pulls already there
wait for their next push. To check one now, ask for it:
```sh
knotless rerun knotless://alice.example.com/my-project --ref refs/heads/main
```
## Write a workflow
Workflows are YAML files in `.knotless/workflows/`. Each says which refs it
runs on, and lists steps. Under Docker, the steps run one after another in
one container of the workflow's image:
```yaml
# .knotless/workflows/ci.yml
on:
push:
refs: [refs/heads/main]
pull:
base: [refs/heads/main]
image: rust:1
cache: [target, /usr/local/cargo/registry]
steps:
- name: test
run: cargo test
```
Push it, and the runner runs it on that commit. `pull` triggers run only on
a runner started with `--pulls`.
Each step can read `KNOTLESS_RUN_REPO`, `KNOTLESS_RUN_REF`,
`KNOTLESS_RUN_SHA`, and, in a pull's run, `KNOTLESS_RUN_PULL` and
`KNOTLESS_RUN_BASE`, the commit the head is merged with. Images a step
leaves in `KNOTLESS_RUN_OUTPUTS` become the check's pictures.
## Run only on some paths
A trigger can name `paths`. The workflow then runs only on a change that
touches one of them:
```yaml
on:
push: {refs: [refs/heads/main]}
pull:
base: [refs/heads/main]
paths: [web/**, harness/layers.mjs, .knotless/workflows/web.yml]
```
`web/**` matches everything under `web`. A workflow a change leaves out
still files its check at once, as `success` with its steps skipped and a
note saying why. Merges and deploys that wait on it go ahead, and the check
list shows it as skipped. [Workflow files](../reference/workflows.md#paths)
has the glob rules.
## Run a workflow only when asked
Some workflows cost too much to run on every push. Give one an `asked`
trigger, and it runs only when the owner asks for it by name:
```yaml
on:
asked: {refs: [refs/heads/main]}
steps:
- run: node harness/scenarios.mjs $TRACKS
timeout: 240
```
```sh
knotless rerun knotless://alice.example.com/my-project --ref refs/heads/main \
--workflow scenarios -e TRACKS="team solo"
```
The owner can ask the same way for a workflow that runs on pushes to that
ref. Variables given with `-e` reach the steps.
A run asked for waits behind every other run. A step's `timeout`, in
minutes, can only shorten the runner's `--step-timeout`, except in a run
asked for, where it may go past it. When the runner stops, as for a deploy,
it does not wait for a run asked for: it cancels it, and runs it again once
it starts.
## Read the results
```sh
knotless checks <url> # newest first
knotless checks <url> --pull <at-uri> --wait 600 --log
knotless checks <url> --commit <sha> --images ./shots
```
`--wait` waits for runs to finish, and fails if none has shown by then.
`--commit` takes a short SHA too, as `git log --oneline` prints it.
`--log` adds the end of each log, and `--images` saves each check's
pictures to a directory. The web pages show
the same: beside a branch, how the newest run of each workflow went, and on
a pull, its checks.
In a repository that names runners, `knotless merge` refuses a pull until
its checks pass. See [Collaborating](collaborating.md#fork-pull-review-and-merge).
To run a ref or a pull again, as the owner:
```sh
knotless rerun <url> --ref refs/heads/main
knotless rerun <url> --pull <at-uri>
knotless rerun <url> --pull <at-uri> --workflow shots
```
The last runs only one of the pull's workflows, leaving the others' checks
as they are.
## Give workflows secrets
A workflow lists the secrets it needs by name:
```yaml
secrets: [DEPLOY_TOKEN]
```
As the repository's owner, give the runners a secret with `knotless
secret`, the value on stdin. It is sealed to each runner the repository
names, so only they can read it, and `--ref` limits it to some refs'
pushes:
```sh
printf %s "$DEPLOY_TOKEN" | knotless secret knotless://alice.example.com/my-project set DEPLOY_TOKEN --ref refs/heads/main
knotless secret knotless://alice.example.com/my-project ls
```
Or add them on the repository's Settings, under Actions, signed in as
its owner. The page seals each value in the browser and lists the
secrets by name, with their refs and the runners they are sealed to.
A runner named later gets each only once it is set again.
A repository that names more than one runner can keep a deploy and its
secrets on one of them: say the hosted runner checks every push and
pull, and a runner of your own deploys. `runners` in the workflow names
the runners that run it, and `--runner` gives the secret to those alone:
```yaml
on: {push: {refs: [refs/heads/main]}}
runners: [deploy.alice.example.com]
secrets: [DEPLOY_TOKEN]
```
```sh
printf %s "$DEPLOY_TOKEN" | knotless secret knotless://alice.example.com/my-project set DEPLOY_TOKEN --ref refs/heads/main --runner deploy.alice.example.com
```
Whoever runs a runner can also grant secrets in its `--secrets` file, per
repository, which wins over one of the same name. A secret can be
granted to the whole repository, or only to pushes to some refs. A branch
can change the workflow, so limit a deploy key to the branch that deploys:
```json
{
"alice.example.com/my-project": {
"API_KEY": "…",
"DEPLOY_TOKEN": { "value": "…", "refs": ["refs/heads/main"] }
}
}
```
Pulls get no secrets. The runner replaces each secret's value with `***` in
the public log, but a step can still leak one transformed, encoded or a
little at a time.
## Mirror to another git host
Git is what knotless shares with every other forge, so a workflow can keep
a copy of a repository on one, such as [Tangled](https://tangled.org). Each
push to the repository then pushes the same ref there. The step has a clone
of the repository, its history included, at the commit pushed.
First make the repository on the other host, and give it a deploy key: an
SSH key made for this alone, its public half added to your account there.
Then give the private half to the runners, for the refs to mirror:
```sh
ssh-keygen -t ed25519 -N '' -C mirror -f mirror
knotless secret knotless://alice.example.com/my-project set MIRROR_KEY \
--ref 'refs/heads/*' --ref 'refs/tags/*' < mirror
```
And add the workflow, with the SSH address the other host's repository page
shows:
```yaml
# .knotless/workflows/mirror.yml
on:
push: {refs: [refs/heads/*, refs/tags/*]}
image: debian:stable-slim
secrets: [MIRROR_KEY]
env: {MIRROR: git@knot.example.com:alice.example.com/my-project}
steps:
- name: mirror
run: |
apt-get update -qq && apt-get install -qq -y git openssh-client >/dev/null
printf '%s\n' "$MIRROR_KEY" > /tmp/key && chmod 600 /tmp/key
git config --global --add safe.directory /work
GIT_SSH_COMMAND='ssh -i /tmp/key -o StrictHostKeyChecking=accept-new' \
git push "$MIRROR" "+$KNOTLESS_RUN_SHA:$KNOTLESS_RUN_REF"
```
The `+` makes the mirror follow the repository even where history was
rewritten. A branch or tag deleted here stays on the mirror until deleted
there. `accept-new` trusts the host's key on first sight. To pin it, put
the host's line from `ssh-keyscan` in the workflow instead.
Only the code is mirrored. Pulls, issues and stars stay on the forge they
were written on.
## Run pulls and the merge queue
Start the runner with `--pulls` to run open pulls too. A pull runs at its
head merged with its base, so it fails if merging it would break the base.
Pulls whose author and head repository's owner may both push to the
repository run as they move. Anyone else's runs only when the owner asks,
with `knotless rerun --pull`.
Start it with `--queue` to land the repository's
[merge queue](collaborating.md#use-the-merge-queue). The runner's account
must be able to push to the branch: the owner, or a collaborator through the
gateway.
Both take the Docker executor.