Declarative CI

An environment is only half of a test run. The ci: block declares what should execute inside it, so a suite can run without an operator watching a terminal for hours.

This page documents the parts that exist today: the schema, the offline plan preview, and the reporting path. Execution is being delivered in stages, and the offline surface is complete and enforced first on purpose — a plan whose cost cannot be reviewed before it runs should not be started.

Declaring a suite

ci:
  suite: demo-harbor-smoke
  budget:
    maxMinutes: 45
  steps:
    - id: stack-health
      run: curl -fsS http://127.0.0.1:8080/ >/dev/null
      timeoutMinutes: 10

    - id: api-migrations
      on: service:api
      needs: [stack-health]
      run: /app/bin/migrate --check
      timeoutMinutes: 10
      collect:
        - path: /app/var/migrate-report.json
          when: always

    - id: teardown-report
      needs: [api-migrations]
      when: always
      run: echo "collected"
      timeoutMinutes: 5

ci: requires cloud.url, because results and evidence have to outlive the compute that produced them.

Steps

Steps form a directed acyclic graph through needs:. With the default maxParallel: 1 execution is a deterministic topological order, so the graph costs nothing in complexity while still expressing two things a flat list cannot: that a step should be skipped when a dependency did not succeed, and that a reporting step should run even when one branch is red.

There is no expression language. when covers the real cases without putting a second language inside YAML strings.

Verifier steps

uses: glyphrun and uses: cairntrace are step types rather than plain shell commands, which lets Chalupa map their documented exit codes onto step status without parsing output, attach their standard evidence paths automatically, and report their results through the existing suite contract.

    - id: terminal-specs
      uses: glyphrun
      specs: [tests/glyph/navigation.yml]

    - id: browser-specs
      uses: cairntrace
      environment: do
      specs: [flows/checkout.yml]
      labels: { path: temporal }

Both drive something that lives on the host — a pseudo-terminal in one case, a browser in the other — so they always run on the droplet and cannot target a service container.

An exit code neither tool documents is reported as errored, never as a test failure. "We do not understand what the tool did" and "your tests failed" are different statements and must not be conflated.

Secrets

ci.secrets declares names only:

  secrets:
    - { name: APP_PASSWORD }
    - { name: OPTIONAL_TOKEN, required: false }

Values never appear in chalupa.yml, and Chalupa never learns where they are stored. Reading them stays the operator's job, outside Chalupa. Names carrying a reserved prefix — including CHALUPA_, DIGITALOCEAN_, TVAULT_ and FILECHEAP_ — are rejected so a project can never shadow a platform or provider credential.

A step only receives the names it lists in its own secrets: array, and every name it lists must be declared at suite level.

Budget

budget.maxMinutes is enforced offline. Validation sums every step's timeout multiplied by its attempts and rejects a configuration whose declared worst case does not fit:

declared worst case is 605 minutes; raise budget.maxMinutes or lower step timeouts

That check is the reason each step needs an explicit timeout. Optional budget.maxUsd bounds spend for the same reason.

Previewing offline

task ci-preview renders the resolved plan without contacting any provider, and is part of the mandatory validate gate:

── ci suite: demo-harbor-smoke   maxParallel: 1   onFinish: keep
── plan (topological, 4 steps):
    1  stack-health      droplet                               10m
    2  api-migrations    service:api           needs: 1        10m
    3  gateway-smoke     service:gateway       needs: 1        10m
    4  teardown-report   droplet               needs: 2,3       5m  when: always
── compose services targeted: api gateway
── secret names required (values are never read here): (none)
── artifact globs: 1  budget 32 MiB  chunk 1024 KiB  retention 7d
── worst case: 35 min of 45 budgeted (78%)

Service targets are checked against the pruned Compose document rather than the services: list, because pruning can remove a service the list still names. Only the post-prune comparison catches a step that would otherwise fail hours into a paid run.

bun src/cli/ci-plan.ts --config <path> emits the same plan as JSON, and --hourly-usd <rate> adds a worst-case cost ceiling.

Reporting results

Results reach the control plane through the existing suite contract, so CI results and manually reported runs share one view:

task report REPORT=cairn-stats.json
bun src/cli/report.ts --format glyphrun --file runs.json --suite terminal

Terminal specs report through --format glyphrun, which accepts a JSON array of run documents and requires an explicit --suite, since those documents carry no suite name of their own.

Idle observation

Idle is measured from established SSH connections. A headless run has none, so declaring both ci: and idleShutdownMinutes raises a warning:

WARNING ci_idle_shutdown_conflict (idleShutdownMinutes): idle is observed from
established SSH connections, and a headless CI run has none

Treat it as a real conflict: an idle window shorter than the suite will report the environment as idle while the suite is still running.

Evidence size

artifacts.maxChunkBytes is capped at the artifact service limit and maxTotalBytes at the session limit, so an oversized policy fails offline instead of partway through a finished run.

Collecting trace or video output raises a warning, because failed browser runs with those enabled reach roughly a gigabyte each. Capture them while debugging a specific red run, not by default.