Declarative CI

An environment is only half of a test run. The ci: block declares what should execute inside it, so a suite can run without an operator watching a terminal for hours.

The schema, the offline plan preview, execution, secret delivery, and reporting all exist today. The offline surface was built and enforced first on purpose — a plan whose cost cannot be reviewed before it runs should not be started — and it stays mandatory even now that a suite can actually run: task ci-preview is still part of the validate gate.

Declaring a suite

ci:
  suite: demo-harbor-smoke
  budget:
    maxMinutes: 45
  steps:
    - id: stack-health
      run: curl -fsS http://127.0.0.1:8080/ >/dev/null
      timeoutMinutes: 10

    - id: api-migrations
      on: service:api
      needs: [stack-health]
      run: /app/bin/migrate --check
      timeoutMinutes: 10
      collect:
        - path: /app/var/migrate-report.json
          when: always

    - id: teardown-report
      needs: [api-migrations]
      when: always
      run: echo "collected"
      timeoutMinutes: 5

ci: requires cloud.url, because results and evidence have to outlive the compute that produced them.

Steps

Steps form a directed acyclic graph through needs:. With the default maxParallel: 1 execution is a deterministic topological order, so the graph costs nothing in complexity while still expressing two things a flat list cannot: that a step should be skipped when a dependency did not succeed, and that a reporting step should run even when one branch is red.

There is no expression language. when covers the real cases without putting a second language inside YAML strings.

Verifier steps

uses: glyphrun and uses: cairntrace are step types rather than plain shell commands, which lets Chalupa map their documented exit codes onto step status without parsing output, attach their standard evidence paths automatically, and report their results through the existing suite contract.

    - id: terminal-specs
      uses: glyphrun
      specs: [tests/glyph/navigation.yml]

    - id: browser-specs
      uses: cairntrace
      environment: chalupa
      specs: [flows/checkout.yml]
      labels: { path: temporal }

Both drive something that lives on the host — a pseudo-terminal in one case, a browser in the other — so they always run on the droplet and cannot target a service container.

An exit code neither tool documents is reported as errored, never as a test failure. "We do not understand what the tool did" and "your tests failed" are different statements and must not be conflated.

Host requirements

ci.requires names the host tools a suite depends on. The engine enforces them at host-prepare, before the first step runs — a missing tool is never a surprise two hours into a paid suite:

  requires:
    - { tool: docker, minVersion: "24.0.0" }

Presence on PATH is the gate. node, docker, tmux and yarn are apt-installed when missing (one bounded apt-get update, then one apt-get install per tool); cairn, glyph, bun and go have no Ubuntu package and must come from compute.image. A declared minVersion is checked against the first line of <tool> --version when the tool answers cheaply; an unreadable version counts as unmet. Anything still unmet after the install pass fails the run in the errored state with the missing tool, its floor version, and the fix (compute.image) named — never silently ignored. A dry run only inspects the host, so it reports a missing apt-installable tool as unmet without installing it.

Delivering the source

ci.source hands a workspace environment its code — the project being tested is not always a pre-built Compose image. The engine clones a shallow checkout into its workspace before the first step and exports two variables to every step:

  source:
    git: https://github.com/acme/widgets.git
    ref: main
    secret: GITHUB_DEPLOY_TOKEN   # optional; a ci.secrets name, never a value

• The URL must be https and credential-free; an ssh URL would depend on agent forwarding, and a token embedded in the URL would be an exfiltration primitive in a reviewed config.
• A deploy token, when declared, is delivered through the ci.secrets SSH channel and fed to git via a transient askpass script owned by the engine — it appears neither in ps, the journal, nor .git/config.
• Steps reach the checkout through $CHALUPA_WORKSPACE (absolute path) and $CHALUPA_SOURCE_COMMIT (the cloned HEAD). Nothing changes cwd semantics: a step that wants the source runs cd "$CHALUPA_WORKSPACE" itself.
• A clone failure ends the run in the errored state before any step, with the redacted git error as the reason; a dry run never clones.

Secrets

ci.secrets declares names only:

  secrets:
    - { name: APP_PASSWORD }
    - { name: OPTIONAL_TOKEN, required: false }

Values never appear in chalupa.yml, and Chalupa never learns where they are stored. Reading them stays the operator's job, outside Chalupa. Names carrying a reserved prefix — including CHALUPA_, DIGITALOCEAN_, TVAULT_ and FILECHEAP_ — are rejected so a project can never shadow a platform or provider credential.

A step only receives the names it lists in its own secrets: array, and every name it lists must be declared at suite level.

Declaring names is only half the setup. Once compute exists, deliver the actual values with task ci:secrets, piping them from stdin in NAME=value form; there is no chalupa ci secrets — delivery reads a payload from stdin, which the allowlisted chalupa actions do not support:

printf 'APP_PASSWORD=%s\n' "$(tvault get --project demo-harbor APP_PASSWORD)" |
  task ci:secrets CONFIG="$DEMO_CONFIG"

The task validates the payload shape, computes its digest locally, and delivers it over the same SSH boundary as ci:arm. Chalupa never learns where the values came from, and the delivered file is written 0600, readable only by the engine's root process. Deliver secrets before task ci:arm; the engine reads /etc/chalupa/ci/secrets.env at plan start and does not poll for a later delivery.

Budget

budget.maxMinutes is enforced offline. Validation sums every step's timeout multiplied by its attempts and rejects a configuration whose declared worst case does not fit:

declared worst case is 605 minutes; raise budget.maxMinutes or lower step timeouts

That check is the reason each step needs an explicit timeout. Optional budget.maxUsd bounds spend for the same reason.

Previewing offline

task ci-preview renders the resolved plan without contacting any provider, and is part of the mandatory validate gate. Like task validate, it belongs to the repository checkout rather than to a running environment:

── ci suite: demo-harbor-smoke   maxParallel: 1   onFinish: keep
── plan (topological, 4 steps):
    1  stack-health      droplet                               10m
    2  api-migrations    service:api           needs: 1        10m
    3  gateway-smoke     service:gateway       needs: 1        10m
    4  teardown-report   droplet               needs: 2,3       5m  when: always
── compose services targeted: api gateway
── secret names required (values are never read here): (none)
── artifact globs: 1  budget 32 MiB  chunk 1024 KiB  retention 7d
── worst case: 35 min of 45 budgeted (78%)

Service targets are checked against the pruned Compose document rather than the services: list, because pruning can remove a service the list still names. Only the post-prune comparison catches a step that would otherwise fail hours into a paid run.

bun src/cli/ci-plan.ts --config <path> emits the same plan as JSON, and --hourly-usd <rate> adds a worst-case cost ceiling.

Running a suite

Once compute exists, the suite is operated through the CLI. The Taskfile names these actions ci:arm, ci:follow, ci:cancel, and ci:report; the CLI accepts the group and the member as two words so no command needs a colon:

chalupa ci arm --config "$DEMO_CONFIG"
chalupa ci follow --config "$DEMO_CONFIG"
chalupa ci cancel --config "$DEMO_CONFIG"
chalupa ci report --config "$DEMO_CONFIG"

Arming is classified as billable: it starts a suite that holds the droplet open for its whole budget, so it requires an interactive terminal and the exact test <environment> phrase. Cancelling requires cancel <environment> and publishes what was already collected; it never sinks compute, which stays a separate chalupa down decision. ci report needs no confirmation phrase — the run is already a fact, and republishing it is inert because the control plane keys on sourceRunId.

There is no chalupa ci status. The engine's live run state is a bounded read-only refresh the operator UI issues for itself; open chalupa tui to watch it, or run task ci:status CONFIG=<path> directly from the checkout for the raw JSON.

Reporting results

The engine itself never publishes anything — it has no provider credential and cannot reach the control plane. Publication is best effort and happens at three points, in order of how soon you learn the outcome:

• chalupa ci report (or task ci:report), run explicitly once a suite settles;
• chalupa ci cancel, which calls ci:report for you after stopping the run;
• chalupa down, which calls ci:report once more immediately before destroying compute — the last moment summary.json still exists — so a finished run that nobody manually published is not lost by teardown.

Publishing is idempotent: the control plane keys on sourceRunId, so any overlap between these three simply republishes the same result.

Terminal specs run outside the ci: engine share the same reporting path:

task report REPORT=cairn-stats.json
bun src/cli/report.ts --format glyphrun --file runs.json --suite terminal

Terminal specs report through --format glyphrun, which accepts a JSON array of run documents and requires an explicit --suite, since those documents carry no suite name of their own.

Finishing the environment

onFinish declares what should happen to compute once a run settles:

ci:
  onFinish: sink-on-success

keep (the default), sink, and sink-on-success are the only values. The engine cannot destroy itself — it holds no provider credential — so onFinish is honored by the local, operator-run teardown runner instead: ci:report records the settled outcome against this deployment's registration, and the next task teardown:run (or the armed, scheduled variant) destroys compute when the recorded outcome satisfies the declared policy. A sinking onFinish therefore does nothing on its own without a registration:

chalupa teardown register --config "$DEMO_CONFIG"

Declaring onFinish: sink or sink-on-success without a session block is a validation error, because the runner that can act on it only ever targets a registered, session-bound deployment.

Be plain about what session.expiresAfterMinutes actually is: control-plane state, not an infrastructure shutdown mechanism. Nothing on the droplet or in Vercel holds a DigitalOcean credential, so expiry cannot destroy anything by itself — it only marks a deployment as past its deadline for anything that later chooses to look. The only thing that ever turns that deadline into a stopped droplet is a registered and armed local teardown runner (task teardown:register, then task teardown:run-armed TEARDOWN_ENABLED=1, or the installed launchd schedule) actually calling task down on your machine. Register without arming, or never install the schedule, and a sinking onFinish policy is declared but inert: the environment keeps billing right through and past its expiry until a human runs task down themselves. This is exactly what the ci_on_finish_external and session_teardown_external validation warnings say, every time.

Reporting from somewhere else

A project whose tests already run elsewhere — in its own pipeline, against a preview deployment — can publish results without a chalupa.yml and without provisioning anything:

bun src/cli/report-ci.ts \
  --summary summary.json \
  --environment my-project-preview \
  --cloud-url https://chalupa.run

Publishing reads two things: the environment slug and the control plane origin. It opens no Compose file, contacts no droplet and needs no SSH identity, so a Compose file the project does not have was never a real requirement. The control plane registers the environment on first report; it stays unknown because no agent ever described a deployment for it, which is the honest state for a run that had no compute of its own.

CHALUPA_INGEST_KEY is still required, and the slug and origin are validated against the same rules chalupa.yml uses.

Idle observation

Idle is measured from established SSH connections. A headless run has none, so declaring both ci: and idleShutdownMinutes raises a warning:

WARNING ci_idle_shutdown_conflict (idleShutdownMinutes): idle is observed from
established SSH connections, and a headless CI run has none

Treat it as a real conflict: an idle window shorter than the suite will report the environment as idle while the suite is still running.

Evidence size

artifacts.maxChunkBytes is capped at the artifact service limit and maxTotalBytes at the session limit, so an oversized policy fails offline instead of partway through a finished run.

Collecting trace or video output raises a warning, because failed browser runs with those enabled reach roughly a gigabyte each. Capture them while debugging a specific red run, not by default.

On-failure diagnose with Monitor

Every step already runs with three correlation variables in its environment, regardless of whether onFailure.investigate is declared:

CHALUPA_ENV=<the environment name>
CHALUPA_DEPLOYMENT_ID=<this droplet's numeric provider id, if registered>
CHALUPA_RUN_ID=<the pipeline run id>

These are the fallback names Monitor's own context-precedence table reads below CHALUPA_CI_ENVIRONMENT and CHALUPA_CI_RUN_ID, which the engine already set. CHALUPA_DEPLOYMENT_ID is absent, not empty, on a droplet whose environment predates registration.

Opt into automatic diagnosis by declaring ci.onFailure.investigate:

ci:
  onFailure:
    investigate:
      enabled: true
      pids: [api, top_cpu]
      timeoutSeconds: 30

pids names candidates to resolve a PID from, tried in order: a Compose service name (resolved via docker compose ps then docker inspect, falling back to monitor processes --filter when no such service exists), or the literal top_cpu, which asks Monitor's own process inventory for the droplet's hottest process instead of naming one. codebase is optional and defaults to the project directory; timeoutSeconds (default 30, max 120) bounds the whole monitor investigate call.

When a step fails or errors, the engine runs monitor investigate on the droplet (or tunnel host) with that PID, entirely best effort: a timeout, a missing Monitor install, or a malformed result never changes the step's own outcome and never blocks the run from finalizing. chalupa ci follow narrates either result:

[INCIDENT] deploy captured (api): fcheap://stash/inc_..., 1s
[INCIDENT] deploy not captured: pid-unresolved

A successful capture produces a monitor.incident ArtifactRefV1 — Monitor's own local file.cheap stash reference, never the profile or incident bytes themselves — and Chalupa attaches it to the run's own evidence, stored alongside the run's other artifacts and reported through ci report the same way step evidence already is. The console's pipeline run page renders it next to per-spec evidence links as View incident.

onFailure.investigate calls Monitor's own companion tools — fcheap, codemap, vecgrep — directly, never through ci.requires, whose tool enum is closed to what compute.image bakes in. Unlike ci.requires — which the engine verifies at host-prepare and fails the run on when unmet (see Host requirements) — these are still declared rather than verified: an image missing one fails the hook at run time, not before, and the hook is best-effort by design, so that failure never blocks the run. task ci-preview says so explicitly:

── onFailure.investigate requires on the droplet (declared, not verified): fcheap>=0.30.0 codemap>=0.40.0 vecgrep>=2.20.0

Performance telemetry (see Performance telemetry) stays identity-free either way: it never contains a PID, a process name, or this run's incident evidence. Correlating a performance window with an incident is done by deployment_id and the run's time window, never by a shared process identifier — Monitor's own contract is explicit that the two lanes must never mix.