Logs and diagnose

The read-only commands on this page are chalupa logs, chalupa diagnose and chalupa evidence. Their flags are listed in the CLI command reference.

Logs, diagnose and evidence

Three read-only verbs answer "what happened on the box?" without a hand-written SSH session. They need a running BYOC environment (managed GPU hosts have no Compose stack) and read it over SSH with the same identity and address as the tunnel. SSH never prompts here, so the box's host key must already be in your known_hosts (running chalupa tunnel once records it); otherwise the read fails with Host key verification failed instead of hanging.

There is no passthrough: each verb builds a fixed command from a short allowlist (journalctl -o json, docker compose logs, docker ps and an inspect limited to lifecycle facts, free, df, cloud-init status, failed units, Monitor's issues --json, and the recipes you declared). Nothing that changes the box can be expressed, container environments are never read, and lines are redacted best-effort before they reach your terminal or a file.

chalupa logs api worker --since 15m --level error
chalupa logs --box --since 1h --grep 'out of memory'
chalupa logs api --follow
chalupa logs box --since 20m

chalupa logs reads the box. It is not chalupa test logs, which prints this machine's own transcripts of a chalupa test run (provisioning, tunnel, results). T is 10m, 2h, now, an ISO 8601 instant or @epoch seconds; a time without a zone is UTC. Services that log to journald are read with journalctl; the rest with docker compose logs. --level is exact for the former and a text heuristic for the latter. --box adds the whole box's journal (warnings and above unless you pass --level). The service name box is not a Compose service: it reads the box's own systemd units (cloud-init's stages, chalupa-compose, chalupa-health, the image-cache mount and reset, overlays and replica-set units) with journalctl -u, at every severity unless you pass --level; it is the place to see why a boot was slow.

chalupa diagnose
chalupa diagnose --since-up --bundle ./diagnose-2026-10-01

diagnose is one bounded read: every container with markers for an OOM kill, restarts and non-zero exits, memory and swap, disk usage, cloud-init status, failed units, Monitor issues when observability.monitor is declared, and every recipe in diagnostics:. --since-up adds this boot's warnings and errors. --bundle DIR writes the report and the latest log lines to a new directory (mode 0700, files 0600) that stays on your machine.

chalupa evidence --last-failed --live
chalupa evidence --run-dir ~/.cairntrace/runs/2026-10-01T14-02-11-000Z_checkout_a1b2c3 --md --out evidence.md

evidence is the box side of a failed test run. It reads a cairn run directory without changing it (only run.json and events.ndjson are opened, so screenshots, network captures and traces never enter the brief), derives the window the run was alive plus or minus 30 seconds, and joins it with the box: containers with OOM and restart markers inside that window, error-first logs from every service and the box, host facts and your diagnostics. Markdown is the default, shaped like cairn's own agent context and kept under 16 KB; --json is the full document. --last-failed looks in the artifact root named by the cairn config in test.config, then ./runs beside chalupa.yml, then ~/.cairntrace/runs; --runs-root DIR overrides that.

By default evidence reads the box live when it answers and otherwise falls back, with a note, to the snapshot that chalupa down takes just before the sink. --live insists on a live read; --snapshot (or --no-live) stays offline. The snapshot is a bounded read (30 seconds at most, and it never delays the destroy) of containers, error-first logs and diagnostics, saved under ~/.local/state/chalupa/evidence/<environment>/. That is what lets you explain a failed round after the box is gone. The snapshot records which session took it, and evidence only joins a snapshot to a run from that same session, so a later box that reused the environment name is never presented as the box of an earlier run. Evidence stays local by default; nothing is uploaded.

Exit codes: 0 on success, 64 for a usage or config refusal, 69 when the box could not be reached.