Control plane and agent

The hosted Chalupa control plane does not control droplets over SSH. The local operator uses SSH once to deliver the ingest key and later to open service tunnels. The agent installed by cloud-init observes the environment and sends data to an HTTPS endpoint. This constraint keeps the hosted control plane compatible with Vercel and reduces its privileges.

Signal flow

1. Pulumi delivers the control-plane URL, agent code, and systemd units without receiving the ingest key.
2. After cloud-init completes, chalupa up streams the ingest key over the operator's existing SSH trust path and stores it atomically. It remains root-only when private log collection is disabled; a logs-enabled stack grants read access only to the isolated chalupa-logs group.
3. The agent observes uptime, containers, disk, and reaper state.
4. It serializes a v1 JSON payload.
5. It signs the method, path, timestamp, nonce, and body hash.
6. It sends the request over HTTPS.
7. The control plane validates the signature and rejects replays.
8. The observation updates the environment detail view.

The normal interval is 30 seconds. Reasonable delays do not imply destruction; an old enough signal changes the state to stale.

Performance telemetry is a separate optional stream. When observability.monitor is configured, the trusted adapter launches a pinned Monitor binary as an unprivileged user, validates each privacy-safe rollup, attaches the deployment identity, and signs it for /api/ingest/telemetry. Monitor never receives the ingest key or control-plane URL. Heartbeats remain the source of current container and lifecycle state; Monitor windows are durable performance history.

The agent service has a systemd path condition and cannot start before /etc/chalupa/ingest.key exists. If SSH delivery fails after the droplet is created, chalupa up exits with a billable-resource warning. Restore the same SSH access used by chalupa tunnel, then retry only delivery and launch reporting:

chalupa agent-bootstrap --config /absolute/path/to/chalupa.yml

This retry is idempotent for the current tvault value. It does not run pulumi up, replace the droplet, or touch persistent storage. It also compares the current config's cloud.url with the provisioned stack output before sending the key. If they differ, use the normal validated chalupa up flow to review and apply that infrastructure change.

Bootstrap waits up to 15 minutes for cloud-init status --wait before it streams either the collector or the key. This prevents an open SSH port from being mistaken for a fully initialized host. A timeout leaves compute intact and potentially billable so you can inspect cloud-init and retry chalupa agent-bootstrap. After the key is installed, bootstrap restarts the heartbeat timer. Its first trigger is relative to that activation, so an earlier condition-skipped attempt cannot consume the only scheduled run.

Verified first-connection identity

StrictHostKeyChecking=accept-new is trust on first use, not verification: the first connection to a spoofed host records the spoof. The droplet's Ed25519 host key fingerprint is public identity material, and the agent reports it in every heartbeat (the hostKeyFingerprint field below), so a single operator can pin it before the ingest credential is streamed.

Fetch the fingerprint the control plane last recorded for this stack, then bootstrap with it:

task ssh:fingerprint CONFIG=/absolute/path/to/chalupa.yml
CHALUPA_TASK_EXPECTED_SSH_FINGERPRINT=SHA256:... task agent-bootstrap CONFIG=/absolute/path/to/chalupa.yml

task ssh:fingerprint is a read-only admin-token call to the control plane; it never touches provider credentials. When no fingerprint has been pinned, agent-bootstrap prints a loud warning that first-connection trust is unverified and still proceeds on the legacy accept-new path, so nothing regresses for an operator who chooses not to pin.

Heartbeat payload

This example is entirely synthetic:

{
  "version": "1",
  "environmentSlug": "demo-harbor",
  "environmentName": "Demo Harbor",
  "observedAt": "2026-07-23T18:20:00.000Z",
  "deployment": {
    "provider": "digitalocean",
    "providerId": "10001",
    "region": "nyc3",
    "sizeSlug": "s-2vcpu-4gb",
    "launchedAt": "2026-07-23T17:50:00.000Z"
  },
  "ip": "192.0.2.44",
  "uptimeSeconds": 1800,
  "containers": [
    {
      "name": "api",
      "state": "running",
      "memoryMiB": 312,
      "cpuPercent": 4.8
    },
    {
      "name": "postgres",
      "state": "running",
      "memoryMiB": 486
    }
  ],
  "reaper": {
    "idleChecks": 2,
    "checkIntervalSeconds": 300,
    "limitMinutes": 45,
    "lastActivityAt": "2026-07-23T18:08:00.000Z"
  },
  "disk": {
    "mountedGiB": 20,
    "usedGiB": 1.4
  },
  "hostKeyFingerprint": "SHA256:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
  "agentVersion": "1"
}

The addresses in this guide come from the RFC 5737 documentation ranges. Do not publish real IP addresses in documentation, metadata, or screenshots.

HMAC signature

Each request includes:

The canonical string is:

v1
POST
/api/ingest
<timestamp>
<nonce>
<sha256-of-body>

The control plane accepts limited clock skew and claims each nonce exactly once. The request signs the exact body bytes; serializing the body again after signing invalidates the request.

The same algorithm signs events, suite runs, and performance windows with their exact endpoint path. A signature for /api/ingest is not valid for /api/ingest/telemetry.

Visible states

Chalupa shows the time of the latest observation. unknown is preferable to inventing a healthy state.

Payload privacy

The heartbeat accepts container names, resource metrics, and deployment data. It does not accept:

• environment variables;
• application logs;
• credentials;
• database contents;
• sensitive arguments;
• artifact files.

If you need to attach larger evidence, report an artifact reference in a suite run.

Monitor telemetry is narrower still: it accepts only eleven fixed numeric metric IDs, explicit availability counts, and allowlisted alert counters. It does not accept container names, host identity, processes, paths, arbitrary labels, raw errors, or profiles. The unprivileged producer has no IP network access, capabilities, ingest key, or access to Chalupa, workload, persistent volume, Docker, common host identity files, mount tables, or network endpoint tables; only the separate root adapter can reach provider metadata and the control plane. See Performance telemetry for the complete boundary.

Explicit events

The agent and lifecycle integrations can send batches to /api/ingest/events to record transitions such as launched, sunk, reaper-warning, reaper-destroyed, or seed-restored. In particular, chalupa down emits sunk after a successful destroy. If Chalupa is unavailable, the task warns you. Subsequent reconciliation can confirm absence only when Chalupa already observed that deployment; verify short-lived missed cycles against the provider invoice.

Each event carries a stable sourceEventId. Repeating the same batch must not duplicate the event.