Suites and artifacts

A suite run answers three questions: which spec ran, in which environment, and how it ended. The control plane retains comparable metrics; large files remain elsewhere and are linked.

The Taskfile examples assume:

export DEMO_CONFIG="/path/to/project/chalupa.yml"

Run model

{
  "version": "1",
  "environmentSlug": "demo-harbor",
  "suiteName": "checkout-e2e",
  "runs": [
    {
      "sourceRunId": "run_demo_20260723_001",
      "specName": "create-order",
      "status": "passed",
      "startedAt": "2026-07-23T18:30:00.000Z",
      "durationMs": 18420,
      "metricName": "workflow-latency",
      "metricMs": 1220,
      "variant": "baseline",
      "artifacts": [
        {
          "kind": "report",
          "store": "fcheap-local",
          "stashId": "stash_demo_report_001"
        },
        {
          "kind": "trace",
          "store": "link",
          "url": "https://artifacts.example.com/traces/demo-001"
        }
      ]
    }
  ]
}

sourceRunId identifies the execution in its source system. It must remain stable and idempotent within the integration.

Statuses

Do not turn infrastructure errors into failed results if that distinction matters during investigation.

Duration and metric

durationMs always describes the total run duration. metricName and metricMs are optional and represent one comparable measurement within the spec.

The runs view calculates p50 only across matching suite, environment, spec, variant, and metric identities. A percentile with one sample is context, not a statistical trend.

Report results

When the task report integration is enabled, send your runner's JSON to the signed endpoint:

task report CONFIG="$DEMO_CONFIG" REPORT="./runs.json" SUITE="checkout-e2e"

The task handles signing and must not print the key. You can also implement a compatible sender by following API and schema.

fcheap-local backend

With fcheap-local, Chalupa stores only the stashId. To retrieve an artifact, the console presents this command:

fcheap restore stash_demo_report_001

The file remains in the operator's local vault. Chalupa does not assume that a remote file.cheap service exists.

Link backend

store: link accepts an HTTP or HTTPS URL without embedded username or password fields. Visibility and expiration depend on the link provider.

Do not put secrets in URL query strings. Prefer separate authorization or short-lived signed URLs when the artifact is not public.

Artifact types

Supported types are:

• video;
• trace;
• report;
• screenshot;
• logs;
• other.

The type helps the interface present the correct action; it does not change the storage policy.

Responsibility boundaries

Chalupa:

• indexes references;
• associates them with runs;
• shows commands or links;
• retains structured metrics.

The artifact store:

• stores bytes;
• defines retention;
• controls access;
• handles file sizes and transfers.