Suites and artifacts

A suite run answers three questions: which spec ran, in which environment, and how it ended. The control plane retains comparable metrics; large files remain elsewhere and are linked.

The Taskfile examples assume:

export DEMO_CONFIG="/path/to/project/chalupa.yml"

Run model

{
  "version": "1",
  "environmentSlug": "demo-harbor",
  "suiteName": "checkout-e2e",
  "runs": [
    {
      "sourceRunId": "run_demo_20260723_001",
      "specName": "create-order",
      "status": "passed",
      "startedAt": "2026-07-23T18:30:00.000Z",
      "durationMs": 18420,
      "metricName": "workflow-latency",
      "metricMs": 1220,
      "variant": "baseline",
      "artifacts": [
        {
          "$schema": "urn:filecheap.dev:artifact-ref:v1",
          "version": 1,
          "provider": "fcheap-local",
          "uri": "fcheap://stash/report_20260723_184500.123456789_0123456789abcdef01234567",
          "artifact_id": "report_20260723_184500.123456789_0123456789abcdef01234567",
          "kind": "cairntrace.run",
          "producer": {
            "tool": "cairntrace",
            "native_schema": "urn:cairntrace.dev:run:v1",
            "native_id": "run_demo_20260723_001",
            "entrypoint": "run.json"
          }
        },
        {
          "$schema": "urn:filecheap.dev:artifact-ref:v1",
          "version": 1,
          "provider": "link",
          "uri": "https://artifacts.example.com/traces/demo-001",
          "kind": "chalupa.report"
        }
      ]
    }
  ]
}

sourceRunId identifies the execution in its source system. It must remain stable and idempotent within the integration.

Statuses

Do not turn infrastructure errors into failed results if that distinction matters during investigation.

Duration and metric

durationMs always describes the total run duration. metricName and metricMs are optional and represent one comparable measurement within the spec.

The runs view calculates p50 only across matching suite, environment, spec, variant, and metric identities. A percentile with one sample is context, not a statistical trend.

Report results

When the task report integration is enabled, send your runner's JSON to the signed endpoint:

task report \
  CONFIG="$DEMO_CONFIG" \
  REPORT="./runs.json" \
  ARTIFACTS="./artifact-sidecar.json" \
  SUITE="checkout-e2e"

The optional sidecar associates complete references with raw Cairn run IDs:

{
  "$schema": "urn:chalupa.run:artifact-sidecar:v1",
  "version": 1,
  "runs": {
    "run_demo_20260723_001": [
      {
        "$schema": "urn:filecheap.dev:artifact-ref:v1",
        "version": 1,
        "provider": "fcheap-local",
        "uri": "fcheap://stash/report_20260723_184500.123456789_0123456789abcdef01234567",
        "artifact_id": "report_20260723_184500.123456789_0123456789abcdef01234567",
        "kind": "cairntrace.run"
      }
    ]
  }
}

Create each reference from a completed stash instead of typing the envelope by hand:

fcheap artifact-ref <stash-id> \
  --kind cairntrace.run \
  --producer-tool cairntrace \
  --native-schema urn:cairntrace.dev:run:v1 \
  --native-id run_demo_20260723_001 \
  --entrypoint run.json \
  --json

The CLI prints one ArtifactRefV1 object. The MCP tool fcheap_artifact_ref returns the same object as structured content. Place that object under the matching raw run ID in the sidecar; do not add Chalupa fields inside the envelope.

Every sidecar key must match a run in the Cairn stats document. Orphan keys, duplicate provider:uri identities, unknown fields, more than 250 keys, or more than 20 references per run fail locally before the report request is sent. The CLI reads at most 4 MiB of raw Cairn JSON and 1 MiB of sidecar JSON, then enforces the 256 KiB normalized request limit. Omitting ARTIFACTS keeps the artifact-free reporting flow unchanged.

The task handles signing and must not print the key. You can also implement a compatible sender by following API and schema.

Attach artifacts on the first ingest. A retry must use a fresh HMAC nonce while resending the exact same normalized run and sidecar. Identical content deduplicates; changing artifacts, producer metadata, status, metrics, or other run facts for the same source run returns 409 run_ingest_conflict and does not overwrite the stored run.

ArtifactRefV1

Chalupa stores and returns the complete envelope. It does not silently reduce it to a stash ID or URL. $schema, version, provider, uri, artifact_id, kind, producer routing metadata, and web_url all survive ingest, storage, and readback whenever they are present.

The validators keep transport locators free of embedded credentials, but they are not a data-loss-prevention scanner. Artifact IDs, kinds, and producer names, versions, commands, and entrypoints can reveal operational details. Producers must keep every metadata field non-sensitive.

Unknown fields are rejected. In particular, do not add an integrity field: file.cheap verifies restored bytes against its own manifest, and ArtifactRefV1 does not duplicate a legacy content hash.

New producers must send ArtifactRefV1. /api/ingest/runs temporarily accepts only the two exact historical locator shapes from older clients and upgrades them before normalization, fingerprinting, or storage. That compatibility path does not accept extra fields, unsafe links, invalid local IDs, or duplicates created by conversion. It is deprecated and must remain only until every deployed producer sends V1 and pre-fingerprint historical rows are remediated.

Conformance and versioning

The vendored schema.json defines the strict JSON shape and provider variants. Chalupa's runtime validators also enforce semantic rules that structural JSON Schema cannot express by itself:

• local and cloud URIs must identify the exact artifact_id;
• HTTP ports must be in the 0..65535 range;
• an opaque urn: value must contain a non-empty identifier;
• embedded credentials, query strings, and unsafe relative entrypoints are forbidden in transport locators.

The complete valid and invalid fixture corpus runs through both the report CLI validator and the Chalupa ingest validator. Its validation semantics are derived from the reviewed file.cheap commit. A provenance manifest records every intentional byte-level transformation and a deterministic SHA-256 checksum of the resulting schema plus fixtures. Chalupa renames two local fixtures, uses a reserved neutral example host, and updates the hosted-provider description to match the reviewed private service. This guards against accidental drift; it does not make the two runtime implementations a shared package.

ArtifactRefV1 is immutable. Any backward-incompatible shape or semantic change requires a new schema URI and contract version instead of silently changing urn:filecheap.dev:artifact-ref:v1.

Ordering and retries

Artifact references are an ordered list. Chalupa preserves their order and includes it in the run's idempotency fingerprint. JSON object key order does not matter, but reordering the artifact list is a semantic change. Resend the same list in the same order on retries; otherwise an existing source run returns 409 run_ingest_conflict.

fcheap-local provider

provider: fcheap-local identifies an opaque stash in a vault on the operator's machine. The uri must exactly match fcheap://stash/<artifact_id>. To retrieve it, the console presents:

fcheap restore report_20260723_184500.123456789_0123456789abcdef01234567

The file remains in the operator's local vault. Chalupa does not assume that a remote file.cheap service exists, does not turn the local URI into a browser link, and cannot determine whether another machine has that vault.

Link provider

provider: link accepts a stable HTTP or HTTPS uri. Credentials, query strings, and fragments are forbidden. Visibility and retention still depend on the external provider. Chalupa opens the link only after an administrator chooses it; it never crawls the URL server-side.

fcheap-cloud provider

provider: fcheap-cloud identifies a committed artifact in the private hosted file.cheap service. file.cheap v0.31.0 or newer can publish a bounded archive through an authenticated plan → PUT → commit exchange and returns a stable, credential-free ArtifactRefV1. Cairntrace v2.1.0 and Glyphrun v0.16.0 use that publisher for complete-run evidence.

Chalupa validates and preserves the reference but never receives the artifact bytes or storage credentials. It may open an optional stable HTTPS web_url; it never resolves the fcheap:// URI in the browser and rejects signed, query-bearing, or credential-bearing stable URLs. Private byte access remains a separate, short-lived, authenticated file.cheap capability.

Artifact kinds

kind is a bounded lowercase namespaced token such as cairntrace.run, glyphrun.run, chalupa.report, or filecheap.stash. It helps the interface present the reference and does not change storage policy.

Responsibility boundaries

Chalupa:

• indexes references;
• associates them with runs;
• shows provenance, complete JSON, restore commands, or stable links;
• retains structured metrics.

The artifact store:

• stores bytes;
• defines retention;
• controls access;
• handles file sizes and transfers.