QA Use Cases

Every tool in the lab gets the same treatment Harbor got first: a set of QA stories — named, persona-scoped use cases with concrete steps and expected results. A story is not a wish. Its expected result is derived from the tool’s own documentation and GitOps config, and the source is cited inline on the story. When a future suite runs a story and reality disagrees with the expectation, that is drift — a finding to file, never something to silently edit away in the test.

The worked example

Harbor’s set was built first and is the only one with an executable suite so far: stories in rhels/lab-agentops (health-suite/harbor-qa/USECASES.md), suite beside them, running daily. Every set in this section mirrors that format, and each set is also mirrored as USECASES.md under health-suite/<tool>-qa/ in the same repo, so a suite can be built from any of them the same way.

Story format

Each story carries:

• ID — <PREFIX>-A<n> for admin stories, <PREFIX>-U<n> for user stories (e.g. VLT-A1, ARG-U2). IDs are stable: a suite reports failures by ID, and an ID is never reused for a different expectation.
• Persona — who acts (below).
• Goal — one sentence: what this story proves.
• Preconditions — the state that must hold before step 1.
• Steps — the actions, in order.
• Expected — the observable result, with the doc/config source cited in (source: …). An expectation no document supports is marked TBD — no documented expectation; the section-wide gaps are collected on each page and in the index below.
• Assert — the machine-checkable form of the expectation: the concrete endpoint, CLI, or kubectl command plus the expected value, status, or pattern. Prose is the spec layer; the Assert line is the code-facing contract. A suite is implemented from the Assert lines without re-interpreting the prose, and any expected value that varies (a set, a threshold, a baseline count) lives in the check’s data manifest — code and data, not paragraphs.
• Mode tag — how a suite would verify it: [config/API] (deterministic checks against APIs and config), [browser] (a real UI flow), or [CLI] (shell-level proof).

The execution contract (standing, for every suite built from these stories)

These stories exist to be run regularly. Any suite built from a set in this section — Harbor’s is the first — follows the same contract the lab’s health suites already run under:

1. Defined as code. One coded check per story ID, asserting exactly what the story’s Assert line says. If an Assert line cannot be implemented as written, the story is wrong or the tool changed: file the drift, don’t loosen the check quietly.
2. Results persist per run, machine-readable. Every run writes one record per check — check ID, status (pass / fail / infra-down / not-run), the measured value the Assert compared, and the duration — to the suite’s report store in rhels/lab-agentops (the health suites’ reports/ pattern: one JSON per run plus a rendered latest.md).
3. Every run reports a regression diff. The report’s first section is the diff against the previous run: new failures, recoveries, and changed measured values — modeled on the health suite’s diff-vs-previous and its SLI history. A run with no diff is a quiet run; reporting follows the house rule (surface failures, new infra-down states, recoveries, and regressions; stay silent on clean, unchanged passes).
4. Manual residue is explicit and minimal. A story whose Assert cannot be expressed as code today is flagged Manual-only in its page’s Gaps, with the reason. Manual is a shrinking list, not a category of convenience: when a tool grows the API or probe that makes a manual story codable, the story graduates and the flag comes off.

Personas

The sets

Harbor’s set lives with its suite (above) and is not repeated here.

Standing rules for every set

• Definitions only until a suite exists. These pages define the stories; nothing here claims a story currently passes. Where a standing check already proves a story (the hourly health suite, the E2E cycle, the check-pro-*.sh probes), the story says so.
• Negative controls are first-class. Every persona boundary has at least one story whose expected result is a denial. A suite that only tests the happy path is not a QA suite.
• Stopped-cluster framing. CRC is intentionally stopped at the time of writing. CRC-dependent stories are written “when CRC is running” and cite the CRC-start checklist; a stopped CRC is the expected state, not a failure.
• No credential values, ever — credentials are referenced by service name via the Service Directory, exactly as the rest of this book does.