RHELS Lab — Agent Docs
This is the deep-reference documentation for the RHELS lab. It is private:
never published publicly, it is served only inside the lab — the showcase
at / and this book at /agent-docs/ behind the docs front door (the
public GitHub Pages path remains gated off — see cicd.md).
This directory is the raw working reference for agents operating the lab.
Tool pages — the coverage matrix
Every tool in the lab has a dedicated page with the same two parts: Part A — how it is set up (deployment, GitOps source, the mechanics that surprise people) and Part B — how to use it (the tasks a reader actually performs).
| Tool | Page | What the parts cover |
|---|---|---|
| OpenLDAP | Identity & Access | A: seed + PostSync ldapadd propagation rules, group machinery · B: add/change/remove users and groups, verify a login |
| Vault | Vault & Secrets | A: deployment, seal model, KV layout, VSO tenants, backup summary · B: read/write secrets, give a workload a secret, rotate |
| cert-manager & Let’s Encrypt | Certificates & DNS | A: the DNS-01-only constraint, the issuer, front doors · B: get/check/renew a certificate, add a front door |
| Tailscale & Cloudflare | Tailscale & Cloudflare | A: the tailnet, the operator, zones and the two API tokens · B: resolve names, add a front door, renew a token |
| Argo CD | Argo CD | A: the two installs, app layout, sync model · B: check health, sync, roll back, add an app |
| Harbor | Harbor | A: chart install, storage, LDAP auth, admin custody · B: log in, pull/push, robot accounts, scan results |
| Grafana | Grafana | A: the three surfaces and their provisioning/auth · B: sign in, find dashboards, Explore, add a dashboard |
| Logging pipeline | Logging pipeline | A: collection, the aggregator’s sinks, the two Lokis · B: query logs, prove delivery, add a sink |
| Prometheus & Alertmanager | Prometheus & Alertmanager | A: three metrics tiers, two Pushgateways, the alert rules · B: query, read firing alerts, silence, add targets/rules |
| GitHub Actions runners | GitHub Actions runners | A: ARC on pro (2026-11-01, lab-monorepo#102), the mini-k8s set, image quirks · B: target the runners, check capacity, debug stuck jobs |
| ServiceNow PDI & MCP | ServiceNow PDI & MCP | A: the instance, catalog-to-git group flow, keep-alive, MCP servers · B: request a group, check a request, use the MCP endpoint |
| Namespace as a Service | Namespace as a Service | A: the agent-first flow, LDAP-bind identity, server-side validation, the PR-only lifecycle · B: the nine tools, a worked request, status, contributing |
| StackRox | StackRox | A: the pro host, mini-Argo external-cluster pattern, Vault-held credentials · B: open Central, check the sensor, re-register, recover |
| OpenShift (CRC) | Workload Cluster | A: operators, apps, platform workloads, conventions · B: console, API via the tunnel, lifecycle rules |
| Mac mini cluster | Tooling Plane | A: namespaces/workloads, disk & memory constraints · B: reach the cluster, day-to-day checks, change discipline |
| This docs system | Docs system | A: one Astro site, the content collection, gates, publish loop · B: edit/add a page, preview, publish |
Chapters
| Page | What it covers |
|---|---|
| Lab Overview | The big picture: tooling plane, workload cluster, conventions |
| Networking | Connectivity pattern between tooling host and workload cluster |
| Vault PKI (planned) | Design only — Vault as the internal CA; not built |
| CI/CD | Current IAC flow and the partial signed pipeline (Harbor/Cosign/Trivy live; digest-pin blocked; Kyverno Audit only; Pages/public path unavailable) |
| Observability | The deep dive: metrics, the dual-Loki pipeline’s measured lessons, delivery verification, the health suite, alerting |
| Disaster Recovery | The stateful inventory classified, the Vault backup state, the cold-rebuild order and every manual seam |
| Runbooks | 13 operations agents can run, with exact steps |
| Glossary | BSN, AppProject, verifyImages, SBOM, Cosign, and friends |
| Agent Protocol | Secrets hygiene + how agents work issues in this lab |
| inventory.json | Machine-readable inventory (generated at build time from the inventory/ snapshot) |
A machine-friendly index of this content is also available as llms.txt — also generated at build time, from the same page sources, so it cannot drift from the book.
How to use these docs as an agent
- Start with the protocol. Read agent-protocol.md before doing anything that touches secrets, issues, or another agent’s work.
- Then the runbooks. If you are here to perform an operation, runbooks.md gives you the exact sequence. Follow it; do not improvise.
- Check freshness. inventory.json carries a
generated_attimestamp and asourcefield (liveorrecorded). Ifsourceisrecorded, the data is the last known-good snapshot, not a fresh read — see “verify inventory freshness” in the runbooks. - Sanitization boundary. Nothing in this tree may contain IPs, hostnames, tailnet names, SSH paths, tokens, kubeconfigs, keys, emails, personal names, or home paths. If you spot one, stop and raise it before continuing.
Source of truth
- Infrastructure as code lives in per-app IAC repos (
rhels/IAC.<bsn>) and the platform reporhels/local-k8s-gitops, reconciled by Argo CD. - These docs describe intent and operations. If the docs and the IAC repo disagree, the IAC repo wins — update the docs.