RHELS LAB
Contents · the agent book

Start here

  1. RHELS Lab — Agent Docs
  2. Service Directory
  3. Lab Overview

Planes

  1. Tooling Plane — the Mac mini cluster
  2. Workload Cluster — CRC / OpenShift
  3. Networking

Tool pages

  1. OpenLDAP — Identity & Access
  2. Vault & Secrets Delivery
  3. Certificates — cert-manager & Let's Encrypt
  4. Tailscale & Cloudflare — the name layer
  5. Agent onboarding — how an agent joins the team
  6. Argo CD — GitOps on both planes
  7. Harbor — container registry
  8. Grafana — dashboards & Explore
  9. Logging pipeline — Vector, Loki, Logstash
  10. Prometheus & Alertmanager — metrics and alerts
  11. GitHub Actions — self-hosted runners
  12. ServiceNow PDI & the MCP servers
  13. StackRox — security platform on pro
  14. This documentation system
  15. Namespace as a Service — the MCP front door
  16. Vault PKI

Operations

  1. CI/CD
  2. Observability — deep dive
  3. Disaster Recovery & Rebuild
  4. SRE: IaC, DR, Upgrades, SLO/SLA
  5. Runbooks
  6. The Agent-Run Lab — A Working Showcase

QA Use Cases

  1. QA Use Cases — how this section works
  2. QA — Vault
  3. QA — Argo CD
  4. QA — Identity (OpenLDAP + Dex)
  5. QA — Grafana, Prometheus, Alertmanager
  6. QA — Logging (Loki, Tempo, Logstash)
  7. QA — ACS (StackRox)
  8. QA — Kyverno
  9. QA — MCP server (Namespace as a Service)
  10. QA — CI runners (ARC)
  11. QA — Front door (ingress, cert-manager, Tailscale operator)
  12. QA — Alert bridge
  13. QA — ServiceNow PDI, sims, group-sync
  14. QA — Lab docs
  15. QA — CRC / OpenShift

Reference

  1. Glossary
  2. Agent Protocol

Agent book · Reference

Glossary

BSN — Build Serial Number. The lab’s app identifier: appDDDD (seven characters, e.g. app1001). Used for namespaces (app1001-dev), IAC repos (rhels/IAC.app1001), and Argo CD Applications.

IAC repo — Infrastructure-as-code repository. Each app has one private repo (rhels/IAC.<bsn>) holding its manifests; the platform itself lives in rhels/local-k8s-gitops. Argo CD reconciles the clusters from these repos. Git is the source of truth — never hand-edit the cluster.

AppProject — An Argo CD construct scoping what an Application may deploy (source repos, destination clusters/namespaces). Lab convention: lowercased names derived from the BSN.

ApplicationSet — An Argo CD construct generating Applications from templates/parameters. The lab uses it to stamp out per-app Applications (namespaces, RBAC, workloads) from the BSN.

verifyImages — A Kyverno policy type that checks container images against signature/attestation rules. Installed on CRC as ClusterPolicy require-signed-images in Audit for app* namespaces (rhels/team#107; GitOps c5d749f586d43640a61591f0bcdb94ac04d98e0d). Enforce (unsigned images rejected at admission) is not live — that is the #107 final mile.

SBOM — Software Bill of Materials. A machine-readable inventory of an image’s components. Optional follow-on to the signed pipeline (generated at build, attached to the image); not claimed live.

Cosign — The signing tool in the live signed pipeline (rhels/team#107): keyless signing of container images using GitHub OIDC identity, so no long-lived signing keys exist to leak.

Tooling plane — The control side of the lab: Docker Desktop Kubernetes on the tooling host, running CI runners, Argo CD, Vault, cert-manager, and Harbor/Loki/Tempo (live, rhels/team#107). It builds and ships; it does not run demo workloads.

Workload cluster — The demo side of the lab: OpenShift 4.22 (CRC), running the appDDDD demo apps in per-app namespaces, managed by Argo CD. It runs workloads; it does not build them.

Dex — The OIDC bridge that fronts OpenLDAP for the tooling-plane Argo CD, so the directory is the only credential store behind it. See identity.md.

Fixture user — A seeded directory identity (dev1, dev2, ops1) used by login checks and day-to-day sign-in. Fixtures are not people; their passwords are low-value test credentials held in Vault. See identity.md.

VaultStaticSecret / VSO tenant — The Vault Secrets Operator pattern for delivering one Vault KV entry into one namespace as a Kubernetes Secret: a dedicated service account + VaultAuth + VaultStaticSecret, backed by a Vault policy and role scoped to exactly that one path. See vault.md.

Aggregator — The standing Vector StatefulSet on CRC (log-agg) that receives forwarded logs and fans them out to the sinks. Its config is one file per sink, so adding a sink is a one-file git change. See observability.md.

Sink — One destination in the aggregator’s fan-out, defined by one file: 10-loki (local Loki), 15-logstash-fallback, 20-grafana-cloud-loki, plus 00-metrics for Vector’s own metrics.

ClusterLogForwarder (CLF) — The OpenShift Cluster Logging object that decides which namespaces’ logs go where. The lab’s CLF covers the log-sim namespace and feeds both Logstash and the aggregator.

DNS-01 — The only ACME challenge type this lab can use: the home ISP permanently blocks inbound 80/443, so HTTP-01 is impossible and all Let’s Encrypt issuance proves control by DNS TXT records via Cloudflare. See certificates.md.

Front door — A service’s single public name on the lab’s own domain (argo., vault., grafana. …), served by the Tailscale operator + ingress on the tooling plane and reachable only from the tailnet. See certificates.md.

Seed Job (PostSync hook) — The Argo CD hook Job that loads the OpenLDAP seed LDIF after a completed sync of the agentops-ldap app. It runs ldapadd -c, so it creates entries that do not exist yet and silently skips ones that do — the single fact behind the directory’s propagation rules (identity.md Part A).

Pushgateway — The Prometheus companion for batch jobs: producers that exit before they can be scraped (CronJobs, one-shot checks) push their result metrics to it, and Prometheus scrapes the gateway. Two run in the lab — one per plane (tools/prometheus-alertmanager.md).

Content collection — The Astro mechanism this book is built with: a typed set of markdown files (site/src/content/docs/) whose frontmatter drives the table of contents, rendered by one dynamic route inside the site’s own layout (tools/docs-system.md).