Puppet Stagehand — Tester's Guide

Who this is for: anyone verifying that a deployed Stagehand console actually does what it claims — QA validating a release, someone testing the installer, or anyone doing a UAT pass before sign-off. Pairs with docs/USER-GUIDE.md (what the feature is, in plain language) — this doc is how you prove it works, walking through the running console itself.

Assumes a deployed console — installed via the Puppet Installer, or an existing running instance — not a source checkout. If you're building from source and want the automated Go/TypeScript test-suite commands, see docs/DEVELOPMENT.md instead.

Status: this guide grows with the console. Every phase of development is required to update it with any new manual-QA steps a UAT pass should exercise (see .planning/HUMAN-TESTS-DEFERRED.md for the standing backlog of what couldn't be verified yet).

This guide (and docs/USER-GUIDE.md) publish to a generated, browsable static site on every push to main via .github/workflows/docs.yml.

---

The Stagehand 1.0 release boundary (read first)

The shipped product runs the core_v1 capability profile (docs/design/core-v1-capabilities.md is the authoritative contract): one local Global Administrator, full Activity Log, Compliance/findings (reinstated 2026-08-17, D-REL-06), Code Management (reinstated 2026-08-17, D-REL-07), Bolt vulnerability scans (reinstated 2026-08-19, D-REL-10), and the deferred surfaces — multi-user RBAC/teams, Approvals, and customer-facing Data Management — hidden behind stable 404 {"error":"feature_not_available", "detail":"deferred_v2_review"} responses registered outside authentication and body parsing.

Two consequences for testers:

1. Release-boundary acceptance is executable:

   cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi \
     -run 'TestDeferredRouteMatrixRegistryTripwire|TestRouteGating' -race
   

   The tripwire asserts BOTH directions — every documented deferred route is registered through requireFeature, and every requireFeature registration appears in the matrix — so the docs and the binary cannot drift apart silently. Manual spot-check: hit any deferred route (e.g. GET /api/v1/users) as the admin and expect the exact envelope above, NOT a 401/403 — authentication errors keep their own semantics.
2. Historical per-phase blocks below for deferred surfaces (RBAC/teams, approvals, Hiera tiers/data) document preserved v2 implementation, not supported 1.0 workflows. Their suites still run — the default test env deliberately enables every capability so the preserved code stays regression-tested (newTestEnv; newCoreTestEnv keeps the production profile) — but nothing they describe is reachable in a shipped 1.0 build. Compliance/findings, Code Management, and Bolt vulnerability scan are the exceptions: they moved out of this deferred category on 2026-08-17 (D-REL-06, D-REL-07) and 2026-08-19 (D-REL-10) respectively, and are now supported 1.0 surfaces — see "Compliance" and "Code Management" below, not the deferred-surface list. The historical "code deploy" block below now documents a shipped surface, not a preserved-but-hidden one — see "Code Management".

Compliance (supported in 1.0)

• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run 'TestCompliance|TestFindings' -race — ingest, dual-write into the findings ledger, summary/node rollup, settings.
• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run TestComplianceRemainsEnabled -race — the reinstatement regression guard: fails loudly if vulnerabilities is ever re-deferred without updating this doc and the decision log together.
• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run TestBoltVulnerabilityScanRemainsEnabled -race — the reinstatement regression guard (D-REL-10) for the distinct bolt_vulnerability_scan capability (the Compliance page's scan-trigger wizard): fails loudly if it's ever re-deferred without updating this doc and the decision log together.
• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run 'TestBoltScan' -race — the scan-launch endpoint itself: session team-scope filtering, zero-resolved rejection, the allowlisted-params regression guard, and the inspector scanner's required-profile validation (TestBoltScanInspectorRequiresProfile/TestBoltScanInspectorLaunchesWithProfile).
• cd frontend && npx vitest run src/components/ComplianceScanWizard.test.tsx src/pages/Compliance.test.tsx — the scan-trigger wizard itself: provider selection, per-provider config gating (Inspector's required profile), the shared TargetSelector, and the POST body shape ({scanner, target, params} — the pre-D-REL-10 wizard shipped a broken {scanner, targets: string[]} shape that always 400'd; these tests pin the correct shape as a regression guard). Also covers the Summary/Heatmap view-switcher: worst-status cell rendering + accessible label, the distinct "not-scanned" cell for a node×benchmark pair with no results, and that clicking a cell opens NodeControls filtered to that node+benchmark.
• Manual: sign in, open Patching & Compliance → Compliance, confirm the page loads (empty state with zero results is fine if no scanner has posted yet), confirm the "Compliance" nav item is visible, then click Run compliance check and walk the wizard: pick each of the three scanners in turn and confirm Next is disabled on the Configure step only when Inspector has no profile entered, pick a target on the Choose targets step (Static/Group/PQL), and confirm Start scan on the Review step either launches (if Bolt is configured) or the whole button is replaced with a "scans: Bolt not configured" note (if not).
• Manual: with at least one node reporting, click Heatmap next to Summary. Confirm the grid shows one row per node and one column per benchmark, each cell showing a glyph + color (never color alone) for that pair's worst status, and a distinct dot glyph for any node×benchmark pair with no results yet. Click a cell and confirm the drilldown panel opens filtered to just that node and that one benchmark (not the node's full control history). Click Summary again and confirm the original dashboard (charts, table, pagination) is unchanged.

Code Management (supported in 1.0 — control repo only)

1.0 ships control-repo management only: team_module and module kind repos cannot be created through the console (attaching a new one 404s with feature_not_available/deferred_v2_review), gated by the code_management_non_control_repos sub-capability (2026-08-18 narrowing — see docs/design/core-v1-capabilities.md). This is checked in-handler in handleRepoCreate, not via a whole-route requireFeature gate, because POST /api/v1/repos must stay reachable for kind=control. Any team_module/module repos that already exist (e.g. seeded test/dev data) keep working normally for every other route — list, fetch, rotate-key, deploy, delete — only creating a new one is blocked.

• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run 'TestRepo|TestCodeDeploy' -race — repo attach/fetch/rotate-key/delete lifecycle, propose, deploy (sync/async-fallback/team-scope/concurrency/audit-replay), deploy-scopes CRUD/RBAC. TestRepoBranches covers the GET /api/v1/repos/{id}/branches deploy-environment-picker endpoint against a local git fixture with multiple branches. TestRepoDeployments (999.3 item 2) covers GET /api/v1/repos/{id}/deployments — each branch's live head commit, and that an environment actually deployed through the console picks up its last-deploy timestamp/actor/run id/status from the existing code_deploy audit trail while a never-deployed environment reports no deploy fields at all.
• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run TestCodeManagementRemainsEnabled -race — the reinstatement regression guard: fails loudly if code_management is ever re-deferred without updating this doc and the decision log together.
• cd backend && go test ./internal/gitops -run TestListBranches -v — ListBranches (git ls-remote --heads, no clone) against a local bare-repo fixture with several branches: names come back sorted with refs/heads/ stripped, and an unreachable remote surfaces as an error rather than an empty list. No PCC_TEST_DATABASE_URL needed (pure gitops-package test); needs git on PATH (skips otherwise).
• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run 'TestCodeManagementNonControlRepoKindsStillDeferred|TestExistingNonControlRepoStaysFunctional' -race — the 2026-08-18 narrowing: team_module/module creation 404s with the standard envelope and never reaches the store (including a team_module payload that's also invalid for another reason, to prove the deferred check wins first), control creation on the same endpoint is unaffected, and a pre-existing team_module repo stays fully listable/rotatable/deletable.
• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/capabilities -race — TestCoreV1Profile asserts code_management_non_control_repos is deferred alongside the other v2 surfaces.
• cd frontend && npx vitest run src/components/Shell.test.tsx src/pages/Setup.test.tsx src/pages/Code.test.tsx src/components/ui/Select.test.tsx — nav visibility, Setup wizard unaffected, Code page behavior including the environment-picker Select (branch discovery, loading/error states, "no control repo attached" blocked state) and Select's per-option disabled rendering (the idiom the repo-kind picker uses for team_module/module).
• Manual: sign in, open Management → Code Management. On the Repositories tab, confirm the repository-kind picker shows "control repo" selectable and "team module (v2)" / "module — tracked (v2)" present but disabled/unselectable. Attach a real control repo with a real git host, paste the shown public deploy key into that host as a read-only deploy key, confirm Fetch succeeds and shows classes/Puppetfile. On the Deployments tab, confirm the top Environments list shows every branch with its commit, and "Never deployed from this console" for any environment you haven't deployed through the console yet; confirm the environment dropdown below it lists that repo's real branches (defaulting sensibly to its default branch), then run Deploy now against an environment and confirm it completes AND that the Environments list above updates with a passed/failed status, timestamp, and actor without needing a page reload; also confirm that with no control repo attached, the Deployments tab clearly says so and Deploy now is disabled. From Reports → Configuration Coverage, confirm the Environments summary card's "View deployments →" link (shown only when a control repo is attached) lands on this same list. This full live round-trip against a real git host is a .planning/HUMAN-TESTS-DEFERRED.md candidate if no real repo is available in the current sandbox — the code-level suites above cover both mechanisms with fake/local fixtures regardless.

Puppetfile Forge version cross-check (docs/design/code-management.md §4, gap closed 2026-08-18; deprecation/endorsement enrichment + DefaultBaseURL hostname fix, 2026-08-19)

• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/forge/... -race — the Forge client (internal/forge): not-configured, invalid-slug, success, 404, 401/403, malformed JSON, no-published-release, the owner/name → owner-name slug normalization, LookupModule's deprecation/endorsement decoding (deprecated + not-deprecated + fields-absent-decodes-to-zero-value cases), and a regression test pinning DefaultBaseURL to https://forgeapi.puppet.com (a prior version guessed a -puppetcore host that never resolved — see forge.go's doc comments for the incident). All against a local httptest.Server — never the real Forge API.
• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run 'TestClassifyForge|TestRepoForgeCheck' -race — classifyForge's current/update_available/unknown/fail-closed-on-bad-semver logic plus deprecation/endorsement field pass-through; the GET /repos/{id}/forge-check endpoint end-to-end (unconfigured → honest unknown, then configured against a fixture, then a deprecated-module fixture); git:/ref:/unpinned entries are skipped; force=true bypasses the 30-minute cache; unknown repo 404s.
• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run TestRepoForgeCheckEndToEnd -race — also asserts the Dashboard's code_management block reflects the same cached result without triggering a second Forge call.
• cd frontend && npx vitest run src/pages/Code.test.tsx — the "Puppetfile Forge cross-check" describe block: update-available pill + latest version rendered, honest "not configured" note + unknown pill (never a false "current"), git:/unpinned rows render — with no pill, "Check Forge now" issues a force=true request, and a deprecated module renders a critical ✕ deprecated pill alongside its status pill with deprecation/superseded-by/endorsement folded into the row's tooltip.
• cd frontend && npx vitest run src/emoji-gate.test.ts src/inline-hex-gate.test.ts — confirms the deprecated pill's glyph rides the Icon name="x" primitive (not a literal ✕ character) and introduces no inline hex.
• cd frontend && npx vitest run src/pages/Dashboard.test.tsx — the "adds/omits a stale-Forge-module action item" cases: the Action Center card appears only when stale_forge_modules > 0 and links to /code.
• Manual: with no Forge key set, confirm the Puppetfile table shows ? unknown pills and a "no Forge API key configured" note, never a false "current". Set a real key under Settings → Packages, click Check Forge now on an attached control repo's Puppetfile card, and confirm a module with a newer published release shows ↻ update available with the newer version number, and that a known-deprecated module (e.g. puppetlabs-rabbitmq) shows the ✕ deprecated pill with a populated tooltip. This full live round-trip against the real Puppet Forge is a .planning/HUMAN-TESTS-DEFERRED.md candidate if no real Forge key is available in the current sandbox — the suites above cover the mechanism against a local fixture regardless. (This manual check is also the live confirmation that PCC_FORGE_BASE_URL/DefaultBaseURL actually resolves — the 2026-08-19 bug shipped past every fixture-based automated test because the fixtures never touch the real hostname.)

Forge key metadata (999.3 item 4) — last-6 reference, attached-at, manual expiration

• cd backend && PCC_TEST_DATABASE_URL=<dsn> go test ./internal/httpapi -run 'TestForgeKey|TestDashboardSurfacesForgeKeyExpiresAt' -race -v — TestForgeKeyMetadataAbsentWhenNoKeySet (every forge_key_* metadata field absent with no key set — never fabricated), TestForgeKeyMetadataLast6AndAttachedAt (last-6-chars + attached-at populate once a key is saved, and the raw key never round-trips in the response body), TestForgeKeyExpiresAtRoundTripsAndValidates (manual date round-trips, a malformed date 422s, saving the key alone leaves a previously-set expiry untouched, and clearing the key clears its expiry too), TestDashboardSurfacesForgeKeyExpiresAt (the Action Center summary carries the same date). This work also fixed a latent test-isolation gap: dataservice_test.go's shared "httpapi" test database wasn't truncating the settings table between tests, so packages.forge_key/packages.forge_key_expires_at could leak across unrelated tests depending on run order — TestForgeKeyMetadataAbsentWhenNoKeySet is what surfaced it (previously nothing in this suite asserted "no key configured" strictly enough to notice). settings is now in the truncation list.
• cd frontend && npx vitest run src/pages/settings/PackagesSettings.test.tsx — reference line rendering/omission, saving only the expiration date never sends forge_key in that request (the pointer-field contract that lets the console update the date without the key ever round-tripping back into the browser, since GET never returns it), and the overdue-badge glyph+label rendering.
• cd frontend && npx vitest run src/pages/Dashboard.test.tsx — the "adds/omits a forge_key_expiring action item" cases: critical when overdue, warn within 30 days, absent otherwise (dates computed relative to "now" so these don't rot).
• Unresolved, not guessed at: whether the real forgeapi.puppet.com exposes key-expiration data was investigated (grepped docs/ and backend/internal/forge for any existing reference — none found) but NOT confirmed either way; this sandbox has no live network access to forgeapi.puppet.com to check directly. The expiration field ships as manual-entry only, with UI copy that says so explicitly, rather than guessing at an API shape. If a human tester has live access, checking whether the Forge API's key-info response includes an expiration field is a .planning/HUMAN-TESTS-DEFERRED.md candidate.
• Manual: under Settings → Packages, save a Forge key and confirm the reference line ("key ending in xxxxxx — attached <date>") appears; save an expiration date in the past and confirm the Action Center shows a critical "Forge API key expired N days ago" card linking back to Settings → Packages; save one 10 days out and confirm it's a warn-tone card instead; save one 90 days out and confirm no card appears at all (only the reference/date fields on the settings page itself).

---

Manual QA — when and how

Some behaviors genuinely can't be exercised in this sandbox (a real SSH host presenting a rotated key, a real deployed+authenticated browser session, live network conditions). Those are:

1. Documented per-phase in that phase's {N}-VALIDATION.md under "Manual-Only Verifications," with concrete repro steps.
2. Tracked centrally in .planning/HUMAN-TESTS-DEFERRED.md until a real environment is available to run them.
3. Written up as a QA report in docs/qa/ once actually exercised — follow the existing convention: a per-feature markdown file with an automated-verification table (command → result → evidence) and a manual-scenario table (scenario → expected result → status).

Existing QA reports in docs/qa/ are the reference examples for format — read one before writing a new one (new-to-puppet-tutorial.md is a good full example of the two-table structure).

Phase 22 manual-QA checklist — scoped code deploy

Scenario	Expected result
Global Administrator clicks "Deploy now" on a mapped/any environment	Button shows "Deploying {environment}…" and disables; within ~45s a Badge (success "passed" / critical "failed") appears with the raw r10k output below it
A deploy exceeds the 45s bound	"Still running — tracking it on Tasks." appears with a working link to /bolt; the run finishes visibly there
Team Administrator deploys an environment with NO code_deploy_scopes mapping	Inline critical Alert: "You can't deploy {environment} — it's not mapped to a team you administer." (deny-by-default)
Team Administrator deploys an environment mapped to a DIFFERENT team	Same denied copy as above — never distinguishes "unmapped" from "someone else's"
Team Administrator deploys an environment mapped to THEIR OWN team	Info Alert: "Submitted for approval — a Global Administrator or Approver needs to sign off before this deploys." — never the busy spinner or a result pill
Bolt.tsx's "Deploy now" shortcut (Brownfield adoption flow) hits a 403	Same denied copy as Code.tsx — never the generic "Couldn't trigger deploy"
Approver opens Approvals and finds a captured code_deploy request	Environment + requested-by are visible above the generic JSON payload (both render — additive, not a replacement)
Global Administrator maps/unmaps an environment↔team pair (Settings → Administration → Deploy scopes)	Mapping list updates; a Team Administrator's next deploy attempt against that environment reflects the change immediately

Deploy environment picker manual-QA checklist — control-repo branch discovery

Scenario	Expected result
Attach a control repo with multiple branches (e.g. production, staging), open Deployments	The environment dropdown lists every branch on the repo, defaulting to the repo's configured default branch when it's among them (else the first branch reported) — no free-text entry
No control repo (kind=control) attached	Deploy now is disabled and a message explains a control repo must be attached first (Repositories tab); no dropdown is shown
Branch-list fetch is slow	The dropdown and Deploy now are disabled with a "Loading branches…" placeholder until it resolves
Branch-list fetch fails (host unreachable, auth problem)	A critical Alert explains the failure with the repo URL, Deploy now stays disabled, and a Retry button re-fetches
Attaching a second control repo, or a repo with zero branches	Console picks a control repo deterministically (first one found) rather than erroring; zero branches shows "No branches found" in the dropdown with Deploy now disabled

Deployments-tab Environments list manual-QA checklist (999.3 item 2)

Scenario	Expected result
Open Deployments on a control repo with an environment never deployed through the console	That row shows its branch's short commit SHA and "Never deployed from this console" — no fabricated date/status
Run "Deploy now" against an environment	That environment's row updates in place (no reload) with a passed/failed status pill, the deploy's timestamp, and the actor who ran it
Environments-list fetch fails (host unreachable, auth problem)	A critical Alert explains the failure with the repo URL and a Retry button re-fetches, independent of the Deploy now form below it
Configuration Coverage → Environments summary card, control repo attached	A second "View deployments →" link appears alongside "View environments →" and opens Code Management directly on the Deployments tab
Configuration Coverage → Environments summary card, no control repo attached	Only "View environments →" appears — no dangling link to an empty Deployments list

Phase 23 manual-QA checklist — scoped task & plan launcher

Scenario	Expected result
PQL preview badge/list vs. actual launch-time resolved set	In the running console, open Runner → PQL tab, enter a query, click Preview, note the matched-count badge and (if expanded) the certname list, then Launch and confirm the run's resolved target count/list matches what the PQL preview showed — requires a live PuppetDB with real node facts (LAUNCH-03, D-05/D-06)
Stagehand visual conformance of the TargetSelector tabs across Runner, Discover, and Scan (/bolt)	Manually inspect Runner, Discover, and Scan against docs/design/handoff-stagehand/ for consistent token usage (no inline hex), hit targets (≥44px), and no emoji — visual/design-system conformance isn't asserted by unit/component tests (LAUNCH-03)
Stagehand visual conformance of the Run a Playbook tab (/bolt?tab=playbooks)	Manually inspect the Run a Playbook tab — a Bolt Tasks & Plans tab again as of 2026-08-19, not a standalone Automation page — against docs/design/handoff-stagehand/ for the same conformance checklist: consistent token usage (no inline hex), hit targets (≥44px), no emoji (PLBK-01, PLBK-02)

Phase 25 manual-QA checklist — Bytnar Graphs delta (Labs gate + failure overlay)

Scenario	Expected result
Open a node with a FAILED latest report, click its Dependencies tab	Resource nodes render colored by status: failed = red, changed = amber, unchanged = green
Click a failed/changed resource node	Its downstream blast radius shows a border/overlay highlight; each highlighted node keeps its own severity fill color (a downstream node that is itself failed stays red, not recolored)
Same click	Side panel shows a Status row with glyph + label (e.g. ✕ Failed) and a "Downstream impact: N downstream resource(s)" line; clicking never navigates away or opens the Triage queue
Open a node with NO Puppet report yet, view Dependencies	Every resource node is gray "Unreported"; the muted note "No report yet — every resource shows Unreported until this node's first Puppet run." appears once near the legend (never green/"clean")
Settings → Puppet Labs, "Node Dependency Graphs" card	Shows "Owner: Matthew Stone"; toggling the flag off hides the "View dependency graph →" link on every node's Inventory page, and the /nodes/<certname>/graph route itself renders the "Dependency graph is in Puppet Labs" off-state (not a 404, not a crash)

Rows 1-4 exercise the Cytoscape canvas itself, which jsdom cannot mount (RESEARCH Pitfall 2, no OverviewTab.test.tsx by design) — these carry to .planning/HUMAN-TESTS-DEFERRED.md's Phase 25 section until walked on a live console with a seeded failed-report node. Row 5's on/off logic is already covered by NodeDetail.test.tsx/NodeGraph.test.tsx/ PuppetLabsSettings.test.tsx; this row's job is a human eyeball on the actual rendered off-state copy and the settings card.

Phase 26.4 manual-QA checklist — NodeDetail tabs, Activity feed, Classes tab, Configuration Coverage

Scenario	Expected result
Open a node's detail page	Three tabs render (Overview / Activity / Classes), Overview selected by default; Overview shows the existing Key facts + Recent reports plus a new "{N} classes applied · View Classes tab →" line that switches to the Classes tab in place (no navigation)
Open the Activity tab on a node with mixed history (audit entries, Bolt runs, Puppet reports)	A single time-ordered table renders all three kinds together; the date-range picker defaults to the last 30 days; toggling a kind-filter chip off hides that kind's rows (at least one chip always stays on); clicking a row's deep-link navigates to the underlying Bolt run (/bolt?run=), report, or audit entry (/activity?certname=&object_type=)
Open the Activity tab on a node with an orphaned/stale Bolt run	That row's outcome badge reads "unreported" (glyph + label, never color-only) instead of "running" or a fabricated failure
Open the Classes tab on a node with classes from both ENC and catalog, plus at least one single-source class	Each class chip shows its source/mismatch badge: "in sync" (both sources), "not yet applied" (ENC-declared only), or "catalog only" (catalog-applied only)
Navigate to Configuration Coverage (Reporting → Configuration Coverage, /class-usage — nav label renamed 2026-08-18, see the ad hoc note below)	Three sections render: Most applied (ranked, proportional bars), Declared but unused, Unclassified nodes (single stat + "View nodes →"); clicking a most-applied or declared-but-unused row opens its dedicated drilldown page (not a modal), listing the matching nodes in the existing Inventory table shape
Stagehand visual conformance across all four new surfaces (NodeDetail tabs, Activity feed, Classes tab, Configuration Coverage + drilldowns)	All colors/spacing/type resolve through frontend/src/theme.css tokens (no inline hex), every interactive element (tab buttons, filter chips, "Apply range", ranked-list/drilldown rows) meets the ≥44px hit-target minimum, and no emoji appears anywhere

All six rows were auto-approved under this run's auto-chain checkpoint protocol (Plans 03/04's Task checkpoints) rather than walked on a live console — see .planning/HUMAN-TESTS-DEFERRED.md's Phase 26.4 section.

Top-level Activity Log run link (2026-08-18, ad hoc): the per-node Activity tab above has always deep-linked its bolt_run-kind rows to /bolt?run=; the top-level Reporting → Activity page (Activity.tsx, backed by /audit, distinct from the per-node /inventory/nodes/{cn}/activity feed) now does the same. runIdFor() resolves the run from object_id directly for bolt_run/puppet_run entries, or from a run_id field embedded in the entry's new payload for entries whose object type isn't itself a run (code_deploy, mcp_deploy) — most other object types never ran through Bolt and correctly get no link. - cd frontend && npx vitest run src/pages/Activity.test.tsx — covers all three cases (object_id-resolved, payload-embedded run_id, no run). - Manual: open Reporting → Activity, filter to object_type=code_deploy (or bolt_run), confirm a view run → link appears on those rows and opens the matching run under Orchestration (in the side panel if that preference is on); confirm rows with no associated run (e.g. settings, secret) show no link.

Class Usage → Configuration Coverage rename + summary cards (2026-08-18, ad hoc): the page grew beyond "which classes are used" into a fleet coverage overview, so its nav label, header title (both derived from Shell.tsx's SECTIONS array), and every other display-name reference across the app and these docs were renamed to Configuration Coverage. The route (/class- usage), component/file names (ClassUsage.tsx, ClassUsageDrilldown.tsx), and backend (classusage.go, GET /api/v1/class-usage) are all unchanged — this is a display-label rename only, so no bookmark or deep-link breaks.

A summary-cards row was added above the existing ranked lists, reusing Dashboard.tsx's MetricTile visual pattern (Card + StatBlock + a single CTA link) and only data already cheaply available on this page or one existing call away — no new backend collection: - Environments (GET /environments) — links to Node Groups' Environments tab (/classification?tab=environment, new query-param support added to Classification.tsx for this deep link). - Classes in use (total_most_applied) — anchors down to the Most applied section on the same page. - Declared but unused (total_declared_but_unused) — anchors down to the Declared but unused section on the same page. - Unclassified nodes (unclassified_count) — links to the existing unclassified-nodes drilldown. - Modules (GET /repos, control-repo puppetfile — Code Management's already-parsed Puppetfile data) — links to Code Management. Omitted entirely when no control repo is attached; shows "Not measured" (never a fabricated 0) when one is attached but hasn't fetched successfully yet.

An "ENC-applied classes" split (ENC-declared-and-applied vs. everything else catalog-applied) was considered and dropped: classusage.go's classUsageResult only returns the declared-but-unused diff, never the declared class name list itself, so the intersection isn't reconstructable from the API response without a new backend field — out of scope for a "cheap data only" summary row.

cd frontend && npx vitest run src/pages/ClassUsage.test.tsx src/pages/Classification.test.tsx src/pages/NodeDetail.test.tsx \
  && npx tsc --noEmit && npx vite build
cd backend && go test ./internal/httpapi/...

• Manual: open Reporting → Configuration Coverage. Confirm five summary cards render (four always, "Modules" only when a control repo is attached under Code Management) above the existing three ranked sections; confirm each card's link/anchor lands where labeled (Environments tab, the two in-page anchors actually scroll, the unclassified drilldown, and Code Management's Puppetfile view). With no control repo attached, confirm the "Modules" card doesn't render at all (not a "0" or "Not measured" card).

Phase 32 manual-QA checklist — Visual Bolt Designer (canvas/CRUD/validate)

Scenario	Expected result
Open Orchestration → Bolt and pick the Designer tab	The tab renders alongside Runner/Discover/Playbook; an empty canvas shows the "No steps yet" state with the searchable palette visible; searching with no matches shows the palette's empty-state copy
Add two steps from the palette, then reorder and delete	Each card shows index, name, and kind; the drag handle and delete button meet the 44px hit target; deleting warns about $stepname references and cannot be undone
Select a step and fill it in	The inspector renders the same typed param form as the Runner (required-first, sensitive params via Secrets) plus the shared TargetSelector; a target choice round-trips into the step
Use the link-glyph inserter in step 2 to reference step 1	A literal $stepname token is inserted as plain text — no wires drawn
Save the draft, then Validate	Save persists; Validate calls real bolt plan show: a valid draft shows the success state (glyph + label, never color alone), an invalid one lists cleaned error rows with the raw Bolt output behind a "Show full error" disclosure; with Bolt unconfigured the response is the fail-closed "Bolt unavailable" state, not a raw error
Open the same draft in two windows, save in one, then save in the other	The second save is blocked by the conflict banner ("changed since you loaded it — another window or session") until Reload adopts the server draft; no silent overwrite, no auto-merge
Stagehand visual conformance across the Designer surface	Theme tokens only (no inline hex), no emoji, glyph+label status pairing, ≥44px hit targets

All seven rows are deferred for live visual sign-off (this session ran the full automated suite including the real-bolt validate integration test, but no human browser walkthrough) — see .planning/HUMAN-TESTS-DEFERRED.md's Phase 32 section.

Phase 33 manual-QA checklist — Designer dry run, publish, undo/redo

Scenario	Expected result
Save a draft whose first targeted step points at a real lab target, then click Dry run	Button shows Running… and disables (no double dispatch); the results panel fills with per-step glyph+label success/failed rows (mono target names, full output behind a disclosure); no target machine is actually changed
As a signed-in Global Administrator, click Publish on a saved draft attached to a repo	The "Publish this plan?" confirm dialog opens with a prefilled pcc/ branch and commit message; confirming shows the "Published — branch pcc/… was pushed to the attached repo." success banner with the proposal link, and the branch really exists on the control repo
Attempt the same publish unauthenticated (expired session) and as a non-admin principal	The server rejects the publish (401/403); the dialog stays open showing "Couldn't publish this plan" with the server's reason — never a silent success, never an approvals queue
Make several canvas edits (add two steps, reorder them, delete one, edit a param, change a target), then Undo repeatedly and Redo back	Each Undo restores the exact prior step list, one edit at a time; Redo re-applies them in order; both buttons disable (not hide) at the ends of the history and show Undo/Redo tooltips
Reload the page mid-editing	The undo history starts fresh (Undo/Redo both disabled) — history is in-session only and the saved draft is untouched
Stagehand visual conformance across the new surfaces	Theme tokens only (no inline hex), no emoji, glyph+label status pairing, ≥44px hit targets on the icon-only Undo/Redo buttons

All rows are deferred for live sign-off — automated evidence stands in (canvas + history suites green; backend TestBoltDesigner* publish/test-run authz coverage), but no live lab dispatch, real repo push, or browser walkthrough happened. See .planning/HUMAN-TESTS-DEFERRED.md's Phase 33 section.

Phase 34 manual-QA checklist — command palette

Scenario	Expected result
Click "Search anything…" in the sidebar (dark theme)	A headerless dialog no wider than 640px opens top-aligned over the scrim with the search input focused; the ⌘K chip renders on the trigger
Type into any form field, press Meta+K then Ctrl+K	Palette opens both times, browser search is suppressed, the field's typed value is unchanged; closing restores focus
Search terms producing many/partial/zero results	Groups render present-only in fixed Issues→Nodes→Sections→Actions order with right-side kind tags; fuzzy ordering favors closer matches; rows are ≥44px; long labels ellipsize; the list scrolls past its cap; zero total matches shows exactly No matches. (non-interactive)
Kill the backend (or block /inventory/nodes) mid-search	Node results degrade silently; sections and actions remain rendered and selectable — no toast, spinner, or error banner
Select a node, a section, and an action; Escape on a fresh open	Each selection navigates to its internal route and closes once; Escape closes with no navigation
Under core_v1 (default)	No Issues group, no findings request in the network tab, no approvals action offered
Repeat visual checks in light theme	Token-driven styling holds in both themes; visible focus rings; no emoji; no color-only status

All rows are deferred for live browser sign-off (automated suite green, including the capability-gating and failure-isolation tests) — see .planning/HUMAN-TESTS-DEFERRED.md's Phase 34 section.

Phase 26.5 manual-QA checklist — novice-friendly Bolt UX

Scenario	Expected result
Open Runner, Discover, and Scan (/bolt) with Advanced collapsed (the default)	Each form is visibly lighter than before — no Group/PQL tab bar — with one prominent primary button and a one-line helper
Open the Playbooks page (/playbooks) with Advanced collapsed (the default)	The form is visibly lighter — no Group/PQL tab bar, no Extra vars/Tags/Skip tags fields — with one prominent primary button and a one-line helper; a picked playbook with a vars: block shows generated fields above Advanced, pre-filled from its declared defaults; a playbook with none shows no generated fields — Advanced is the only vars-editing surface
Click "Show advanced options" on each form, then reload the page	Group/PQL targeting appears (and on Playbooks, Extra vars/Tags/Skip tags); the open/closed choice persists per page across reload
Compare each form's default-visible vs. Advanced field set against the UI-SPEC Per-Page Composition Contract table	Playbooks' check_mode + install method stay default-visible; extra_vars/tags/skip_tags + Group/PQL move behind Advanced; the same holds for Runner/Discover/Scan's field sets
Open a finished run with a very long stdout target on RunDetail, click "View output"	Raw output was hidden until clicked; once expanded, a very long output renders without breaking the page layout (no new max-height/scroll constraint — inherits the existing <pre> overflow CSS)
Complete one task/plan run using only the default (non-Advanced) path, as a first-time user	Run launches and completes without ever needing to open Advanced

All five rows were auto-approved under this run's auto-chain checkpoint protocol rather than walked on a live console — see .planning/HUMAN-TESTS-DEFERRED.md's Phase 26.5 section.

Phase 27 manual-QA checklist — Gibson licensing sign-off + fallback render

Scenario	Expected result
Read docs/legal/gibson-font-license.md and check whether a Perforce/Canada Type Gibson purchase record exists, and which license category it grants (D-02)	A recorded determination: either CONFIRMED (the record covers app-embedding) or OPEN/NEGATIVE (keep the fallback default)
Run cd frontend && npm run dev and view a couple of screens (headings + body)	Default renders in IBM Plex Sans, not Gibson
Optionally set VITE_FONT_MODE=gibson at build time (or data-font="gibson" on <html> in devtools)	Gibson renders, confirming the opt-in override still works

This checklist carries a gate="blocking-human" requirement (D-09) — it was never auto-approved, including under this run's active auto-chain protocol (workflow._auto_chain_active: true). RESOLVED 2026-08-14: Matthew walked this checklist and recorded the determination as OPEN/NEGATIVE, final — no Perforce/Canada Type purchase record was found covering app-embedding/public self-hosted web distribution. IBM Plex Sans is now the permanent public-build default; Gibson stays dev/demo-only. See docs/legal/gibson-font-license.md and .planning/HUMAN-TESTS-DEFERRED.md's resolved Phase 27 entry.

Phase 28 manual-QA checklist — self-update lifecycle, live progress & rollback

Scenario	Expected result
Global Administrator changes the update channel (Test Pilots/Beta/Stable), then restarts the console process	The Self-Update page reopens with the same channel selected — the choice survived the restart, not just a page reload
A fixture/real manifest with a newer version than the running build is reachable on the selected channel	The nav shows a glyph+label "Update" Badge next to Self-Update within one background-poll interval, without requiring a page visit to trigger the check
Global Administrator clicks "Update to v{target}"	The confirmation Dialog shows the non-destructive Copywriting-Contract body text verbatim, explicitly naming (platform: VM) — never framed as a destructive/data-loss action, since auto-rollback is the whole point
Confirm the update ("Update now") and watch it run to completion	The lifecycle stage list advances live (snapshotting → applying → verifying → complete), each stage's copy matches the Copywriting Contract, and completed stages always show the success glyph regardless of their own
During the literal VM restart window (the process is briefly down while the helper swaps binaries)	The page shows "Reconnecting…" — never a red error/failure state — and recovers on its own once the console is back
Trigger an update that fails its post-restart health check (or inspect a past rolled_back history row)	The stage list shows the rolled-back terminal state with a truncated failure-reason preview and a working "Show details" expand revealing the full server-supplied reason
Open the update-history table with zero/loading/error/populated data	Each state matches its Copywriting-Contract copy ("No updates yet" / 3 skeleton rows / "Couldn't load update history" + Try again / newest-first rows with glyph+label outcome badges)
A non-Global-Administrator visits /self-update	Only the "Only a Global Administrator can manage console updates." gate message renders — no channel picker, no trigger button, no history table

This checklist was not walked on a live console this session (no systemd-capable VM/console deployment available in this planning/execution sandbox) — see .planning/HUMAN-TESTS-DEFERRED.md's Phase 28 section, which also carries the one item this checklist deliberately excludes: the live systemctl restart under a real KillMode=process unit (SELFUP-07), which needs a real systemd host, not just a live browser session.

Phase 29 manual-QA checklist — signature verification rejection banners

Scenario	Expected result
Point the console at a fixture manifest signed with the wrong channel's key (or an unrecognized channel with no stamped trust root) as a Global Administrator	The Self-Update page shows the signature-specific banner ("failed signature verification…"), not the generic "Couldn't check for updates"
Point the console at a fixture manifest whose version is older than or equal to the running version	The Self-Update page shows the version-rejection banner ("rejected because its version is not newer…"), not the generic fallback
Point the console at an unreachable/erroring manifest URL (network failure, non-200)	The Self-Update page shows the existing generic "Couldn't check for updates" banner — the generic fallback still works for everything that isn't a signature/version rejection
Confirm none of the three banners convey status by color alone	Each has distinct copy text, not just a red-vs-red-vs-red Alert with no differentiating words

This checklist was not walked on a live console this session (no live browser/console deployment in this planning/execution sandbox) — see .planning/HUMAN-TESTS-DEFERRED.md's Phase 29 section, which also carries the live tag-push signed-round-trip item (RESEARCH Pitfall 4) this checklist deliberately excludes — that item needs a real GitHub Release, not just a live browser session.

Phase 41 manual-QA checklist — EYAML profile wizard

Scenario	Expected result
As a Global Administrator, generate a real EYAML key pair against a live console, watching the browser's Network tab	The create request/response never contains the private key or any BEGIN ... PRIVATE KEY PEM text; only the public key and metadata (name, algorithm, timestamps) appear anywhere in the response body
Upload a real hiera-eyaml-generated key pair (if available) via mode=upload	The console accepts it, self-validates the modulus match before storing, and the profile appears in the list with the correct algorithm
Use the "encrypt a value" helper and manually paste the resulting ENC[PKCS7,...] value into a real Hiera YAML file read by a live Puppet Server with hiera-eyaml configured	The value decrypts correctly at catalog-compile time (puppet lookup/puppet apply resolves it back to the original plaintext) — this specific item is a genuine interop gap this sandbox cannot close (D-01's flagged wire-format assumption): see .planning/HUMAN-TESTS-DEFERRED.md's Phase 41 section
Navigate to Configuration → Create Secret from a fresh session	The EYAML wizard opens directly — no Dashboard detour, no Action Center card anywhere on the Dashboard. Generate/upload a profile, encrypt a value, close back to "/"

The live hiera-eyaml gem interop row above was not verified in this sandbox — no live, connected Puppet Server with hiera-eyaml configured is available here. It must not be claimed as verified anywhere in this phase's SUMMARY. See .planning/HUMAN-TESTS-DEFERRED.md's Phase 41 section for the concrete repro steps.

Phase 35 manual-QA checklist — Estate Viewer, instance registry, health polling

Scenario	Expected result
Register a second real or fixture instance through Estate Viewer's "Add instance" wizard	The confirm step is genuinely unreachable until instance-connection-test returns a success — a failed test shows classified copy (unreachable / cert rejected / generic fallback), never a generic "failed" string, and never unlocks confirm
Switch the global "Active instance:" selector under the logo to the new instance	Inventory (or any other existing page) now shows that instance's data; Estate Viewer's own "Showing:" dropdown is UNCHANGED by that switch — the two scopes never sync
Point one registered instance at an unreachable host, then wait out a poll interval	Estate Viewer shows "? unreported" for that instance (never "0", never blank) and the instance is never dropped from the Instances total
With 2+ reachable instances and "Showing: All instances" selected, reload the page	The "Agent versions across the estate" overlay bar assigns each instance a stable, distinct color across the reload (not re-randomized)

Automated evidence stands in for all four rows independent of this table (the full backend instance-registry/poller/aggregate-endpoint suite, the frontend wizard/switcher/page/CompositionBar suite, and a stable-color-index unit test proving color_index is deterministic) — but this table's rows themselves require an actual second instance and a real browser, so they are walked as this plan's own blocking human-verify checkpoint rather than a prior automated pass. See .planning/phases/35-estate-viewer/35-09-SUMMARY.md for this session's outcome, and .planning/HUMAN-TESTS-DEFERRED.md's Phase 35 section if that walkthrough was deferred rather than completed live.

Console SBOM manual-QA checklist — docs/adr/0008-console-sbom.md

Historical note (999.3 item 3, 2026-08-19): the page below moved from Administration → SBOM to Settings → SBOM; rows below quoting the old path/test file name are the original session's record, left as-is.

Scenario	Expected result	Walked this session?
Visit Administration → SBOM on a console built via docker compose build (real SBOM generation, not the committed placeholder)	Version/channel/commit and a non-zero component count render	Verified via curl against a running container (GET /api/v1/sbom → component_count: 19), not a real browser visit
(999.3) Open Settings → SBOM on a console with a real (non-placeholder) SBOM	The Components table lists every Go/npm dependency with name, version, license; the filter box narrows by name or license substring	Not walked live — covered by SbomSettings.test.tsx's mocked component list, not a real browser against a real built SBOM
Click "Download App SBOM (CycloneDX)"	A puppetui-console-<version>.cdx.json file downloads immediately, with no network request visible in devtools — it's served straight from the running binary	Verified the underlying endpoint directly (GET /api/v1/sbom/download → correct Content-Type/Content-Disposition, valid CycloneDX body); the frontend button itself was exercised only by Sbom.test.tsx's mocked fetch, not a real click in a real browser
Validate the downloaded App SBOM	Valid CycloneDX 1.6; both Go and npm production dependencies are present, no devDependencies	Verified — 19 correctly-deduped components spanning Go modules (with license/hash evidence) and npm packages, metadata.component correctly stamped puppetui-console
Console has no self-update manifest configured (or the configured URL is unreachable)	"Download Full Container SBOM" renders disabled with the "not available yet for this build" copy — no error banner, no broken link	Verified the backend half (container_sbom_url correctly absent from the response with no manifest configured); the frontend disabled-state rendering is covered by Sbom.test.tsx, not a real browser
Cut a real tagged release through release.yml end-to-end	sbom-container.cdx.json + .sha256 + .minisig are attached to the GitHub Release; the Administration → SBOM page of a console running that build shows "Download Full Container SBOM" enabled and pointing at that exact asset	Not walked — needs a real tag push
Verify the container SBOM's signature	minisign -V -p <published channel public key> -m sbom-container.cdx.json -x sbom-container.cdx.json.minisig succeeds	Not walked — needs a real release
Validate the container SBOM	Contains OS packages from the ubuntu:24.04 runtime layer, Puppet Bolt's Ruby gems (or its apt package, depending on build path), and every component the App SBOM lists	Not walked — syft was never actually run against a real image in this session

This session's docker compose build && up pass caught and fixed four real bugs the initial code review missed (wrong cyclonedx-gomod install path, a missing -main cmd/console flag, a Dockerfile multi-line RUN parse error, and a missing git repository for version detection) — see .planning/HUMAN-TESTS-DEFERRED.md's Console SBOM entry for the full list and what's still unverified (mainly: the container-tier/release.yml path, and a real browser visit to the page rather than direct API calls).

Phase 42 — Guided Console UI and Results

cd frontend && npx vitest run src/pages/HierascopeList.test.tsx src/pages/HierascopeNew.test.tsx \
  src/pages/HierascopeDetail.test.tsx src/components/EyamlWizardDrawer.test.tsx \
  src/hierascopeErrors.test.ts src/hierascopeConfidence.test.ts \
  src/components/Shell.test.tsx src/pages/Dashboard.test.tsx
cd frontend && npm run build && npx tsc --noEmit

Covers: nav/routing to /hierascope (Shell.tsx's new Configuration section); the guided setup flow's capped scope preview (blocks Start at >100,000 active nodes, matches SCOPE-01's server-side cap exactly); the live 3s poll against GET /api/v1/hierascope/jobs/{id} until a terminal status; the Evidence/Failures/Inputs tabs (confidence badges, certname drill-down, JSON/CSV export links) and job-list pagination; Cancel/Restart against a running/terminal job with the shared Error Code Dictionary (hierascopeErrors.ts) mapping every backend code to specific copy — including the 16 Runner/artifact-verification codes added 2026-08-18 after live debugging surfaced the gap (see that date's ad hoc entry above); the EYAML Action Center card's fail-closed readiness fetch and the wizard Drawer's Generate/Upload/Verify/Delete flow and one-way "encrypt a value" helper with its no-persistence guarantee (cleared on Drawer close/reopen).

Phase 42 manual-QA checklist — guided Hierascope + EYAML wizard walkthrough

Scenario	Expected result
Open Configuration → Hierascope on a live console with an attached repo	Job list renders (empty state if none exist), "Start analysis run" is visible
Start a guided run: pick a repo, two refs, preview scope, then start	A real active-node count and scope-cap meter render before commit; the new job lands on its own detail page
Watch a queued/running job's detail page	Progress polls automatically every ~3s with no manual refresh; a determinate bar renders once the first progress event arrives
Open Evidence/Failures/Inputs on a completed job with real differences	Confidence badges, Before/After rendered values, and certname drill-down all render; Download JSON and Download CSV both produce real files
Cancel a running job, then Restart a terminal job	Cancel stops the real subprocess; Restart creates a new job with byte-identical frozen inputs and lands on its detail page
Generate a real EYAML key pair on a live console, watching the Network tab	The private key never appears in any request/response body
Click Verify on a profile in this release	Honest "not yet configured" result, never a false pass
Delete a profile	Irreversibility warning dialog appears before the delete is confirmed
Use the "encrypt a value" helper, then close and reopen the wizard Drawer	Ciphertext renders once with a Copy button and copy-once notice; plaintext/ciphertext fields are empty again on reopen
Across all five new screens	Stagehand six-pillar conformance: theme tokens only (no inline hex), no emoji, IBM Plex Sans/Mono typography, 24px card insets, every status pill pairs a glyph with a text label

The live hiera-eyaml gem interop item (pasting a real ENC[PKCS7,...] value into a live Hiera YAML file read by a live Puppet Server) is the same gap Phase 41 already carries forward — see .planning/HUMAN-TESTS-DEFERRED.md's Phase 41 section rather than a duplicate entry here; nothing new to re-verify.

Phase 43 manual-QA pointer — live VM/Compose installer smoke tests

Phase 43's manual-only items (a live VM bolt apply_prep/apply walkthrough exercising stagehand::console's hierascope params against a real target, a live docker compose up smoke test of the Compose path's staged binary and env vars, and the not-yet-published linux_amd64 hierascope release asset) are recorded — not duplicated — in .planning/HUMAN-TESTS-DEFERRED.md's "Phase 43 — Installer Integration and Release Gate" section, with concrete replay steps for each. The Helm/ Kubernetes deployment gap is recorded there distinctly as a permanent, out-of-phase-scope capability gap rather than a deferred test.

Phase 45 manual-QA checklist — Playbooks feature refresh

Scenario	Expected result
Attach a Playbook Repo via the Playbooks page's "Attach repo" link	Console navigates to Code Management's attach form with the "Playbook Repo" kind pre-selected; attaching and fetching a real fixture repo discovers its playbooks, which then appear in the Playbooks page's picker
Run a Designer-published multi-step plan containing one pcm::run_playbook task step alongside another (non-playbook) step	The run's detail view shows the full ordered step list (kind badge + name + the run's own aggregate outcome badge per row); the playbook step's row renders the identical recap-strip/per-target-output body a standalone playbook run uses, inline — never generic raw output for that step
Run a Designer-published plan containing EXACTLY one (unnamed) playbook step and no other steps	The run collapses in the run log to "Playbook — {name}" — indistinguishable in look from a standalone playbook run, with no "1-step plan" framing anywhere

The multi-step scenario above only recognizes a playbook step whose Designer-authored name is left as the literal pcm::run_playbook source string (i.e., unnamed) — a step given a custom name (e.g. "Configure webserver") is NOT currently recognized as a playbook step and renders as a generic task row instead. This is a known, documented, and automated-tested limitation, not a gap this checklist is expected to catch — see .planning/HUMAN-TESTS-DEFERRED.md's Phase 45 section and frontend/src/components/RunDetail.test.tsx's isPlaybookStep tests.

Manual QA checklist template

Phase 37-10 automated patch-provider gates

Run from backend/:

GOCACHE=/tmp/phase37-10-go-cache go test ./internal/patching ./internal/store -count=1
GOCACHE=/tmp/phase37-10-go-cache go test -race ./internal/patching ./internal/store -count=1

The focused coverage checks preview-token drift, provider transition locking, provider persistence, and normalized provider contracts. HTTP integration tests also require PCC_TEST_DATABASE_URL.

For a deployed-console manual pass, sign in as the primary Global Administrator, open GET /api/v1/patching/providers, preview a destination, and confirm that the switch cannot proceed with a changed token or stale state version. Verify that a run remains blocked while a switch is in its verification/cleanup span.

Phase 37-12 supported-modules manifest + provider contract gates

Run from backend/ (HTTP tests need PCC_TEST_DATABASE_URL):

go test ./internal/patching ./internal/httpapi -run 'Test(SupportedModules|OSProviderManifestReadiness|PatchProviderManifestHTTP)' -count=1
go test ./internal/httpapi -run 'Test(PatchProviderDocumentationContract|PatchProviderRoundTripHistory|PatchProviderContentBoundary)' -count=1

What the automated matrix proves:

• Manifest readiness (fail closed). The embedded supported_modules.json binds os_patching to module souldo-soup (fork of voxpupuli/puppet-os_patching), fact soup_patching, task soup::patch_server, contract 1, release state pre_release with no pinned version/SHA-256/signature. OS Patching readiness, switch preview toward it, and POST /api/v1/patching/run all fail with the stable code supported_module_release_unbound; the native engine is never invoked. Loader negative cases: every missing identity field, malformed checksum/URL/version/commit, duplicate provider, unknown contract version, identity-tuple mismatch with the adapter, unsupported runtime product, and a pre-release entry that tries to pin release values.
• Directed transitions. All six pairs are the same machinery: os_patching↔pe_basic, os_patching↔pe_advanced, pe_basic↔pe_advanced. Terminal migration states are exactly verified, rolled_back, and migration_attention_required; the store rejects any other transition.
• History preservation. Patch groups and migration records survive an OS → PE Basic → OS state round trip; every migration keeps its immutable integer ID and stays readable at GET /api/v1/patching/provider-migrations/{id}.
• Security checks. Preview/switch/run/configuration/cleanup/recovery require the Global Administrator session; anonymous requests are refused. Mutations require an idempotency_key and the current state_version (stale_provider_state on drift, preview_changed on token drift — covered by the 37-10 gates above). Runs and cleanup are refused with migration_attention_required while a migration needs attention; cleanup can only ever touch verified provider/console-owned resources.
• Content assertions. The current patching docs (README, USER-GUIDE, tiers, design contracts) name OS Patching / PE Basic / PE Advanced, eligibility vs selection, preview/confirm, recovery, and the Activity Log — and their provider sections make no promises about deferred governance, scheduling, advisory-correlation, or the deferred v2 management surfaces.

Manual (deployed console) additions to the 37-10 pass above: confirm the Patching Providers screen reports OS Patching as release-unbound rather than executable, and that the Activity Log records preview/switch/configuration events with the acting administrator.

Phase 37-19/37-20 — Soup real-node acceptance evidence + signed release pipeline (code-complete, live run deferred per D-REL-01)

The substantive automated coverage for this work lives in two sibling repositories, not in puppet-console itself — this repo only tracks a pointer to the Soup source commit the fail-closed manifest was checked against. Run from backend/:

go test ./internal/patching/... -count=1
go test -race ./internal/patching/... -count=1

This proves supported_modules.json's pre_release fail-closed gate still holds (no pinned version/SHA-256/signature) and confirms the manifest's source_commit matches soup-os_patching@main's actual current HEAD.

The rest of this work's automated proof runs in the other two repos:

• /Users/matthew/Code/soup-os_patching — ruby -c on every acceptance spec (spec/acceptance/soup_spec.rb, spec/acceptance/migration_spec.rb, spec/acceptance/windows_soup_spec.rb) and on lib/soup/evidence_validator.rb; ruby -ryaml parses .github/workflows/soup-acceptance.yml and soup-release.yml; a standalone Ruby harness exercises Soup::EvidenceValidator's fail-closed behavior and its coverage-union check_matrix logic end to end (the real bundle exec rspec suite needs a PDK/Puppet gem toolchain not available in every environment — see 37-19-SUMMARY.md's "Issues Encountered"). Executing the acceptance specs themselves against real Litmus/CI targets is the actual deferred item; see below.
• /Users/matthew/Code/puppet-installer — go test ./internal/vendored/... and bash scripts/vendor-modules.sh --verify-config confirm the provider_modules fail-closed vendoring contract for souldo-soup still refuses to fetch an unpinned entry.

What's deferred and why: the live Litmus/CI real-node acceptance matrix run (Linux + Windows) and the actual signed public souldo-soup v0.1.0 GitHub/Forge release both need external infrastructure and credentials this environment does not have (disposable test targets, LITMUS_INVENTORY, an authorized Windows runner, FORGE_API_TOKEN, SOUP_SIGNING_KEY_ID). Per decision D-REL-01, both are carried in .planning/HUMAN-TESTS-DEFERRED.md's "1.0 release-gate exceptions" section with the exact commands a human runs once that infrastructure exists — start there, not here, when it's time to actually execute this matrix or cut the release.

Ad hoc console owner request (2026-08-19) — Action Center service health, Bolt tab consolidation, Tabs redesign

Three related IA/visual changes, all forward-only (no historical retrofit of earlier phases' own docs/checklists):

1. Action Center: the "Recent task runs" mini-list is gone from the Dashboard, replaced in the exact same spot by ServiceHealthPanel — the former standalone Service Status page's connections/Bolt/readiness data, recast as a small card grid instead of a table. The Service Status page (pages/ServiceStatus.tsx, route /service-status) and its sidebar nav entry are both gone; /service-status now redirects to /. Recent runs still have two homes: Bolt's own run history table (every tab) and Reporting → Run Reports.
2. Bolt tab consolidation: the "Designer" tab is relabeled "Plan Builder" (same ?tab=designer value, unchanged). Playbooks and Bolt Credentials — previously their own top-level Automation sidebar items — are now Bolt tabs ("Run a Playbook", "Credentials"), reusing PlaybookForm/BoltCredentials directly with no duplicate run-history table. /playbooks and /bolt-credentials both redirect to the matching /bolt?tab= value.
3. Tabs.tsx redesign: the shared components/ui/Tabs primitive (used by Bolt, Code Management, Node Groups, Hierascope, Node Detail, Node Graph) now renders a larger, more visually obvious active-tab state — a tinted --primary pill background plus a 3px underline plus bolder/larger text, up from a thin 2px underline alone. Purely a Tabs.css change; no prop/ API change, so every consumer picks it up automatically.

Frontend (frontend/) — no backend changes in this pass:

cd frontend && npx tsc --noEmit && npx vitest run \
  src/pages/Dashboard.test.tsx src/components/ServiceHealthPanel.test.tsx \
  src/pages/Bolt.test.tsx src/components/Shell.test.tsx \
  src/components/ui/Tabs.test.tsx src/App.test.tsx

Covers: ServiceHealthPanel rendering "not configured"/"unreachable"/ "running" states honestly per card, a working "Test now" action, and the Bolt runtime readiness ready/total summary, all degrading quietly (no crash, no competing role=status element) when every fetch rejects; Dashboard no longer rendering a dashboard-recent-runs element under any summary shape; Bolt's tab bar exposing "Plan Builder" (not "Designer") and "Run a Playbook" (not a bare "Playbook"), ?tab=designer / ?tab=playbooks / ?tab=credentials deep links each selecting the right tab, and the shared run-history table staying visible across every tab; Shell's Automation section down to exactly Puppet Agent + Bolt Tasks & Plans, with no Playbooks/Bolt Credentials/Service Status links anywhere in the sidebar. Full existing suite re-run (npx vitest run, no filter) to confirm no other Tabs consumer regressed from the CSS-only redesign — 109 files / 1028 tests passing at the time of this change.

docs/USER-GUIDE.md updated: Dashboard section gained a "Service health" paragraph; the Bolt section's Designer references renamed to "Plan Builder", and every "Playbooks page" reference renamed to "Run a Playbook tab" (including the "Structuring a repo for..." subheading and the Code Management cross-reference); the Bolt credential-pairs paragraph now points at the Credentials tab; Glossary gained "Credential pair (Bolt)" and "Service health" entries and renamed every "(Designer)" glossary term to "(Plan Builder)".

Manual-QA checklist (visual-only — not asserted by unit/component tests):

Scenario	Expected result
Load the Action Center	Service health cards render below the attention queue, in the exact spot the old "Recent task runs" list used to occupy; sidebar has no "Service Status" item anywhere
Open Bolt Tasks & Plans	Tab bar reads Runner, Discover, Plan Builder, Run a Playbook, Credentials — each with the redesigned tinted-pill + underline active state, visibly larger/more obvious than a plain underline
Click through Code Management, Node Groups, Hierascope, a Node Detail page, and a Node Graph's Summary/Triage/Dependencies tabs	Every one picks up the same redesigned Tabs treatment automatically (shared primitive) — no page looks unstyled/mismatched against the others
Stagehand visual conformance of the redesigned Tabs active state	Compare against docs/design/handoff-stagehand/ tokens (no inline hex, --primary-derived tint, radii per spec) in both light and dark themes

No live dev-server browser walkthrough was performed for this pass (see the final report for this exact limitation) — the above checklist is deferred to a human/browser pass; see .planning/HUMAN-TESTS-DEFERRED.md if it isn't already there.

When you do have a real deployed instance to test against, use this shape (copy into a new docs/qa/{feature}.md):

# {Feature} — QA report

## Automated verification
| Command | Result | Evidence |
| --- | --- | --- |

## Requirement coverage
| Approved outcome | Automated test file(s) | Manual scenario | Manual status |
| --- | --- | --- | --- |

## Deployed manual QA checklist
| Scenario | Expected result | Status |
| --- | --- | --- |

## Release disposition

Ad hoc console owner request (2026-08-19) — standalone Create Secret nav entry

The EYAML "encrypt a value" flow was previously only reachable through a "Manage EYAML keys" button on a Dashboard Action Center card. That card is gone; Configuration → Create Secret is now the one entry point, auto-opening the same wizard. See the updated Phase 41 checklist above for the core encrypt/generate/upload scenarios — this entry covers only what changed about getting there.

Scenario	Expected result
Load the Dashboard	No EYAML card anywhere — Action Center shows no "Manage EYAML keys" button, readiness banner, or dismiss affordance
Click Configuration → Create Secret in the sidebar	Navigates to /create-secret; the EYAML wizard opens immediately (no extra click)
Close the wizard (Escape, scrim click, or the X)	Returns to / (Dashboard), not a blank page

Ad hoc (2026-08-19) — EULA gate (real text, installation-wide acceptance)

Replaces the EULA placeholder with the final legal text and adds a mandatory full-screen acceptance gate shown after login until any user accepts it. Acceptance is installation-wide (not per-user, not versioned) — once any user accepts, no session sees the gate again. See docs/DEVELOPMENT.md for the automated coverage.

Manual-only: whether the gate correctly blocks every route path a determined user might try (deep-linking to an authenticated URL, browser back/forward) before the SPA's refresh() resolves is UI-timing-dependent and not covered by the automated suite — see .planning/HUMAN-TESTS-DEFERRED.md.

Ad hoc (2026-08-19) — Sidebar "Active instance:" label wrong color in non-default themes

Console owner report: the "Active instance:" label (the global instance switcher, Shell.tsx) looked black regardless of theme, only reading correctly in the default dark theme. Root cause: a hardcoded content-area color token used on a sidebar-background element; fixed with a sidebar-scoped override. See docs/DEVELOPMENT.md for the automated coverage.

Manual verification (not automated — this is a rendered-color check, not something the vitest/jsdom suite evaluates): built the console image with the fix, ran it against a fresh instance, and screenshotted the sidebar in all 5 themes (Settings → Appearance) with localStorage theme switching. "Active instance:" is legible against the sidebar background in every theme, matching the style of sibling labels like "REPORTING"/"AUTOMATION".

Ad hoc (2026-08-19) — Settings → Connections and open pages weren't consistently scoped to the active instance

Console owner report: with more than one registered instance, Settings → Connections could show and silently edit a different instance's PuppetDB/Puppet Server connection than the one actually active, and pages already open when you switched the active instance kept showing the previous instance's data until you navigated away and back. Root causes and fixes: - GET /settings/connections returned every instance's connection rows unfiltered; the page collapsed them by kind and could display an arbitrary instance's row. Now scoped to the active instance (ListConnectionsByInstance). - PUT /settings/connections always wrote instance #1's literal id=kind row regardless of which instance was active. Now resolves (and, for a brand-new row, mints) the id for the active instance's connection of that kind. - The Dashboard's "Test now" service-health cards consume the same list endpoint, so they inherit the same fix — they now test the active instance's connections, not an arbitrary one. - In-memory caches behind the Dashboard summary, Class Usage, and ENC/classification lookups are cleared on every active-instance switch, so a switch can no longer keep showing a few seconds of the previous instance's cached data. - The sidebar content area now remounts on an active-instance switch, so any page you already had open re-fetches immediately instead of only refreshing on next navigation.

Manual verification: with 2+ registered instances (Estate Viewer demo instances work fine for this), switch the active instance while sitting on Inventory or the Dashboard — the page's data updates without navigating away. Then open Settings → Connections: it shows an "Editing connections for the active instance: …" note naming the currently active instance, and its PuppetDB/Puppet Server host/port match that instance's, not instance #1's. Switch the active instance again and reopen Settings → Connections — the form now shows the other instance's connection details, and saving there does not change instance #1's connection (spot-check via a second browser tab left on instance #1's Dashboard "Test now" card). See docs/DEVELOPMENT.md for the automated coverage.

Ad hoc (2026-08-19) — Compliance page Heatmap view

A new "Heatmap" view alongside the existing Compliance summary, showing a node x benchmark grid (worst-status-wins per cell, with a distinct "not scanned" state for coverage gaps).

Scenario	Expected result
On the Compliance page with real scan data ingested, click "Heatmap"	Grid renders: one row per node, one column per benchmark; each cell shows a glyph (✓/!/✕/?/·) with a matching color, never color alone
A node has a benchmark with at least one failing control among passes	That cell shows "✕" (fail), not an average/majority-vote of the controls
A node has never been scanned against a particular benchmark that exists elsewhere in the fleet's data	That cell shows "·" (not-scanned), visibly distinct from a passing "✓" cell
Click any cell	Opens the same node-controls panel as the Summary view's per-node drilldown, but pre-filtered to only that cell's benchmark
Switch back to "Summary"	The original charts/table/pagination render exactly as before this change

Ad hoc (2026-08-19) — Action Center still showed the previous instance's data right after switching

Console owner follow-up report: even with the fix above, switching the active instance while sitting on the Dashboard/Action Center could still show the previous instance's numbers. Root cause was a genuine ordering bug, not a caching gap: the sidebar's "Active instance:" selector updated its own state (which remounts every page and fires its data fetch) before the PUT /settings/active-instance request that actually flips the backend's active instance had resolved — so the freshly-remounted Action Center's fetch reached the backend while it was still pointed at the old instance, got a genuine (but stale-relative-to-the-switch) answer, and cached it. Nothing then told the page to fetch again once the switch actually landed a moment later.

Fixed by waiting for the switch to be confirmed by the backend before updating the selection (and therefore before any page remounts/re-fetches). Also hardened the Dashboard's in-memory cache to record which instance each cached result was computed for, so even a slow, unrelated in-flight request from just before a switch can never repopulate the cache as if it were the new instance's data.

Manual verification: with 2+ registered instances, sit on the Dashboard/Action Center and switch the active instance. The selector should show the old instance for a brief moment (the switch is confirming), then the page updates to the new instance's numbers — never showing the new selection with the old instance's data. Repeat switching back and forth several times in quick succession; the numbers should always match whichever instance is currently selected.

Ad hoc (2026-08-19) — Estate Viewer's Add-instance wizard was missing the Puppet Server step; no Edit/Delete instance UI existed

Console owner report: the "Add instance" wizard only ever captured a PuppetDB connection — there was no way to configure an instance's Puppet Server (CA) connection at creation time, and once created, an instance's name/edition/connections could never be changed or removed from the UI at all (the backend's PUT/DELETE /api/v1/instances/{id} already existed but had no frontend caller).

Scenario	Expected result
Click Add instance, fill in name/product type/PuppetDB, test it successfully, click Next	A new "Connect to Puppet Server (optional)" step appears before Confirm
Leave the Puppet Server step's host blank and click Next	Next is enabled with no test required; the Confirm step shows "Not configured" for the Puppet Server host
Fill in a Puppet Server host on that step	Next becomes disabled again until "Test connection" succeeds for THAT connection — the same rule already enforced for PuppetDB
Complete registration with both connections configured	GET /api/v1/instances/{id} shows two connection rows (kind puppetdb, kind puppetserver); the Certificates page and CSR counts work for this instance without any extra Settings → Connections step
On an instance row in Estate Viewer, click Edit	A drawer opens (matching the Add-instance wizard's chrome) pre-filled with that instance's name and product type
Change the name/product type and Save	The row updates immediately; the instance's connections are untouched
In that same Edit drawer, click Manage connections	The active instance switches to this one and you land on Settings → Connections, already showing this instance's connections
Click Delete on a non-active instance row	A confirm dialog names the instance; confirming removes it from the list
Click Delete on the CURRENTLY ACTIVE instance's row	The confirm dialog's Delete fails with "Switch the active instance before removing this one." — the instance is NOT removed

See docs/DEVELOPMENT.md for the automated coverage.

Ad hoc (2026-08-19) — Create Secret defaults to Encrypt a value; key management moved to Manage keys

EyamlWizardDrawer (opened by Create Secret) restructured from a single Generate/Upload tab pair with "encrypt a value" as a closed-by-default Collapsible, into two top-level tabs: Encrypt a value (new default — the Select/Input/Encrypt-button form, always visible, no show/hide trigger) and Manage keys (Generate/Upload sub-tabs + the Existing profiles list, everything that used to sit at the top level). With zero EYAML profiles, the Encrypt tab's controls render disabled with a message pointing at Manage keys. The Generate sub-tab also gained an inline warning: a Generated key's private half never leaves the console, so Puppet cannot decrypt values encrypted under a Generated profile unless a matching key pair is separately installed on Puppet's own infrastructure — see docs/USER-GUIDE.md's "EYAML profiles & the secret wizard" section for the full explanation.

Manual-QA checklist:

Scenario	Expected result
Open Configuration → Create Secret on a console with zero EYAML profiles	Lands on "Encrypt a value"; Profile/Plaintext/Encrypt controls are disabled with a message pointing at "Manage keys"; clicking through switches tabs
Open Create Secret on a console with at least one profile	Lands on "Encrypt a value" with the controls already enabled — no extra click needed
Switch to "Manage keys" → "Generate"	The private-key-stays-in-the-console warning is visible near the Generate button, before you click it
Generate a key pair, then try to actually decrypt an ENC[PKCS7,...] value encrypted under it via a real Puppet apply/lookup against a live Puppet Server with hiera-eyaml configured	Decryption fails — this is expected and matches the new inline warning, not a regression. This specific check requires real Puppet infrastructure outside this repo's automated suite; see .planning/HUMAN-TESTS-DEFERRED.md's Phase 41 section, which now also notes the Generate-mode-specific limitation

See docs/DEVELOPMENT.md for the automated coverage.

Ad hoc (2026-08-19) — Novice-facing diagrams in USER-GUIDE.md; EULA plain-language appendix + tl;dr link on the acceptance gate

docs/USER-GUIDE.md gained three Mermaid diagrams (Push vs Pull, the EYAML Generate-vs-Upload decrypt round trip, and a small Node Dependency Graph example tied to the existing "Downstream impact / blast radius" Glossary entry) plus a new "Appendix: EULA in Plain Language" section summarizing the real legal text at frontend/src/legal/eulaText.ts in plain English, with an explicit "not a substitute, not legal advice" disclaimer. mkdocs.yml now configures pymdownx.superfences's Mermaid custom fence so these diagrams render on the published docs site (and on GitHub.com, which supports Mermaid fences natively). The mandatory EULA acceptance gate (EulaGate.tsx) gained a link to the new appendix, for anyone who wants the plain-language version before reading the full agreement.

Manual-QA checklist:

Scenario	Expected result
Open the published docs site's User Guide page, scroll to "Two Ways Things Get Done", "EYAML profiles & the secret wizard", and "Node Dependency Graph"	Each section shows a rendered diagram (boxes and arrows), not a raw code block starting with flowchart
Scroll to "Appendix: EULA in Plain Language"	A bulleted plain-language summary appears, ending with a disclaimer that it isn't a substitute for the real agreement
On a fresh installation, sign in for the very first time	The EULA acceptance page shows a "Read the tl;dr on the docs site →" link near the top, opening the Appendix section in a new tab

See docs/DEVELOPMENT.md for the automated coverage.

---

Per-phase requirement (standing rule)

Every phase must, before being marked complete:

1. Check whether it changed anything a user directly sees or does — if so, update docs/USER-GUIDE.md (plain-language, ELI10, newbie-to-Puppet-and-automation perspective; add new concepts to the Glossary).
2. Update docs/TESTER-GUIDE.md with any new manual-only verification items (UAT steps against a deployed console), and update docs/DEVELOPMENT.md with any new automated test commands or changes to the full-suite gate command.
3. Any behavior genuinely deferred to a human pass gets recorded in .planning/HUMAN-TESTS-DEFERRED.md — never silently dropped.

This is enforced the same way all project-specific conventions are: it's a binding rule in the repo root CLAUDE.md, which every planning and execution agent reads before doing phase work.

---

Where things live

What	Where
Per-phase validation strategy (sampling rate, per-task verification map)	.planning/phases/{N}-*/{{N}}-VALIDATION.md
Deferred human-only tests	.planning/HUMAN-TESTS-DEFERRED.md
Feature-level QA reports	docs/qa/*.md
Security threat models / audits	.planning/phases/{N}-*/{{N}}-SECURITY.md (when /gsd-secure-phase has run) — e.g. 21-SECURITY.md, the first phase to exercise this convention end-to-end (17/17 threats closed, ASVS L1, 2 accepted risks)
Local dev environment setup	RUN.md
Backend test DB env var	PCC_TEST_DATABASE_URL
Automated test-suite commands (build-from-source)	docs/DEVELOPMENT.md

---

This guide is maintained alongside the codebase. See docs/USER-GUIDE.md for the end-user-facing companion document.