Guide conformance

This document records the strict architecture audit of the AI engineering system against Developing-in-Agentic-AI-Systems-Learning-Paths.md. The guide is the non-negotiable authority; this document reports conformance and cannot override it.

Audit identity

Item	Audited value
Audit date	2026-09-23
Canonical-text guide SHA-256	c247b45ed53bb7b901954611c3bc03a37294d9adfb02a6542d71522f694f62be
Framework baseline	webmaxru/ai-engineering-system@9c8be3c plus this audit and non-normative bookkeeping repairs
Audited reference baseline	webmaxru/northstar-orders-api-demo@b65c2de5c8224342c72c37eeed7ef9f965ad8a2c; current main, known defective
Earlier repairs	Terminology, workflow syntax and artifact handoff merged in webmaxru/northstar-orders-api-demo#7, webmaxru/northstar-orders-api-demo#10, and webmaxru/northstar-orders-api-demo#12
Current reference work	On 2026-09-28, the owner closed system-maintenance issues #14, #16, #20, #22, and #24 and their maintenance PRs to retain demo-only open work. Demo issues #4, #17, and draft PR #19 remain open. The closures are scope cancellations, not accepted fixes.
Current conformance decision	Blocked: the locked baseline remains defective; maintenance candidates were closed unmerged, and hosted acceptance/trusted publication remain unestablished
Experimental comparison	reference/ai-engineering-system-agent-hooks@cbb22f1e90f8edcce8e019c4c867af8daebe7605

A change to the guide content or hash invalidates this conclusion until the framework and reference are audited again.

architecture-lock.json records this identity and intentionally blocks a successful release verification while required reference changes remain pending. Run pwsh -File tools/verify-architecture.ps1 to check the guide, extension coverage, non-self-governance boundary, one-way Northstar reference, snapshot, and conformance decision.

The earlier parser/terminology blockers are resolved in the inspected source; the workflow now creates jobs. This does not imply release acceptance. The current audit records 220 passing unit tests, eight passing PostgreSQL acceptance tests, additional separate-process application proof, five Poutine errors despite exit zero, and two moderate dependency findings. A real cloud planner stopped before producing a plan because live task authority was absent. A missing-artifact probe exposed a false-positive readiness decision.

The 117-file snapshot matches the exact audited revision for inspection. It is not approved for adoption. The owner closed the system-maintenance workstreams to retain demo-only open work; none of their plans or implementations was merged or accepted. The remaining demo tasks do not repair the locked baseline, and hosted acceptance and trusted publication remain unestablished. The historical candidate evidence below does not change the locked main baseline.

Demo and closed-maintenance status (2026-09-28)

Work	Current evidence	Remaining gate
Repair and combined-mode controller: issue #14, PR #18	Historical head 17e7a5c5f1fbf88a92351043c675f555f4c7f04f; independent review exists; hosted acceptance tests passed	Issue and PR were closed by the owner for scope on 2026-09-28; repository-controls and evidence failed; unmerged and unaccepted
Parallel isolation: issue #16, plan PR #26, implementation PR #27	Historical head fc59deefc31213b2c596bd0384008f77a9955c4f; 500 unit tests, 10 PostgreSQL acceptance tests, scope/merge checks and two read-only CLI identity canaries passed	Issue and PRs were closed by the owner for scope; full validate:all fails on 84 Zizmor findings; hosted evidence/controls fail; AC9 cloud isolation was not proved
Trusted acceptance: issue #24, plan PR #25, implementation PR #28	Historical head 0260fe99fdd48672d83f83d45849ac9487d38fe2 received exact-head approval; hosted human-review and evidence passed	Issue and PRs were closed by the owner for scope; repository-controls failed, so trusted acceptance was not established
Repository controls: issue #22, plan PR #23	Plan approval existed for base 17e7a5c5…	Issue and PR were closed by the owner for scope; no ruleset setting was changed
Workflow scanner: issue #20, plan PR #21	Prior plan approval existed for base 2e3cd083…	Issue and PR were closed by the owner for scope; the base was stale and no scanner findings were fixed
WI-1842 rehearsal: issue #4, PR #13	The issue remains as the plan-first demo task; PR #13 recorded the planner stopping because the cached task contract was absent	PR #13 was closed as a stale, zero-file rehearsal; it does not prove a successful cloud run
Endpoint demonstration: issue #17, PR #19	Head cf7622166e48cdc543121adaa37a0ab57dcb4c45; exact-head approval and hosted human-review pass; local validate (470 unit tests), PostgreSQL acceptance (12/12), scope and merge checks pass	PR #19 remains draft; full validate:all fails at Zizmor (86 findings, 65 errors/21 notes), hosted repository-controls fails, and no VS Code/cloud run is proven

The local CLI canaries establish only that two separately owned worktrees loaded matching task, contract, plan, base, session and owner identities. They do not prove every VS Code lifecycle hook, a successful cloud task, or hosted acceptance. The system-maintenance closures do not change these findings. The cloud attempt is recorded as a blocked negative result, not a pass.

Interpretation rules

The guide contains requirements, choice-based designs, conditional technologies, recommendations, and examples. Strict conformance means:

• Required architecture is implemented without contradiction or substitution.
• Choice-based guidance keeps the guide’s valid alternatives and records which alternative is used for each risk class.
• Conditional technology is required when the corresponding capability is enabled; it is not forced into repositories that do not use that capability.
• Recommended guidance is explicitly identified and may be adopted when it fits the repository; adopting it does not turn the recommendation into a universal guide requirement.
• Examples demonstrate a valid shape but do not freeze owner names, action versions, labels, file contents, or command syntax unless the surrounding text makes them mandatory.
• A stricter mechanism is acceptable only when it preserves the guide-defined behavior and is registered in TECHNICAL-EXTENSIONS.md.

Status values in this audit:

• Conformant - the source implementation matches the guide requirement.
• Conformant + extension - the guide requirement is present and a documented mechanism makes it concrete or stricter.
• Choice applied - Northstar selects one of the guide’s supported patterns.
• Recommended / adopted - Northstar adopts a guide recommendation without representing it as mandatory architecture.
• Conditional / not applicable - the guide requires the behavior only when a capability is used, and Northstar does not use that variant.
• Blocked - source or hosted evidence exposes a defect that prevents the reference from supporting the conformance claim.
• Hosted not verified - source expresses the required control, but only live GitHub state can prove enforcement.

Requirement matrix

Guide basis	Guide architecture and terminology	Framework and Northstar implementation	Status	Extension
Unit 1 and Unit 3, lines 34-58 and 138-210	plan → act → evaluate is a visible loop; evaluation uses system signals rather than confidence	Design exposes the phases; missing-artifact and stale-report paths undermine trustworthy evaluation	Design aligned; execution blocked	EXT-006, EXT-010
Unit 4, lines 212-288	GitHub is the system of record and control plane	Issues, pull requests, commits, reviews, workflows, checks, CODEOWNERS, rulesets, and environments own durable state and enforcement; conversations are not authority	Conformant; hosted enforcement not verified	EXT-013, EXT-018
Unit 5 and Unit 6, lines 307-430	Humans remain accountable and agent work is evaluated through the contributor model	Agents may plan, implement, and review, but cannot accept their own output; pull requests are evaluated for intent, scope, evidence, ownership, policy, and fallback	Conformant; live review evidence not verified	EXT-006, EXT-007
Unit 2, lines 508-533 and 2296-2326	Agents have narrow SDLC responsibilities and bounded scopes	Planner, implementer, dependency, security-reviewer, and risk-reviewer roles have distinct tools, instructions, and handoffs	Conformant	EXT-004
Unit 3 and Unit 5, lines 553-624 and 3454-3598	A task contract defines inputs, outputs, scope, constraints, and observable success criteria	Live issue parser works when explicitly invoked; actual cloud bootstrap missed authority and local task switching exposed stale context	Host bootstrap blocked	EXT-001
Unit 4 and Unit 5, lines 635-867	Planning, execution, and validation are separated; use a plan-first workflow or plan + execution workflow based on risk; planning is read-only	Read-only planning preserved; implementation authorizer requires approved plan/branch even at lower risk, so advertised plan + execution is not proved	Choice advertised but incompletely implemented	EXT-001, EXT-003
Unit 2 and Unit 4, lines 3835-3967 and 4165-4223	Risk-based autonomy is required; the guide recommends low, medium, high, and critical classifications and stronger controls at higher-impact boundaries	Four-level model exists; catch-all medium rule masks low-risk documentation paths	Recommended model adopted; classifier blocked	EXT-002
Unit 4, Unit 5, and Unit 7, lines 244-278, 1196-1200, and 3998-4307	Required reviews, required checks, CODEOWNERS, rulesets or branch protection, environments, explicit permissions, and least privilege constrain work	Source policy, workflows, CODEOWNERS, role toolsets, protected-environment design, and GitHub App permission boundaries implement the model	Conformant; hosted settings not verified	EXT-004, EXT-008, EXT-009
Unit 7 and Unit 4, lines 1145-1183, 1748-1836, 2893-2907, and 4048-4070	Hooks provide pre-action blocking, post-action/error logging, and human escalation; custom agents declare tools, instructions, and handoffs	Native wiring exists; host payload/branch incompatibility, failure-event classification and raw diagnostic echo remain	Blocked; see surface matrix	EXT-004, EXT-005
Unit 6, Unit 7, and workflow units, lines 916-1100, 1334-1441, 1915-1980, and 2365-2591	GitHub Actions expose triggers, contexts, outputs, permissions, concurrency, orchestration, logs, and artifacts	Repaired workflow creates jobs; examined run fails repository-controls, human-review, plan-approval and evidence	Jobs demonstrated; acceptance blocked	EXT-006, EXT-011
Unit 5, Unit 6, and Unit 7, lines 340-373, 1016-1094, 2708-2816, and 4338-4392	Meaningful actions produce attributable, run- and commit-linked workflow outputs and artifacts; missing evidence is failure	Envelopes exist but absent artifact/null digest can produce readiness; base/attempt and stale Stop report checks are incomplete	Blocked, not strict fan-in	EXT-005, EXT-006
Memory and continuity units, lines 3088-3439	Issues, pull requests, documents, workflow outputs, logs, and artifacts form external memory and a durable source of truth	Fixture trust stays false; live resolution and cache freshness across hosts are not reliable	Design aligned; bootstrap blocked	EXT-001, EXT-018
Security units, lines 259-278, 402-413, 1127-1145, and 4038-4048	Code scanning, dependency signals, secret scanning, push protection, protected secrets, and environment approvals remain blocking signals	SARIF {} passes; read errors can skip scan inputs; workflow scanner errors do not propagate; hosted settings unverified	Blocked	EXT-008, EXT-015
MCP units, lines 1103-1145 and 1447-1691	MCP servers expand capability and must be governed through registries and organization/enterprise allow lists with bounded tools and protected credentials	Policy requires approved registries and an MCP allow list; Northstar’s local gh-aw adapter exposes named tools only and stores no credentials	Conformant in source; hosted registry/allow list policy not verified	EXT-014
Agentic Workflow units, lines 1260-1274, 1349-1441, and 2365-2591	GitHub Agentic Workflows express bounded intent in Markdown, compile to a lock workflow, use explicit triggers/tools/permissions/safe outputs, and augment rather than replace CI/CD	Source uses bounded staged output; examined hosted agent failed on model availability	Source aligned; hosted operation blocked	EXT-011
Reliability units, lines 1052-1076, 1183-1200, 2047-2108, 2834-2881, and 4432-4447	Recovery uses bounded retries, rollback, and human escalation; policy and security failures are not retried away	Failure-signature helper tested in isolation, not integrated into Stop-loop persistence/enforcement	Integration blocked	EXT-010
Governance units, lines 3822-3967 and 4412-4457	Governance is continuous; the guide recommends weekly review of failures, monthly review of permissions and secret scopes, and quarterly review of rules, ownership, environments, retention, and evidence	Cadence adopted; actual hosted audit could not authenticate and returned unavailable, not ready	Recommended / adopted; live audit blocked	EXT-009

Named technology disposition

Guide-named technology or feature	Northstar disposition
GitHub Issues, branches, commits, pull requests, reviews	Implemented
Required checks, CODEOWNERS, rulesets or branch protection	Source policy implemented; live enforcement not verified
Protected environments and required reviewers	Source workflows and policy implemented; live configuration not verified
GitHub Actions triggers, contexts, outputs, permissions, concurrency, artifacts	Real jobs started; required acceptance checks failed
CodeQL/SARIF, dependency signals, secret scanning, push protection	Source present; malformed SARIF/read-error gaps; two moderate dependencies; hosted settings not verified
GitHub Copilot agents, Copilot CLI hooks, prompts, instructions, handoffs	Partially exercised; see COPILOT-SURFACES.md
GitHub Agentic Workflows / Continuous AI / gh-aw	Compiles; scanner errors and hosted model failure prevent operational claim
MCP servers, registry, MCP allow list, GitHub MCP server	Policy and bounded tools implemented; organization/enterprise registry and allow list settings not verified
GITHUB_TOKEN and GitHub App tokens	Explicitly scoped in workflows; live App installation and permission state not verified
Personal access tokens	Not used by the canonical reference
Custom MCP Registry v0.1 or Azure API Center registry	Conditional option not used by Northstar

Example handling

The reference does not treat illustrative snippets as universal requirements. In particular:

• action versions from guide snippets are not frozen requirements;
• sample CODEOWNERS entries, labels, titles, and reviewer names are replaced with repository-specific values;
• the example plan-gate.yml and pull-request template are implemented with equivalent, stricter contracts rather than copied as the only valid shape;
• the gh agent-task and Copilot CLI command examples remain operational examples, not required system interfaces;
• custom MCP registry hosting and Azure API Center remain optional because Northstar does not operate a custom registry.

Gap-filling implementation disposition

The audit records each identified non-guide implementation in TECHNICAL-EXTENSIONS.md. The register covers:

• versioned contracts, digests, risk floors, branch conventions, and exact pre-tool authorization;
• payload-minimized hook records, evidence envelopes, readiness states, validation authority, trusted publication, identity separation, and maintenance manifests;
• governance drift queries, failure signatures, the reference toolchain, supplemental scanners, and merge validation;
• the PostgreSQL proving workload, repository separation, single README reference, inert snapshot locking, controlled bootstrap, MCP adapter, instruction synchronization, and the Agent Hooks experiment.

No extension replaces a guide-required GitHub control. When a source file can only describe a hosted setting, the corresponding status remains hosted not verified. architecture-lock.json provides machine-readable extension coverage; semantic completeness still requires human architecture review.

Current hosted evidence boundary

The following claims require live GitHub evidence and are not established by the repository snapshot alone:

• ruleset or branch-protection enforcement and required status bindings;
• current CODEOWNERS review enforcement;
• protected-environment reviewers, branch restrictions, self-review prevention, and administrator-bypass settings;
• GitHub App installation IDs, permissions, identity separation, and private key locations;
• secret scanning and push protection enablement;
• current pull-request reviews and immutable-head status;
• workflow run provenance, artifact retention, and the stable trusted-acceptance status;
• organization or enterprise MCP registry and allow list policy;
• successful hosted execution of the compiled Continuous AI workflow.

These controls must be reported as not verified until the corresponding GitHub API query or workflow run supplies direct evidence.

Audit conclusion

The architecture has no identified conceptual conflict with the guide, selects guide-supported choices by risk, distinguishes adopted recommendations, and keeps conditional technologies conditional. No guide requirement is intentionally replaced.

The current release is nevertheless not conformant. The earlier workflow syntax and terminology blockers were repaired in merged Northstar PRs #7, #10 and #12; they are no longer the release blocker. The locked main revision still has task/bootstrap, evidence-integrity, risk-routing, recovery and scanner defects, while the newer repair and isolation candidates have not produced trusted hosted acceptance. The architecture lock therefore correctly fails closed. Conformance may be restored only after the dependent Northstar changes are accepted, exact evidence is published for an immutable accepted commit, the inert snapshot and locks are refreshed to that commit, and pwsh -File tools/verify-architecture.ps1 passes without bypassing its release gate.