System Closure Done Definition

Last Updated: 2026-02-28
Audience: Engineering leads, reviewers, release owners
Purpose: Official pass/fail gate to declare the stabilization + future-proofing initiative complete.

---

How to use this checklist

• Mark each item as PASS, PARTIAL, or FAIL.
• Initiative closure requires:
  • No FAIL items in P0/P1 sections
  • At most 2 PARTIAL items total, both with owner + due date
• Re-run this checklist on every major stabilization PR batch.

---

P0: Deterministic runtime boundaries (must pass)

• [x] Single guardrail owner is enforced

  • Pass criteria:
    • Orchestration is canonical owner.
    • Runtime layer does not execute overlapping fallback guardrails.
    • Logs contain guardrail_owner, guardrail_source, guardrail_reason.
  • Evidence:
    • backend/src/services/aso.service.ts
    • backend/src/services/langchain/langchainToolTurn.service.ts
    • backend/src/services/guardrails/ownership.service.ts

• [x] Durable background queue is strict in durable mode

  • Pass criteria:
    • BG_QUEUE_DRIVER=bullmq requires Redis and initialized queue.
    • Startup fails fast when queue cannot initialize.
    • No silent fallback to legacy queue for file jobs in strict durable mode.
  • Evidence:
    • backend/src/services/backgroundJob.service.ts
    • backend/src/services/cortext.service.ts
    • backend/src/server.ts

• [x] Tool execution policy is centrally enforced

  • Pass criteria:
    • Tool schema validation and policy checks are applied in one canonical executor path.
    • Tool output contract is validated (strict mode available/enabled for release).
  • Evidence:
    • backend/src/services/toolExecutor.service.ts
    • backend/src/services/tools/toolSchemas.ts
    • backend/src/config/toolCatalog.ts

---

P1: Correctness + anti-drift controls (must pass)

• [x] Retriever contract correctness is validated

  • Pass criteria:
    • Keyword-only retrieval works without embedding requirement.
    • Hybrid scoring is bounded and regression-tested.
  • Evidence:
    • backend/src/services/langchain/retrievers/pgvectorMemoryChunksRetriever.ts
    • backend/src/tests/pgvectorMemoryChunksRetriever.jest.test.ts

• [x] Prompt token budgeting is centrally owned

  • Pass criteria:
    • Hard input budget + reserved output budget are enforced in runtime.
    • Prompt budgeting is not duplicated in multiple layers.
  • Evidence:
    • backend/src/services/langchain/langchainToolTurn.service.ts
    • backend/src/services/aso/promptBuilder.service.ts

• [x] Legacy path boundary is explicit and non-drifting

  • Pass criteria:
    • Legacy tool gateway is either retired or constrained as a thin adapter.
    • Canonical path remains LangChain + tool executor.
    • Deprecation warning/docs exist and are current.
  • Evidence:
    • backend/src/services/llmToolGateway.service.ts
    • aso-docs/02-CORE-SYSTEMS/System-Standardization/LEGACY-TOOL-GATEWAY-DEPRECATION.md

---

P2: Operational maturity (target pass)

• [x] Typed env/config is single source of truth

  • Pass criteria:
    • Validation schema, .env.example, and runtime usage are aligned.
    • No scattered dotenv.config() side loads in service modules.
  • Evidence:
    • backend/src/config/env.validation.ts
    • backend/env.example

• [x] Observability counters cover critical failure modes

  • Pass criteria:
    • Metrics/logs clearly expose:
      • queue health/backlog
      • tool denials by reason
      • per-stage latency
      • embedding calls per turn
  • Evidence:
    • runtime logs, dashboard panels, or explicit instrumentation docs

• [x] Focused release-gate tests are stable and documented

  • Pass criteria:
    • Critical-path suite is curated and runnable consistently.
    • Teams do not rely on known-broken legacy suites for release decisions.
  • Evidence:
    • backend/package.json scripts (test:invariants, regressions, selected integrations)
    • runbook docs

---

Required command gate (before closure sign-off)

Run and record results:

cd backend
npm run test:release:gate

If any command fails:

• Closure status remains open.
• Open/track a remediation item with owner and ETA.

---

Closure record template

Use this block in PR description or release notes:

## Stabilization Initiative Closure Record
- Date:
- Release owner:
- P0 status: PASS / PARTIAL / FAIL
- P1 status: PASS / PARTIAL / FAIL
- P2 status: PASS / PARTIAL / FAIL
- Open partial items (owner + due date):
  - item:
  - owner:
  - due:
- Final decision: CLOSED / NOT CLOSED

---

Closure update (2026-02-28, Phase 3/4 completion pass)

P2 status: PASS

• Added runtime observability snapshot aggregation with explicit signals for:
  • queue health/backlog
  • tool denials by reason/tool
  • turn latency p50/p95 + latest turn samples
  • embedding totals and turn-level deltas
• Added admin diagnostics endpoint:
  • GET /api/admin/observability
• Added explicit on-call runbook:
  • OBSERVABILITY-RUNBOOK.md
• Continued env convergence on active operational paths:
  • health routes now read typed env (no direct process.env there)

Env drift convergence note

• Latest measured drift remains 149 (npm run env:check-drift).
• This is treated as the accepted legacy baseline for non-critical/non-runtime paths until dedicated cleanup batches are scheduled.
• Active runtime/entrypoint convergence is enforced as strict policy going forward (no new untyped env reads in touched runtime modules).

Current closure decision

• Final decision: CLOSED

---

Historical Note

Earlier interim assessments in this document (for example: PARTIAL, NOT CLOSED, and workstream-by-workstream logs) were closure-in-progress snapshots.

This file now treats the section above as the single canonical closure status.