System Invariants and Smoke Tests

Last Updated: 2026-02-28
Content-Type: Reference (operational)
Audience: Engineering + QA + On-call

---

Purpose

This document defines the “must always be true” invariants for the platform, plus a minimal smoke-test checklist to run after any PR affecting chat, tools, prompts, retrieval, or infra.

---

Invariants (must always hold)

Tool Platform invariants (backend)

1. Central enforcement: all tool executions pass through the canonical tool executor path and are policy enforced.
2. Policy completeness: access policies in tool catalog are fully enforced:
   • protocol allowlists
   • enablement (runtime overrides)
   • target scopes: self_only, explicit_user_selected, etc.
3. No hidden tools: any callable tool is registered in the canonical tool catalog (even if marked internal).
4. Observability: every tool execution emits an audit event and includes: toolId, userId, protocolId, allow/deny reason.

Prompt Contract invariants (backend)

1. No brittle stripping: runtime must not depend on template section stripping to be correct.
2. Structured runtime context is parse-safe: if a structured context channel exists, it must be explicitly delimited, schema validated, and fail closed (not silently).

Retrieval invariants (backend)

1. One embedding per turn when chunk retrieval is enabled (unless a specific feature requests more).
2. Conversation scoping: when structured history exists, retrieval must avoid re-injecting transcript chunks unless explicitly configured.
3. No ingestion data loss: chunk ingestion must be atomic (never “delete all chunks then fail insert”).

Background work invariants (backend)

1. Bounded concurrency: background jobs are globally capped and deduped by key.
2. Time-bounded external calls: embedding/web fetch operations have explicit timeouts.

Frontend chat invariants (aso-portal)

1. Single vs group correctness: UI sends to the correct transport and updates the correct history.
2. No stuck state: on any upload/send failure, status resets from thinking to error or idle and UI surfaces a consistent error.
3. Payload parity: if the UI exposes preferences (web search toggles), those preferences are transmitted for both single and group or explicitly disabled.

Env/config invariants

1. Single source-of-truth: env is loaded and validated once; other modules import typed env rather than reading process.env directly.
2. Templates match reality: .env.example matches validation and actual usage.
3. No committed secrets: repo does not track any env-like “copy” files.

---

Smoke-test checklist (minimal)

Backend (runtime)

• [ ] Server starts cleanly with only .env (or .env.example + documented overrides).
• [ ] Tool list endpoint works (if applicable) and reflects enablement + policy.
• [ ] Denied tool execution returns 403 with a clear reason (policy/protocol/enablement).
• [ ] Chat request completes with tools enabled and produces a response.

Backend (retrieval)

• [ ] One chat turn triggers at most one embedding generation (verify logs/metric).
• [ ] Chunk retrieval returns results and does not duplicate structured conversation history (if enabled).

Backend (ingestion)

• [ ] A memory ingestion run does not drop chunks on transient failure (transaction/UPSERT behavior).

Portal

• [ ] Single chat: send text, receive response; status transitions idle → thinking → idle.
• [ ] Single chat streaming (if enabled): placeholder message replaced; tool progress renders.
• [ ] Group chat: suggested prompt sends to group; message appears in group history.
• [ ] Group chat preferences: toggles affect requests, or toggles are hidden/disabled with a clear UI message.
• [ ] Upload failure and socket failure do not leave UI stuck in thinking.

Docs/dev workflow

• [ ] Setup guide references .env.example only.
• [ ] No env copy files tracked in git.