LangChain Upgrade Guide

Last Updated: 2026-02-17
Content-Type: How-to / Roadmap
Audience: Developers, Architects

---

Purpose

This document is a phase-by-phase roadmap for adopting more of LangChain’s capabilities in the ASO/Kirakira backend. It explains the current usage, what each phase adds, where to change code, and how to do it safely.

---

Table of Contents

1. Current Status
2. LangChain Capabilities (Summary)
3. Prompt Flow Diagram
4. Phase 1: Low-Risk Wins
5. Phase 2: Deeper Integration
6. Phase 3: Full Pipeline (Optional)
7. Environment Variables Reference
8. File and Code Reference
9. Tradeoffs and Checklist

---

Current Status

What Is Already Using LangChain

Component	Location	Purpose
LangChain client	backend/src/services/langchain/langchainClient.service.ts	Builds chat model using ONE_API_URL/ONE_API_KEY or direct provider when USE_ONE_API=false.
Agent + tools	backend/src/services/langchain/langchainToolTurn.service.ts	createReactAgent loop: LLM → tool calls → execute → repeat until final answer. Used when orchestrator flag is on.
LCEL chain	backend/src/services/langchain/chains/	asoResponseChain (prompt → model → parser); aso.service invokes it when USE_LANGCHAIN_ORCHESTRATOR is true.
LangChain memory	backend/src/services/langchain/memory/asoDbChatMessageHistory.ts	Custom BaseChatMessageHistory backed by ASO Memory table; agent gets history from DB when memoryConfig is set.
PG vector retriever	backend/src/services/langchain/retrievers/pgvectorMemoryChunksRetriever.ts	Custom RAG: embeds query, queries pgvector memory chunks, returns ranked chunks. Not a LangChain chain; it’s our code in the langchain folder.

When the LangChain Orchestrator Runs

The ASO chat runtime is LangChain-only:

• The previous native/legacy tool-call paths and the flags USE_PROVIDER_NATIVE_TOOLS, USE_JSON_TOOL_FALLBACK, and USE_LANGCHAIN_ORCHESTRATOR were removed.
• backend/src/services/aso.service.ts always calls runChatRuntime() and backend/src/services/aso/chatRuntime.service.ts always routes through asoResponseGraph (LangGraph).

Memory RAG (Separate from Orchestrator)

• Context building: backend/src/services/aso/context.service.ts calls retrieveMemoryChunks from langchain/retrievers/pgvectorMemoryChunksRetriever when USE_MEMORY_CHUNKS !== "false" and user input exists.
• This feeds retrieved chunks into the main ASO context; it does not use LangChain runnables or chains.

NPM Packages (backend)

• langchain
• @langchain/core
• @langchain/openai
• @langchain/langgraph (agent loop prebuilt, StateGraph for Phase 3.2)

---

LangChain Capabilities (Summary)

Understanding what LangChain offers helps decide what to adopt:

Capability	Description	Our Current Use
Model abstraction	One interface for OpenAI, Anthropic, local APIs, etc.	Yes — ChatOpenAI + One-API (or direct provider when USE_ONE_API=false).
Tools & tool calling	Define tools with schemas; model returns tool calls; runtime executes.	Yes — in langchainToolTurn.service.ts; agent can call tools multiple times.
Chains (LCEL)	Compose steps: e.g. context → prompt → LLM → parse.	Yes — asoResponseChain (prompt → model → parser); aso.service invokes it when orchestrator is on.
Agents	Multi-step loop: LLM → tools → observe → repeat until final answer.	Yes — createReactAgent in langchainToolTurn.service.ts; multi-step tool loop.
Memory	Conversation buffers, summaries, vector memory.	Yes — AsoDbChatMessageHistory backs the agent; history from DB when memoryConfig is set.
RAG	Document loaders, splitters, vector stores, retriever + LLM chains.	Partial — MemoryChunksRetrieverAdapter; RAG injected in context and in LangChain tool turn.
Observability	LangSmith tracing, evals, datasets.	Yes — LangSmith env vars supported; traces when path runs.
LangGraph	Multi-step or multi-agent graphs with state and branching.	Yes — asoResponseGraph (respond node runs LCEL chain); extensible for branching.

---

Prompt Flow Diagram

How the system prompt is built and where the LangChain template layer (Phase 1.3) sits:

Flow in words:

1. Sources: Template text comes from DB (prompt_templates), JSON fallback, or protocol definition (string or reference-based with base + overrides).
2. Composition: Protocol routing picks the protocol; composePromptTemplate(protocol) loads the base, injects , and appends protocol rules.
3. Build: buildToolCallPrompt(...) builds promptData (npcName, shortTermMemory, worldKnowledge, asoCoreMemories, sharedMemories, systemWideKnowledge*, etc.), then fillTemplate(template, promptData) produces basePrompt. Protocol directives (and optional system-knowledge block) are prepended to get the final system string (contextWithoutOutputFormat).
4. Usage: That string is used as-is for the native path. For the LangChain path it is passed into runLangChainToolTurnWithTrace. Phase 1.3 adds a ChatPromptTemplate that accepts this string as systemContent and produces the SystemMessage(s) for the LangChain invoke, so “one major prompt” is managed via a LangChain template without changing the existing composition pipeline.

---

Phase 1: Low-Risk Wins

Goal: add observability, align RAG with LangChain patterns, and introduce prompt templates without changing core behaviour.

1.1 Observability (LangSmith) ✅ Implemented

What: Send traces (LLM calls, tool calls, latency) to LangSmith so you can inspect and debug runs.

Why first: No refactor of business logic; only configuration and optional SDK usage.

Implemented (codebase):

• backend/src/config/env.validation.ts — LANGCHAIN_TRACING_V2, LANGCHAIN_API_KEY, and LANGCHAIN_PROJECT are now optional env vars (typed and validated).
• backend/env.example — LangSmith section added with the three variables and comments; also documented USE_LANGCHAIN_ORCHESTRATOR and USE_STRUCTURED_CHAT_MESSAGES for triggering the LangChain path.

LangChain’s SDK reads these from process.env automatically; no change to aso.service.ts or langchainToolTurn.service.ts is required.

Steps (you):

1. Sign up and get an API key at LangSmith.
2. Add to backend/.env:

   LANGCHAIN_TRACING_V2=true
   LANGCHAIN_API_KEY=your_langsmith_api_key
   LANGCHAIN_PROJECT=aso-kirakira   # optional; names the project in LangSmith

3. To see traces: Ensure the LangChain path runs: set USE_LANGCHAIN_ORCHESTRATOR=true and USE_PROVIDER_NATIVE_TOOLS=true. Any message should create a trace. (For tool traces, use cognitive depth multi_stage/deep; single_pass is routed to a single model call with no tools.)
4. Confirm: Open LangSmith, select project aso-kirakira (or your project name), and confirm traces appear for those runs.

Success criteria: Traces visible in LangSmith for the LangChain tool path.

Test script (personal vs service key): From backend/ you can run a minimal trace without starting the full app to see whether the key works and what error LangSmith returns:

cd backend
# Use key from .env
npm run test:langsmith-trace

# Or pass key explicitly (e.g. personal vs service)
npx ts-node scripts/test-langsmith-trace.ts --key=lsv2_YOUR_PERSONAL_KEY
npx ts-node scripts/test-langsmith-trace.ts --key=lsv2_YOUR_SERVICE_KEY

Any 403/401 or "Failed to send multipart request" will appear in the output so you can tell if the issue is on your side or Smith’s. See backend/scripts/test-langsmith-trace.ts for details.

Note: A personal API key with a project name (e.g. LANGCHAIN_PROJECT=aso-kirakira) is enough for development and traces work. Service keys are intended for production and may require a workspace ID (not org ID) in LangSmith; if you don’t have a workspace set up or don’t need service keys yet, using the personal key is fine until you switch later.

---

1.2 RAG with LangChain Abstractions ✅ Implemented

What: Use LangChain’s retriever and/or chain patterns so RAG is consistent and easier to extend (e.g. hybrid search, reranking later).

Implemented:

• BaseRetriever wrapper: backend/src/services/langchain/retrievers/memoryChunksRetrieverAdapter.ts defines MemoryChunksRetrieverAdapter extending LangChain’s BaseRetriever. In _getRelevantDocuments(query) it calls retrieveMemoryChunks with the existing options (playerUserId, npcUserId, topK, etc.) and maps RetrievedChunk[] to LangChain Document (pageContent + metadata: tier, memoryId, score, etc.). createMemoryChunksRetriever(options) and documentsToChunks(docs) are exported.
• Integration: context.service.ts and langchainToolTurn.service.ts use createMemoryChunksRetriever (and where needed documentsToChunks) instead of calling retrieveMemoryChunks directly, so RAG runs through the LangChain retriever in both paths.

Original context: The underlying retrieveMemoryChunks lives in pgvectorMemoryChunksRetriever.ts. The adapter wraps it so the same behaviour is available as a LangChain retriever for chains and clearer code paths.

Success criteria: Same or better RAG behaviour; retriever usable inside LangChain chains; code paths that use RAG are clearer. ✅ Met.

---

1.3 Prompt Templates ✅ Implemented

What: Move critical prompts (system prompt, tool instructions, etc.) into LangChain ChatPromptTemplate / PromptTemplate with named variables so they’re easier to version and reuse.

Where prompts live today: Built in aso.service.ts and related services (e.g. protocol prompts, tool-call instructions). Some come from the DB via prompt.service.ts and promptComposition.service.ts.

Steps:

1. Choose one high-value prompt (e.g. the system prompt passed to the LangChain tool turn, or the “native” system prompt for the main ASO path).
2. Create a ChatPromptTemplate (or PromptTemplate) that takes variables such as persona, shortTermMemory, toolsDescription, etc.
3. Replace the inline string building with template.formatMessages(...) or template.invoke(...).
4. Repeat for other prompts incrementally.

Files to touch:

• New: e.g. backend/src/services/langchain/prompts/ (or under existing prompt.service.ts / config).
• backend/src/services/aso.service.ts and/or backend/src/services/langchain/langchainToolTurn.service.ts where those prompts are built.

Implemented: backend/src/services/langchain/prompts/asoSystemPrompt.ts defines a ChatPromptTemplate with {systemContent}; langchainToolTurn.service.ts uses formatAsoSystemMessages({ systemContent: systemPrompt }) so the system prompt is managed via the template. The content is still produced by the existing pipeline (see Prompt Flow Diagram).

Success criteria: At least one major prompt is managed via LangChain template; behaviour unchanged; future prompts can follow the same pattern. ✅ Met.

---

Phase 2: Deeper Integration

Goal: multi-step tool use (agent loop) and optional LangChain-backed conversation memory.

2.1 Agent Loop (Multi-Step Tool Use) ✅ Implemented

What: Replace the current “one tool turn” (LLM → tools → one more LLM call) with a proper agent loop: LLM → tool calls → execute tools → append results to messages → LLM again, until the model returns a final answer without tool calls.

Implemented behaviour: backend/src/services/langchain/langchainToolTurn.service.ts uses LangGraph’s prebuilt createReactAgent(...) (multi-step loop) so the model can call tools multiple times in one turn. Tool execution is still routed through executeTool(...), so permissions and behavior remain consistent with the rest of the system.

Iteration cap / safety: The max iterations are capped by LANGCHAIN_AGENT_MAX_ITERATIONS (1–50). The runtime additionally selects a smaller limit for single_pass and a higher limit for deep to avoid runaway while still allowing complex turns.

Guardrail: If the model claims it will “search/check” but produced no tool calls, the system can do a single web-search fallback pass (when web tools are allowed) to prevent “hang tight” responses that never actually search.

Success criteria: With USE_LANGCHAIN_ORCHESTRATOR=true, the model can call tools multiple times in one turn; traces and logs still work.

---

2.1.1 Unified path selection and multimodal (LangChain path)

What: When USE_LANGCHAIN_ORCHESTRATOR is true, use only the LangChain path for all turns. Path selection no longer depends on cognitive depth or attachments. Cognitive depth continues to control behavior within the path (e.g. RAG injection, perception/salience). The LangChain path supports multimodal (images) so that turns with attachments are handled by the same path.

Implemented:

• Path selection: In aso.service.ts, the condition is if (LLM_FLAGS.USE_LANGCHAIN_ORCHESTRATOR) { ... runLangChainToolTurnWithTrace ... } else { ... native path ... }. No !hasAnyAttachments or cognitiveDepth !== "single_pass" in the condition.
• Multimodal (required): runLangChainToolTurnWithTrace accepts imageUrls (same shape as native path). When present, the last user message is built as multimodal content (text + image_url parts) so the model receives images. Image count is limited (e.g. MAX_PROMPT_IMAGE_URLS); URLs are cleaned before use.
• Deprecation plan: The native tool path can be deprecated once the LangChain path supports all current use cases. Phase 2.1.2 (file support) is the concrete step to get there; images are already covered in 2.1.1. Deprecation is deferred until you explicitly decide to deprecate the native path; both paths remain available until then.

Success criteria: Flag on → LangChain path only; attachments (images) work on the LangChain path; no regression for text-only or image turns.

---

2.1.2 File support on LangChain path (parity for deprecation)

What: Add support for non-image file attachments (e.g. PDFs, documents) on the LangChain path so that when USE_LANGCHAIN_ORCHESTRATOR is true, file uploads are handled the same way as on the native path. This completes “all current use cases” so the native tool path can be deprecated when you choose to.

Implemented:

• fileHandlingConfig: Computed once in aso.service.ts (before the LangChain vs native branch) and passed into both paths. runLangChainToolTurnWithTrace accepts fileHandlingConfig and imageUrls (which can include type === "file" items).
• Last user message: When there are file attachments and fileHandlingConfig?.preferDirectUpload is true, the last user message is built via buildMultimodalPayloadNormalized (same as the fallback path): canonical content (text + images + file references) is normalized to provider format (OpenAI file / Gemini fileData, etc.) and used as the HumanMessage content. When there are no files (or no preferDirectUpload), the existing text-only or text+image logic is unchanged (no regression for image-only or text-only).
• Native path: Not deprecated; deprecation is deferred until you explicitly request it. Both paths remain available.

SSE note (important): For direct-upload file references to work over the SSE endpoint, the request must preserve file metadata (internalFileId, fileId, fileUri, mimeType, etc.). The backend schema for /api/aso/chat/stream now accepts and passes these through, so both Socket and SSE transports can carry provider-native file references.

Why: The native path uses fileHandlingConfig and provider-specific handling (e.g. OpenAI file_id, Gemini file_uri) for type === "file" attachments. The LangChain path previously only sent images. 2.1.2 closes that gap.

Steps:

1. In runLangChainToolTurnWithTrace, accept fileHandlingConfig (or equivalent) and treat imageUrls-style items with type === "file" when building the last user message (e.g. provider-normalized file parts alongside text and image parts).
2. Reuse existing file-normalization behaviour (e.g. from multimodalPayload.service / fileNormalization.service) so the LangChain message shape matches what the native path sends for files.
3. Pass fileHandlingConfig and any file-specific context from aso.service.ts into the LangChain call when the flag is on.

Files to touch:

• backend/src/services/langchain/langchainToolTurn.service.ts: extend last user message to include file parts when present; optional: shared helper for multimodal message building.
• backend/src/services/aso.service.ts: pass fileHandlingConfig (and file attachments) into runLangChainToolTurnWithTrace when using the LangChain path.

Success criteria: With the flag on, turns that include only file attachments (or files + images) are handled on the LangChain path with correct provider format; no regression for image-only or text-only turns.

---

2.1.3 LLM transport: One API vs direct provider (global, dual path)

What: Support two LLM transport paths behind a global env flag: (A) current path via One API (single endpoint; all chat/completion goes through One API), (B) path without One API using direct provider APIs (e.g. LangChain’s ChatOpenAI / ChatGoogleGenerativeAI with each provider’s base URL and API keys). The flag controls all chat/completion calls — the LangChain tool path, the native tool path, and every caller of generateChatResponse / generateChatCompletionMessage. Both transports coexist; deprecation of one path is deferred until you decide.

Why: Single global choice keeps behavior consistent (no mixing One API and direct in the same app). Enables comparison of latency, cost, and behavior; clear rollback by flipping the flag; explicit deprecation later.

Flag scope (Option B): One flag for the whole app. When “direct”, all chat/completion uses provider-specific clients (e.g. LangChain models or provider SDKs) and provider API keys. When “One API”, all chat/completion uses One API as today. Suggested name: USE_ONE_API=true|false or LLM_TRANSPORT=one_api|direct.

Steps (outline):

1. Introduce the env flag and a small config (e.g. in llmFlags.ts or env validation).
2. LangChain path: When flag is “direct”, getLangChainChatModel() returns a provider-specific model (e.g. ChatOpenAI with OpenAI base URL for GPT, ChatGoogleGenerativeAI for Gemini) based on model name; when “One API”, keep current (One API base URL).
3. Non-LangChain path: When flag is “direct”, generateChatResponse and generateChatCompletionMessage in embedding.service.ts (or a shared chat client they use) call direct provider APIs instead of oneApiClient; route by model name to the right provider client. When “One API”, keep current behavior.
4. Ensure images/files and tool-calling work on both paths (message format and file upload may differ per provider when direct).

Files to touch:

• backend/src/config/ (or env): add flag; optional llmFlags.ts entry.
• backend/src/services/langchain/langchainClient.service.ts: branch on flag; return One API–backed or direct provider–backed model.
• backend/src/services/embedding.service.ts: branch on flag in generateChatResponse / generateChatCompletionMessage; when direct, use provider-specific client(s) or LangChain models by model name.

Success criteria: With flag “One API”, behavior unchanged (all traffic via One API). With flag “direct”, all chat/completion goes to provider APIs; no regression. Deprecation of one path is deferred.

---

2.2 LangChain Memory ✅ Implemented

What: Use LangChain’s memory abstractions (e.g. ConversationBufferWindowMemory) or a custom memory that reads/writes your existing Memory table and short-term format, so the rest of the app can treat “conversation history” as a LangChain memory.

Why: Standardising on LangChain’s memory interface improves conversation flow, gives one place to evolve history (windowing, summarisation), and keeps the agent’s context aligned with existing storage.

Implemented:

• Custom history: backend/src/services/langchain/memory/asoDbChatMessageHistory.ts defines AsoDbChatMessageHistory extending LangChain’s BaseChatMessageHistory. It loads messages via getShortTermMemory (same source as before), parses the stored text into HumanMessage/AIMessage, and implements addUserMessage/addAIMessage via addMemory so writes go to the existing Memory table.
• Integration: runLangChainToolTurnWithTrace accepts an optional memoryConfig (userId, conversationId, npcUserId, npcUsername, optional limit/maxMessages/maxCharsPerMessage). When memoryConfig is set, the agent’s history is built from AsoDbChatMessageHistory.getMessages() instead of the history array.
• Call site: In aso.service.ts, when USE_LANGCHAIN_ORCHESTRATOR is true, memoryConfig is passed so the LangChain path always uses DB-backed memory for conversation history. The post-response pipeline continues to write the new turn to Memory as before.

Files touched:

• New: backend/src/services/langchain/memory/asoDbChatMessageHistory.ts, backend/src/services/langchain/memory/index.ts.
• backend/src/services/langchain/langchainToolTurn.service.ts: LangChainMemoryConfig type, memoryConfig on args, history from memory when memoryConfig present.
• backend/src/services/aso.service.ts: pass memoryConfig into runLangChainToolTurnWithTrace on the LangChain path.

Success criteria: Agent uses LangChain memory that reflects existing conversation storage; no regression in context quality. ✅ Met.

---

Phase 3: Full Pipeline (Optional)

Goal: express the main ASO response pipeline as LangChain runnables (LCEL) or as a LangGraph, for uniform streaming, batching, and observability.

3.1 LCEL Chain for ASO Response ✅ Implemented (prompt → model → parser)

What: Compose the full path (prompt building → model with tools → parser) into one LCEL chain:

asoPromptRunnable.pipe(asoModelRunnable).pipe(asoParserRunnable)

Implemented:

• Chain state and runnables: backend/src/services/langchain/chains/ defines:
  • Types: AsoChainInput, AsoChainOutput, AsoChainInputWithPromptStep (optional toolCallPrompt / sanitizedInput / npcInfo / useStructuredChatMessages so the prompt runnable can derive systemPrompt and userPrompt).
  • Prompt runnable: asoPromptRunnable — when state has toolCallPrompt and sanitizedInput, derives systemPrompt and userPrompt (OUTPUT FORMAT strip + system knowledge block); otherwise passes through when systemPrompt / userPrompt are already set.
  • Model runnable: asoModelRunnable — calls runLangChainToolTurnWithTrace and returns state with content and trace.
  • Parser runnable: asoParserRunnable — identity over final state (explicit end step).
  • Composed chain: asoResponseChain = asoPromptRunnable.pipe(asoModelRunnable).pipe(asoParserRunnable).
• Integration: When USE_LANGCHAIN_ORCHESTRATOR is true, aso.service.ts builds lcArgs as AsoChainInputWithPromptStep and invokes asoResponseGraph.invoke({ payload: lcArgs }), then reads payload.content and payload.trace. Prompt derivation runs inside the LCEL chain (the graph nodes call the chain). Benefits: LangGraph is now the production entry point for the LangChain response path, so routing/branching is safe and incremental.

Files touched:

• backend/src/services/langchain/chains/types.ts, promptRunnable.ts, modelRunnable.ts, parserRunnable.ts, asoResponseChain.ts, index.ts.
• backend/src/services/aso.service.ts: import asoResponseChain, AsoChainInputWithPromptStep; pass toolCallPrompt / sanitizedInput / npcInfo / useStructuredChatMessages into the chain.

Success criteria: ASO response path runs through LCEL (prompt → model → parser); behaviour and performance acceptable; traces in LangSmith show the chain. ✅ Met.

---

3.2 LangGraph (Optional) ✅ Implemented

What: Model the ASO pipeline (or parts of it) as a graph: nodes (e.g. “gather context”, “route by depth”, “run agent”, “post-process”) and edges (conditional or fixed). LangGraph manages state and cycles (e.g. agent loop with tool calls).

Implemented:

• Graph: backend/src/services/langchain/graph/asoResponseGraph.ts — StateGraph with state payload (chain input + content + trace). Includes a router and two response nodes:
  • respond: invokes asoResponseChain (agent/tools)
  • respond_single_pass: invokes asoResponseChainSinglePass (single model call, no tools) Path: START → router → (respond | respond_single_pass) → END.
• Usage: asoResponseGraph.invoke({ payload: lcArgs }); result is in finalState.payload.content and finalState.payload.trace.

When to extend: Add conditional edges (e.g. route by depth), human-in-the-loop, or more nodes (context, post-process) by editing the graph builder.

Files:

• backend/src/services/langchain/graph/asoResponseGraph.ts, graph/index.ts.

Success criteria: A non-trivial part of the ASO flow runs as a LangGraph; state and branching behave as designed. ✅ Met (chain runs inside graph).

---

Environment Variables Reference

Variable	Purpose	Phase
USE_LANGCHAIN_ORCHESTRATOR	If true, use the LangChain response path (with USE_PROVIDER_NATIVE_TOOLS). Selection of LangChain vs native does not depend on attachments/depth (Phase 2.1.1), but the graph can route internally (e.g. single_pass → single-pass model call).	Current
USE_PROVIDER_NATIVE_TOOLS	If true, use native-tools block (LangChain or native path inside it).	Current
USE_STRUCTURED_CHAT_MESSAGES	If true, pass structured history to LangChain/native tool path.	Current
USE_MEMORY_CHUNKS	If not false, use memory-chunk RAG in context and in LangChain tool turn.	Current
ONE_API_URL	Base URL for LLM (used by LangChain client).	Current
ONE_API_KEY	API key for LLM.	Current
LANGCHAIN_TRACING_V2	Set to true to enable LangSmith.	Phase 1.1
LANGCHAIN_API_KEY	LangSmith API key.	Phase 1.1
LANGCHAIN_PROJECT	Project name in LangSmith (optional).	Phase 1.1
LANGCHAIN_AGENT_MAX_ITERATIONS	Max agent loop steps (default 10; 1–50). Passed as recursionLimit.	Phase 2.1
USE_ONE_API	When true (default), all chat/completion uses One API. When false, uses direct provider APIs.	Phase 2.1.3
OPENAI_API_KEY	Required for direct OpenAI when USE_ONE_API=false and model is OpenAI.	Phase 2.1.3
GEMINI_API_KEY or GOOGLE_API_KEY	Required for direct Gemini when USE_ONE_API=false and model is Gemini.	Phase 2.1.3
DEEPSEEK_API_KEY	Required for direct DeepSeek when USE_ONE_API=false and model is DeepSeek.	Phase 2.1.3
XAI_API_KEY	Required for direct Grok (xAI) when USE_ONE_API=false and model is Grok.	Phase 2.1.3

Add new vars as needed for RAG (e.g. retriever topK) or graph config.

---

File and Code Reference

Area	File(s)	Notes
LangChain client	backend/src/services/langchain/langchainClient.service.ts	getLangChainChatModel().
Single tool turn	backend/src/services/langchain/langchainToolTurn.service.ts	runLangChainToolTurnWithTrace, buildTools.
LCEL chains (3.1)	backend/src/services/langchain/chains/	asoResponseChain (agent/tools) and asoResponseChainSinglePass (no tools), plus asoResponseChainWithContext (context → toolCallPrompt → prompt → model → parser) and runnables. Used via the graph when USE_LANGCHAIN_ORCHESTRATOR.
Prompt builder	backend/src/services/aso/promptBuilder.service.ts	buildToolCallPrompt (extracted from aso.service for reuse and LCEL).
LangGraph (3.2)	backend/src/services/langchain/graph/	asoResponseGraph (router → respond/respond_single_pass). Invoked from aso.service when USE_LANGCHAIN_ORCHESTRATOR.
PG vector retriever	backend/src/services/langchain/retrievers/pgvectorMemoryChunksRetriever.ts	retrieveMemoryChunks.
ASO entry point	backend/src/services/aso.service.ts	LangChain vs native vs self-monitoring selection inside the USE_PROVIDER_NATIVE_TOOLS block.
Context (RAG)	backend/src/services/aso/context.service.ts	Calls retrieveMemoryChunks for main context.
LLM flags	backend/src/config/llmFlags.ts	USE_LANGCHAIN_ORCHESTRATOR, USE_STRUCTURED_CHAT_MESSAGES.
Tools	backend/src/services/toolExecutor.service.ts, backend/src/config/toolCatalog.ts	Real tool execution and catalog.
Chat request schema (portal + SSE)	backend/src/schemas/asoChat.schema.ts	Zod schema for chat payload (attachments include file metadata for direct-upload references).
Streaming endpoint	backend/src/controllers/aso.controller.ts, backend/src/routes/aso.routes.ts	POST /api/aso/chat/stream (SSE): streams token + tool_start/tool_end, then done.

---

Tradeoffs and Checklist

Benefits of Adopting More LangChain

• Observability: LangSmith gives a single place to inspect LLM and tool behaviour.
• Standard patterns: RAG, agents, and memory are well-documented and easier to onboard.
• Multi-step tools: Agent loop allows richer tool use without hand-rolled loops.
• Future-proofing: Easier to add new chains, agents, or graphs later.

Costs and Risks

• Dependency: Upgrades and breaking changes in LangChain can require code changes.
• Refactor: Phases 2 and 3 touch the core of aso.service.ts; need tests and gradual rollout.
• Flexibility: Some custom behaviour may need to be expressed in LangChain’s abstractions.

Phase-by-Phase Checklist

• [x] Phase 1.1 — LangSmith: env vars set, traces visible (env validation, env.example, test script).
• [x] Phase 1.2 — RAG: retriever wrapped (MemoryChunksRetrieverAdapter); integrated in context.service and langchainToolTurn.
• [x] Phase 1.3 — Prompts: at least one major prompt in LangChain template (asoSystemPrompt.ts + langchainToolTurn).
• [x] Phase 2.1 — Agent: multi-step tool loop (createReactAgent) wired behind USE_LANGCHAIN_ORCHESTRATOR.
• [x] Phase 2.1.1 — Unified path (flag on → LangChain only) and multimodal (images) support in LangChain path.
• [x] Phase 2.1.2 — File support on LangChain path (parity so native path can be deprecated when you choose; deprecation deferred).
• [x] Phase 2.1.3 — LLM transport dual path (One API vs direct provider; global flag for all chat/completion; deprecation deferred).
• [x] Phase 2.2 — Memory: custom LangChain memory backed by existing DB (AsoDbChatMessageHistory; agent gets history from memory when memoryConfig is set).
• [x] Phase 3.1 — LCEL: ASO response path runs through asoResponseChain (prompt → model → parser; aso.service passes toolCallPrompt into chain).
• [x] Phase 3.2 — LangGraph (optional): ASO response runs as asoResponseGraph (respond node wraps LCEL chain); extensible for branching.

---

Related Documentation

• CURRENT-STATE-AND-NEXT-STEPS.md — Where things stand after all phases; suggested next steps and checklist for building from here.
• Memory System — RAG and memory chunks.
• Prompt System — Prompt templates and composition.
• Tool System — Tool execution and catalog.
• 06-ARCHITECTURE/Data-Flow — High-level data flow.