Context Engineering: Why Your AI Agent Needs a Database, Not a Prompt

The "24% problem" identifies a significant performance plateau for production AI agents. Despite extensive prompt engineering—including chain-of-thought, few-shot prompting, and system prompt tuning—agents frequently default to hallucinations when handling long-horizon tasks. This failure stems from treating agent memory as a stateless prompt issue rather than a structural data management requirement.

Reliance on static prompts fails because language models cannot natively reconcile deprecated APIs, outdated internal policies, or complex multi-hop dependencies. When memory is implemented merely as a vector search index of conversation history, the agent lacks the relational structure necessary to distinguish between current state and superseded data. This lack of architectural rigor results in the following systemic weaknesses:

• Attention Dilution: In excessively long context windows, critical instructions often suffer from the "lost in the middle" phenomenon, where information positioned deep in the prompt is prioritized less effectively by the model's attention mechanism.
• Stateless Retrieval: Standard Retrieval-Augmented Generation (RAG) processes each turn in isolation, lacking the ability to accumulate knowledge or track changes to the underlying state over time.
• Structural Poverty: Naive implementations fail to differentiate between different data access patterns, attempting to resolve entity state, semantic similarity, and hierarchical knowledge through a single, inefficient lookup method.

To move beyond this ceiling, engineering teams must transition toward context engineering, which replaces flat, static prompts with a tiered data architecture. A production-grade context environment requires four distinct organization forms to maintain accuracy:

• Vector Stores: Used specifically for fuzzy recall and semantic similarity across unstructured data.
• Filesystems: Utilizing protocols for path-based navigation to mirror how human-navigated documentation is structured.
• Relational Tables: Providing strict, auditable storage for the current state of entities, such as API versioning or user permission levels.
• Knowledge Graphs: Enabling multi-hop reasoning by explicitly mapping relationships between disparate artifacts.

By implementing these forms within a three-tier loading pattern—where L0 abstracts provide orientation, L1 provides structure, and L2 supplies granular detail only upon request—engineers can mitigate the token-consumption issues inherent in long-horizon tasks while ensuring the agent possesses the most current, verified context.

AI agent performance frequently plateaus due to a reliance on monolithic, unstructured prompts rather than a disciplined engineering approach to context. To move beyond this, we must categorize the six primitives currently driving agent behavior and identify their structural limitations:

• In-Context Window: While offering low-latency access, it suffers from the "lost in the middle" phenomenon, where critical information buried in large token sequences is ignored by attention mechanisms.
• Retrieval-Augmented Generation (RAG): Current RAG implementations are stateless, leading to precision failure during multi-hop queries where relationships across disparate chunks are required.
• Web Search / Tools: These provide real-time grounding but are stateless by design; they lack historical accumulation and, if improperly sandboxed, present significant security risks.
• MCP Tools: Standardized via the Model Context Protocol, these allow structured function calling, yet they remain transactional and ephemeral.
• Short-Term Memory: A session-bound scratchpad that grows unboundedly, forcing destructive truncation that sacrifices reasoning continuity.
• Long-Term Memory: Most implementations mistakenly treat this as a simple vector search index. By relying solely on semantic similarity, they fail to provide the provenance, structure, and relational integrity required for high-horizon tasks.

The failure of these primitives in production stems from treating a database problem as a search problem. A naive vector_db implementation—which maps strings to embeddings without maintaining entity state, relational schema, or hierarchical updates—cannot support agents that require consistent, multi-turn reasoning. Structural integrity requires a transition from flat indexes to a multi-modal organization strategy incorporating vectors (semantic similarity), filesystems (hierarchical structure), tables (structured state), and knowledge graphs (relational traversal).

Engineers should move toward a tiered loading architecture. By deploying an L0 abstract summary (a "catalog card" of approximately 100 tokens) alongside L1 structured overviews, systems can provide the model with essential orientation without saturating the context window. This prevents token-driven latency and allows for context-aware, highly precise retrieval only when the agent’s specific task warrants deeper investigation.

Agent context must be organized so that each access pattern is served by the data structure that matches its semantics. The four storage forms—vector stores, hierarchical filesystems (exposed via the viking:// protocol), relational tables, and knowledge graphs—together provide fuzzy recall, deterministic lookup, stateful tracking, and multi‑hop reasoning.

• Vector Store (semantic similarity)
  • Good for: retrieving passages, code snippets, or prior conversation turns that are conceptually related to a query.
  • Bad for: exact key‑value lookups or relational joins.
  • Example: an agent asks “How did we resolve payment‑gateway timeouts last quarter?” and the vector store returns similar incident reports and resolution notes.
• Filesystem (hierarchical structure) – viking:// protocol
  • Good for: navigating large, well‑organized knowledge bases such as project documentation, source trees, or ADR collections.
  • Bad for: ad‑hoc fuzzy search across unrelated files.
  • Example path: viking://project/architecture/decisions/adr-042-database-choice.md lets the agent load the abstract summary (L0) first and fetch the full document (L2) only when needed.
• Relational/Table Store (structured state)
  • Good for: precise lookups, aggregations, and maintaining the current state of entities such as user profiles, task queues, or API response caches.
  • Bad for: unstructured text or semantic similarity.
  • Example schema:

  CREATE TABLE agent_context_entities (
      entity_id TEXT PRIMARY KEY,
      entity_type TEXT NOT NULL,
      state JSONB,
      last_updated TIMESTAMPTZ,
      session_count INT DEFAULT 0,
      confidence FLOAT
  );
  CREATE TABLE agent_context_relations (
      from_entity TEXT REFERENCES agent_context_entities(entity_id),
      relation_type TEXT,
      to_entity TEXT REFERENCES agent_context_entities(entity_id),
      evidence TEXT,
      strength FLOAT
  );

• Knowledge Graph (relational reasoning)
  • Good for: multi‑hop queries, inferring implicit connections, and traversing relationships among concepts, people, and code artifacts.
  • Bad for: large‑scale fuzzy retrieval; graph traversal can become expensive.
  • Example query: “Find all services that depend on a database that was migrated in the last release and that have open security tickets.” The graph resolves the chain of dependencies without scanning raw text.

By combining these forms, an agent can first use a vector store for quick semantic grounding, then resolve exact identifiers via the relational tables, navigate detailed documentation through the viking:// filesystem, and finally perform complex reasoning over the knowledge graph. This tiered approach satisfies the requirements of precision, provenance, and scalability that a production‑grade AI system demands.

The OpenViking architecture addresses the "attention dilution" and latency overhead associated with naive RAG implementations by implementing a hierarchical, progressive loading strategy. By decoupling context storage from immediate prompt injection, the system ensures that the model's limited context window is reserved for the most pertinent information, optimizing token consumption while maintaining the agent's reasoning capability.

The architecture organizes knowledge units—accessed via the viking:// protocol—into three distinct tiers, each serving a specific phase of the agent's decision-making process:

• L0 (Abstract Summary): A high-density, ~100-token header designed for persistent orientation. These summaries are loaded at session initiation for the entire relevant knowledge space, providing the agent with a global map of available resources without incurring significant latency. Example: viking://payments/summary might return, "Service: Payment Gateway; Status: Active; Last Audit: 2026-08-01; Pending Schema Changes: 2."
• L1 (Structured Overview): A 2,000-token intermediate layer that provides granular structural data. This tier is pulled dynamically only when the L0 assessment confirms topical relevance. It typically includes API contracts, architectural constraints, and dependency mappings, allowing the agent to perform informed planning without triggering a full retrieval of exhaustive documentation.
• L2 (Full Details): The high-fidelity source document or raw state. Access to L2 is restricted to targeted "just-in-time" retrieval. By fetching L2 only when the agent specifically requests deep inspection, the system minimizes unnecessary context bloat, significantly reducing the per-turn token spend and avoiding the performance degradation seen when saturating the context window.

This strategy effectively transforms the agent's memory from a flat search index into a structured, tiered database system. By utilizing the viking:// virtual filesystem, engineers can implement lazy loading patterns where the agent navigates through L0/L1 summaries as a navigation tree, only consuming L2 tokens for the specific, high-resolution facts required to execute complex, multi-hop reasoning tasks.

Transitioning agent memory from ephemeral, unstructured prompts to an auditable, governed data discipline represents the formalization of context engineering. This shift mirrors the evolution of Site Reliability Engineering (SRE), where previously manual, ad-hoc operational tasks were codified into measurable, automated systems. By treating context as a managed data asset rather than an amorphous prompt, engineering teams can implement rigorous observability, lineage tracking, and compliance standards.

Context engineering formalizes memory through structured storage, moving away from stateless RAG patterns. This shift is essential for meeting enterprise compliance requirements, such as those defined under SOC 2 or ISO 27001, which necessitate strict controls over data access, integrity, and provenance. Without an auditable memory layer, agents risk violating these standards by hallucinating deprecated policies or leaking unauthorized data.

To implement this governance model, enterprise systems should adopt the following strategies:

• Provenance Tracking: Every piece of retrieved context must include metadata identifying its source, timestamp, and modification history, enabling audit trails similar to database transaction logs.
• Stateful Entity Management: Move beyond simple vector search by using relational schemas to store agent state. This allows for precise, auditable updates to entity knowledge, ensuring that agents do not rely on stale or contradictory information.
• Tiered Access Control: Apply granular permissions at the context storage layer. By utilizing a tiered loading architecture (e.g., L0 for orientation, L1 for overview, L2 for deep retrieval), engineers can enforce least-privilege access, ensuring agents only retrieve context authorized for the specific session scope.
• Deterministic Versioning: Treat context as code. Use immutable snapshots of knowledge bases for specific agent tasks to ensure repeatability, allowing for debugging and forensic analysis when an agent exhibits incorrect behavior in production.

By shifting to this database-centric approach, teams gain the ability to measure "context health" using standard SRE metrics, such as retrieval precision, latency per tier, and staleness of cached information. This infrastructure transforms agent reasoning into a deterministic, verifiable process, effectively treating the information environment as a production service subject to the same oversight and reliability engineering principles applied to core infrastructure.