The Disconnected Vector Store — Why AI Storage Needs to Move Inside the Boundary

The previous article challenged the assumption that binary content belongs in object storage, and argued that the governance gap it creates becomes a security vulnerability when agents are the callers. This article examines the same problem in the infrastructure built specifically to serve those agents: the vector store and the agent memory store.

The pattern has become so common it barely registers as a decision. An organisation builds a RAG pipeline: chunk the documents, generate embeddings, store them in a vector database, retrieve by similarity at query time. An agent framework is added: conversation history goes to Redis, long-term memories to Pinecone or Chroma, retrieved context assembled at prompt time from multiple sources. The operational database — where the organisation’s actual data lives, with its access controls, its audit trail, its transactional integrity — sits to one side, connected to this machinery by application code that nobody has formally designated as an integration layer.

The vector store and the agent memory store are disconnected. They were deployed by default, not by design. And the consequences of that disconnection — from the transactional boundary, from the governance model, from the access policy — are what this article examines.

A Pattern That Normalised Before It Was Examined

There is a precise parallel with the saga pattern discussed in an earlier article. Sagas were a rational engineering response to a real problem — enforcing consistency across distributed microservices when a single transaction boundary was no longer available. They were sound in context, adopted widely, and became normalised to the point where the conditions that justified them stopped being examined in each new case.

The disconnected vector store followed the same trajectory, but faster. When organisations began building RAG pipelines in earnest, native vector support was either absent or immature in most general-purpose databases. The specialist vector databases — Pinecone, Weaviate, Chroma, Qdrant — filled a genuine gap. The workaround made sense. What did not follow was the re-examination of that choice as the gap closed. Native vector support has since arrived in PostgreSQL via pgvector, in Oracle via AI Vector Search, and in varying forms across most major database platforms. The external vector store remained anyway, because it was already there, the pipelines were built around it, and inertia is a more powerful architectural force than most engineering teams acknowledge.

The result is an infrastructure pattern adopted for reasons that no longer fully apply, carrying costs that were not part of the original calculus.

The Security Problem

The previous article established that object storage lacks fine-grained access control, and that an agent with access to an object store bucket can read everything in it — constrained only by coarse bucket-level policy. The disconnected vector store has exactly the same problem, applied to the data that agents use most directly: the context they reason on.

A vector store is a copy of data in a different form — embeddings derived from documents, records, and content that lives in the operational database. The access controls defined in that database — which identities can see which records, what is masked, what is subject to retention policy — do not automatically extend to the vector store. The embedding of a document that a given agent identity should not be able to retrieve may nonetheless be retrievable through semantic similarity search, because the vector store’s access model is less granular, or enforced against a different permissions model entirely.

This is not a theoretical exposure. A RAG pipeline that retrieves context from a disconnected vector store will return results based on semantic similarity alone. It has no awareness of the row-level security policies, data classifications, or customer-level access restrictions that the operational database enforces. An agent reasoning on that context is reasoning on data it may not have been entitled to see — and acting on it, writing results derived from it, propagating that entitlement violation downstream without any audit trail that would make the violation visible.

The governance gap that the previous article identified in object storage is, if anything, more consequential here. Object storage holds the assets. The vector store holds the meaning extracted from those assets — the semantic representation that agents use to reason. Uncontrolled access to meaning is at least as dangerous as uncontrolled access to the raw content it was derived from.

The Cost of Disconnection

Beyond security, the costs of disconnection fall into three further categories: consistency, operational overhead, and agent memory.

There is a pre-AI version of this problem that I have seen in CRM systems. A telecoms call-centre application had a recommendation engine designed to prompt agents with upsell offers. The recommendations were driven from a cache that refreshed daily.

That sounded reasonable until a customer placed an order and then called back the same day, perhaps to change something or ask a follow-up question. The operational CRM knew the order had been placed. The recommendation cache did not. So the call-centre agent was prompted to offer the same product again to someone who had just bought it.

This was not catastrophic. No database was corrupted, and no system went down. But it was a very poor customer experience, because the company appeared not to know what it had just sold. The problem was not the recommendation engine in isolation. It was the gap between operational truth and derived context.

Consistency. An embedding is derived from data. When the data changes, the embedding is stale until the pipeline that regenerates it runs again. In a synchronous application workflow this lag is manageable — the window is short, the consequences are bounded, and the retrieval results are understood to reflect a near-current state. In an agent workflow the consequences are neither bounded nor understood. An agent querying a disconnected vector store for context will retrieve whatever the index contains, without any indication of whether that content reflects the current state of the source data or a state from days ago before a significant update. The embedding may surface a document that has since been revised, superseded, or deleted from the authoritative store. The agent reasons on that context and acts. The action is based on information the database no longer considers true.

This is not a theoretical edge case. It is the default behaviour of any architecture where the vector store is not updated in lockstep with the database — which is to say, almost every architecture where they are separate systems.

Operational overhead. A disconnected vector store is a separate system to operate: separate index refresh schedules, separate monitoring, separate backup, separate restore, separate access policy management. Each of these is a place where things can fail, diverge, or be misconfigured independently of the operational database. The backup consistency problem identified in the previous article applies here too — a database restore and a vector index restore are separate operations with no native coordination, leaving the possibility of agents retrieving context from an index that no longer corresponds to the state of the data it was built from.

Agent memory. The problem compounds when agent memory is also externalised. An agent that stores its beliefs, its working context, and its understanding of prior interactions in a separate memory store — Redis, a document database, a specialist agent memory platform — is building a model of the world that can diverge from the transactional reality of the database it acts against. It may remember that an order was placed that was subsequently cancelled. It may carry forward a belief about a customer’s status that the database has since updated. It will act on those beliefs with confidence, because nothing in its architecture tells it that its memory and the database are out of sync.

Where the Source Data Lives

The previous article challenged the assumption that binary content belongs in object storage, and established that much of the content organisations externalise — text fields, JSON, incidental images, attachments, documents — belongs in the database. This has a direct implication for the vector store argument.

Where the source data lives in the database, the question of where to store the embedding is straightforward: the database. The embedding is derived from data that is already subject to the database’s access controls, transactional guarantees, and retention policies. Keeping the embedding alongside the source data is not an architectural preference — it is the natural consequence of keeping data together. The consistency problem largely disappears: when the source record changes, the embedding can be regenerated within the same transaction. The security problem disappears: the same access policy that governs the source data governs the embedding.

Where the source data is genuinely external — binary content that belongs in object storage for legitimate scale or access pattern reasons — the embedding still belongs in the database. The embedding is not the asset. It is a fixed-size numerical representation that participates in retrieval operations and carries metadata about its provenance. It belongs inside the governance boundary regardless of where the asset it was derived from lives.

The architectural principle is therefore consistent: embeddings belong in the database. The question of where the source content lives is a separate question, addressed in the previous article. The answer to that question does not change the answer to this one.

The Scale Objection, Honestly Assessed

The standard argument for the external vector store is scale. Specialist vector databases are optimised for approximate nearest-neighbour search at high dimensionality and high volume. General-purpose databases were not designed for this workload, and at sufficient scale the performance characteristics diverge.

This is true. It is also, in most enterprise deployments, less relevant than it is presented to be. The scale at which the performance gap between an in-database vector index and a specialist vector database becomes operationally significant is higher than most organisations are operating at. The benchmarks cited in favour of specialist vector databases tend to reflect workloads at the upper end of what enterprises encounter — and even there, the gap is closing as in-database implementations mature.

The more important point is about architectural default. The question is not whether there exist workloads at sufficient scale to justify a specialist vector database. There clearly are. The question is whether that justification should be the starting assumption or the conclusion of an analysis. Currently it tends to be the starting assumption, applied to workloads that have not been evaluated against it. The external store is deployed first; the scale case is made afterwards, if it is made at all.

Native Vector Support: Where Things Stand

pgvector brought vector search to PostgreSQL and has seen rapid adoption. It is a genuine capability, though with limitations at scale and in query sophistication that more recent implementations have addressed. Oracle AI Vector Search, now generally available, integrates vector storage and similarity search directly into the Oracle database engine — which means vector queries can be combined with relational filters, subject to Oracle’s access control model, and executed within the same transaction boundary as the rest of the database workload. Oracle AI Agent Memory takes the same approach to agent memory specifically: storing agent beliefs and working context inside the database rather than in an external store, with the consistency and governance properties that entails.

These are not arguments for Oracle specifically. They are illustrations of a direction the market is moving in — one that closes the gap that disconnected external stores were filling. The relevant question for any architecture is whether the specialist external store is still justified given what in-database alternatives now offer, or whether it is a decision made when those alternatives did not exist and not revisited since.

The Default Trap

The deepest problem with the disconnected vector store is not technical. It is that it becomes permanent not because it is right but because it is there.

Infrastructure decisions that ship as defaults acquire institutional inertia at speed. The pipelines are built around them. The operational processes — index refresh schedules, monitoring, backup — are built around them. The teams are organised around maintaining them. Questioning the decision requires unpicking all of that, and the cost of unpicking is always visible in a way that the cost of staying put is not — until the security incident, the consistency failure, or the audit finding makes the invisible cost suddenly very legible.

This is precisely what happened with sagas and eventual consistency, and with the object storage default examined in the previous article. The pattern made sense when it was introduced. It became the default. The conditions under which it made sense stopped being examined.

Disconnected vector stores are earlier in the same trajectory. The moment to examine the decision is now, before the infrastructure calcifies further and before the agent workloads that depend on it scale to the point where the governance and consistency gaps become production problems rather than architectural concerns.

What This Means for Architecture

The disconnected vector store is not an inevitable feature of AI architecture. It is a default that emerged when the alternatives did not exist, and it has persisted through inertia rather than considered design.

The series has now traced four versions of the same underlying failure: transactional integrity pushed out of the database into saga-managed application code; binary content pushed out of the database into object storage without adequate governance; business rules pushed out of the database into application-layer validation; and AI storage deployed outside the database entirely. In each case the drift was rational at the moment it happened. In each case the agent era makes the cost of that drift considerably more visible.

The architectural principle is consistent across all four: integrity, governance, and state need to live where the caller cannot bypass them and where the transactional boundary can hold. For AI storage — embeddings, agent memory, retrieved context — that means inside the database, unless there is a specific, examined, and scale-justified reason why it cannot be.

That completes the diagnosis. Across this series one failure has recurred in every form: integrity, governance and state pushed out of the database into layers an agent can bypass. What follows is the answer. The next article turns from what agents break to what the architecture must become — and it begins where the exposure is sharpest, the write path. It is the first article of Designing the Stack, the point where this series’ diagnosis becomes a design: the database stops trusting the caller and starts owning its own contract.