Simon Griffiths

I’ve been looking at some of the recent AI architecture models coming out of NVIDIA. They’re polished, coherent, and very strong on infrastructure. But something stood out immediately: they largely skip over the hardest part of enterprise AI — the data.

At first glance, that feels like a major omission. In most organisations I’ve worked with, AI isn’t constrained by compute first. It’s constrained by data: where it lives, how it’s governed, and how hard it is to move.

But the more I looked at it, the more it became clear — this isn’t a mistake. It’s a boundary.

The Centre of Gravity: Compute, Not Data

NVIDIA’s worldview is built around accelerated compute: GPUs, high-speed interconnects, memory bandwidth, and the runtimes that drive training and inference. Even their higher-level constructs — “AI factories”, reference architectures — are fundamentally compute-centric.

Running through all of it is an implicit assumption: that the data is already available, already governed, and can be moved to where compute happens. That assumption holds up in certain scenarios — model training pipelines with curated datasets, greenfield AI applications, smaller-scale or departmental use cases. It starts to fall apart in large enterprises, because in most real environments none of those assumptions are reliably true.

Enterprise AI Is a Data Gravity Problem

The constraint in most large organisations isn’t compute — it’s data. Data is distributed across dozens of systems, governed by overlapping policies and regulations, expensive and slow to move, and tightly coupled to the business meaning held by the systems that produced it. The real question for enterprise AI isn’t how to run inference efficiently. It’s how to bring inference to data — or data to inference — without breaking governance, cost models, or operational stability.

Typical AI reference diagrams don’t reflect this. They tend to show a clean linear pipeline: data → embeddings → inference → output. The work that sits before that pipeline even starts — discovering where the data actually is, understanding its structure and semantics, enforcing access controls, transforming and normalising across systems, maintaining lineage — is the bulk of the effort in most enterprises. It’s exactly the part the diagrams don’t show.

The reason this matters is that NVIDIA’s reference architectures rest on an assumption: that a clean, governed data plane exists, ready to be consumed. In most large organisations it doesn’t. What exists is multiple overlapping data platforms, inconsistent metadata and lineage, brittle pipelines, and partial or fragmented governance. When the architecture says “data is ready”, the reality is that getting data ready is the majority of the problem.

Why NVIDIA Stays Out of It

This isn’t an oversight. It’s a strategic decision.

If NVIDIA tried to “solve” the data layer, they would immediately step into territory owned by database platforms like Oracle, cloud data platforms like Snowflake, and the wider ecosystem of governance, cataloguing, and integration tools. That’s not an extension of their stack — it’s a completely different control plane.

Staying focused on compute gives them three clear advantages: neutrality across clouds and data platforms, a clean and widely adoptable narrative, and focus on the layer where they have unambiguous technical dominance. So instead of trying to solve the data layer, they assume one exists and optimise everything beneath it.

Two Architectural Directions Emerging

What’s starting to emerge is a split in how AI systems are being designed.

The first direction — compute-first AI, aligned with NVIDIA — moves data toward models, optimises for throughput and performance, and works well for training and greenfield systems. The second — data-first AI, which is the more common enterprise reality — moves models toward governed data, embeds AI inside existing data platforms, and prioritises security, lineage, and integration. You can see this second approach in how platforms like Oracle and Snowflake are evolving, pushing AI capabilities directly into the data layer rather than expecting data to leave it.

The Real Problem We Haven’t Solved Yet

The gap between these two worlds is where things get interesting. The questions still open are architectural rather than computational: how to run inference inside governed data environments, how to avoid copying sensitive data into GPU pipelines, how to unify structured, unstructured, and operational data in real time, and how to enforce fine-grained security during AI execution.

This isn’t primarily a compute challenge. It’s a data architecture problem — with AI embedded into it.

Final Thought

NVIDIA hasn’t ignored the data problem. They’ve simply drawn a line around it, and that’s reasonable — it’s not their domain. But for anyone working in enterprise systems, that boundary is exactly where the real complexity begins.

The next phase of AI won’t be defined just by faster GPUs. It will be defined by how well we close the gap between data governance and inference execution. That’s the part that still isn’t cleanly solved.