The New Blueprint

for AI-Ready Data:

Vector Search Meets
Distributed SQL

An Intellyx White Paper by Jason English

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 1
Approaching a fork in the road for critical data due
to AI? Maybe it’s a merge.
In the past, when a general-purpose relational SQL database wasn’t the right tool for the job, teams
turned to specialized alternatives. If you needed to ingest and analyze logs for observability, you’d
choose a time-series database. Building a catalog? You might opt for a NoSQL object store. And so on.

As cloud infrastructure (and hyperscale platforms) took off, these distinctions began to blur. New
cloud databases based on Postgres-like architectures appeared on the horizon. Distributed SQL
systems such as CockroachDB emerged to preserve transactional integrity while adding resilience
and elastic scale beyond what a single-node Postgres could handle. They were performance-tuned
for cloud native applications that need to run anywhere in the world.

Now AI-driven applications are adding a new level of complexity and urgency to the data landscape.
Users and digital agents alike demand speed, but we can’t afford to sacrifice resilience or scalability
to get it. We also don’t want to revert to less resilient, less scalable databases just to support AI.

Instead, we can keep the benefits of distributed SQL and add high-performance vector capabilities.
This presents a chance to interact with data in a completely new way: combining the strengths of
distributed SQL with vector search to power high-performance AI applications (including generative
AI, agentic AI, and AI Ops).

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 2
Why vector databases are useful for AI

AI-based applications such as large language models

(LLMs), popularized by platforms like ChatGPT and Gemini,
rely on enormous datasets to train and operate effectively. “AI systems today often need to merge vector
Each query to a chatbot interface can spark a cascade of
embeddings with transactional records. By
inferences across billions of model parameters. These
isolating vector search in its own database,
models depend on vector representations – arrays of
you push the responsibility for combining
high-dimensional floating-point values that capture
data into the application layer — adding
semantic meaning – to power everything from natural
complexity and creating potential staleness
language understanding to recommendation engines.
and consistency headaches.”
Unlike traditional datatypes such as strings, numbers,
or timestamps, vectors are fundamentally different. Peter Mattis
Co-Founder and Chief Technology Officer,
Their complexity makes them difficult to manage with Cockroach Labs
conventional relational databases – for example, they can
cause indexing performance issues at scale.

AI innovators recognized that handling vector data

This is where a new approach becomes critical: Distributed
required a different approach. As a result, we’ve seen
SQL databases architected for the cloud era can bring
the emergence of vector databases purpose-built to
transactional and vector workloads together, providing the
support similarity search and massive vector workloads.
resilience, elastic scalability, geo-locality, and operational
However, placing vector data in a silo separate from the
simplicity that modern AI applications demand.
transactional systems and operational logic of the business
has introduced new problems.

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 3
Vector search vs. traditional search

In contrast to traditional search methods that rely on exact matches, vector search finds results based on similarity. Instead
of looking for perfect keyword matches, it surfaces content that is similar to the query, delivering much more nuanced
results. This approach is essential for AI applications such as image search, document retrieval, recommendation systems,
natural language processing, and LLM integration.

For example, in text search, vector embeddings capture

the semantic meaning of words, phrases, or even entire
documents. A user searching for “classical piano music” could “Vector operations such as similarity search
also be shown results like “instrumental piano pieces” or demand that databases support vector
“baroque keyboard compositions” even if those exact phrases data types, enable indexing, and handle
don’t appear in the query. Vector search understands that dynamic updates. Indexing millions or
those terms are closely related in meaning. billions of high-dimensional vectors isn’t
trivial. Many early AI systems handled this
Behind the scenes, deep learning models encode items workload in a separate vector database,
(words, images, etc.) as vectors in a high-dimensional apart from the main transactional datastore.
space where proximity indicates similarity. This effectively But that separation came at a cost, in
transforms seemingly distinct challenges like image both complexity and performance.
recognition and semantic text search into the same core
problem: finding the nearest vectors. Accepting this tradeoff amounts to a false
choice. Why should data architects sacrifice
Storing and querying these embeddings efficiently, the resilience, consistency, and scalability of a
however, is a challenge. distributed SQL system as they move toward
AI‑native workloads? They don’t have to.”

Andy Kimball
Cockroach Labs Fellow
& Vector Indexing Team Lead

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 4
Why vector indexing is hard in a distributed environment

At first look, vector indexing in a distributed SQL  Network latency: Distributed queries involve
environment seems daunting. The process presents network hops between nodes. If a vector search
several unique hurdles that must be overcome, including: requires numerous sequential round-trips (e.g. 10–20
network calls at ~0.5–1ms each), those milliseconds
 Fully distributed indexing: In a multi-node cluster, an add up quickly. And if the process can’t be parallelized,
index can’t rely on a single “leader” node to coordinate performance becomes unpredictable.
all queries. The index itself must be fully distributed
across nodes, with no single point of control or failure. An ideal approach minimizes cross-node
Otherwise, the database would lose the resilience and communication and keeps the number of network
scalability benefits of distributed SQL. round-trips small and consistent for each query.

 Index freshness: Many indexing approaches  Index build performance: Indexes often need
assume a static dataset and struggle with to be built (or rebuilt) in the background without
ongoing updates. In reality, new vectors will be disrupting live traffic. That lower prioritization
inserted (and old ones deleted) continuously. usually means using only a fraction of available
Without a way to update the index incrementally, CPU and memory, which can stretch build times
you’d be forced into slow, resource-intensive to hours or days for very large indexes.
index rebuilds or risk degrading accuracy.
Instead, enterprises need an indexing algorithm that
Maintaining a high-quality index that can absorb builds faster and produces smaller, more efficient
constant changes – without central coordination – is indexes inline with other database processes. Faster
much harder in a distributed system. index builds mean new data can be incorporated
quickly and without lengthy downtime.
 Resource efficiency: Some vector indexing algorithms
require large in-memory data structures (potentially To address these challenges, in May 2025, Cockroach Labs
many GBs in size). In a distributed SQL setup, introduced a new indexing protocol: C-SPANN, to handle
duplicating a multi-gigabyte index on every node is large-scale vector workloads while preserving the core
wasteful, and in a serverless environment, impractical. resilience and elasticity of distributed SQL.

 Scalability: The indexing algorithm needs to

handle arbitrarily large data volumes and query
throughput. It should be disk-friendly and work
with the database’s native sharding. In practice,
huge vector indexes must partition automatically
across the cluster (aligning with data ranges)
without manual sharding or “unsplittable” chunks.

As data grows, the index should distribute

itself, and operations like splitting or merging
data ranges should happen automatically,
avoiding hot spots or bottlenecks.

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 5
How it works: tuning in with distributed vector indexing

In 2024, CockroachDB added support for multi-dimensional vectors with syntax, data types, and functions compatible with
the pgvector Postgres extension. Combined with C-SPANN, this enables storage and similarity search on vectors with
hundreds or thousands of dimensions.

“C-SPANN adapts the SPANN and SPFresh algorithms

for CockroachDB’s distributed architecture – that
enables efficient, low-latency, approximate nearest
neighbor (ANN) queries with high accuracy and fresh
results. These algorithms use partitioning strategies
inspired by K-means clustering: assigning vectors
to the nearest centroid so that vectors with similar
characteristics are grouped together.”

Dikshant Adhikari
Senior Product Manager,
Cockroach Labs

Here’s why distributed vector indexing matters for real-world  Business benefit: Faster index builds mean
AI performance and enterprise outcomes: less downtime, quicker iteration, and reduced
infrastructure costs. These are critical factors for
 Freshness: The SPFresh approach allows the index to scaling AI in production.
be updated incrementally as new vectors are added or
removed, without the need for global rebuilds.  Scale: When business data scales past what a single
 Business benefit: Ensures AI applications operate machine can handle, it can cause significant problems
on the most current data, reducing hallucinations for the enterprise. This is because there must be an
and enabling real-time personalization and decision- efficient path to logically divide (shard) the data onto
making. two separate machines.
 Business benefit: Horizontal scalability to
 Latency: Hierarchical search trees minimize the accommodate large datasets helps prevent downtime
number of network roundtrips required to serve a or complexity due to data management workloads, a
query, with predictable performance across searches. critical factor in lowering cost of ownership.
 Business benefit: Faster and more consistent
responses lead to better user experiences, higher Taken together, these capabilities allow CockroachDB
conversion rates, and more responsive automation. to support high-performance AI workloads on a
distributed SQL platform without compromising
 Build performance: Data quantization and index operational stability or inflating infrastructure costs.
partitioning reduces the number of links and memory Instead of choosing between transactional reliability,
overhead needed to build the index, enabling resilience, horizontal scalability and AI readiness,
background builds that don’t impact transactional enterprise teams can unify both under one architecture,
workloads. to accelerate innovation without sacrificing control.

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 6
Advantages of co-locating vector data on distributed SQL

While the old adage that ‘the cloud is just someone Freshness: Unlike traditional vector indexes that require
else’s computer’ still holds, performance and user full rebuilds to incorporate changes, C-SPANN supports
experience hinge on minimizing distance – physically and incremental updates through local partitioning. This
architecturally – between data and application logic. enables real-time ingestion and retrieval of the most
current data without interrupting critical services.
Freshness helps keep AI from falling behind ongoing
business activity, so product recommendations, fraud
detection systems, or customer support chatbots can stay
“Whether you’re serving users or powering up-to-date with the latest signals.
AI agents, you want the data infrastructure
as close to the workload as possible. In Efficiency: C-SPANN is engineered to run lean. By
C-SPANN, Cockroach Labs has invented a reducing memory overhead, optimizing index size, and
new way to index vector data that accelerates avoiding duplication across nodes, it enables massive
AI performance while preserving the scale without spiraling infrastructure costs. It’s also built
foundational benefits of distributed SQL. to minimize contention and recover quickly in the event of
The C-SPANN indexing framework excels regional disruptions.
at powering high-volume, high-dimensional
vector workloads.” The challenge becomes that when you put data in a separate
database to accomplish these goals of accuracy, freshness,
David Bressler and efficiency, you lose certain useful capabilities that are
Staff PMM, important to building AI applications.
Cockroach Labs

This co-location approach delivers two important advantages:

Operates in concert with structured data. The ability

to run SQL queries that filter, join, and aggregate vector
Why vector data is different
data in concert with structured data is critical to high-
Clearly, a distinct database approach is required for the performance AI applications.
unique needs of AI. Let’s recap why vectors make unique
demands on the database: For example, an ad-serving platform might search for the
most relevant ads to serve to a user, but filtered by the
Accuracy: C-SPANN leverages advanced partitioning and user’s location and also the ads that user has been served
reranking strategies that retain high recall rates while before. It’s even possible that the results of that query are
minimizing resource consumption. The result is vector then joined with other information about the user that
search accuracy that rivals leading standalone vector needs to be included in the response.
databases, without introducing complexity.
Updates are transactional. This is related to freshness,
For enterprise AI, answers not only need to be fast, but best explained with an example:
but correct to a high degree of certainty. That’s
essential as LLMs and agents will increasingly If a new catalog item is added, its metadata can be added
influence strategic and tactical decisions, to the database in the same transaction as the vector
transactions, and customer-facing workflows. that makes it available via semantic search. This is also

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 7
true for when an item is deleted. Without native access to  Ease of adoption and operation: Developers and
transactions, it’s possible for separate databases to get out data scientists can use familiar SQL to manage both
of sync, which becomes a bottleneck for applications trying transactional and vector data. There’s no need to bolt
to serve correct information to users. on specialized systems or deal with brittle pipelines.
Instead there’s just one operational model, with built-in
Co-location aligns AI innovation efforts with the needs of scale and automation.
the business for predictable performance, lower TCO, and
confidence that new AI features won’t bring down the stack.  Global consistency: Distributed SQL ensures that
AI workloads access consistent, compliant data across
geographies. That’s critical for enterprise teams
navigating data sovereignty, security, and latency-
“By embedding these indexing capabilities sensitive use cases.
directly into CockroachDB’s distributed
SQL foundation, organizations can unify  Optimized resource usage: With efficient indexing
transactional and AI workloads – without and minimal duplication, organizations can control

duct-taped architectures or compromises the memory and compute cost of AI, especially as
LLM and vector workloads increase. The result: better
to the additional resilience of a distributed
performance without runaway cloud bills.
architecture. Teams can innovate faster,
operate more efficiently, and scale AI across
use cases with a single system of record.”

Dikshant Adhikari
Senior Product Manager,
Cockroach Labs

There are several broader benefits realized by bringing

vector data into a distributed SQL database like
CockroachDB, beyond indexing performance alone:

 Resilience and scalability: CockroachDB was built

for cloud native environments, offering multi-region
availability, fault tolerance, and elastic scale. That same
architecture now powers vector search, ensuring your
AI apps perform reliably even under global load.

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 8
Vectors and distributed SQL: Convergence use cases

Combining conventional transactional data with vector  Insurance claims adjusting image search:
data is a major convergence, and it’s fast becoming the Analyzing the content of images for context is even
norm. In fact, all the major database vendors are now more challenging than generating human-like text, but
adding vector support to their offerings. It’s a clear signal adds a lot of value when considering insurance claims.
that vector search and SQL databases are converging. Similarity search is the key: This article demonstrates
how it works by extracting image features with a
convolutional neural network (CNN) and enriching
them with time-series data, LLM-generated text, audio
signals, and more.
“Rather than shuffling data between separate
systems, it’s far more intuitive to bring vector
 Banking (RAG) chatbot: An AI-powered banking
indexing into the distributed SQL database
chatbot isn’t very useful if it can’t securely access live
you already trust. With a unified architecture,
data, like customers’ account balances. A chatbot
you eliminate the headaches of duplicating
can integrate retrieval-augmented generation (RAG)
data and keeping it in sync across silos. There’s
techniques by indexing vectors alongside core banking
no more worrying about data movement,
data without compromising the primary application’s
consistency, protection, compliance, security,
resilience or scalability, or shipping any private data off
access control, and all the other complexity
to another vector resource for AI training and inference
that comes from coordinating multiple
workloads.
platforms. One platform means a simpler,
more secure and efficient data landscape.”

Andy Kimball
Cockroach Labs Fellow
& vector indexing team lead

If you’re looking to explore this approach in action, here

are three application areas to consider:

 Retail recommendations: See how vector search

and e-commerce data combine to deliver tailored
product recommendations for a fashion retailer in a
demo video. This example shows AI rapidly generating
dynamic, personalized results for distinct customer
segments.

The New Blueprint for AI-Ready Data: Vector Search Meets Distributed SQL 9
Where to learn more
To continue learning about distributed SQL and vector
search, visit the Cockroach Labs website for in-depth About the Author
documentation, blog articles, and demos. You can also Jason “JE” English is
explore CockroachDB’s tutorials or get in touch with Director & Principal Analyst
Cockroach Labs to discuss your specific use case. at Intellyx. Drawing on
expertise in designing,
The Intellyx Take
marketing and selling
You no longer have to make an either-or choice between
enterprise software and services, he is
databases when selecting a data architecture to support AI
versus transactional workloads. When you’re ready to get focused on covering how agile collaboration

serious about applying AI in your business, having vector between customers, partners and
data right alongside your critical operational data offers a employees accelerates innovation.
natural approach. A writer and community builder with more
than 25 years of experience in software dev/
CockroachDB brings forward all the benefits of distributed
test, cloud and supply chain companies, JE
SQL with mission-critical transactional readiness, while
led marketing efforts for the development,
addressing the shortcomings that once made distributed
SQL incompatible with vector data used by AI. testing and virtualization software company
ITKO from its bootstrap startup days, through
There’s no need to search far and wide for a different data a successful acquisition by CA in 2011.
store for each new AI project, when you can do it all on a Follow him on LinkedIn at /jasonenglish.
single platform — with the resilience, scalability, geo-locality,
and ease of operations modern enterprises demand.

About

Intellyx is the first and only industry analysis, advisory, and training firm focused on customer-driven, technology-empowered
digital transformation for the enterprise. Covering every angle of enterprise IT from mainframes to cloud, process
automation to artificial intelligence, our broad focus across technologies allows business executives and IT professionals to
connect the dots on disruptive trends. Read and learn more at [Link] or follow them on LinkedIn.

About

Cockroach Labs is a pioneering software company at the forefront of database technology, dedicated to delivering resilient
and scalable database solutions to run mission-critical workloads for the world’s most important businesses. The company’s
clients include Form3, Hard Rock Digital, and Shipt, Fortune 50 global financial institutions as well as retail and media industry
leaders. With a mission to scale when others fail, Cockroach Labs is revolutionizing the way businesses manage their data
with its innovative cloud native distributed SQL database, CockroachDB.

©2025 Intellyx B.V. Intellyx is editorially responsible for this content. No AI was used to write this document.
At the time of writing, Cockroach Labs is an Intellyx customer. Image credits: Cockroach Labs.