Simplifying Data Integration: SLM-Driven Systems

for Unified Semantic Queries Across Heterogeneous

tlin280@[Link]

Abstract—The integration of heterogeneous databases into a This paper addresses three critical gaps in existing method-
unified querying framework remains a critical challenge, partic- ologies:
ularly in resource-constrained environments. This paper presents
a novel Small Language Model (SLM)-driven system that syner- • Efficiency vs. Accuracy Trade-offs: State-of-the-art
gizes advancements in lightweight Retrieval-Augmented Genera- Retrieval-Augmented Generation (RAG) pipelines, such
tion (RAG) and semantic-aware data structuring to enable effi- as those employed in systems like EVAPORATE [6],
cient, accurate, and scalable query resolution across diverse data
formats. By integrating semantic-aware heterogeneous graph often involve multi-stage processes including dense vec-
indexing and topology-enhanced retrieval with SLM-powered tor retrieval, reranking, and context augmentation. While
structured data extraction, our system addresses the limitations effective, these pipelines incur significant computational
of traditional methods in handling Multi-Entity Question An- overhead due to repeated LLM inference passes and
swering (Multi-Entity QA) and complex semantic queries. The large-scale vector indexing [7]. For instance, processing a
introduction of semantic entropy as an unsupervised evaluation
metric provides robust insights into model uncertainty. Together, terabyte-scale data lake with a conventional RAG system
these innovations establish a domain-agnostic, resource-efficient may require hundreds of GPU hours, limiting real-time
paradigm for executing complex queries across structured, semi- applicability.
structured, and unstructured data sources, aiming at foun- • Multi-Entity QA Limitations: Existing systems struggle
dational advancement for next-generation intelligent database to resolve queries that span multiple entities across struc-
systems.
Index Terms—Small-scale Language Models, Heterogeneous tured and unstructured data [8]. Consider a query such
Graph Indexing, Semantic Entropy, Multi-Entity QA. as, “Compare the efficacy of Drug A (from clinical trial
tables) with patient-reported side effects (from unstruc-
I. I NTRODUCTION tured forums).” Traditional Text-to-SQL engines fail to
parse the unstructured component [9], while LLM-based
The rapid proliferation of heterogeneous databases, en-
QA systems often hallucinate plausible but ungrounded
compassing structured relational tables (e.g., SQL databases),
comparisons due to missing cross-modal context.
unstructured text (e.g., clinical notes, customer reviews), and
• Evaluation Uncertainty: Conventional metrics like
semi-structured formats (e.g., JSON logs, XML configura-
BLEU, ROUGE, and exact-match F1 scores provide lim-
tions), has created a pressing need for systems capable of
ited insight into a model’s confidence or semantic consis-
executing unified semantic queries across these disparate
tency, particularly for open-ended queries. This shortcom-
modalities [1] [2]. Such systems must balance computational
ing is exacerbated in unsupervised settings where labeled
efficiency with analytical precision, particularly in resource-
validation data is scarce, such as in legal document
constrained environments such as edge computing or real-
analysis or industrial maintenance logs.
time business intelligence platforms. Traditional approaches to
this challenge, which often rely on Large Language Models To bridge these gaps, we propose a lightweight architecture
(LLMs) for semantic parsing or manual schema alignment by that synergizes innovations from two recent advancements:
domain experts, face fundamental limitations [3] [4]. LLM- MiniRAG [10], a resource-efficient RAG framework optimized
based methods, while powerful, demand substantial computa- for Small Language Models (SLMs), and SLM-driven struc-
tional resources for inference and fine-tuning, rendering them tured data extraction techniques. Our system employs hetero-
impractical for applications requiring low-latency responses geneous graph indexing mechanism that unifies text chunks,
or deployment on devices with limited memory (e.g., smart- named entities, and latent relational cues (e.g., temporal or
phones or IoT sensors) [5]. Manual schema alignment, on the spatial dependencies) into a single topological structure. For
other hand, is labor-intensive, error-prone, and inherently un- example, in a healthcare dataset, nodes might represent patient
scalable in dynamic environments where data schemas evolve IDs (structured), medication mentions in clinical notes (un-
frequently, such as in healthcare electronic health records structured), and lab result timestamps (semi-structured), with
(EHRs) or e-commerce product catalogs. edges encoding relationships such as “Patient X received Drug
Y on Date Z.” This graph-based approach reduces reliance • Relational Cues: Inferred entity relationships (e.g., “Cus-
on computationally expensive dense retrieval by leveraging tomer X purchased Product Y”).
sparse, topology-guided traversal (e.g., breadth-first search This graph reduces reliance on complex semantic parsing by
from anchor entities) to identify relevant context. encoding hierarchical and topological relationships, enabling
II. R ELATED W ORK efficient knowledge discovery through graph traversal.
A. Retrieval-Augmented Generation B. Topology-Enhanced Retrieval
RAG systems have emerged as a promising approach to Our system utilizes graph properties, including centrality
enhance language-based querying [11] [12] [13]. However, and connectivity, to efficiently prioritize nodes and edges that
deploying Small Language Models (SLMs) in existing RAG are most relevant to a given query. Centrality measures help
frameworks faces challenges due to SLMs’ limited semantic identify influential nodes, while connectivity ensures robust
understanding and text processing capabilities. MiniRAG [10], traversal across the graph, facilitating comprehensive data
for example, addresses these issues with a semantic-aware integration. For instance, when responding to a query such
heterogeneous graph indexing mechanism and a lightweight as “Compare sales trends for Products A and B in Q2”, the
topology-enhanced retrieval approach. This mechanism com- system dynamically assesses and connects nodes representing
bines text chunks and named entities in a unified structure, the sales data of Products A and B, as well as any associated
reducing the need for complex semantic understanding during temporal or channel-related nodes. This approach not only
retrieval. optimizes graph traversal but also enhances query precision
by focusing on the most pertinent data, thereby reducing
B. Semantic Operators and Querying computational overhead and improving response times.
The concept of semantic operators extends the relational
model to perform semantic queries over datasets [14]. These C. SLM-Driven Structured Data Extraction
operators enable operations like sorting or aggregating records For unstructured documents, the Small Language Model
using natural language criteria, providing a more intuitive (SLM) undertakes two pivotal and distinct tasks that are
way to query data. Existing works have demonstrated the fundamental to enabling advanced data processing and query-
effectiveness of semantic operators in various applications, answering capabilities within the proposed system:
such as fact-checking, extreme multi-label classification, and • Relational Table Generation: The first task, Relational
search. Table Generation, is a crucial step in transforming the
unstructured nature of free-text data into a more organized
C. Measuring Uncertainty in Language Models
and analyzable format. In a real-world business scenario,
Measuring the uncertainty in language models, especially consider a sales report in free-text form such as “Q2
in question-answering tasks, is crucial for determining the sales increased 20%”. The SLM uses a combination of
reliability of their outputs. Semantic entropy [15] offers an natural language processing techniques, including part-
unsupervised method to measure this uncertainty, taking into of-speech tagging and named-entity recognition (NER).
account the semantic equivalence of different sentences. It has For instance, it first identifies the relevant entities in the
been shown to be more predictive of model accuracy compared sentence, like “Q2” as a time-related entity and “sales”
to traditional baselines. as a business-related entity, and the numerical value
III. P ROPOSED S YSTEM A RCHITECTURE “20%” as a measure of change. By leveraging pre-trained
language models and semantic analysis algorithms, the
A. Semantic-Aware Heterogeneous Graph Indexing
SLM can then convert this free-text into a structured table.
Semantic-Aware Heterogeneous Graph Indexing is an inno- The table might have columns such as “Quarter”, “Sales
vative approach designed to address the challenges of inte- Metrics”, and “Change Percentage”, with the correspond-
grating and querying diverse data formats by constructing a ing values “Q2”, “Sales”, and “20%” populated in the
unified graph structure. Drawing inspiration from the Mini- rows. This structured representation allows for easier data
RAG framework, this methodology interlinks three primary comparison, aggregation, and further analysis.
components: text chunks, named entities, and relational cues. • Semantic Operator Synthesis: The second task, Se-
Text chunks are the foundational segments derived from raw mantic Operator Synthesis, focuses on the translation
documents, serving as the basic nodes within the graph. These of natural language queries into executable operations.
segments are crucial for maintaining the contextual integrity When a user poses a natural language query, the SLM
of the data. Named entities, on the other hand, are identified needs to understand the semantic meaning behind the
through a lightweight tagging process utilizing Small Lan- words and map them to appropriate operations in a query-
guage Models (SLMs). Inspired by MiniRAG, we construct processing system. For example, if the query is “Find
a unified graph structure that interlinks: the total sales of all products in Q3”, the SLM uses
• Text Chunks: Raw document segments. semantic parsing algorithms to break down the query.
• Named Entities: Extracted via lightweight SLM-based It identifies the entities “total sales”, “all products”, and
tagging. “Q3”. Then, it maps these to SQL-like operations such
 as aggregations (e.g., SUM for calculating the total sales) instance, in a medical context, if an SLM consistently answers
and filtering operations (to select data related to Q3). “What are common influenza symptoms?” with responses like
Operations like SQL joins can also be synthesized when “Fever, cough, fatigue” and “Symptoms include sore throat
the query requires combining data from multiple tables. and body aches”, semantic analysis (e.g., clustering via embed-
For instance, if the data is stored in a product table and dings like BERT) groups these into a single semantic cluster.
a sales table, and the query is to find the sales of specific Low entropy arises because all answers align with the same
products, the SLM can generate a join operation to link core meaning (influenza symptoms), indicating high reliability.
the two tables based on a common key, such as product This consistency reflects the model’s confidence and reduces
ID. ambiguity, making it trustworthy for critical domains like
• Enabling Complex Multi-Entity QA through Hybrid healthcare. Semantic entropy thus evaluates how reproducibly
Pipelines. The combination of these two capabilities, the SLM conveys factual or domain-specific knowledge, rather
Relational Table Generation and Semantic Operator Syn- than judging the specificity of a single answer.
thesis, empowers the system to handle complex Multi- Conversely, high semantic entropy reveals uncertainty or
Entity Question Answering (Multi-Entity QA) through inconsistency in the SLM’s outputs, often due to conflicting
hybrid pipelines. Starting with unstructured data, the training data or ambiguous queries. For example, when asked,
SLM first converts it into structured tables through Re- “Can I be sued for sharing a photo on social media?” an
lational Table Generation. These generated tables then SLM might generate divergent responses like “Yes, if copy-
serve as the input for TableQA engines. For example, righted”, “No, unless consent is violated”, or “It depends on
in a large-scale e-commerce data lake with unstructured jurisdiction”. These answers form multiple semantic clusters
customer reviews, product descriptions, and sales records, (e.g., ”yes,” ”no,” ”conditional”), leading to high entropy.
the SLM can transform relevant unstructured data into This signals unreliability, as the model fails to converge on
tables. When a complex query like “Compare the average a coherent answer, exposing gaps in its training or the query’s
customer satisfaction ratings of products from different inherent complexity. High entropy prompts systems to flag
manufacturers that had a sales increase of more than 15% such outputs for human review or model retraining, ensuring
in the last quarter” is posed, the SLM-generated tables users receive actionable insights in domains like law, where
are used by the TableQA engine. The engine can then ambiguity carries significant risk. Crucially, semantic entropy
utilize the semantic operators synthesized by the SLM to focuses on variability across responses, not the vagueness of
perform operations like filtering the sales data for the last a single reply, making it a robust metric for evaluating model
quarter and products with a sales increase over 15%, and confidence and contextual understanding.
then joining the relevant tables to calculate and compare
the average customer satisfaction ratings. This end-to- IV. C ONCLUSION AND F UTURE W ORK
end process showcases how the dual-task capabilities of
Our proposed SLM-driven system for unified semantic
the SLM enable the handling of complex queries across
queries across heterogeneous databases simplifies data inte-
diverse data sources.
gration and demonstrates significant performance advantages
D. Semantic Entropy for Uncertainty Quantification in complex query scenarios. By integrating multiple advanced
techniques, we have opened up a new technical path for the
Semantic entropy, a concept rooted in information theory,
development of database query systems. For future work, we
is integrated into the framework to quantify the semantic
plan to further optimize the retrieval mechanism to handle
consistency of a Small Language Model’s (SLM) responses in
even larger and more diverse datasets. Additionally, we aim
question-answering tasks. Unlike traditional accuracy metrics,
to explore the integration of more advanced language model
which rely on predefined ground truths, semantic entropy ad-
architectures into the system to further enhance its semantic
dresses a core challenge in natural language processing (NLP):
understanding and query answering capabilities. We also in-
evaluating answer quality for open-ended questions where
tend to expand the application domains of the system, such as
unambiguous “correct” answers may not exist. By analyzing
applying it to real-time data analytics and knowledge database
the variability in meaning across multiple generated responses
construction.
(e.g., clustering answers by semantic similarity), this metric
captures the model’s uncertainty and reliability. For instance,
R EFERENCES
in subjective or context-dependent scenarios (e.g., legal advice
or creative writing), semantic entropy reveals whether the SLM [1] G. Trappolini, A. Santilli, E. Rodolà, A. Halevy, and
produces coherent, stable answers or diverges into conflicting F. Silvestri, “Multimodal neural databases,” in Proceed-
interpretations—a critical measure of trustworthiness in real- ings of the 46th International ACM SIGIR Conference
world applications. on Research and Development in Information Retrieval,
1) Measuring Uncertainty in SLM-Generated Answers: Se- 2023, pp. 2619–2628.
mantic entropy quantifies the reliability of answers generated [2] M. Urban and C. Binnig, “Caesura: Language mod-
by a Small Language Model (SLM) by measuring consistency els as multi-modal query planners,” arXiv preprint
in meaning across multiple responses to the same input. For arXiv:2308.03424, 2023.
 [3] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai,
J. Sun, M. Wang, and H. Wang, “Retrieval-augmented
generation for large language models: A survey,” 2024.
[Online]. Available: [Link]
[4] S. Pan, Y. Zheng, and Y. Liu, “Integrating graphs with
large language models: Methods and prospects,” IEEE
Intelligent Systems, vol. 39, no. 1, pp. 64–68, 2024.
[5] Z. Liu, C. Zhao, F. Iandola, C. Lai, Y. Tian,
I. Fedorov, Y. Xiong, E. Chang, Y. Shi,
R. Krishnamoorthi, L. Lai, and V. Chandra, “Mobilellm:
Optimizing sub-billion parameter language models
for on-device use cases,” 2024. [Online]. Available:
[Link]
[6] S. Arora, B. Yang, S. Eyuboglu, A. Narayan, A. Hojel,
I. Trummer, and C. Ré, “Language models enable
simple systems for generating structured views of
heterogeneous data lakes,” 2025. [Online]. Available:
[Link]
[7] C. Liu, M. Russo, M. Cafarella, L. Cao, P. B. Chen,
Z. Chen, M. Franklin, T. Kraska, S. Madden, and
G. Vitagliano, “A declarative system for optimizing ai
workloads,” arXiv preprint arXiv:2405.14696, 2024.
[8] T. Lin, “Mebench: Benchmarking large lan-
guage models for cross-document multi-entity
question answering,” 2025. [Online]. Available:
[Link]
[9] X. Liu, S. Shen, B. Li, P. Ma, R. Jiang, Y. Luo, Y. Zhang,
J. Fan, G. Li, and N. Tang, “A survey of nl2sql with
large language models: Where are we, and where are we
going?” arXiv preprint arXiv:2408.05109, 2024.
[10] T. Fan, J. Wang, X. Ren, and C. Huang,
“Minirag: Towards extremely simple retrieval-
augmented generation,” 2025. [Online]. Available:
[Link]
[11] W. Fan, Y. Ding, L. Ning, S. Wang, H. Li, D. Yin, T. S.
Chua, and Q. Li, “A survey on rag meeting llms: Towards
retrieval-augmented large language models,” 2024.
[12] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin,
N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel
et al., “Retrieval-augmented generation for knowledge-
intensive nlp tasks,” Advances in Neural Information
Processing Systems, vol. 33, pp. 9459–9474, 2020.
[13] X. He, Y. Tian, Y. Sun, N. V. Chawla, T. Laurent,
Y. LeCun, X. Bresson, and B. Hooi, “G-retriever:
Retrieval-augmented generation for textual graph un-
derstanding and question answering,” arXiv preprint
arXiv:2402.07630, 2024.
[14] L. Patel, S. Jha, C. Guestrin, and M. Zaharia, “Lo-
tus: Enabling semantic queries with llms over tables
of unstructured and structured data,” arXiv preprint
arXiv:2407.11418, 2024.
[15] L. Kuhn, Y. Gal, and S. Farquhar, “Semantic uncertainty:
Linguistic invariances for uncertainty estimation in
natural language generation,” 2023. [Online]. Available:
[Link]
This figure "[Link]" is available in "png" format from:

[Link]