Paper Summary: Survey & Landscape
Topic: Large Language Model
Week 1-2: Research Foundation
Summary by: Lê Văn Pháp, Nguyễn Diệu Hương, Nguyễn Nhựt An, Hà Hữu Phát, Bùi
Hải Yến (Team RESE006)
Summary of Key Literature and Surveys
on LLMs and Augmented Language Models
Citation
Publication
Paper Title Count Key Taxonomy
Year/Venue
(Approx)
Retrieval-Augmented LLM:
Retrieval-Augmented Retrieval module, Generation
Generation for Large Language 2023 / arXiv 3028 module, Reranking - post
Models: A Survey processing, Retrieval methods
(Sparse, Dense, Hybrid)
Adaptation Stages: Pre-
A Comprehensive Overview of
2024 / arXiv 2654 training, Fine-tuning,
Large Language Models
Prompting/Utilization
Model Compression:
A Survey on Model
Quantization, Pruning,
Compression for Large 2024/ arXiv 639
Knowledge Distillation, Low-
Language Models
Rank Factorization
Efficient Large Language Efficient LLMs: Model-Centric
2024 / arXiv 343
Models: A Survey vs. Data-Centric
From System 1 to System 2: A
2026 / IEEE Reasoning in LLM: System 1 vs.
Survey of Reasoning Large 286
TPAMI System 2
Language Models
Agentic RAG: Reasoning-
Towards Agentic RAG with
Enhanced RAG, RAG-
Deep Reasoning: A Survey of
2025 / EMNLP 50–80 Enhanced
RAG-Reasoning Systems in
Reasoning, Synergized RAG-
LLMs
Reasoning
Reasoning RAG via System 1
or System 2: A Survey on
Predefined Reasoning, Agentic
Reasoning Agentic Retrieval- 2025 18
Reasoning
Augmented Generation for
Industry Challenges
1. The Taxonomy & Classification
Instead of adopting a static hierarchical structure, recent studies propose a multi-dimensional
taxonomy to comprehensively map the Large Language Model (LLM) ecosystem. Specifically,
models are analyzed and systematized through three core lenses: (1) Adaptation Stages, (2)
Reasoning Capabilities, and (3) Model Compression & Efficiency Methods aimed at
addressing computational resource constraints.
Classification by Adaptation Stages (Naveed et al.): This dimension evaluates LLMs
based on their transformational lifecycle, from their raw state to the final user-facing product. It
encompasses: Pre-training -> Fine-tuning (via Instruction/Alignment) -> Prompting/Utilization
(end-user exploitation).
Classification by Model Compression & Efficiency Methods: This lens encompasses
"compressed models" and "efficiency methods." The primary objective of this category is to
mitigate hardware barriers and operational costs (cost, memory, latency). It includes the
following sub-branches:
Model Compression (Zhu et al.): Reduces the storage footprint and computational
overhead through four main techniques:
● Quantization: Reduces the bit precision of parameters. Includes PTQ (Post-
Training Quantization, requiring no retraining) and QAT (Quantization-Aware
Training, requiring retraining to compensate for precision loss).
● Pruning: Eliminates redundant components. Includes Structured Pruning
(removing entire neurons/layers) and Unstructured Pruning (sparsely removing
individual parameters).
● Knowledge Distillation (KD): Forces a smaller model (Student) to learn from a
larger model (Teacher). Includes Black-box KD (learning only from the outputs of
closed-source LLMs) and White-box KD (direct access to the parameters of open-
source LLMs).
● Low-Rank Factorization: Decomposes massive weight matrices into smaller
ones.
Data-Centric Methods (Zhong et al.): Optimizes performance by controlling the input
information flow rather than modifying the model's weights:
● Data Selection: Curating high-quality datasets to accelerate and reduce costs for
both the Pre-training and Fine-tuning phases.
● Prompt Engineering: Designing input instructions to guide LLMs toward
accurate outputs. Includes Few-shot prompting, Prompt compression, and
Prompt generation.
Classification by Reasoning Capabilities (Z. Li et al.): The reasoning model ecosystem is
systematized through four strategic pillars:
Foundations: Comprises the core building blocks for cultivating cognitive abilities, such
as foundational LLMs, symbolic logic systems, Monte Carlo Tree Search (MCTS), and
Reinforcement Learning (RL).
Blueprinting: The focal point of inference-stage reasoning. This branch delves into core
methods for navigating thought processes, including Structure Search, Reward
Modeling, Macro Action, and Self-Improvement mechanisms.
Benchmarking: Standardizes the measurement of reasoning capabilities by defining
testing datasets, evaluation metrics, and empirical performance comparisons.
Extended Techniques: Expands the boundaries of reasoning models into applied aspects,
including infrastructure optimization, trustworthiness enhancement, developing LLMs
into autonomous Agents, and Efficient Reasoning.
Currently, as highlighted by Minaee et al., the most widely used LLM architectures are encoder-
only, decoder-only, and encoder-decoder. Most of them are based on the Transformer
architecture, which remains the dominant and absolutely prevailing developmental direction
due to its superior text generation capabilities and excellent scaling properties. However, these
models still face significant challenges regarding "hallucinations" stemming from a lack of
memorization capacity (context window and static parameters) and the
Compositionality Gap (Mialon et al.). Consequently, augmentation techniques have emerged
to enhance three primary capabilities of a language model: reasoning, tool use, and acting.
This ecosystem is decomposed into three main strategic branches:
Reasoning Capability Branch: This branch focuses on guiding the model to decompose
complex tasks into logical interpretive chains to avoid "guessing". It marks a crucial
paradigm shift (Z. Li et al.): transitioning from System 1 (fast reflexes, next-token prediction)
to System 2 (slow, systematic thinking). Reasoning is no longer viewed as an emergent,
accidental byproduct of scaling up the model size. Instead, it is architected as a composable
process encompassing exploration, verification, planning, and iterative refinement. This process
is defined by five core methodological approaches: structure search (to explore thought
branches), self-improvement (iterative learning), macro action (abstract reasoning
operations), and the dominance of reinforcement-based methods (such as reward
modeling and Reinforcement Fine-Tuning - RFT), which maintain long, step-by-step reasoning
chains without interruption.
To implement this philosophy in practice, the system directs the model to break down
complex tasks into logical interpretive chains through the following techniques:
● Eliciting via Prompting: Awakening cognition through instructions. This
includes Zero-shot CoT (no examples provided) and Few-shot CoT (providing
detailed solution templates). This group's reliability is enhanced through variants
like Self-consistency (generating multiple paths and taking the majority vote),
Self-ask (generating follow-up questions), and ReAct (interleaving reasoning with
physical actions).
● Recursive Prompting: Applying a "divide and conquer" strategy, such as
Least-to-most prompting. The model splits a large problem into smaller sub-
problems, solves them sequentially, and uses the previous step's answer as
factual input for the next, thereby overcoming the compositionality gap.
● Explicitly Teaching: Embedding multi-step thinking into the model's
"instincts" via direct Fine-tuning on datasets containing detailed rationales.
Notable techniques include creating a Scratchpad, utilizing Working Memory, or
Bootstrapping (self-generating reasoning data).
Tool & Act Capability Branch: When internal reasoning is insufficient (especially for large-
scale calculations or retrieving up-to-date information), Augmented Language Models (ALMs)
invoke external modules.
Calling other models: The system automatically iteratively calls LMs to outline and
self-correct (e.g., Re3, PEER), or integrates visual/audio encoders (Flamingo, Socratic)
to process multimodal data.
Information Retrieval / RAG: Offloading memory by fetching factual data (via
Sparse/Dense retrievers). Architecturally, this is classified into: Jointly fine-tuning both
the retriever and the generative model (REALM, RAG, Atlas), or freezing the retriever to
save computational resources (RETRO).
A prominent intersection between RAG and Reasoning (RAG-Reasoning
Integration) is highlighted by Y. Li et al. This represents the most advanced direction for
RAG, categorized into three interactive paradigms:
• Reasoning -> RAG (Reasoning-Enhanced RAG): Utilizing the LLM's reasoning
capability to optimize the RAG pipeline (e.g., automatically analyzing questions
to generate smarter search queries, or filtering out noisy documents).
• RAG -> Reasoning (RAG-Enhanced Reasoning): Injecting external knowledge
into the reasoning chain to "anchor" the model's thinking to factual reality, filling
logic gaps and eliminating hallucinations (e.g., IRCoT).
• RAG <=> Reasoning (Synergized RAG-Reasoning): The most advanced and
complex paradigm. Reasoning and retrieval operate in a continuous loop:
reasoning determines what needs to be searched -> the returned information acts
as a springboard for the next reasoning step -> the process repeats until the final
answer is reached (e.g., ReAct or Self-RAG).
Search & Navigate: Level 1 allows the model to autonomously generate search queries
upon detecting a knowledge deficit (LaMDA, BlenderBot). Level 2 is more advanced,
transforming the AI into an agent that freely browses the web, clicks, and interacts with
user interfaces like a human (WebGPT, WebShop).
Code Interpreters: Offloading difficult mathematical and physical tasks to specialized
tools. ALMs translate the problem into Python code or formal logic languages, allowing
the computer to calculate absolutely accurate results (e.g., PAL, Mind's Eye, Codex).
Action Optimization Branch (Reinforcement Learning - RL) Because tool invocation or
web navigation operations cannot be optimized using traditional differentiable algorithms,
ALMs utilize Reinforcement Learning to align their behaviors.
Hard-coded Reward: Automated machine scoring based on clear-cut rules (e.g., a
perfect score if the correct product specifications are purchased on WebShop). The
advantage is high automation, though it can be rigid and mechanical.
Reinforcement Learning from Human Feedback (RLHF): Training an auxiliary
Reward Model based on human-ranked data. This technique helps ALMs learn abstract
standards such as politeness, safety, and the ability to accurately cite sources (WebGPT,
GopherCite).
Among these branches, reinforcement-based approaches (including Reinforcement
Fine-Tuning - RFT and Reward Modeling) are currently the dominant methods in
contemporary large-scale systems. They serve as the backbone operating the most advanced
reasoning-oriented models, enabling them to maintain long, step-by-step chains of reasoning
coherently and without disruption.
2. Evolution of the Field (Timeline)
The evolution of Augmented Language Models (ALMs) reflects a transition from standalone
language models toward systems capable of reasoning, retrieval, and tool interaction. Early large
language models relied solely on parametric knowledge stored in model weights, which limited
their ability to update knowledge, perform multi-step reasoning, and reduce hallucinations. The
introduction of retrieval-augmented methods marked the first major shift, enabling models to
access external knowledge sources. However, early RAG systems remained static and retrieval-
only, lacking reasoning capabilities. Subsequent research integrated reasoning mechanisms
such as Chain-of-Thought prompting and iterative retrieval, leading to agentic retrieval systems
capable of multi-step planning and adaptive information acquisition. More recent work further
advances this paradigm by decoupling reasoning from tool interaction and optimizing inference
efficiency, reflecting a shift from capability enhancement toward scalable and efficient ALM
deployment.
Several seminal works have played a crucial role in shaping this evolution. Chain-of-Thought
Prompting (Wei et al., 2022) demonstrated that step-by-step reasoning significantly improves
performance on complex tasks, forming the foundation of reasoning-based ALMs. ReAct (Yao et
al., 2023) extended this paradigm by integrating reasoning and tool interaction through an
iterative Thought–Action–Observation loop, enabling agent-like behavior. Self-RAG (Asai et al.,
2023) introduced self-reflective retrieval, allowing models to adaptively decide when to retrieve
information and critique generated outputs, thereby improving factual accuracy. IRCoT (Trivedi
et al., 2023) proposed interleaving retrieval with reasoning steps, marking the emergence of
synergized retrieval–reasoning frameworks. More recently, Search-R1 (Jin et al., 2025)
leveraged reinforcement learning to enable autonomous retrieval planning and reasoning,
representing the current frontier of agentic ALM systems. In parallel, system-level research has
focused on improving efficiency and scalability. ReWOO (Xu et al., 2023) proposed decoupling
reasoning from observations to reduce token redundancy and computational cost, while
INFERCEPT (Abhyankar et al., 2024) introduced optimized inference scheduling and memory
management for multi-tool ALM workflows. These developments highlight a shift from
improving reasoning capability to optimizing execution efficiency, indicating the maturation of
Augmented Language Models toward real-world deployment.
3. Current State-of-the-Art (SOTA)
3.1. Standard Benchmark Datasets
At the foundational level, classical open-domain question answering benchmarks remain
widely used. These include Natural Questions, TriviaQA, WebQuestions, and CuratedTrec, along
with MS-MARCO for generative QA and FEVER for fact verification. These datasets primarily
evaluate factual correctness and retrieval grounding in knowledge-intensive tasks.
More recent developments emphasize reasoning-intensive evaluation.
● Multi-hop benchmarks such as HotpotQA, 2WikiMultiHopQA, and MuSiQue have
become central to assessing compositional reasoning and iterative retrieval. Additional
datasets, including ARC-Challenge and StrategyQA, further probe logical inference
capabilities.
● Long-form generation benchmarks such as ASQA and ELI5 evaluate synthesis,
ambiguity resolution, and discourse-level coherence.
The emergence of agentic RAG systems introduces evaluation in open-world and interactive
settings. Benchmarks such as BrowseComp and WebWalkerQA simulate real-world
information-seeking scenarios requiring adaptive retrieval and multi-step reasoning.
Furthermore, ALM-oriented benchmarks extend evaluation beyond NLP tasks to system-level
and tool-integrated environments. These include GSM8K (and GSM8K-XL) for
mathematical reasoning, ToolBench for API-based interactions, ALFWorld for embodied
decision-making, and ShareGPT for conversational workloads. Emerging multimodal
benchmarks such as MMDocBench and ViRAG further reflect the shift toward multimodal
retrieval and reasoning systems.
3.2. Evaluation Metrics
At the core level, standard metrics for question answering and fact verification tasks include
Exact Match (EM), F1-score, and Accuracy. For generative tasks, particularly long-form
responses, metrics such as ROUGE and BLEU are commonly employed, often complemented by
human evaluation to assess fluency, coherence, and informativeness.
With the rise of reasoning-centric systems, evaluation increasingly incorporates semantic and
process-level metrics. These include semantic accuracy, multi-hop reasoning correctness,
and the validity of intermediate reasoning steps. In addition, citation faithfulness and
attribution accuracy have become critical, especially in systems that integrate external
knowledge sources.
The agentic RAG survey further highlights the importance of retrieval-specific metrics,
including Recall@k, NDCG, and Hits@N, which are often optimized directly in training-based
agentic systems via reinforcement learning. Task success rate is also commonly used to evaluate
end-to-end performance in interactive environments.
At the system level, ALMs introduce efficiency-oriented metrics as first-class evaluation
criteria. These include end-to-end latency, normalized latency, time-to-first-token (TTFT),
throughput, token consumption, and execution cost. Such metrics are essential for assessing
deploy ability in real-world settings where multi-step reasoning and tool interactions introduce
significant computational overhead.
3.3. Performance Ceiling and Current Limitations
Despite substantial progress, current systems exhibit clear performance ceilings that differ
across benchmark settings and real-world scenarios.
On structured benchmarks such as HotpotQA, SOTA synergized RAG-reasoning systems achieve
performance approaching human-level accuracy. Similarly, efficient ALM frameworks such as
ReWOO demonstrate that smaller models (e.g., 7B parameters) can match or approximate the
performance of significantly larger models (e.g., GPT-3.5) while improving efficiency, suggesting
that accuracy on closed benchmarks is nearing saturation.
However, this ceiling does not generalize to real-world environments. In open-domain and
agentic settings, performance degrades significantly when faced with ambiguous queries,
conflicting evidence, or incomplete information. Even advanced training-based agentic
reasoning systems (System 2), while achieving strong performance through adaptive planning
and iterative retrieval, remain limited by robustness and generalization challenges in dynamic
environments.
A central bottleneck lies in retrieval. Standard RAG systems often employ indiscriminate
retrieval strategies, introducing irrelevant or noisy context that degrades reasoning
performance. Even adaptive frameworks such as Self-RAG highlight that retrieval quality and
decision-making about when to retrieve remain unresolved challenges.
In addition, system-level constraints impose a critical ceiling on practical deployment. Although
recent systems such as INFERCEPT and LAMPS significantly improve throughput and latency,
they remain constrained by GPU memory limitations, scheduling inefficiencies, and the
complexity of multi-step workflows.
4. Open Challenges & Future Directions
Unsolved Problems:
● The first and most fundamental challenge is Tool Interaction Limitations. Current
systems interact with search engines in the most basic way possible — submitting a
query and receiving results, with no control over anything in between. Agents cannot
fine-tune retrieval configurations, cannot specify preferred source types, and cannot
adjust search depth based on the current reasoning step. This aligns with the observation
in the ALM Survey that most tool-using systems still treat tools as black boxes —
providing input and receiving output without any ability to control the intermediate
process.
● The second challenge is Reward Design Limitations. RL-based systems such as
Search-R1 rely on simple outcome-based reward signals — correct answer yields a
reward, incorrect answer yields a penalty. This approach fails to reflect the quality of
individual reasoning steps and intermediate retrieval decisions. The agent receives no
signal indicating which search steps were redundant, which were misdirected, and which
were genuinely effective — resulting in slow learning and unstable behavior. Both the
RAG-Reasoning Survey and Self-RAG identify this as a core weakness, yet no general
solution has been proposed.
● The third challenge is Retrieval Efficiency. Agents frequently perform redundant or
excessive retrieval operations — repeating queries already executed, retrieving
information irrelevant to the current reasoning step, or failing to stop when sufficient
information has already been gathered. This issue is partially addressed at the planning
level by ReWOO and at the system level by INFERCEPT and LAMPS, but remains
unsolved at the agent policy level. Since computational cost scales directly with the
number of tool calls, this is not only a quality problem but also a practical deployment
cost problem.
● The fourth challenge is Generalization and Robustness. Models trained on
structured benchmarks consistently fail when encountering unseen tools, dynamically
changing environments, conflicting information sources, or open-ended questions
without definitive answers. The Agent Survey identifies this as a systemic limitation of
current SOTA — models learn to perform well in familiar settings but lack the
adaptability required when conditions shift.
Limitations of Current SOTA Methods:
● Traditional RAG operates as a static, one-directional pipeline with no feedback loop
between retrieval and generation. Self-RAG improves upon this through adaptive
retrieval but remains entirely dependent on retriever quality. ReWOO significantly
reduces token cost through decoupled planning but lacks any mechanism to self-correct
when the initial plan is flawed. INFERCEPT and LAMPS effectively optimize the serving
system layer but do not address reasoning quality at the model level. RL-based systems
such as Search-R1 achieve the strongest benchmark performance but rely on overly
simplistic reward functions, require expensive training, and generalize poorly to open-
domain real-world tasks. Cutting across all of the above, virtually every current system is
limited to plain text input, while real-world data encompasses images, tables, and a wide
range of non-textual formats.
Emerging Trends:
● The most actively pursued research direction is fine-grained reward modeling —
moving beyond binary outcome-based rewards toward reward signals that evaluate each
individual decision made by the agent: whether a search was necessary, whether the
query was well-formed, and whether the retrieved result was effectively utilized. This is
widely regarded as the key to training agents that learn genuinely good behavior rather
than simply learning to score well on structured benchmarks.
● Closely related is the direction of advanced tool configuration — equipping agents
with finer-grained control over the retrieval process, including source type selection,
search depth adjustment, and flexible multi-tool orchestration adapted to the current
reasoning context. Building on insights from the Tool Learning Survey and ALM Survey,
this represents a progression from simple tool-using toward true tool-orchestration.
● On the efficiency front, retrieval efficiency optimization is emerging as a direction
with dual impact — simultaneously improving reasoning quality by reducing noise and
reducing operational cost by minimizing unnecessary API calls. This direction naturally
bridges the agent policy layer (Search-R1) and the system layer (INFERCEPT, LAMPS),
creating a cross-layer research opportunity that remains largely unexplored.
● Taken together, the most prominent emerging trends in the field are converging around
reinforcement learning-based agents, multi-tool reasoning, and real-world
autonomous research agents — reflecting a broader shift from static RAG pipelines
toward genuinely autonomous agents capable of planning, searching, and reasoning
adaptively in the dynamic and complex conditions of real-world deployment.
5. Potential Niche Selection
From the five surveyed works, three research niches emerge as independently compelling, each
targeting a distinct structural limitation of current RAG-reasoning systems. Rather than
converging on a single direction, the group identifies all three as viable research contributions
occupying different axes of the field: reasoning architecture, inference efficiency, and learning
signal design.
5.1. Small Adaptive Language Models: From Chain-Based to Graph-Based
Reasoning
The foundational RAG framework (Lewis et al., NeurIPS 2020) and its self-reflective successor
Self-RAG (Asai et al., 2023) both operate on linear, chain-based retrieval-generation pipelines.
While effective for single-hop queries, such architectures impose a sequential bottleneck on tasks
requiring simultaneous conditioning over multiple interdependent evidence sources—a limitation
documented empirically on HotpotQA and 2WikiMultiHopQA, where multi-hop performance
consistently degrades as reasoning depth increases. The broader agentic RAG survey traces this
trajectory explicitly, noting that the field has progressively evolved from chain-based to tree-based
to graph-based reasoning workflows, with graph-based approaches such as LightRAG and Think-
on-Graph demonstrating superior handling of relational multi-hop queries precisely because they
model entity-level dependencies rather than token-level sequences.
ReWOO (Xu et al., 2023), establishes a critical empirical anchor: decoupling reasoning plans from
tool observations allows a 7B LLaMA-based model to match GPT-3.5 (175B) on zero-shot multi-
hop benchmarks, with a 5x reduction in token consumption. This finding grounds the proposed
direction in demonstrated evidence rather than speculation—reasoning specialization, not
parameter scale, is the primary performance driver. The niche extends this insight by replacing
ReWOO’s linear plan-execute structure with a directed acyclic graph (DAG) workflow, wherein
nodes represent discrete retrieval or inference operations and edges encode data dependencies.
This enables parallel execution of independent subqueries, conditional branching based on
intermediate results, and more faithful modeling of the relational structure inherent in complex
knowledge-intensive tasks.
5.2. Lazy Retrieval and Context Distillation in Multi-API Request Settings
The ALM survey (Mialon et al.) identifies the memorization-retrieval trade-off as a core
unresolved challenge, noting that current agentic frameworks resolve it naively: at each inference
step, all prior API responses—JSON payloads, SQL results, log outputs—are concatenated into the
prompt in full, producing rapidly growing context windows. In a realistic multi-API agent
workflow involving five to ten sequential tool calls, this accumulation can exhaust the context
budget of a 32k-token model within a single task execution, a phenomenon the survey terms
“cumulative memory” overflow. The problem is qualitatively different from standard long-
document RAG because the retrieved content is dynamic and structured rather than static prose,
making existing KV cache compression techniques designed for text sequences a poor fit.
The proposed niche addresses this through two components operating at the input level: (1) a lazy
retrieval policy that defers API calls until the current reasoning state demonstrably requires new
information, extending the on-demand retrieval criterion of Self-RAG to structured data streams;
and (2) a context distillation module that compresses prior API responses into compact semantic
summaries before appending them to subsequent steps. This is orthogonal to—and composable
with—the KV cache scheduling optimizations of INFERCEPT and LAMPS: those works reduce
GPU memory waste at the hardware layer, while the proposed niche reduces semantic redundancy
at the prompt layer. The practical target is a 90% reduction in cumulative context size for long-
horizon agentic tasks, enabling deployment on sub-cloud infrastructure and substantially
reducing per-query API costs.
5.3. Reinforcement Learning with Custom Reward Design for Agentic RAG
The current frontier of RL-trained agentic RAG—exemplified by Search-R1 and R1-Searcher—
relies exclusively on outcome-based rewards: the agent receives a positive signal only upon
producing the correct final answer, with no intermediate feedback on the quality of individual
retrieval decisions or reasoning steps. Both the Synergized RAG-Reasoning survey and the
broader reasoning agentic RAG survey identify this as a primary open challenge, noting that
outcome-only rewards “fail to capture complex reasoning behavior and multi-step decision
processes.” The consequence is a learning signal that cannot distinguish a principled, evidence-
grounded inference chain from one that arrives at the correct answer through coincidental
retrieval—providing no gradient to reinforce the former over the latter.
The proposed direction involves designing dense, process-level reward functions that evaluate
intermediate steps alongside final outputs. Three candidate signals are particularly tractable:
retrieval precision relative to the current reasoning subgoal (computable via embedding similarity
between the retrieved passage and the active chain-of-thought step); logical coherence of the
generated reasoning chain, assessed by a frozen critic model; and evidence utilization rate,
measuring the proportion of retrieved content demonstrably referenced in the final generation.
These signals could be operationalized via a lightweight critic trained on human-annotated
reasoning traces, or through self-play over multiple candidate paths scored by the outcome
reward. The ALM survey’s discussion of RLHF and hard-coded reward instability provides direct
motivation: fine-grained, process-aware rewards address precisely the training instability that
arises when sparse outcome signals fail to propagate useful gradients through long reasoning
chains.
6. Personal Critical Reflection
6.1. Developments Not Fully Addressed by the Surveyed Literature
Three significant gaps stand out. First, and most consequentially, the emergence of large
reasoning models such as OpenAI o1 and DeepSeek-R1 is largely absent from all five surveys.
These models internalize extended chain-of-thought as a latent inference process prior to
generating outputs, effectively internalizing what the surveyed frameworks implement through
explicit retrieval-reasoning loops. This raises a non-trivial architectural question that none of the
papers address directly: if a model’s parametric reasoning is sufficiently deep, does external
retrieval remain the appropriate design primitive, or should it be reconceptualized as one tool
among many within a unified reasoning loop? This tension bears directly on the viability of the
niches identified in Section 5—particularly graph-based reasoning and custom reward design—
and warrants explicit engagement in future work.
Second, the surveyed works collectively underrepresent multimodal retrieval as a first-class
research problem. The agentic RAG survey acknowledges that real-world evidence is rarely text-
only—biomedical queries invoke imaging data, legal analysis requires structured tables, and
scientific reasoning spans figures and datasets—yet the architectures and benchmarks discussed
remain predominantly text-centric. The context distillation niche proposed in Section 5.2 is
particularly exposed to this gap: compression strategies designed for prose API responses may
not generalize to structured or visual data streams without non-trivial extension. Third,
adversarial robustness and source verification receive only superficial treatment across all five
papers. As retrieval-augmented agents are deployed in production environments, the ability to
detect and reject poisoned or factually erroneous documents becomes a safety requirement that
accuracy-focused benchmarks such as HotpotQA and TriviaQA do not capture.
6.2. Entry Barriers and Research Feasibility
The field presents a stratified landscape of entry difficulty that maps unevenly onto the three
proposed niches. Graph-based reasoning for small ALMs (Section 5.1) carries a moderate barrier:
the core contribution is architectural and can be developed and evaluated using existing open-
source models and standard multi-hop benchmarks without requiring large-scale compute. The
main challenge is the absence of established graph-structured evaluation protocols for RAG
systems, which means evaluation methodology must itself be partially constructed. Lazy retrieval
and context distillation (Section 5.2) is similarly accessible at the algorithmic level, but the
absence of standardized multi-API agent benchmarks—a gap noted implicitly in the ALM survey—
means that realistic evaluation requires custom experimental infrastructure. RL reward design
(Section 5.3) presents the highest barrier: implementing process-level reward functions requires
annotated reasoning traces, familiarity with policy gradient methods and their instabilities, and
sufficient compute for RL training. Recent advances in parameter-efficient fine-tuning (LoRA)
and open-source reasoning models have reduced the minimum viable compute requirement, but
this niche remains the most technically demanding of the three.
Taken together, these gaps and barriers reinforce the research directions identified in Section 5.
The absence of process-level feedback in current RL systems, the opacity of retrieval decisions in
multi-API contexts, and the rigidity of chain-based reasoning architectures are not peripheral
limitations—they are structural deficiencies that the surveyed literature acknowledges but has not
resolved. Each proposed niche targets one of these deficiencies directly, with entry costs calibrated
to the available infrastructure of an early-stage research group.