Information Retrieval

Syllabus
• Introduction to Information Retrieval
• 1.1 Introduction to Information Retrieval, Basic Concepts, Information Versus Data, Trends and research
issues in information retrieval.
• 1.2 The retrieval process, Information retrieval in the library, web and digital libraries.
• Modeling in Information Retrieval
• 2.1 Taxonomy of Information Retrieval models, Classic Information Retrieval, Alternate set: Theoretical
model, Alternative Algebraic models, Alternative Probabilistic models
• 2.2 Structured text Retrieval models, Models for browsing
• Query and Operations in Information Retrieval
• 3.1 Query structures, Keyboard based querying, Pattern matching, Structured queries
• 3.2 User relevance feedback, Automatic local analysis, Automatic global analysis
Syllabus
• Indexing and Scoring in Information Systems
• 4.1 Introduction, Inverted Files, Other Indices for Text, Boolean queries and Introduction to Sequential
searching
• 4.2 Scoring, term weighting and the vector space model, Parametric and zone indexes, Weighted zone
scoring, Learning weights, The optimal weight, Term frequency and weighting, Inverse document frequency,
Tf-idf weighting. The vector space model for scoring, Queries as vectors, Computing vector scores, Efficient
scoring and ranking, Inexact top K document retrieval
• Evaluation of Information Retrieval Systems
• 5.1 Information retrieval system evaluation, Standard test collections, Evaluation of unranked retrieval
sets, Evaluation of ranked retrieval results, Assessing and justifying the concept of relevance
• 5.2 System quality and user utility, System issues, Refining a deployed system
• Applications of Information Retrieval Systems
• 6.1. Introduction to Multimedia Information Retrieval
• 6.2 Introduction to Distributed Information Retrieval
Course Outcomes
• CO1 Describe the basic concepts of the Information retrieval system.
• CO2 Design the various modeling techniques for information retrieval systems.
• CO3 Perform query operations on information retrieval systems
• CO3 Analyze the indexing and scoring operation in information retrieval systems
• CO5 Apply IR principles to design real-world applications
Textbooks:

•1 Modern information retrieval, Baeza-Yates, R. and Ribeiro-Neto, B., 1999. ACM press.
•2 Introduction to Information Retrieval By Christopher D. Manning and PrabhakarRaghavan, Cambridge
University Press
•3 Information Storage & Retrieval By Robert Korfhage – John Wiley & Sons
References:
•1 Storage Network Management and Retrieval, VaishaliKhairnar
•2 Introduction to Modern Information Retrieval. G.G. Chowdhury. NealSchuman
•3 Natural Language Processing and Information Retrieval by Tanveer Siddiqui, U.S Tiwarey
Case study presentation activity

•Students were divided into group of 3 and select topic on recent advancements in IR.
•On the selected topic students are asked to select 3 research papers each from reputed journals and conferences
from recent three years .
•They have to study all the 9 research papers in a group, understand the recent advancements in IR .
•Students are also asked to make a report on studied papers with columns having methodology and findings .
The Terrier Project is an open-source
Information Retrieval (IR) platform
developed at the University of
Glasgow, specifically by the
Information Retrieval Research
Group in the School of Computing
Science.

Terrier stands for Terabyte

Retriever.
It is a high-performance, flexible,
and scalable IR platform designed
for large-scale text retrieval tasks
— including academic research,
search engine development, and
experimentation in NLP and IR.
Difference between IR and Chatgpt ?
 What They Are
Information
Feature ChatGPT
Retrieval (IR)
A system that finds A generative AI
documents or model that can
Definition information matching understand and
a query from a large generate human-like
database or index text
Google Search, ChatGPT, GPT-4,
Example
Library search engines Claude, Gemini
How They Work
Feature Information Retrieval ChatGPT

Uses keyword matching,

Uses a pretrained large
ranking algorithms (like
language model to predict and
Core Method BM25, TF-IDF), and semantic
generate responses based on
search to retrieve relevant text
input text using deep learning
from existing sources

Fetches existing Generates new text using

Output Type text/documents from a learned patterns from huge
database or the web training data
 Purpose
Information
Feature ChatGPT
Retrieval
Help users find Help users converse,
relevant existing summarize, explain,
Goal
documents or or generate text
information intelligently
Writing, summarizing,
Searching articles,
tutoring, code
Best For facts, web content,
generation, chatting,
PDFs, etc.
reasoning
Use Cases

IR Examples ChatGPT Examples

- Search engine for research papers (e.g.,

- Writing an essay or report
Semantic Scholar)

- Legal or academic search systems - Explaining a concept in simple terms

- Creating a story, poem, or chatbot

- Product search on e-commerce sites
dialogue

- FAQ retrieval systems - Debugging or writing code snippets

Intelligence Level
Feature IR ChatGPT
Limited — matches based Deep understanding —
Language Understanding on keywords or semantic models sentence meaning,
vectors tone, context
✅ Yes — can infer,
Can Reason/Infer? ❌ No — only retrieves deduce, and explain
reasoning
✅ Yes — might generate
❌ Rare — returns
Can Hallucinate? incorrect but plausible
existing info
text
Summary Table
Aspect Information Retrieval ChatGPT (LLM)

Retrieves or Generates? Retrieves Generates

Pretrained knowledge from

Source of Information Existing documents
datasets

Interactivity One-shot queries Multi-turn dialogue

Creative, inferential,
Answers Factual and from source
possibly incorrect

Assistance, creativity,
Best Use Case Search and fact-finding
explanation
Difference between Information Retrieval
(IR) and Data Retrieval (DR)
Aspect Data Retrieval (DR) Information Retrieval (IR)

Retrieve exact data from structured Retrieve relevant information from

Purpose
sources unstructured/semi-structured text

Structured (e.g., databases: rows, Unstructured (e.g., documents,

Data Type
columns) articles, web pages)
Vague, natural language (search
Query Type Exact match (SQL queries, filters)
queries, keywords)
SQL: SELECT * FROM customers Google Search: “best mobile phones
Examples
WHERE age > 25 under 20k”
Rank documents by relevance using
Search Process Match data based on field values
scoring algorithms
Relevance — get most meaningful
Accuracy Goal Precision — get the exact data
information
Common Tools MySQL, PostgreSQL, Oracle Lucene, Elasticsearch, Solr
Use Cases

Use Case Data Retrieval Information Retrieval

Search for policy
Banking system Get account details
guidelines
Library system Get due date of a book Find books on AI ethics
Search reviews on battery
E-commerce Get product price
performance
Search for similar past
Medical records Fetch patient report
diagnoses
Structured Data
UnStructured Data
Document Retrieval
 Comparison Table

Computer-Centered Human-Centered
Aspect
Perspective Perspective
Algorithms, models,
Focus Users, needs, behavior
performance
Satisfaction, usability,
Evaluation Precision, recall, efficiency
success
Key Concern System optimization User experience
User can formulate good Users may need help
Assumption
queries expressing need
Maximize objective Maximize perceived
Goal
performance usefulness
Information Retrieval in Traditional Libraries
• Characteristics:
• Collections are manually curated (e.g., books, journals).
• Documents are often structured and cataloged using formal metadata (e.g., author, title, subject headings).
• Retrieval is based on controlled vocabularies (e.g., Library of Congress Subject Headings).

• Retrieval Approach:
• Users search through catalogs, not full-text content.
• Searches are precise and based on known-item retrieval (e.g., a specific book or author).
• Interfaces are often form-based with Boolean logic.
• Strengths:
• High-quality metadata and expert indexing.
• Useful for academic or archival research.
• Limitations:
• Limited to physical holdings or specific databases.
• Not suited for exploratory or broad search behavior.
Information Retrieval on the Web
• Characteristics:
• The Web is a vast, dynamic, and unstructured collection of documents.
• Content is heterogeneous (e.g., blogs, images, videos, PDFs).
• There is no centralized control or consistent metadata.

• Retrieval Approach:
• Based on full-text indexing of web pages (e.g., search engines like Google).
• Heavy use of link analysis (e.g., PageRank) to estimate document importance.
• Users typically use short, keyword-based queries.

• Challenges:
• Ambiguity of queries (users often type 2-3 words).
• Need to handle multi-lingual and multi-modal content.

• Techniques:
• Web crawlers and indexers.
• Ranking algorithms (e.g., BM25, PageRank).
• Use of click data and user behavior for personalization.
Information Retrieval in Digital Libraries
• Characteristics:
• Combines strengths of traditional libraries and web technologies.
• Documents are often digitized, structured, and include rich metadata.
• Collections are organized and preserved digitally (e.g., institutional repositories, digital archives).
• Retrieval Approach:
• Supports both metadata-based and full-text search.
• Often includes semantic search, faceted navigation, and advanced filtering.
• Strengths:
• High-quality content, often curated and peer-reviewed.
• Rich metadata enhances precision and recall.
• Easier integration with academic and scientific tools.
• Challenges:
• Ensuring interoperability across different systems (e.g., using standards like OAI-PMH).
• Handling rights management, access control, and preservation.
• Examples:
• Digital libraries like arXiv, PubMed, Europeana, or Google Scholar.
What Is OAI-PMH?

• OAI-PMH stands for Open Archives Initiative Protocol for

Metadata collection.
• It is a standard way for one system to collect metadata (like title,
author, date, etc.) from another system.
• You can think of it like:
• System A: “Here’s my list of books.”
• System B: “Thanks! I’ll add them to my catalog.”
Example Use Case

• A university’s digital library wants to share research papers with a

national archive.
• Both use OAI-PMH.
• The national archive (collects) metadata from the university’s system
regularly.
• Why It Matters
• Makes searching across many libraries or archives easier
• Reduces duplication and manual work
• Ensures compatibility across platforms
 Comparison Summary
Traditional
Feature Web IR Digital Library
Library
Unstructured and
Structure Highly structured Semi-structured
noisy
Controlled Rich and
Metadata Sparse or missing
vocabularies standardized
Scale Small to moderate Massive Moderate to large
Centralized or
Control Centralized Decentralized
federated
Metadata-based Full-text +
Retrieval method Metadata + full-text
search link-based
Researchers, Scholars,
Users General public
librarians professionals
 Research Issues in Information Retrieval
1. Query Understanding
•Handling vague, ambiguous, or short queries.
•Research into query expansion, reformulation, and intent detection.
2. Evaluation Metrics
•Improving evaluation beyond precision and recall.
•Designing user-centric metrics (e.g., satisfaction, engagement).
•Simulating user behavior for offline testing of IR systems.
3. Relevance and Ranking
•Better modeling of relevance — it’s not binary, it’s graded and dynamic.
•How to combine multiple signals (textual, behavioral, contextual) into effective ranking.
4. Fairness, Bias, and Ethics
•Addressing bias in ranking and filter bubbles.
•Ensuring fairness, transparency, and privacy in search algorithms.
Research Issues in Information Retrieval
5 Cross-Lingual and Multilingual IR
•Allowing users to search in one language and retrieve documents in another.
•Requires machine translation and cross-lingual alignment.
6. Zero-shot and Low-resource IR
•Building IR systems for low-resource languages or domains with little data.
•Exploring transfer learning and few-shot learning techniques.
7. Real-Time and Streaming IR
•Designing systems that can index and retrieve in real time.
•Useful for applications like news search, social media monitoring, and event
detection.