Scribd
Upload
6 views5 pages

Inverted Index Construction & Search

The document is an assignment from the Walchand Institute of Technology focusing on constructing an inverted index for information retrieval. It includes a program that processes text files to create a data structure mapping words to their document IDs and frequencies. The assignment also features a search functionality to query the inverted index for specific document and line numbers.

Uploaded by

vinayostwal707
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
6 views5 pages

Inverted Index Construction & Search

The document is an assignment from the Walchand Institute of Technology focusing on constructing an inverted index for information retrieval. It includes a program that processes text files to create a data structure mapping words to their document IDs and frequencies. The assignment also features a search functionality to query the inverted index for specific document and line numbers.

Uploaded by

vinayostwal707
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

WALCHAND INSTITUTE OF TECHNOLOGY, SOLAPUR

INFORMATION TECHNOLOGY
2021-22 SEMESTER - II
ASSIGNMENT - 6
Subject: Information Retrieval

Name: Ayush Pande Roll no: 74 Class: Final Year Btech IT

Title: Construction pf Inverted Index & Searching using inverted index

Theory: The Inverted Index is the data structure used to support full text search
over a set of documents. It is constituted by a big table where there is one entry
per word in all the documents processed, along with a list of the key pairs:
document id, frequency of the term in the document.

Program Code:
#6

from [Link] import word_tokenize


import nltk
from [Link] import stopwords
from pprint import pprint

dict={}
def collect(j):
global dict
file = open('C://Users//anike//Desktop//Btech//IR//IR Assignments//IR Assignments//ir
files//ir'+str(j)+'.txt', encoding='utf8')
read = [Link]()
[Link](0)

line = 1
for word in read:
if word == '\n':
line += 1
print("Number of lines in file is: ", line)

# create a list to
# store each line as
# an element of list
array = []
for i in range(line):
[Link]([Link]())

punc = '''!()-[]{};:'"\, <>./?@#$%^&*_~'''


for ele in read:
if ele in punc:
read = [Link](ele, " ")

# to maintain uniformity
read = [Link]()

for i in range(1):
# this will convert
# the word into tokens
text_tokens = word_tokenize(read)

tokens_without_sw = [
word for word in text_tokens if not word in [Link]()]

# to obtain the
# number of lines

for i in range(line):
check = array[i].lower()
for item in tokens_without_sw:
if item in check:
if item not in dict:
dict[item] = []

if item in dict:
if i + 1 not in dict[item]:
m=[]
[Link](j)
[Link](i+1)
if(m not in dict[item]):
dict[item].append(m)

for i in range(0,10):
collect(i+1)

pprint(dict)

s = input("Enter search query (doc no,line no): ")


# print([Link]())
# for i in range(len(dict)):
m1=[]
if s in dict:
print([Link](s))
else:

2
print("QUERY NOT FOUND")

Screenshots/Output

3
4
5

You might also like