Search Engine refers to a huge database of internet resources such as web pages,
newsgroups, programs, images etc. It helps to locate information on World Wide
Web.
User can search for any information by passing query in form of keywords or phrase.
It then searches for relevant information in its database and return to the user.
A search engine is an online answering machine, which is used to search,
understand, and organize content's result in its database based on the search query
(keywords) inserted by the end-users (internet user). To display search results, all
search engines first find the valuable result from their database, sort them to make
an ordered list based on the search algorithm, and display in front of end-users. The
process of organizing content in the form of a list is commonly known as a Search
Engine Results Page (SERP).
Google, Yahoo!, Bing, YouTube, and DuckDuckGo
[Link] 1
Search Engine Components
Generally there are three basic components of a search engine as listed below:
[Link] Crawler
[Link]
[Link] Interfaces
2
Web crawler
Web Crawler is also known
as a search engine bot,
web robot, or web spider.
It is mainly a software
component that traverses
on the web, then downloads
and collects all the
information over the
Internet. It is a software
component that traverses
the web to gather
information.
Content can vary — it could
be a webpage, an image, a
video, a PDF, etc. — but
regardless of the format,
content is discovered by
links. 3
Search Engine Components
Database
By hopping along this path of links, the crawler is able to find new content and add it to
their index called Caffeine — a massive database of discovered URLs — to later be
retrieved when a searcher is seeking information that the content on that URL is a
good match for. All the information on the web is stored in database. It consists of
huge web resources with Indexing.
Some most popular search engine databases are Amazon Elastic Search Service
and Splunk.
Search Interfaces
This component is an interface between user and the database. It helps the user to
search through the database. It basically helps users to search for queries using the
database. Search Interfaces that affect the search results -
• Operators
• Phrase Searching
• Truncation
4
Working of Search Engine
Web crawler, database and the search interface are the major component of a search
engine that actually makes search engine to work. Search engines make use of Boolean
expression AND, OR, NOT to restrict and widen the results of a search. Following are
the steps that are performed by the search engine:
Step 1 : The search engine looks for the keyword in the index for predefined database
instead of going directly to the web to search for the keyword.
Step 2: It then uses software to search for the information in the database. This software
component is known as web crawler.
5
Working of Search Engine
Step 3: Once web crawler finds the pages, the search engine then shows the relevant
web pages as a result. These retrieved web pages generally include title of page, size of
text portion, first several sentences etc.
Step4 :The retrieved information is ranked (It uses query and indexes to create ranked
list of documents) according to various factors such as frequency of keywords, relevancy
of information, links etc.
6
meta search engines, search fundamentals, search strategies
MSE search several engines at once; it does not crawl the web or maintain a
database of web pages; instead, they act as a middle agent, passing the user’s query
simultaneously to other search engines or web directories, returning the results,
collecting them, remove the duplicate links, merge and rank them into a single list and
display it to the user. Some samples of MSEs are Vivisimo, MetaCrawler, Dogpile,
Mamma, and Turbo10.