Module 3.

1
Query Languages
Outline
■ Keyword-Based Querying
■ Pattern Matching
■ Structural Queries
■ Query Protocols
Keyword-Based Querying
■ A query is formulation of a user information need.
■ In its simplest form, a query is composed of keywords and
the documents containing such keywords are searched for.
■ Keyword-based queries are popular because they are
intuitive, easy to express, and allow for fast ranking.

1. Single-Word Queries
2. Context Queries
3. Boolean Queries
4. Natural Language Queries
1. Single-Word Queries
■ A query is formulated by a word.
■ A document is formulated by long sequences of words.
■ A word is a sequence of letters surrounded by separators
■ A definition of letter and separator is flexible. e.g hyphen - can
be defined as a letter or a separator e.g on-line

■ The result of word queries is the set of documents containing

word of the query.
■ Further, the resulting documents are ranked according to a
degree of similarity to the query.
■ To support ranking, two common statistics on word occurrences
inside texts are commonly used: 'term frequency' and 'inverse
document frequency' (TF-IDF)
2. Context Queries
■ Definition
- Search words in a given context that is near other words.
- We may want to form phrases of words or find words which are proximal in the text.
-Words which appear near each other may signal a higher likelihood of relevance than
if they appear apart.
-Cotextual queries help to capture meaning and context of words within the
documents, making search results more precise.
Types
Phrase (some flexibility on how the words are separated)
>a sequence of single-word queries
>e.g. enhance retrieval, red apple
Proximity ( a relaxed version of phrase query)
>a sequence of single words or phrases, and a maximum allowed distance
between them are specified.
>e.g,within distance (enhance, retrieval, 4) will match ‘…enhance the
power of retrieval…’
This is useful when you are looking for concepts that might be expressed by
multiple different phrases. e.g Curriculum Theories
Boolean Queries
▪ Definition
▪ A syntax composed of atoms that retrieve documents, and of
Boolean operators which work on their operands
▪ e.g, translation AND syntax OR syntactic

■ Fuzzy Boolean(Partial Matching)

Retrieve documents appearing in some operands (The AND
may require it to appear in more operands than the OR)
=
Natural Language
■ Generalization of “fuzzy Boolean”
■ A query is an enumeration of words and context queries
■ All the documents matching a portion of the user query
are retrieved.
■ Higher ranking is assigned to those documents matching
more parts of the query.
■ The negation can be handled by letting the user express
that some words are not desired.
■ A threshold may be selected so that the documents with
very low weights are not retrieved.
■ Under this scheme we have completely eliminated any
reference to Boolean operations and entered into the field
of natural language queries.
Pattern Matching
■ A pattern is a set of syntactic features that must
occur in a text segment.
/OCR
‘pro(blem|tein)(s|є)(0|1|2)*’->’problem2’ and ‘proteins’
8. Extended Patterns:
■ Extended patterns are a user friendly way to represent complex search queries.
■ EPs are subsets of REs which are expressed with simpler syntax.
■ Retrieval systems can internally convert external patterns into REs or search them
with specific algorithm.
■ Each system supports its own set of EPs and hence no formal definition exists for
EPs.
Structural Queries
■ Mixing contents and structure in queries
- contents: words, phrases, or patterns
- structural constraints: containment, proximity, or other
restrictions on structural elements present in the
document.

■ The Boolean queries can be built on top of the

structural queries, so that they combine the sets of
documents delivered by those queries.

■ Three main structures

- Fixed structure (Filled form)
- Hypertext structure (web pages)
- Hierarchical structure (books, articles,legal documents)
1. Fixed Structure
■ The documents had a fixed set of fields, much like a filled form.
■ Each field had some text inside.
■ Some fields were not present in all documents.
■ Only rarely could the fields appear in any order or repeat across a document.
■ A document could not have text not classified under any field.
■ Fields were not allowed to nest or overlap.
■ The retrieval activity allowed on them was restricted to specifying that a given basic
pattern was to be found only in a given field.
■ Most current commercial systems use this model.
■ This model is reasonable when the text collection has a fixed structure.
■ EX: a mail has a sender, a receiver, a date, a subject and a body field.
User can search for the mails sent to a given person with “football” in the Subject field.

■ Drawback:The model is inadequate to represent the hierarchical structure present

in an HTML document,
 2. Hypertext
■ Hypertexts probably represent the maximum freedom with respect to structuring power.
■ A hypertext is a directed graph where the nodes hold some text and the links represent
connections between nodes or between positions inside the nodes .

■ Initially, the retrieval from a hypertext began as a merely navigational activity. That is,
the user had to manually traverse the hypertext nodes following links to search what he
wanted.

■ It was not possible to query the hypertext based on its structure. Even in the Web
one can search by the text contents of the nodes, but not by their structural connectivity.

■ An interesting proposal to combine browsing and searching on the Web is WebGlimpse.

■ It allows classical navigation plus the ability to search by content in the neighborhood of
the current node.
 Hypertext : WebGlimpse
WebGlimpse: combine browsing and searching on the Web
■ Querying hypertexts based on both their content and structure requires specialized tools
and techniques that can handle the complexity of hypertext documents.

■ Hypertexts, such as web pages, are interconnected through links and contain a mixture of
text, images, multimedia, and other elements.

■ The following tools and techniques are commonly used for this purpose:

1. XPath

● Purpose: XPath is a query language for selecting nodes from an XML document, which
can also be applied to HTML documents.
● Functionality: It allows users to query hypertexts based on their structure by
navigating through the document's hierarchy, selecting elements, attributes, and text.
● Example: Extracting all links (<a> tags) from a webpage or selecting all paragraphs
within a specific section.

2. CSS Selectors

● Purpose: CSS selectors are used to select elements in HTML documents based on their
attributes, classes, IDs, or relationships to other elements.
● Functionality: Similar to XPath, but more focused on the styling attributes of HTML
elements, making it useful for querying content based on structural relationships.
● Example: Selecting all headings (<h1>, <h2>, etc.) within a specific <div>.

3. Regular Expressions (Regex)

● Purpose: Regex is a powerful tool for searching and manipulating strings based on
patterns.
● Functionality: It can be used to query content within hypertexts by identifying patterns
within text, such as specific keywords, phrases, or even more complex patterns.
4. SQL/XML or XQuery

● Purpose: SQL/XML and XQuery are query languages designed for querying XML data.
● Functionality: XQuery can be used to query both the content and structure of XML-based hypertexts.
SQL/XML integrates SQL queries with XML content, allowing for complex querying of structured
documents.
● Example: Retrieving specific elements from an XML document based on hierarchical relationships
and content.

5. Web Scraping Libraries (e.g., BeautifulSoup, Scrapy)

● Purpose: These libraries are designed to extract data from web pages by parsing HTML or XML
content.
● Functionality: They combine content and structural querying by allowing users to traverse the
document tree, extract elements, and analyze the content within those elements.
● Example: Extracting all articles from a news website and analyzing their structure to identify the most
common layout patterns.
3) Hierarchical Structure
Hierarchical Structure
Samples of Hierarchical Model
■ PAT Expressions
■ Overlapped Lists
■ Lists of References
■ Proximal Nodes
■ Tree Matching

link : Samples of Hierarchical Models

Query Protocols
■ These are query languages that are used automatically by software applications to
query text databases.
■ Some of them are proposed as standards for querying CD-ROMs or as intermediate
languages to query library systems. Because they are not intended for human use,
we refer to them as protocols rather than languages.
■ Some of the query protocols are:
Z39.50
WAIS (Wide Area Information Service)
CCL (Common Command Language)
CD-RDx (Compact Disk Read only Data exchange)
SFQL (Structured Full-text Query Language)
Z39.50
■ It is a protocol approved as a standard in 1995 by ANSI and NISO.

■ This protocol is intended to query bibliographical information using a standard

interface between the client and the host database manager which is independent
of the client user interface and of the query database language at the host.

■ The database is assumed to be a text collection with some fixed fields (although it
is more flexible than usual).

■ The Z39.50 protocol is used broadly and is part, for instance, of WAIS.

■ The protocol does not only specify the query language and its semantics, but also
the way in which client and server establish a session, communicate and exchange
information, etc.

■ Although originally conceived only to operate on bibliographical information (using

the Machine Readable Cataloging Record (MARC) format), it has been extended to
query other types of information as well.
WAIS(Wide Area Information Service)
■ WAIS is a suite of protocols that was popular at the beginning of the 1990s before the boom
of the Web. The goal of WAIS was to be a network publishing protocol and to be able to
query databases through the Internet.
 Other Query protocols….
■ In the CD-ROM publishing arena, there are several proposals for query protocols. The
main goal of these protocols is to provide 'disk interchangeability.' This means more
flexibility in data communication between primary information providers and end users.
It also enables significant cost savings since it allows access to diverse information
without the need to buy, install, and train users for different data retrieval applications.
We briefly cover three of these proposals:
a. CCL (Common Command Language) is a NISO proposal (Z39.58 or ISO 8777)
based on Z39.50. It defines 19 commands that can be used interactively. It is
more popular in Europe, although very few products use it. It is based on the
classical Boolean model.
b. CD-RDx (Compact Disk Read only Data exchange) uses a client-server
architecture and has been implemented in most platforms. The client is
generic while the server is designed and provided by the CD-ROM publisher
who includes it with the database in the CD-ROM. It allows fixed-length fields,
images, and audio, and is supported by such US national agencies as the CIA,
NASA, and GSA.
Other Query protocols….
c. SFQL (Structured Full-text Query Language)
■ It is based on SQL and also has a client-server architecture.
■ SFQL has been adopted as a standard by the aerospace community (the Air Transport
Association/Aircraft Industry Association).
■ Documents are rows in a relational table and can be tagged using SGML.
■ The language defines the format of the answer, which has a header and a variable length
message area.
■ The language does not define any specific formatting or markup. For example, a query in
SFQL is:
Select abstract from [Link] where title contains
"text search"
■ The language supports Boolean and logical operators, thesaurus, proximity
operations, and some special characters such as wild cards and repetition. For
example:
where paper contains "retrieval" or like "info %"
and date > 1/1/98
■ Compared with CCL or CD-RDx, SFQL is more general and flexible, al-though it is
based on a relational model, which is not always the best choice for a document
database.