Program 9:
Take the Institution name as input. Use Pydantic to define the schema for the desired output
and create a custom output parser. Invoke the Chain and Fetch Results. Extract the following
Institution-related details from Wikipedia: The founder of the Institution, when it was
founded. The current branches in the institution. How many employees are working in it. A
brief 4-line summary of the institution.
Aim
To develop a Python program that takes an institution name as input and extracts relevant
details such as founder, year of establishment, branches, number of employees, and summary
from Wikipedia using Pydantic schema.
Objective
To understand web data extraction using the Wikipedia library
To implement data validation using Pydantic
To process unstructured text using Regular Expressions
To organize extracted data into a structured format
Software Requirements
Python 3.x
Libraries: wikipedia, pydantic, re
Algorithm
1. Start the program
2. Install required libraries (Wikipedia, Pydantic)
3. Import necessary modules
4. Define a Pydantic schema (InstitutionInfo) with required fields
5. Accept institution name as input from the user
6. Call function get_wiki_text(name)
o If page exists → retrieve content
o If disambiguation error → select first option
o If page not found → return error message
7. Pass retrieved text to extract_info(text)
8. Use Regular Expressions to extract:
o Founder
o Founded year
o Employees
9. Assign a default value for branches
10. Extract the first 4 lines as a summary
11. Store extracted data in the Institution Info model
12. Display the extracted details
13. Stop the program
Theory
Wikipedia provides a Python library to fetch content from Wikipedia pages. However, the data
retrieved is unstructured text.
To convert this into structured information:
Regular Expressions (re) are used to extract specific fields
Pydantic BaseModel is used to define a schema and validate the extracted data
The program extracts:
Founder name
Year of establishment
Number of employees
Branch information
Summary (first few lines)
Procedure
1. Install required libraries using pip
2. Import necessary modules
3. Define a Pydantic schema (InstitutionInfo)
4. Fetch Wikipedia page content using the Wikipedia library
5. Handle exceptions like:
o DisambiguationError
o PageError
6. Use regular expressions to extract required fields
7. Store extracted data into the schema
8. Display formatted output
Program Description
get_wiki_text() → Fetches Wikipedia content
extract_info() → Extracts required details using regex
InstitutionInfo → Ensures structured and validated output
Source Program:
Observation
The program successfully fetched Wikipedia content
Extracted founder and year using pattern matching
Employee count was extracted when available
Summary displayed the first few lines of content
Branch information is generalized
Output:
Result
The program successfully extracted and displayed institution details from Wikipedia using
Python.
Conclusion
This experiment demonstrates how unstructured web data can be processed into a structured
format using Python, Regex, and Pydantic. It is useful for real-world applications like
information extraction and NLP systems.
Advantages
Easy to implement
Automates data extraction
Uses structured validation (Pydantic)
Useful for data analysis
Limitations
Regex may not always give accurate results
Depends on the Wikipedia page format
Branch data is not precisely extracted
Viva Questions
1. What is Pydantic?
A: Library used for data validation using Python type hints.
2. What is a Regular Expression?
A: A pattern used to search and extract text.
3. What is a Disambiguation Error?
A: Occurs when multiple Wikipedia pages match the input.
4. What type of data is fetched from Wikipedia?
A: Unstructured data.
5. What is the use of schema?
A: To organize and validate extracted data.