Scribd
Upload
7 views2 pages

Python Task

The Python ETL assignment for trainees at Pro Stack Academy focuses on real-world data engineering tasks, including REST API consumption and data handling using Python. Trainees are required to invoke APIs to extract user and product data, transform it, and load it into MongoDB, MySQL, JSON, and CSV formats, with an emphasis on error handling and logging. Bonus requirements include using environment variables for database credentials and creating a modular project structure.

Uploaded by

sajjanjyothi103
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views2 pages

Python Task

The Python ETL assignment for trainees at Pro Stack Academy focuses on real-world data engineering tasks, including REST API consumption and data handling using Python. Trainees are required to invoke APIs to extract user and product data, transform it, and load it into MongoDB, MySQL, JSON, and CSV formats, with an emphasis on error handling and logging. Bonus requirements include using environment variables for database credentials and creating a modular project structure.

Uploaded by

sajjanjyothi103
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Python ETL Assignment​ ​ ​ ​ ​ Pro Stack Academy

______________________________________________________________
Python ETL Assignment for Trainees

1. Objective The objective of this assignment is to help trainees understand


real-world data engineering tasks such as REST API consumption, data
extraction, transformation, loading (ETL), file handling, and database
integration using Python.

2. Tools & Technologies -


●​ Python 3.x -
●​ requests, pandas - pathlib,
●​ os - MySQL (pymysql) -
●​ MongoDB (pymongo)

Note:
OS and pathlib Modules Explain the purpose and usage of:
os module -
pathlib module

Provide examples for file creation and directory handling

Task 1:
API URL:[Link]
Method:GET
Required Fields:None

Task:
invoke Rest API and write data(user_id,user_name,age,city)
into
a)Mongodb - users:collection
b)Mysql DB - users Table
c)new Json File - [Link]
d)new CSV File - [Link]

Task 2:
API URL:[Link]
Method:GET
Required Fields:None

Task:
invoke Rest API and write data(pid,pname,category,price,rating)
Python ETL Assignment​ ​ ​ ​ ​ Pro Stack Academy
______________________________________________________________
into
a)Mongodb - products :collection
b)Mysql DB - products Table -
c)new Json File
d)new CSV File

Note: with Error handling options

Task 3:
--------------------------------------------
REST URL: [Link]
Method Method:GET
Access Type:Public
Required Fields: None

1. Extract Data: from Rest API/FS/DB/Cloud


2. Tranform - Based on Required.
beauty products(pid,pname,price,categroy,rating)
3. Load into
1)[Link] file
2)[Link] file
3)mysql Product table
4)MongoDB Product collection.
Note: with Error handling options

Expected:
Bonus Requirements
Use logging instead of print statements.
- Use environment variables for database credentials.
- Create a modular project structure.
- Add retry logic for API calls.

Deliverables
- Python source code files
- Generated CSV and JSON files
- Database tables and collections
- Documentation explaining the solution

You might also like