0% found this document useful (0 votes)
4 views6 pages

DataEng Interview

The document provides guidance for a Python/Data Engineering/Data Analyst internship interview, emphasizing the need for clear and practical explanations of projects. It includes sample answers for common interview questions about a healthcare data cleaning pipeline project using Python and Pandas, detailing the workflow, data validation, and the importance of generating a JSON report. Additionally, it offers advice on what interviewers look for in candidates, such as understanding their project and the ability to identify improvements.

Uploaded by

pradeep
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
4 views6 pages

DataEng Interview

The document provides guidance for a Python/Data Engineering/Data Analyst internship interview, emphasizing the need for clear and practical explanations of projects. It includes sample answers for common interview questions about a healthcare data cleaning pipeline project using Python and Pandas, detailing the workflow, data validation, and the importance of generating a JSON report. Additionally, it offers advice on what interviewers look for in candidates, such as understanding their project and the ability to identify improvements.

Uploaded by

pradeep
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Since this is for a **Python/Data Engineering/Data Analyst internship**,

interviewers usually expect **clear, practical explanations** rather than


very advanced ones. They want to know if you understand your own
project and can explain your decisions confidently.

Here are internship-level answers you can memorize.

---

## 1. Explain your project.

**Answer (45–60 seconds):**

"My project is a DailyMed Healthcare Data Cleaning Pipeline developed


using Python and Pandas. The main objective is to clean and validate
healthcare data stored in a CSV file. The script reads the raw dataset,
removes empty and duplicate rows, cleans text fields, extracts IDs from
URL columns, validates important fields, and generates two outputs: a
cleaned CSV file and a JSON validation report. This project helped me learn
data preprocessing, validation, and working with real-world datasets."

---

## 2. Explain the project workflow.

**Answer:**

"The workflow is simple:

1. Read the CSV file.

2. Remove empty rows.

3. Remove duplicate records.


4. Clean text fields by removing extra spaces and newlines.

5. Extract IDs from URL columns.

6. Clean multi-value fields.

7. Validate each row by checking required fields.

8. Mark each row as valid or invalid.

9. Save the cleaned CSV.

10. Generate a JSON report containing statistics about the cleaned data."

---

## 3. Why did you use Pandas?

**Answer:**

"I used Pandas because it makes working with CSV files very easy. It
provides DataFrames, which allow me to clean, filter, validate, and
manipulate data efficiently. Compared to plain Python, Pandas requires
much less code and is faster for handling tabular data."

---

## 4. How did you remove duplicates?

**Answer:**

"I used the `duplicated()` function in Pandas. It identifies duplicate rows,


and I kept the first occurrence while removing the remaining duplicate
records. This helps improve data quality."

---
## 5. How did you remove empty rows?

**Answer:**

"I checked for rows where all columns were empty using
`isnull().all(axis=1)` and then removed those rows. This prevents blank
records from appearing in the final dataset."

---

## 6. How did you clean text?

**Answer:**

"I created a function that removes extra spaces, tabs, and newline
characters and trims leading and trailing spaces. This makes the text
more consistent and easier to process."

---

## 7. How did you extract IDs from URLs?

**Answer:**

"Some columns contained complete URLs instead of IDs. I used a regular


expression to extract only the last part of the URL, which is the unique
identifier. This makes the data easier to work with."

---

## 8. What validation rules did you implement?


**Answer:**

"I checked whether important fields like UMLS ID, drug label, section ID,
setID, and header ID were present. I also checked whether the text field
contained meaningful content. If any validation failed, I marked the record
as invalid and recorded the reason."

---

## 9. Why did you generate a JSON report?

**Answer:**

"The JSON report summarizes the cleaning process. It includes the number
of input rows, output rows, duplicates removed, empty rows removed,
valid and invalid records, and counts of missing fields. This makes it easy
to monitor data quality."

---

## 10. What improvement would you make to this project?

**Answer (Recommended):**

"While reviewing the project, I noticed one improvement. I created a list of


required output columns but forgot to use it before saving the CSV. I would
fix that by selecting only those columns before writing the output file. I
would also improve the project by adding logging, better error handling,
and unit tests."

---
# Possible Follow-up Questions

### Q: What is Pandas?

**Answer:**

"Pandas is a Python library used for reading, cleaning, analyzing, and


manipulating tabular data such as CSV and Excel files."

---

### Q: What is a DataFrame?

**Answer:**

"A DataFrame is a two-dimensional table in Pandas with rows and


columns, similar to an Excel spreadsheet."

---

### Q: What is JSON?

**Answer:**

"JSON stands for JavaScript Object Notation. It is a lightweight format used


to store and exchange structured data."

---

### Q: What did you learn from this project?

**Answer:**
"I learned how to clean real-world datasets, validate records, use Pandas
for data manipulation, generate reports, and write modular Python code. I
also learned the importance of checking data quality before analysis."

## Interview Advice

For an internship, interviewers are not expecting perfection. They are


looking for candidates who:

* Understand the project they built.

* Can explain the code in simple terms.

* Can identify improvements or bugs.

* Show a willingness to learn.

If you can confidently answer the questions above and explain your
project step by step, you'll be well prepared for an internship interview.

You might also like