WRITING JSON WITH
PYTHON Module-3
Reading and Writing CSV
Files
using Pandas DataFrames
A Practical Guide
Python Data Analysis
Pandas DataFrame
Intro
Python Pandas DataFrame is a powerful 2D data structure used for real-world
data analysis and manipulation tasks. Here are highlighted points:
• Two-dimensional labeled data structure with rows (index) and columns.
• Columns can contain different data types: integers, floats, strings, booleans, etc.
• Conceptually similar to an Excel sheet or SQL table in memory
Pandas DataFrame
Key features
Labeled axes (row index and column names) for intuitive data selection and
alignment.
Size is mutable: rows and columns can be added or dropped dynamically.
Supports heterogeneous data types across columns in the same DataFrame.
Rich built-in methods for filtering, sorting, grouping, merging, joining, and reshaping
data.
Pandas DataFrame
Intro
Data cleaning and transformation
• Handling of missing data using methods like isna(), dropna(), and fillna().
• Easy replacement of values, type conversion, and column-wise operations
(vectorized operations).
• Supports reshaping operations such as pivot, melt, stack, unstack, and groupby
aggregations.
Introduction to CSV Files
What is a CSV file?
• A simple, widely used format for storing tabular data.
• Data is organized in rows and columns, with values separated by commas (or other delimiters).
• Plain text format, easily readable by humans and machines.
Why use CSV files?
• Portability across applications and OS
• Simplicity and easy editing
• Small file size compared to other formats
Introduction to Pandas DataFrames
What is Pandas?
• Powerful Python library for data analysis
• Provides DataFrames and Series structures
Why use Pandas for CSV?
• Simplified reading and writing of CSV data
• Powerful data manipulation tools
• Handles missing values and type conversions
Reading CSV Files into Pandas
The read_csv() function is the primary tool for reading CSV data:
import pandas as pd
df = pd.read_csv('[Link]')
print(df)
Reads '[Link]' and creates a DataFrame for analysis
Key read_csv() Parameters
Essential Parameters:
• sep/delimiter: Specify column separator (default: ',')
• header: Row number for column names (default: 0)
• names: Custom column names list
• dtype: Specify column data types
Advanced read_csv() Parameters
• index_col: Use specific column as row index
• usecols: Read only selected columns
• chunksize: Read large files in chunks
• encoding: Specify file encoding (UTF-8, latin-1)
• na_values: Recognize missing value indicators
Handling Missing Values
Missing values represented as empty strings, 'NA', or 'NaN'
df = pd.read_csv('[Link]', na_values=['NA', 'Missing'])
df = [Link](0) # Fill with 0
df = [Link]([Link]()) # Fill with column mean
Writing DataFrames to CSV Files
Use to_csv() function to write DataFrames:
data = {'col1': [1, 2, 3], 'col2': ['A', 'B', 'C']}
df = [Link](data)
df.to_csv('[Link]', index=False)
index=False prevents DataFrame index from being written
Key to_csv() Parameters
• sep: Column delimiter (default: ',')
• na_rep: String for missing values
• header: Write column names (default: True)
• index: Write row index (default: True)
• columns: Select specific columns to write
• encoding: Specify file encoding
Appending to CSV Files
Use mode parameter to control write behavior:
data = {'col1': [4, 5, 6], 'col2': ['D', 'E', 'F']}
df = [Link](data)
df.to_csv('[Link]', mode='a', header=False, index=False)
Set header=False when appending to avoid duplicate headers
Handling Different Encodings
Common encodings:
df = pd.read_csv('[Link]', encoding='utf-8')
df = pd.read_csv('[Link]', encoding='latin-1')
df = pd.read_csv('[Link]', encoding='cp1252')
df.to_csv('[Link]', encoding='utf-8')
UTF-8 recommended for international characters
Processing Large Files with Chunksize
Read files exceeding available memory:
for chunk in pd.read_csv('large_file.csv', chunksize=10000):
processed_chunk = chunk[chunk['value'] > 50]
processed_chunk.to_csv('[Link]', mode='a', header=False)
Process data incrementally without loading entire file
Best Practices and Tips
• Always specify data types using dtype for better performance
• Use index_col to set proper index columns
• Specify encoding explicitly to avoid errors
• Use chunksize for large files (>1GB)
• Use usecols to select only needed columns
Common Issues and Solutions
UnicodeDecodeError:
Try different encodings: 'utf-8', 'latin-1', 'cp1252', 'iso-8859-1'
Memory Issues:
Use chunksize parameter for large files
Key Takeaways
Use read_csv() and to_csv() for efficient CSV handling
Manage missing values and encodings properly
Use chunksize for large file processing
Thank You!
Writing JSON with Python
Data Serialization and File Handling
A Comprehensive Guide
Python JSON Module • January 2026
What is JSON?
JSON (JavaScript Object Notation)
• Lightweight, text-based data format
• Human-readable and machine-parseable
• Language-independent, widely supported
Why use JSON?
• Standard for web APIs and REST services
• Configuration files and data storage
• Data interchange between applications
JSON Data Types
• Object: Unordered key-value pairs (Python dict)
• Array: Ordered collection of values (Python list)
• String: Text enclosed in double quotes
• Number: Integer or floating-point
• Boolean: true or false
• Null: Empty value (Python None)
Python to JSON Type Conversion
Python JSON
dict object
list, tuple array
str string
int, float number
True true
False false
None null
[Link]() - Convert to String
Serialize Python object to JSON string:
import json
user_data = {'name': 'John', 'age': 30, 'active': True}
json_string = [Link](user_data)
print(json_string)
Output: {"name": "John", "age": 30, "active": true}
[Link]() - Write to File
Write Python object directly to JSON file:
import json
user_data = {'name': 'John', 'age': 30, 'city': 'New York'}
with open('[Link]', 'w') as f:
[Link](user_data, f, indent=4)
Creates formatted JSON file with 4-space indentation
dump() vs dumps()
[Link](obj, fp)
• Writes directly to file object
• Better for large objects (memory efficient)
• Use when writing to file
[Link](obj)
• Returns JSON string
• Better for sending via API or network
• Use when string manipulation needed
Pretty Printing JSON
Make JSON human-readable with indent parameter:
data = {'name': 'Alice', 'skills': ['Python', 'SQL']}
# Compact format
print([Link](data))
# Pretty format with 2-space indent
print([Link](data, indent=2))
Improves readability for debugging and storage
Essential [Link]() Parameters
• indent: Number of spaces for indentation
• sort_keys: Sort keys in output (True/False)
• ensure_ascii: Escape non-ASCII characters
• separators: Custom (item, key) separators
• default: Function for non-serializable objects
• skipkeys: Skip non-string keys
Sorting JSON Keys
Use sort_keys parameter for consistent output:
data = {'name': 'Bob', 'age': 25, 'city': 'Boston'}
with open('[Link]', 'w') as f:
[Link](data, f, indent=2, sort_keys=True)
Output keys in alphabetical order for consistency
Custom Serialization with default
Handle non-serializable objects:
from datetime import datetime
def custom_serializer(obj):
if isinstance(obj, datetime):
return [Link]()
raise TypeError(f"Object not serializable")
data = {'created': [Link]()}
[Link](data, f, default=custom_serializer)
Reading JSON with [Link]()
Parse JSON file into Python object:
import json
with open('[Link]', 'r') as f:
data = [Link](f)
print(data)
print(type(data))
Converts JSON file to Python dict or list
Handling JSON Errors
Manage common JSON exceptions:
try:
with open('[Link]', 'r') as f:
data = [Link](f)
except FileNotFoundError:
print("File not found")
except [Link]:
print("Invalid JSON format")
Handle FileNotFoundError and JSONDecodeError
Best Practices
• Always use indent for human-readable files
• Use context managers (with statement) for file handling
• Handle exceptions for robust code
• Use sort_keys=True for consistency
• Validate JSON schema when reading untrusted data
• Use [Link]() instead of dumps() for files
Real-World Example
Configuration file management:
config = {
'database': 'postgresql',
'host': 'localhost',
'port': 5432
}
with open('[Link]', 'w') as f:
[Link](config, f, indent=2, sort_keys=True)
What is JSON?
JSON (JavaScript Object Notation)
• Lightweight, text-based data format
• Human-readable and machine-parseable
• Language-independent, widely supported
Why use JSON?
• Standard for web APIs and REST services
• Configuration files and data storage
• Data interchange between applications
JSON Data Types
• Object: Unordered key-value pairs (Python dict)
• Array: Ordered collection of values (Python list)
• String: Text enclosed in double quotes
• Number: Integer or floating-point
• Boolean: true or false
• Null: Empty value (Python None)
Python to JSON Type Conversion
Python JSON
dict object
list, tuple array
str string
int, float number
True true
False false
None null
[Link]() - Convert to String
Serialize Python object to JSON string:
import json
user_data = {'name': 'John', 'age': 30, 'active': True}
json_string = [Link](user_data)
print(json_string)
Output: {"name": "John", "age": 30, "active": true}
[Link]() - Write to File
Write Python object directly to JSON file:
import json
user_data = {'name': 'John', 'age': 30, 'city': 'New York'}
with open('[Link]', 'w') as f:
[Link](user_data, f, indent=4)
Creates formatted JSON file with 4-space indentation
dump() vs dumps()
[Link](obj, fp)
• Writes directly to file object
• Better for large objects (memory efficient)
• Use when writing to file
[Link](obj)
• Returns JSON string
• Better for sending via API or network
• Use when string manipulation needed
Pretty Printing JSON
Make JSON human-readable with indent parameter:
data = {'name': 'Alice', 'skills': ['Python', 'SQL']}
# Compact format
print([Link](data))
# Pretty format with 2-space indent
print([Link](data, indent=2))
Improves readability for debugging and storage
Essential [Link]() Parameters
• indent: Number of spaces for indentation
• sort_keys: Sort keys in output (True/False)
• ensure_ascii: Escape non-ASCII characters
• separators: Custom (item, key) separators
• default: Function for non-serializable objects
• skipkeys: Skip non-string keys
Sorting JSON Keys
Use sort_keys parameter for consistent output:
data = {'name': 'Bob', 'age': 25, 'city': 'Boston'}
with open('[Link]', 'w') as f:
[Link](data, f, indent=2, sort_keys=True)
Output keys in alphabetical order for consistency
Custom Serialization with default
Handle non-serializable objects:
from datetime import datetime
def custom_serializer(obj):
if isinstance(obj, datetime):
return [Link]()
raise TypeError(f"Object not serializable")
data = {'created': [Link]()}
[Link](data, f, default=custom_serializer)
Reading JSON with [Link]()
Parse JSON file into Python object:
import json
with open('[Link]', 'r') as f:
data = [Link](f)
print(data)
print(type(data))
Converts JSON file to Python dict or list
Handling JSON Errors
Manage common JSON exceptions:
try:
with open('[Link]', 'r') as f:
data = [Link](f)
except FileNotFoundError:
print("File not found")
except [Link]:
print("Invalid JSON format")
Handle FileNotFoundError and JSONDecodeError
Best Practices
• Always use indent for human-readable files
• Use context managers (with statement) for file handling
• Handle exceptions for robust code
• Use sort_keys=True for consistency
• Validate JSON schema when reading untrusted data
• Use [Link]() instead of dumps() for files
Real-World Example
Configuration file management:
config = {
'database': 'postgresql',
'host': 'localhost',
'port': 5432
}
with open('[Link]', 'w') as f:
[Link](config, f, indent=2, sort_keys=True)