Scribd
Upload
16 views27 pages

Python CSV Data Analysis Techniques

This document discusses reading and writing CSV files in Python using the csv module. It covers reading data from a CSV file, accessing specific columns, writing data to a CSV file, and various formatting options like delimiters, quoting styles, and escape characters.

Uploaded by

Suman Das
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
16 views27 pages

Python CSV Data Analysis Techniques

This document discusses reading and writing CSV files in Python using the csv module. It covers reading data from a CSV file, accessing specific columns, writing data to a CSV file, and various formatting options like delimiters, quoting styles, and escape characters.

Uploaded by

Suman Das
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

100 DAYS OF DATA

WORKING WITH CSV


DATA ANALYSIS OF TITANIC DATA SET INCLUDED

Data analysis from a CSV file in Python


Learn to read and write CSV files in Python

•
COPY

name,age,height(cm),weight(kg)
Lenin,30,188,90
Phil,42,178,76
Claire,40,165,54
Alex,18,140,46

\t

csv

pandas
[Link]

COPY

import csv

with open('my_family.csv') as input:


csv_reader = [Link](input, delimiter=',')
line_count = 0
for row in csv_reader:
if line_count == 0:
print(f'Header row - {", ".join(row)}')
line_count += 1
else:
print(f'{row[0]} is {row[1]} years old, {row[2]} cm tal
line_count += 1
print(f'Total: {line_count} lines')

COPY

Header row - name, age, height(cm), weight(kg)


Lenin is 30 years old, 188 cm tall and 90 kg heavy
Phil is 42 years old, 178 cm tall and 76 kg heavy
Claire is 40 years old, 165 cm tall and 54 kg heavy
Alex is 18 years old, 140 cm tall and 46 kg heavy
Total: 5 lines
line_count

[Link] next(reader object,

None)

COPY

import csv

with open('my_family.csv') as input:


csv_reader = [Link](input, delimiter=',')
line_count = 0
next(csv_reader, None) #ignore the header
for row in csv_reader:
print(f'{row[0]} is {row[1]} years old, {row[2]} cm tall an
line_count += 1
print(f'Total: {line_count} lines')

COPY

Lenin is 30 years old, 188 cm tall and 90 kg heavy


Phil is 42 years old, 178 cm tall and 76 kg heavy
Claire is 40 years old, 165 cm tall and 54 kg heavy
Alex is 18 years old, 140 cm tall and 46 kg heavy
Total: 4 lines
Total: 4 lines

[Link]

DictReader

COPY

import csv

with open('my_family.csv') as input:


csv_reader = [Link](input, delimiter=',')
for row in csv_reader:
print(f'{row["name"]} is {row["age"]} years old, {row["heig
print(f'Total: {csv_reader.line_num} lines')

csv_reader.line_num

[Link]

[Link]
[Link]

COPY

import csv

header = ['Name', 'Age', 'Height(cm)', 'Weight(kg)']

data = [ ['Phil', 42, 178, 76],


['Alex', 18, 140, 46],
['Claire', 40, 165, 54] ]

filename = "my_family.csv"

with open(filename, 'w') as output:


csvwriter = [Link](output)

# Write a single list


[Link](header)

# Writing a list of lists


[Link](data)
COPY

Name,Age,Height(cm),Weight(kg)
Phil,42,178,76
Alex,18,140,46
Claire,40,165,54

writerow

writerows

[Link] ,

delimiter

COPY

import csv

header = ['Name', 'Age', 'Height(cm)', 'Weight(kg)']

data = [ ['Phil', 42, 178, 76],


['Alex', 18, 140, 46],
['Claire', 40, 165, 54] ]

filename = "my_family.csv"
with open(filename, 'w') as output:
csvwriter = [Link](output, delimiter = '|')

# Write a single list


[Link](header)

# Writing a list of lists


[Link](data)

COPY

Name|Age|Height(cm)|Weight(kg)
Phil|42|178|76
Alex|18|140|46
Claire|40|165|54

DictWriter

fieldnames

COPY

import csv

header = ['Name', 'Age', 'Height(cm)', 'Weight(kg)']

data = [
data = [
{"Name":"Phil", "Age": 42, "Height(cm)":178, "Weight(kg)":76},
{"Name":"Claire", "Age": 40, "Height(cm)":165, "Weight(kg)":54}
{"Name":"Alex", "Age": 18, "Height(cm)":140, "Weight(kg)":46}
]

filename = "my_family.csv"

with open(filename, 'w') as output:


csvwriter = [Link](output, fieldnames=header)
[Link]()
for row in data:
[Link](row)

COPY

Name,Age,Height(cm),Weight(kg)
Phil,42,178,76
Claire,40,165,54
Alex,18,140,46

writerows

COPY

import csv

header = ['Name', 'Age', 'Height(cm)', 'Weight(kg)']


data = [
{"Name":"Phil", "Age": 42, "Height(cm)":178, "Weight(kg)":76},
{"Name":"Claire", "Age": 40, "Height(cm)":165, "Weight(kg)":54}
{"Name":"Alex", "Age": 18, "Height(cm)":140, "Weight(kg)":46}
]

filename = "my_family.csv"

with open(filename, 'w') as output:


csvwriter = [Link](output, fieldnames=header)
[Link]()
[Link](data)

COPY

Name,Age,Height(cm),Weight(kg)
Phil,42,178,76
Claire,40,165,54
Alex,18,140,46

"
"

COPY

Name,Age,Height(cm),Weight(kg),Address
Phil,42,178,76,'Gryffindor room, Hogwarts'
Claire,40,165,54,'Snapes room, Hogwarts'
Alex,18,140,46,'4 Private Drive, Little Whinging'

quotechar

COPY

import csv

filename = "my_family.csv"

with open(filename, 'r') as output:


csvreader = [Link](output, quotechar="'")
for row in csvreader:
print(row)
COPY

['Name', 'Age', 'Height(cm)', 'Weight(kg)', 'Address']


['Phil', '42', '178', '76', 'Gryffindor room, Hogwarts']
['Claire', '40', '165', '54', 'Snapes room, Hogwarts']
['Alex', '18', '140', '46', '4 Private Drive, Little Whinging']

quoting

csv.QUOTE_MINIMAL

csv.QUOTE_ALL

csv.QUOTE_NONNUMERIC

csv.QUOTE_NONE

COPY

import csv

filename = "my_family.csv"

header = ['Name','Age','Height(cm)','Weight(kg)','Address']

data = [
['Phil',42,178,76,'Gryffindor room, Hogwarts'],
['Claire',40,165,54,'Snapes room, Hogwarts'],
[ , , , , p , g ],
['Alex',18,140,46,'4 Private Drive, Little Whinging']
]

with open(filename, 'w') as output:


csvwriter = [Link](output, quotechar="'", quoting=csv.QUOTE_A
[Link](header)
[Link](data)

csv.QUOTE_ALL

COPY

'Name','Age','Height(cm)','Weight(kg)','Address'
'Phil','42','178','76','Gryffindor room, Hogwarts'
'Claire','40','165','54','Snapes room, Hogwarts'
'Alex','18','140','46','4 Private Drive, Little Whinging'

csv.QUOTE_NONE

COPY

import csv
filename = "my_family.csv"

header = ['Name','Age','Height(cm)','Weight(kg)','Address']

data = [
['Phil',42,178,76,'Gryffindor room, Hogwarts'],
['Claire',40,165,54,'Snapes room, Hogwarts'],
['Alex',18,140,46,'4 Private Drive, Little Whinging']
]

with open(filename, 'w') as output:


csvwriter = [Link](output, quotechar="'", quoting=csv.QUOTE_N
[Link](header)
[Link](data)

COPY

Traceback (most recent call last):


File "[Link]", line 16, in <module>
[Link](data)
_csv.Error: need to escape, but no escapechar set

csv.QUOTE_NONE csv

escapechar
\

COPY

import csv

filename = "my_family.csv"

header = ['Name','Age','Height(cm)','Weight(kg)','Address']

data = [
['Phil',42,178,76,'Gryffindor room, Hogwarts'],
['Claire',40,165,54,'Snapes room, Hogwarts'],
['Alex',18,140,46,'4 Private Drive, Little Whinging']
]

with open(filename, 'w') as output:


csvwriter = [Link](output, quotechar="'", quoting=csv.QUOTE_N
[Link](header)
[Link](data)

COPY

Name,Age,Height(cm),Weight(kg),Address
Phil,42,178,76,Gryffindor room\, Hogwarts
Claire,40,165,54,Snapes room\, Hogwarts
Alex,18,140,46,4 Private Drive\, Little Whinging
\

COPY

Name, Age, Height(cm), Weight(kg), Address


Phil, 42, 178, 76, 'Gryffindor room, Hogwarts'
Claire, 40, 165, 54, 'Snapes room, Hogwarts'
Alex, 18, 140, 46, '4 Private Drive, Little Whinging'

skipinitialspace

COPY

import csv

with open('my_family.csv', 'r') as f:


csv_reader = [Link](f, quotechar="'")

for line in csv_reader:


print(line)
p ( )

COPY

['Name', ' Age', ' Height(cm)', ' Weight(kg)', ' Address']


['Phil', ' 42', ' 178', ' 76', " 'Gryffindor room", " Hogwarts'"]
['Claire', ' 40', ' 165', ' 54', " 'Snapes room", " Hogwarts'"]
['Alex', ' 18', ' 140', ' 46', " '4 Private Drive", " Little Whingi

skipinitialspace

True

COPY

import csv

with open('my_family.csv', 'r') as f:


csv_reader = [Link](f, quotechar="'", skipinitialspace=True

for line in csv_reader:


print(line)

COPY

['Name', 'Age', 'Height(cm)', 'Weight(kg)', 'Address']


['Phil', '42', '178', '76', 'Gryffindor room, Hogwarts']
['Claire', '40', '165', '54', 'Snapes room, Hogwarts']
[ , , , , p , g ]
['Alex', '18', '140', '46', '4 Private Drive, Little Whinging']

COPY

import pandas as pd

df = pd.read_csv('my_family.csv')
print(df)

COPY

Name Age Height(cm) Weight(kg)


0 Phil 42 178 76
1 Claire 40 165 54
2 Alex 18 140 46
COPY

import pandas as pd

df = pd.read_csv('my_family.csv')

print(type(df['Age'][0]))
print(type(df['Height(cm)'][0]))
print(type(df['Weight(kg)'][0]))

COPY

<class 'numpy.int64'>
<class 'numpy.int64'>
<class 'numpy.int64'>

names
pd.read_csv()

COPY

Phil,42,178,76
Claire,40,165,54
Alex,18,140,46

COPY

import pandas as pd

df = pd.read_csv('my_family.csv',
index_col='Name',
names=['Name', 'Age', 'Height(cm)', 'Weight(kg)']
)
print(df)

COPY

Age Height(cm) Weight(kg)


Name
Phil 42 178 76
Claire 40 165 54
Alex 18 140 46
df.to_csv

COPY

import pandas as pd

df = pd.read_csv('my_family.csv',
index_col='Name',
names=['Name', 'Age', 'Height(cm)', 'Weight(kg)']
)
df.to_csv('my_new_family.csv')

COPY

Age Height(cm) Weight(kg)


Name
Phil 42 178 76
Claire 40 165 54
Alex 18 140 46
COPY

import pandas as pd

#load the csv file


df = pd.read_csv('[Link]')

# Column Names
print([Link])

# Count unique values in Sex column


print(df['Sex'].value_counts())

# Percentage of male and female passengers


print(df['Sex'].value_counts(normalize=True))
COPY

Index(['PassengerId', 'Survived', 'Pclass', 'Name', 'Sex', 'Age',


'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked'],
dtype='object')
male 577
female 314
Name: Sex, dtype: int64
male 0.647587
female 0.352413
Name: Sex, dtype: float64

COPY

import pandas as pd

#load the csv file


df = pd.read_csv('[Link]')

# Column Names
print([Link])

# Count unique values in Sex column


print(df[df["Survived"] == 1]['Sex'].value_counts())
# Percentage of surviving male and female passengers
print(df[df["Survived"] == 1]['Sex'].value_counts(normalize=True))

COPY

Index(['PassengerId', 'Survived', 'Pclass', 'Name', 'Sex', 'Age',


'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked'],
dtype='object')
female 233
male 109
Name: Sex, dtype: int64
female 0.681287
male 0.318713
Name: Sex, dtype: float64

value_counts()

COPY

import pandas as pd

#load the csv file


df = pd.read_csv('[Link]')

# median age of each sex


median_age_men=df[df['Sex']=='male']['Age'].median()
median_age_women=df[df['Sex']=='female']['Age'].median()

print(f"The median age of men is {median_age_men}")


print(f"The median age of women is {median_age_women}")

COPY

The median age of men is 29.0


The median age of women is 27.0

Common questions

Powered by AI

The 'quotechar' parameter specifies the character used to quote fields in CSV files, which affects how text that contains delimiters or newlines within a field is handled. For both writing and reading CSVs, specifying a 'quotechar' helps encapsulate complex fields such that embedded characters don't incorrectly split the data into additional fields. This is crucial when fields contain commas or spaces, such as addresses in the provided data .

When loading a CSV file into a Pandas DataFrame with the intention of preserving column names, using the 'header' parameter in 'pd.read_csv()' is crucial. By default, Pandas will interpret the first row of the CSV as the header if no 'header' argument is specified .

To ensure that only data lines are processed and not the header line when using 'csv.reader()', you can skip the header by invoking 'next(csv_reader, None)' prior to the loop that iterates over the data. This method reads the first line and moves the reader object to the next line, thereby excluding the header from the iteration .

The inclusion of headers when writing CSV files can be controlled using the 'csv.writer()' and 'csv.DictWriter()' methods. When using 'csv.writer()', a separate call for 'writerow()' must be made with the header list before writing the data rows . With 'csv.DictWriter()', 'writeheader()' can be used to automatically write the header row based on the provided 'fieldnames' .

To validate the presence of a header in a CSV file, the 'next(reader)' method can be used. This method is commonly used to skip the header line by calling 'next(csv_reader, None)' before iterating over the remaining lines. This impacts the reading process by ensuring that the subsequent operations handle the correct data rows rather than including the headers as part of the data .

Indexing a Pandas DataFrame can be customized during the CSV import by using the 'index_col' parameter in the 'pd.read_csv()' function, allowing a specific column to be treated as the DataFrame's index. Customizing the index improves data access efficiency and readability, allowing quicker data retrieval based on the indexed field, such as using names or unique identifiers in the DataFrame .

The primary difference between 'csv.DictReader()' and 'csv.reader()' lies in how they provide access to CSV data. 'csv.DictReader()' reads the CSV file into dictionaries, where each row is an ordered dictionary with keys corresponding to column names, enabling easy accessibility and manipulation of specific fields by name . In contrast, 'csv.reader()' returns each row as a list, requiring positional index access which can be less intuitive, especially when dealing with unknown or dynamic CSV structures .

Using 'csv.QUOTE_NONE' implies that no quoting is performed on fields and relies solely on delimiters to separate values. Issues arise when field data contains delimiters or special characters which can split fields improperly or cause parsing errors. To address these issues, an 'escapechar' must be set to handle these cases by escaping the problematic characters, thus ensuring successful reading and writing operations .

Spaces within CSV data can affect the reading and writing process by unintentionally expanding field boundaries or leading to misinterpretation of data. The 'skipinitialspace' parameter in the 'csv.reader()' method can be used to mitigate these issues by ignoring spaces following delimiters, thus ensuring clean separation of fields. This is particularly useful when data fields, such as addresses, include spaces .

The 'fieldnames' parameter in the 'csv.DictWriter()' class defines the sequence of dictionary keys that the 'DictWriter' will serialize into the CSV file. It influences the output by determining the order of columns in the CSV file and ensuring consistency between the headers and the values being written . Without setting 'fieldnames', the output file may not align the data correctly with its intended headers, especially when dictionaries contain fields in arbitrary order.

You might also like