Introduction to Computer Science (I)
File Processing
Yen-Ru, Lai
yrlai@[Link]
Department of Civil Engineering, National Chung Hsing University
Introduction to Computer Science (I) 9. File Processing
◼ Python File Processing
OS module
The OS module provides many methods for operating files and directories,
accessing environment variables, and performing system-level operations.
It is an essential module for managing system resources and file system
operations.
9-2
Introduction to Computer Science (I) 9. File Processing
◼ Python File Processing
os Module 【 File processing 】
function illustration syntax return
exists() Check if the file exists [Link]('[Link]') Boolean
isfile() Check if it's a file [Link]('[Link]') Boolean
remove() Remove the file [Link]('[Link]')
[Link]('old_name.txt', 'new_name.txt')
rename() Rename the file # Change file name from old_name.txt to
new_name.txt
import shutil
copy() #shutil module Copy the file [Link]('[Link]', '[Link]') #
Copy [Link] as [Link]
9-3
Introduction to Computer Science (I) 9. File Processing
◼ Example # 2. Check if it is a file
if [Link](file_path):
os Module 【 File processing 】
print(f"'{file_path}’ is a file")
Please add a new file on the path: [Link] else:
print(f"'{file_path}’ is not a file")
import os
import shutil # 3. Copy the file
if [Link](file_path):
# File name and path settings # New file appears on desktop : copied_example.txt
file_path = "[Link]" [Link](file_path, copy_file_path)
new_file_path = "renamed_example.txt"
copy_file_path = "copied_example.txt" # 4. Rename the file
if [Link](file_path):
# 1. Check if the file exists #[Link] is renamed renamed_example.txt
if [Link](file_path): [Link](file_path, new_file_path)
print(f”file '{file_path}’ exists")
else: # 5. Remove the file
print(f”file '{file_path}’ does not exist, create a new file ") if [Link](copy_file_path):
with open(file_path, "w") as f: #Remove copied_example.txt
[Link](" This is a sample file \n") [Link](copy_file_path)
9-4
Introduction to Computer Science (I) 9. File Processing
◼ Python File Processing
os Module 【 Path processing 】
function illustration syntax
[Link]('folder', 'subfolder',
[Link]() Combine path
'[Link]')
[Link]() Separate path and file name [Link]('/path/to/[Link]')
[Link]() Get path name (without file name) [Link](path)
[Link]() Extract file name (without path) [Link](path)
[Link]() Get the full path (resolve symbolic links or [Link](relative_path)
[Link]() relative paths) [Link](relative_path)
9-5
Introduction to Computer Science (I) 9. File Processing
◼ Example
os Module 【 Path processing 】 # 3. Get path name
import os # Get the directory name in the path
directory_name = [Link](full_path)
# Set base path and file name print(f" directory name：{directory_name}") #
base_dir = "example_directory" example_directory/sub_folder
sub_dir = "sub_folder"
file_name = "[Link]" # 4. Get absolute path
#Relative paths (full_path) will return a full absolute
# 1. Combine path path base on the current working directory.
# Create a full path absolute_path = [Link](full_path)
full_path = [Link](base_dir, sub_dir, file_name) print(absolute_path)
print(f" The combined full path :{full_path}")
# example_directory/sub_folder/[Link] # 5. Get real path
# Resolve symbolic links and return the final true path
# 2. split path real_path = [Link](full_path)
# Separate path and file name print(f" real path :{real_path}")
path_part, file_part = [Link](full_path)
print(f" Path part:{path_part}") # example_directory/sub_folder
print(f" File part:{file_part}") # [Link]
9-6
Introduction to Computer Science (I) 9. File Processing
◼ Python File Processing
os Module 【Directory processing】
function illustration syntax return
getcwd() Get the current working directory [Link]()
chdir() Get working directory [Link](path)
listdir() List directory contents [Link](path)
mkdir() # Create a single directory [Link](path)
makedirs() # Create multi-level directories [Link](path, exist_ok=True)
rmdir() Delete directory [Link]('empty_folder')
isdir() Check if it is a directory [Link]('folder')) Boolean
Recursively delete multiple directories(The [Link]('1(outermost layer)/2(middle
removedirs()
premise is that the directory must be empty) layer)/3(innermost layer)')
[Link]('nested/folder/structure',
makedirs() Create multi-level directories recursively
exist_ok=True)
9-7
glob() import glob
List files matching a specific pattern List
#glob module [Link](‘*.txt’) #List txt format files
Introduction to Computer Science (I) 9. File Processing
◼ Example # 6. Create multi-level directories
os Module 【Directory processing】 nested_dirs = "level0/level1/level2"
[Link](nested_dirs, exist_ok=True)
import os
import glob # 7. List matching files or directories
# Create test files
# 1. Get the current working directory
with open("[Link]", "w") as f:
current_dir = [Link]()
[Link](" This is test file 1 。\n")
# 2. Switch working directory with open("[Link]", "w") as f:
new_dir = [Link](current_dir, "test_directory") [Link](" This is test file 2 。\n")
if not [Link](new_dir): # Use glob to match .txt files
[Link](new_dir) # If the directory does not exist, create it first txt_files = [Link]("*.txt")
[Link](new_dir) # Change working directory print(txt_files)
# 3. List directory contents # 8. Remove
print([Link]()) # Directory is currently empty # Remove subdirectory
[Link](sub_dir)
# 4. Create subdirectory
sub_dir = "subdir" # Remove multi-level directories
if not [Link](sub_dir): [Link](nested_dirs)
[Link](sub_dir)
# 9. Return to original working directory
# 5. Check if it is a directory [Link](current_dir)
[Link](sub_dir) 9-8
Introduction to Computer Science (I) 9. File Processing
◼ Python File Processing
exception handling
When using the OS module for file and directory operations, exception handling can effectively handle
various potential errors or abnormal situations, thereby ensuring program stability, maintainability, and
easy debugging.
The syntax is: try:
# The code block tried to execute
# Code that may have potential errors
except <ExceptionType> as e:
# Exception handling code
# Capture the specified exception type and execute the corresponding processing logic
else:
# If there is no exception inside the try, execute this part of the code
finally:
# Code that will be executed regardless of whether an exception occurs
# Usually used for cleaning work 9-9
Introduction to Computer Science (I) 9. File Processing
◼ Python File Processing
exception handling
#In the case where there is no file old_name.txt #In the case where the file [Link] exists (please add
import os [Link] yourself)
try:
try: file = open('[Link]', 'r', encoding='utf-8')
[Link]('old_name.txt', 'new_name.txt') content = [Link]()
except FileNotFoundError: except FileNotFoundError as e:
print(" Error: The file to be renamed cannot be found. ") print(f”Error:{e}")
except PermissionError: else:
print(" Error: Insufficient permissions to perform this print(" File read successfully! ")
operation. ") finally:
except Exception as e: print(" This part of the code will be executed regardless of
print(f" An unknown error occurred:{e}") whether an exception occurs. ")
if 'file' in locals():
[Link]() # Ensure the file is closed
9 - 10
Introduction to Computer Science (I) 9. File Processing
◼ Python File Processing
csv Module
The csv module is one of Python's built-in modules, specifically used to process files in csv
format.
CSV is a common format for storing data. Each column represents a piece of data, and fields
are usually separated by commas (,).
9 - 11
Introduction to Computer Science (I) 9. File Processing
◼ Csv File Processing
Method/Category use syntax
Used to read CSV files line by line and parse each line of
[Link]() [Link](file)
data into a string or list.
Used to write CSV files line by line and supports
[Link]() [Link](file)
formatted output.
Write data in one-dimensional mode
[Link]() ex. [Link](['Ken', 180, 70])
[Link](list)
[Link]() Write data in two-dimensional mode
ex. [Link]([['Ken', 180, 70], ['Allen', 185, 66]])
Parse each row of data into a dictionary, where Key is the
[Link]() field name of the first row and Value is the content of the [Link](file)
corresponding field.
Writes a dictionary into a CSV file, allowing specifying
[Link]() [Link](dict, fieldnames=fieldnames)
fieldnames.
Used to specify the delimiter symbol of the CSV file, such [Link](file, delimiter=',')
delimiter parameter
as comma (,), semicolon (;) or other custom symbols.
Ignore excess white space at the beginning of the field [Link](file, skipinitialspace=True)
skipinitialspace parameter
content when reading. 9 - 12
Introduction to Computer Science (I) 9. File Processing
◼ Csv File Processing
◼ Example [Link]
Name, Age, City
Alice, 25, New York
Bob, 30, Los Angeles
Charlie, 35, Chicago
David , 40, Houston
"Emily, Jr.", 28, "San Francisco"
Frank, 50, Miami
9 - 13
Introduction to Computer Science (I) 9. File Processing
◼ Example
Csv File Processing The reader() of the csv module must match the reading mode of open()
import csv
# Read CSV file # Write into CSV file
import csv with open('[Link]', 'w', encoding='utf-8', newline='') as file:
writer = [Link](file)
with open('[Link]', 'r', encoding='utf-8') as file: [Link](['Name', 'Age'])
reader = [Link](file) [Link]([['Alice', 25], ['Bob', 30],['Ken', 50]])
# Iterate reader ([Link])
for row in reader: #Write dictionary into CSV file
print(row) import csv
data = [{'name': 'Alice', 'age': 25},
{'name': 'Bob', 'age': 30},
{'name': 'Karina', 'age': 24},
[Link] is part of the csv module in Python, used to {'name': 'Julie', 'age': 44}]
with open('[Link]', 'w', encoding='utf-8', newline='') as file:
read CSV files. It is similar to the object information of fieldnames = ['name', 'age’]
<_csv.reader object at 0x...> . It is an iterator object, not # Write dictionary into CSV file, specifying field names
writer = [Link](file, fieldnames=fieldnames)
the actual content. Therefore, it is necessary to iterate # write title
[Link]()
this [Link] to view the contents of the CSV file. [Link](data) 9 - 14
Introduction to Computer Science (I) 9. File Processing
◼ Example delimiter=';' tells the program "fields are separated by semicolons", but the file actually uses
Csv File Processing commas as delimiters. This causes [Link] to be unable to correctly parse the field content
and instead treat the entire line of text as the value of a single field.
import csv
# Read semicolon (;) delimited CSV
with open('[Link]', 'r', encoding='utf-8') as file:
reader = [Link](file, delimiter=';')
for row in reader:
print(row) x
import csv
# Read comma (,) delimited CSV
with open('[Link]', 'r', encoding='utf-8') as file:
reader1 = [Link](file, delimiter=',')
for row1 in reader1:
print(row1) o
9 - 15
Introduction to Computer Science (I) 9. File Processing
◼ Example
Csv File Processing
# Remove CSV with extra spaces at the beginning
import csv
with open('[Link]', 'r', encoding='utf-8') as file:
reader = [Link](file, skipinitialspace=True)
for row in reader:
print(row)
【before deleting】 【 after deleting 】
['Name', ' Age', ' City'] ['Name', 'Age', 'City']
['Alice', ' 25', ' New York'] ['Alice', '25', 'New York']
['Bob', ' 30', ' Los Angeles'] ['Bob', '30', 'Los Angeles']
['Charlie', ' 35', ' Chicago'] ['Charlie', '35', 'Chicago']
['David ', ' 40', ' Houston'] ['David ', '40', 'Houston']
['Emily, Jr.', ' 28', ' "San Francisco"'] ['Emily, Jr.', '28', 'San Francisco']
['Frank', ' 50', ' Miami'] ['Frank', '50', 'Miami']
9 - 16
Introduction to Computer Science (I) 9. File Processing
◼ Example
CSV files Batch processing of multiple CSV files
[Link] [Link]
Name,Age Name,Age
Karen,13 Alice,15
Liam,33 Bob,30 • Please use notepad to save the data on the left as
Mia,21 Charlie,2 [Link], [Link]
Noah,19 David,29
• Please add the csv_files folder and save the files
Olivia,30 Ella,34
[Link] and [Link]
Paul,35 Frank,17
Quinn,8 Grace,31
Rachel,16 Hannah,8
Steve,32 Ian,24
Tina,4 Jack,16
Victor,27
9 - 17
Introduction to Computer Science (I) 9. File Processing
◼ Example
CSV files Batch processing of multiple CSV files
• Read each file
Purpose • Only select rows containing age greater than 18
• Write results into a new CSV file
import csv
import os
import glob
# Define source and target directories
input_dir = 'csv_files' # Folder to store original CSV
output_dir = 'processed_csv_files' # The folder to store the processed CSV
# If the target directory does not exist, create it
if not [Link](output_dir):
[Link](output_dir)
# Use glob to match all CSV files in the source directory
csv_files = [Link]([Link](input_dir, '*.csv')) # Match all .csv files
9 - 18
Introduction to Computer Science (I) 9. File Processing
◼ Example
CSV files Batch processing of multiple CSV files
# Traverse through all matching CSV files
for input_path in csv_files:
file_name = [Link](input_path) # Extract file name (without path)
output_path = [Link](output_dir, f'processed_{file_name}')
with open(input_path, 'r', encoding='utf-8') as infile, \ \ : The next line of code that follows
open(output_path, 'w', newline='', encoding='utf-8') as outfile: is a continuation of this line
reader = [Link](infile)
writer = [Link](outfile)
# Read header row
headers = next(reader)
[Link](headers) # Write headers into output file
# Filter data rows that meet criteria
for row in reader:
try:
age = int(row[1]) # Assume the second column is age
if age > 18: # Conditions: Age greater than 18
[Link](row)
except ValueError:
# If encounter age data that cannot be converted to an integer, skip the line
print(f"Cannot process line in file {file_name} :{row}")
The results are stored in the processed_csv_files folder! 9 - 19
continue