Scribd
Upload
7 views1 page

Gutenberg Project Book Data Scripts

The document outlines three Python scripts related to the Gutenberg project: one for downloading plain text versions of books, another for extracting book data into a CSV file, and a third for visualizing this data using Dash and Plotly. Key functionalities include handling author names and dates, cleaning data for visualization, and ensuring accurate data representation through callbacks and index alignment. The scripts address various challenges such as data formatting and interaction issues to facilitate effective data management and analysis.

Uploaded by

jh.larsen2301
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views1 page

Gutenberg Project Book Data Scripts

The document outlines three Python scripts related to the Gutenberg project: one for downloading plain text versions of books, another for extracting book data into a CSV file, and a third for visualizing this data using Dash and Plotly. Key functionalities include handling author names and dates, cleaning data for visualization, and ensuring accurate data representation through callbacks and index alignment. The scripts address various challenges such as data formatting and interaction issues to facilitate effective data management and analysis.

Uploaded by

jh.larsen2301
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Gutenberg project

Download Plaint texts:


1.Scraping_and_get_plain_texts.py. This script will create a folder in the same directory where the Python
file is stored and download all the books that offer a Plain Text version on their page. Please note, not all
books will have the "Plain Text" option. For those that do not have this option, no HTML file will be
generated.

Books data in CSV file:


2.Get_data_of_books_in_csv.py: This script gathers key information for each book, including the title,
author, birth and death dates, subjects, language, the name of the file where the plain text was saved, and
the plain text URL. Here are some important considerations for this process:
•Author Names and Birth/Death Dates: In most cases, this information is found in a single line, separated
by commas, following the structure: "Last name, First name, Birth date - Death date." However, not all books
follow this format, so conditional logic was applied to prevent mixing dates with names during the split. Some
authors may only have a first or last name, so the following conditions were implemented:
• Author Names: If the first two blocks, after splitting by commas, contain numbers, the block
with numbers will be excluded from the author's name.
• Dates: The last block is considered for dates. Since some dates refer to periods "Before Christ"
(BCE) or contain characters like "?" or "BCE," these characters were first removed. The position
of the dash ("-") was identified, and the numbers before and after it were extracted. If no dash
was found and only numbers exist in the last block, that number is treated as the birth date.

Dash:
3.Dash_visualization.py creates an interactive dashboard using Dash and Plotly to visualize book
information from a CSV file “[Link]”. Here the main obstacles were:
• Data Cleaning: The CSV file stored lists (e.g., languages, subjects) as strings (e.g., "[ 'English', 'Spanish'
]"). This required cleaning to remove unnecessary characters like brackets and quotes, and convert the
string into actual Python lists.
• Separation with explode: After cleaning, some columns (e.g., language) contained multiple values in a
single cell. explode was used to split these lists into separate rows, making it easier to count and analyze
individual values like languages or subjects.
• Callback and [Link] :To ensure smooth data updates and interactions, [Link] was used to store
the filtered data from the table. This allows the filtered data to be accessed in different callbacks without
reloading the entire dataset. The callback function updates the stored data whenever the table's content
changes.
• Misalignment of indices: There was an issue where clicking on a book showed incorrect details due to a
misalignment of indices after filtering the table. Derived_virtual_indices was used to fix this. It ensures
the correct book details are displayed by mapping the clicked row to the correct index in the filtered data.

You might also like