Cleaning bank marketing campaign data
Split and tidy bank_marketing.csv, storing as three DataFrames
called client, campaign, and economics, each containing the columns outlined in
the notebook and formatted to the data types listed.
Save the three DataFrames to csv files, without an index,
as [Link], [Link], and [Link] respectively.
Detecting Anomalous transactions
Compute an anomaly score for each transaction and add it to
the transactions DataFrame in a new column named Anomaly_Score.
Which transactions are flagged as anomalies? Add a boolean column to
the transactions DataFrame named Anomaly where True indicates an anomalous
transaction.
Create a summary of anomalous transactions. Save this as a pandas DataFrame
called anomalies_summary, containing the following
columns:TransactionID,TransactionAmount,TransactionDuration,AccountBalance
.
What is the distribution of TransactionAmount for normal and anomalous
transactions? Generate and save a histogram as anomalies_histogram.png that
visualizes these two groups with distinct colors. Look for differences in the
distribution of transaction amounts between normal and anomalous transactions.
3. Clustering Antartic penguin species (KMeans)
Utilize your unsupervised learning skills to clusters in the penguins dataset!
Import, investigate and pre-process the "[Link]" dataset.
Perform a cluster analysis based on a reasonable number of clusters and collect
the average values for the clusters. The output should be a DataFrame
named stat_penguins with one row per cluster that shows the mean of the
original variables (or columns in "[Link]") by cluster. stat_penguins should
not include any non-numeric columns.
4. Assessing Customer Churn using Machine Learning
Load the two CSV files into separate DataFrames. Merge them into a
DataFrame named churn_df. Calculate the proportion of customers
who have churned, and identify the categorical variables in churn_df.
Convert categorical features in churn_df into features_scaled. Perform
feature scaling separating the appropriate features and scale them.
Define your scaled features and target variable for the churn prediction
model.
Split the processed data into training and testing sets giving names
of X_train, X_test, y_train, and y_test using an 80-20 split, setting a
random state of 42 for reproducibility.
Train Logistic Regression and Random Forest Classifier models,
setting a random seed of 42. Store model predictions
in logreg_pred and rf_pred.
Assess the models on test data. Assign the model's name with higher
accuracy ("LogisticRegression" or "RandomForest")
to higher_accuracy.
5. Predicting Credit Card Approvals
Use supervised learning techniques to automate the credit card approval
process for banks.
Preproccess the data and apply supervised learning techniques to find
the best model and parameters for the job. Save the accuracy score
from your best model as a numeric variable, best_score. Aim for an
accuracy score of at least 0.75. The target variable is the last column of
the DataFrame.
6. Customer Analysis: preparing data for modeling
The Head Data Scientist at Training Data Ltd. has asked you to create a
DataFrame called ds_jobs_transformed that stores the data
in customer_train.csv much more efficiently. Specifically, they have set the
following requirements:
Columns containing categories with only two factors must be stored as
Booleans (bool).
Columns containing integers only must be stored as 32-bit integers
(int32).
Columns containing floats must be stored as 16-bit floats (float16).
Columns containing nominal categorical data must be stored as
the category data type.
Columns containing ordinal categorical data must be stored as ordered
categories, and not mapped to numerical values, with an order that
reflects the natural order of the column.
The DataFrame should be filtered to only contain students with 10 or
more years of experience at companies with at least 1000 employees,
as their recruiter base is suited to more experienced professionals at
enterprise companies.
7. Will this customer purchase your product?
The marketing team asked you to analyze the behavior of online customers
during November and December, the busiest months for shoppers.
What are the purchase rates for online shopping sessions by customer
type for November and December? Store the result in a dictionary
called purchase_rates in the format below using the exact names for
keys.
purchase_rates = {"Returning_Customer": 0.254, "New_Customer": 0.276}
What is the strongest correlation in total time spent among page types
by returning customers in November and December? Store the result
in a dictionary called top_correlation in the format below using the
exact names for keys.
top_correlation = {"pair": (x_duration, y_duration), "correlation": 0.345}
A new campaign for the returning customers will boost the purchase
rate by 15%. What is the likelihood of achieving at least 100 sales out
of 500 online shopping sessions for the returning customers? Store the
result in a variable called prob_at_least_100_sales. Optional: plot a
binomial probability distribution chart to visualize your chances.
[Link] data for the department of energy -Building an ETL pipeline
1. First, define an extract_tabular_data() function to ingest tabular data.
This function will take a single parameter, file_path. If file_path ends
with .csv, use the pd.read_csv() function to extract the data.
If file_path ends with .parquet, use the pd.read_parquet() function to
extract the data. Otherwise, raise an exception and print the message:
"Warning: Invalid file extension. Please try with .csv or .parquet!".
2. Create another function with the name extract_json_data(), which takes
a file_path. Use the json_normalize() function from the pandas library
to flatten the nested JSON data, and return a pandas DataFrame.
3. Next, we'll need to build a function to transform the electricity sales
data. To do that, we'll create a function
called transform_electricity_sales_data() which takes a single
parameter raw_data. raw_data should be of type [Link].
The transform_electricity_sales_data() needs to fullfil some
requirements that are described below in the docstring following the
function definition.
4. To load a DataFrame to a file, we'll define one more function
called load(), which takes a DataFrame and a file_path. If
the file_path ends with .csv, load the DataFrame to a CSV file. If
instead the file_path ends with .parquet, load the DataFrame to a
Parquet file. Otherwise, raise an exception that outputs a message in
this format: "Warning: {filepath} is not a valid file type. Please try again!
_"