RMIT Classification: Trusted
Step 1: Load Required Libraries
[Link](“tidyverse”)
[Link](“moments”)
Step 2: Load Required Librariesin
# Load necessary libraries
library(dplyr)
library(ggplot2)
library(moments) # For skewness and kurtosis
Step 3: Load the Dataset
# Load dataset (Replace “your_dataset.csv” with actual file name)
data <- [Link]("your_dataset.csv")
# View first few rows
head(data)
# Check for missing values
sum([Link](dataset))
Step 4: Handling Missing Values
# Count missing values per column
colSums([Link](dataset))
# Option 1: Remove rows with missing values (use with caution)
data_cleaned <- [Link](dataset)
# Option 2: Impute missing values with the mean (for numerical data)
dataset$variable[[Link](dataset$variable)] <- mean(dataset$variable)
# Option 3: Impute missing values with median
data$variable[[Link](dataset$variable)] <- median(dataset$variable)
Step 5: Detecting and Handling Outliers
Using Boxplot to Identify Outliers
RMIT Classification: Trusted
# Boxplot to visualize outliers
boxplot(dataset$variable, main = "Boxplot of Variable", col = "blue",
horizontal = TRUE)
Using IQR (Interquartile Range) to Remove Outliers
# Compute Q1 (25th percentile) and Q3 (75th percentile)
Q1 <- quantile(dataset$variable, 0.25)
Q3 <- quantile(dataset$variable, 0.75)
IQR_value <- IQR(dataset$variable)
# Define lower and upper bounds for outliers
lower_bound <- Q1 - 1.5 * IQR_value
upper_bound <- Q3 + 1.5 * IQR_value
# Remove outliers
#The %>% operator in R is called the pipe operator. It is primarily used in the tidyverse
package, especially in dplyr, to make code more readable and intuitive by passing the
output of one function as the first argument to the next function.
data_filtered <- dataset %>% filter(variable >= lower_bound & variable <=
upper_bound)
# Check new boxplot after outlier removal
boxplot(data_filtered$variable, main = "Boxplot of Variable (After Removing
Outliers)", col = "red", horizontal = TRUE)
Step 6: Checking and Correcting Skewness
# Compute skewness
skew_value <- skewness(data_filtered$variable, [Link] = TRUE)
cat("Skewness:", skew_value, "\n")
# Apply Log Transformation if right-skewed (skewness > 1)
if (skew_value > 1) {
RMIT Classification: Trusted
data_filtered$variable <- log1p(data_filtered$variable) # log1p(x) =
log(x+1) to handle zero values
# Apply Square Root Transformation if moderately skewed
if (skew_value > 0.5 & skew_value <= 1) {
data_filtered$variable <- sqrt(data_filtered$variable)
# Compute kurtosis
kurtosis_value <- kurtosis(data_filtered$variable, [Link] = TRUE)
cat("Kurtosis:", kurtosis_value, "\n")
# Visualization to check distribution
hist(data_filtered$variable, main = "Histogram of Sales (After
Transformation)", col = "lightblue", breaks = 30)
# If kurtosis is too high, further transformation can be applied
if (kurtosis_value > 3) {
data_filtered$variable <- log1p(data_filtered$variable)
Note: Interpretation:
Skewness > 0 means right-skewed (longer right tail).
Skewness < 0 means left-skewed (longer left tail).
Kurtosis > 3 indicates a sharp peak (leptokurtic), while <3 suggests a
flatter distribution (platykurtic).
Final Step: Save As Cleaned Data