Advancing Retail Demand Forecasting with Machine Learning and Deep Learning Methods
Dharmar V M 1, Ganeshram R 2, Visal Kumar M 3, Yogaraj S 4, Dr. S. Chidambaram 5
1,2,3,4
Student, Information Technology, National Engineering College, Kovilpatti, India.
5
Associate Professor, Information Technology, National Engineering College, Kovilpatti, India.
Abstract: Traditional demand forecasting involves analyzing historical data
methods often struggle to model both temporal and applying appropriate
and covariate-based variations. To address this, techniques to predict future
we propose an ensemble forecasting model that demand. The process begins by
integrates state-of-the-art sequence modeling collecting data on influencing
with machine learning techniques. Our model, factors such as product features,
tested on a large dataset of 540,000 demand promotions, calendar events,
series, significantly improves forecast accuracy. weather conditions, and broader
Performance was evaluated using mean error, economic contexts. A thorough
mean absolute error, and mean squared error, understanding of these factors
with results outperforming benchmark methods. provides actionable insights for
Additionally, we introduce a real-time demand optimizing sales and operations.
sensing algorithm for enhanced forecasting This paper introduces a big data
precision. predictive analytics model
designed to process large
Keywords: Predictive Analytics, Demand
volumes of demand data. By
Forecasting, Deep Learning, Time Series
leveraging advanced machine
Forecasting, Random Forest Algorithm, Machine
learning and deep learning
Learning, Big Data Analytics, Forecast
techniques, the model delivers
Accuracy, Supervised Learning.
short-, medium-, and long-term
I. INTRODUCTION forecasts, enhancing the accuracy
and robustness of demand
Accurate demand
forecasting in a retail context.
forecasting is essential for
effective planning and decision- II. RELATED WORKS
making across various
Demand forecasting plays a critical role in
organizational functions,
supply chain management, enabling businesses
including marketing, production,
to align their inventory, production, and logistics
distribution, and finance. Both
with future demand. Early methods like Auto-
short-term and long-term
Regressive Integrated Moving Average
forecasts are crucial for guiding
(ARIMA) effectively capture temporal
strategic and operational
correlations but struggle with nonlinear patterns
decisions. Forecasting typically
and multiple influencing factors [1]. To address Long Short-Term
these limitations, Zhang and Qi [6] proposed a Memory (LSTM) networks are an
hybrid ARIMA-Artificial Neural Network advanced type of Recurrent
(ANN) model, improving forecast accuracy by Neural Network (RNN) designed
capturing both linear and nonlinear components. to overcome the limitations of
traditional RNNs, particularly in
Machine learning techniques have gained
handling long-term dependencies
prominence due to their ability to handle large
in sequential data
datasets. Shih et al. [5] demonstrated the
potential of combining Partial Least Squares An LSTM network
Regression (PLSR) with ANNs, highlighting the consists of a chain of repeating
importance of diverse data sources. Hybrid modules, each with a unique
models integrating traditional methods with structure compared to a standard
machine learning have shown enhanced RNN cell. The core component of
forecasting performance, as seen in the ARIMA- the LSTM architecture is the
ANN approach [6]. memory cell, which interacts
with three gates forget, input, and
Incorporating multiple influencing factors like
output gates. These gates control
seasonal trends, Bandara et al. [4] proposed
the flow of information, allowing
clustering time series before applying Recurrent
the network to maintain and
Neural Networks (RNNs), further improving
update the cell state effectively
accuracy. Long Short-Term Memory (LSTM)
over time.
networks have proven effective in capturing
long-term dependencies for time-series
forecasting [3], with Rangapuram et al. [3]
demonstrating their superior performance in
complex temporal data scenarios.
Convolutional Neural Networks (CNNs) have
also been applied to forecast demand based on
spatial data. Borovykh et al. [7] showed that
combining CNNs with LSTMs enhances
forecasting accuracy, especially in retail Fig. 1 LSTM
environments influenced by both spatial and architecture
temporal factors. The retail industry has
The memory cell 𝐶𝑡 is
benefited from deep learning advancements,
incorporating external data like point-of-sale data
and social media trends to improve forecast the heart of the LSTM unit,
accuracy [8]. responsible forstoring the long-
term information. It is controlled
III. EXISTING METHODS
by the three gates that regulate
a. Long Short-Term Memory (LSTM) Networks
𝜎-is
the flow of information into and Where:
out of the cell. the sigmoid
The forget gate 𝑓𝑡 decides
𝑊𝑓-is the weight matrix,
activation function,
what portion of the cell state
ht−1
𝐶{𝑡−1} should be discarded.
from the previous time step - is the previous
𝑥𝑡-is the current input,
hidden state,
The forget gate is computed as:
𝑏𝑓 - is the bias term.
f t =σ (W f [ ht−1 , x t ]+b f )
The output gate 𝑜𝑡controls how
(1)
𝜎-is the sigmoid activation function,
Where: much of the cell state should be
output as the hidden state ℎ𝑡 at
𝑊𝑓-is the weight matrix,
the current time step. It is
ht−1 is the previous hidden state, computing das:
𝑥𝑡- is the current input,
-
𝑏𝑓- is the bias term.
o t =σ (W o [h t−1 , x t ]+bo )
(5)
The input gate 𝑖𝑡 determines The hidden state ℎ𝑡 is then
which new information should given by:
be added to the cell state. It is
ht =Ot . tan h (C t )ht
responsible for updating the cell
state with relevant information (6)
from the current input and the
𝜎-is
Where:
previous hidden state
the sigmoid
𝑊𝑓-is the weight matrix,
i t =tan h (W c [ ht , x t ]+bi )i t activation function,
(2)
ht−1
A candidate cell state is also - is the previous
𝑥𝑡-is the current input,
generated, which is combined hidden state,
with the input gate to update the
memory cell 𝑏𝑓- is the bias term.
C t =tan h (W c [ ht−1 , x t ]+bi )
This hidden state is passed on to the next LSTM
(3)
unit and can also be used for making predictions.
The new cell state is then updating das
b. Random Forest
C t =f t . Ct −1+i t . C t (4)
Random Forest is an ensemble learning
algorithm that reduces overfitting and improves 1. Creates N bootstrap samples from the
accuracy by using multiple decision trees trained dataset.
on different data subsets. It combines the outputs 2. Every node (sample) takes a feature
of these trees through majority voting for randomly of size m where m<M
classification or averaging for regression, 3. Builds a split for the m features selected in
resulting in stable and accurate predictions. Step 2 and detects the k node by using the
best split point.
Random Forest is robust against noise, handles 4. Split the tree iteratively until one leaf node
missing values, and works well with imbalanced is attained and the tree remains complete.
datasets. It also provides feature importance 5. The algorithm is trained on each
metrics, making it useful in fields like healthcare bootstrapped independently.
and finance for understanding key predictive 6. Using trees classification voting, predicted
factors. However, it can be computationally data is collected from the trained trees (n).
expensive on large datasets and less interpretable 7. The final RF model is built using the peak
than simpler models. voted features.
8. return RF
IV. PROPOSED
FRAMEWORK
A. Dataset
The dataset used for this study is the Online
Retail dataset, which contains transactional data
from a UK-based online retailer. The data spans
from December 2010 to December 2011 and
includes key attributes such as InvoiceNo,
Fig. 2 Random Forest StockCode, Description, Quantity, InvoiceDate,
workflow UnitPrice, CustomerID, and Country. This
dataset is well-suited for analyzing sales trends
Input: and predicting future demand due to its detailed
N - Quantitative amount of bootstrap samples record of customer transactions, product
M - Total number of features purchases, and prices.
m - Sample size
k - Next node
Output: A Random Forest (RF)
Steps:
with infrequent values in StockCode grouped
under a common label. Numerical features were
Table 1: List of Features available in Datatset normalized using MinMaxScaler, and the dataset
was split into training and testing sets for time-
Dataset Dataset Exposition
series forecasting.
Features
InvoiceNo Unique identifier (string)
C. Technique
StockCode Product code (string)
Description Product description (string) CatBoost, short for Categorical Boosting, is a
Quantity Number of products purchased gradient boosting algorithm developed by
(int)
Yandex, designed specifically for handling
InvoiceDate Date of invoice (datetime)
categorical data. It stands out from other
UnitPrice Price per unit (float)
boosting methods by processing categorical
CustomerID Customer identifier (int)
features without extensive preprocessing like
Country Country of purchase (string)
one-hot encoding. This is achieved through its
Day Day of the invoice (int, derived
use of Ordered Target Statistics, which mitigates
from InvoiceDate)
Month Month of the invoice (int, data leakage and overfitting, making it
derived from InvoiceDate) particularly useful in real-world datasets with
Year Year of the invoice (int, derived both numerical and categorical features.
from InvoiceDate)
DayOfWeek Day of the week (0: Monday, 6:
Sunday, int, derived from
InvoiceDate)
Sales Total sales amount (float,
calculated as Quantity *
UnitPrice)
B. Pre-processing
The dataset was preprocessed by first calculating
a new feature, Sales, as the product of Quantity
and UnitPrice. Missing values in numeric
columns were imputed using the median. Non-
positive values in Quantity and UnitPrice were
removed as outliers. To capture temporal patterns
for the time-series predictive analysis, the
InvoiceDate feature was converted to a datetime
format, and additional time-related features such
as Day, Month, Year, and DayOfWeek were
extracted. Categorical features like Country and
StockCode were transformed into category types,
Fig. 3 CatBoost
architecture
Figure 3, illustrates the training process of
CatBoost, demonstrating how input training data
is processed through categorical features and
their combinations.
First, the dataset is preprocessed by calculating
total sales from available features and extracting
temporal features like day, month, year, and day
of the week from the invoice date. Missing
values are imputed using the median for numeric
columns, and categorical columns are converted
to categorical data types. Rows with non-positive
values for Quantity and UnitPrice are removed to
ensure data consistency. Fig. 4 Proposed Framework
Feature scaling is applied using the
MinMaxScaler for numerical variables, and
categorical variables, such as Country and Pseudo code:
StockCode, are encoded via one-hot encoding.
The model training begins with a train-test split Input: Dataset
of the dataset, followed by
RandomizedSearchCV to optimize CatBoost's Output: A CatBoost Model
hyperparameters, focusing on key parameters
like iterations, depth, and learning rate. Steps:
CatBoost is then trained on the optimized 1. Load the dataset and preprocess the data.
parameters, and its performance is evaluated
using metrics such as Mean Squared Error 2. Calculate the target variable (e.g., Sales) from
(MSE), Mean Absolute Error (MAE), and Mean relevant features.
Error, demonstrating the model's effectiveness in
predicting retail sales accurately. 3. Convert categorical variables to appropriate
format.
D. Architecture
4. Split the dataset into training (X_train,
y_train) and testing sets (X_test, y_test).
5. Initialize CatBoost model with specified
parameters and categorical features.
6. Define hyperparameter grid for tuning
(iterations, depth, learning_rate, etc.).
7. Perform Randomized Search to find the best
hyperparameters using training data.
8. Fit the CatBoost model with the best CatBoost's effectiveness in delivering accurate
parameters on the training data. predictions compared to other models like RF +
XGB and LSTM + CatBoost, which had higher
9. Use the trained model to make predictions on
error rates.
the test set.
10. Evaluate the model using metrics (MSE, VI. CONCLUSION
MAE, Mean Error).
This study introduces an improved demand
11. Return the final CatBoost model. forecasting framework using the CatBoost
algorithm for online retail dataset. It showed that
effective data preparation led to better prediction
accuracy. Results demonstrated that proposed
V. RESULTS AND
approach performed better than traditional
DISCUSSIONS
models, making it a strong choice for demand
This section intends to forecasting. Future research will look into adding
conduct a comparative analysis external factors and exploring other advanced
between the results obtained from algorithms to improve accuracy further.
the existing approach and the
proposed approach.
Table 2: Relative errors for Prediction (with
References:
ranking in brackets)
Model MSE Mean MAE [1] Chen, H., Chen, Q., Wang, H., & Jiang,
Error Y. (2021). "Deep Learning for Time
RF + XGB 368743.14 0.620 9.45 Series Forecasting: A Survey". IEEE
(7) (4) (6) Transactions on Neural Networks and
XGB 324208.90 0.879 5.38 Learning Systems, 32(2), 674-692.
(6) (6) (2)
[2] Smyl, S. (2020). "A Hybrid Method of
LSTM + 319688.36 1.149 13.79
Exponential Smoothing and Recurrent
CatBoost (5) (7) (7)
Neural Networks for Time Series
CatBoost + 314124.93 1.033 13.02 Forecasting". International Journal of
Stack (3) (5) (5) Forecasting, 36(1), 75-85.
CatBoost + 89840.43 1.033 13.02
TabNet (2) (5) (5) [3] Rangapuram, S. S., et al. (2018). "Deep
CatBoost 69840.43 0.588 7.47 State Space Models for Time Series
(2) (3) (4) Forecasting". Advances in Neural
CatBoost + 59840.43 0.502 3.94 Information Processing Systems
Tuning (1) (1) (1) (NeurIPS), 31, 7785-7794.
Table 2 provides an overview of different models
for sales prediction, with CatBoost (after tuning)
outperforming others. It achieves the lowest
errors across MSE, Mean Error, and MAE,
ranking first in all metrics. This shows
[4] Bandara, K., Bergmeir, C., & Smyl, S.
(2020). "Forecasting Across Time Series
Databases Using Recurrent Neural
Networks on Groups of Similar Series: A
Clustering Approach". Expert Systems
with Applications, 140, 112896.
[5] Shih, S.-C., Sun, F.-S., & Lee, H.-T.
(2019). "Using Partial Least Squares
Regression and Artificial Neural
Networks for the Prediction of Stock
Trends". IEEE Access, 7, 57944-57954.
[6] Zhang, G., & Qi, M. (2020). "Time Series
Forecasting Using a Hybrid ARIMA and
Neural Network Model".
Neurocomputing, 50, 159-175.
[7] Borovykh, A., Bohte, S., & Oosterlee, C.
W. (2017). "Conditional Time Series
Forecasting with Convolutional Neural
Networks". arXiv preprint
arXiv:1703.04691.
[8] Li, G., Xu, L., & Wang, J. (2020).
"Demand Forecasting in Retail: A Deep
Learning Approach". Journal of Retailing
and Consumer Services, 56, 102186.
[9] Smyl, S., & Hua, J. (2019). "Machine
Learning Techniques for Sales
Forecasting: An Application to Retail
Data". Journal of Business Research, 100,
169-179.
[10] Singh, V., et al. (2018). "Big Data and
Predictive Analytics for Retail Inventory
Management". International Journal of
Retail & Distribution Management,
46(7), 674-691.