Machine Learning in Crop Yield Forecasting
Machine Learning in Crop Yield Forecasting
ABSTRACT
The utilization of machine learning has become increasingly important in the prediction of crop yields
for facilitating decisions regarding crop cultivation and management during the growing season.
Numerous machine learning and data mining algorithms have been developed to support research in
crop yield forecasting. In this study, a systematic literature review (SLR) was conducted on research
published between 2016 and 2021 to investigate the use of machine learning in crop yield forecasting. A
total of 261 relevant studies were identified from five electronic databases, out of which 15 studies were
selected for further analysis based on inclusion and exclusion criteria. The selected studies were
thoroughly examined, and their methods and features were analyzed, to provide suggestions for future
research. The results showed that evapotranspiration, temperature, precipitation, and soil type were the
most commonly used features in crop yield forecasting, while RMSE, MSE, MAE, and R2 were the most
commonly used evaluation parameters. The challenges include selecting appropriate input variables,
handling missing data and outliers, and capturing non-linear relationships between variables. The
authors discuss various techniques such as feature selection, regularization, imputation, non-linear
machine learning, data preprocessing, and data augmentation to address these challenges. The Support
Vector Machine, Linear Regression, Artificial Neural Network (ANN), and Long-Short Term Memory
(LSTM) were identified as the most commonly used algorithms in these models.
KEYWORDS: Crop yield forecasting, Data mining, Deep learning, Systematic review
[Link]
54
Crop yield forecasting using machine learning techniques - a systematic literature review
55
Crop yield forecasting using machine learning techniques - a systematic literature review
advises looking broadly to find other parameters that Naftali Slob, Cagatay Catal and Ayalew Kassahun
contribute to crop yield (Elavarasan et al., 2018). conducted a review and provided an overview of
what has been done on the use of ML in the dairy
Beulah conducted a survey of various data mining sector. (Naftali Slob. et al., 2020).
techniques used to predict crop yields and concluded
that crop yield forecasts could be solved using data Thomas van Klompenburg, Ayalew Kassahun and
mining methods (Beulah, 2019). Cagatay Catal conducted a review of Crop yield
prediction using machine learning to investigate to
Table 1: Summary of selected literature
what extent deep learning algorithms were used for
Reference Goal Research questions
crop yield prediction. (Van Klompenburg, T. et al.,
2020).
Van Review of Crop RQ1- Which
Klompenburg, yield prediction machine learning
T. et al., 2020 using machine algorithms have been
According to this SLR, the above are the important
learning to used in the literature review articles presented in this section.
investigate to what for crop yield
extent deep prediction?
learning RQ2- Which features 2. METHODOLOGY
algorithms were have been used in
used for crop yield literature for crop
prediction. yield prediction In this research process, we followed the guidelines
using machine suggested by Kitchenham et al., (2007).
learning?
RQ3- Which
evaluation
parameters and 2.1. Planning the review
evaluation
approaches have
been used in
Initially, the research inquiries are delineated, and
literature for crop subsequently, relevant studies are chosen by utilizing
yield prediction? databases. Specifically, the ACM Digital Library,
RQ4- What are the
challenges in the IEEE Xplore, ResearchGate, SpringerLink, and
field of crop yield ScienceDirect databases were employed for the
prediction using
machine learning? present study. Once the pertinent research is
identified, it undergoes a filtering process, and is
Naftali Slob. Provide an RQ1 - What kind of
et al., 2020 overview of what problems are solved assessed using quality standards. The pertinent data
has been done on using ML and what from the selected studies are then extracted and
the use of ML in ML tasks are these
the dairy sector. problems mapped synthesized in accordance with the research
into? questions.
RQ2 - What
independent and
dependent variables 2.1.1. Research questions
are used to build the
ML models?
RQ3 - What ML The objective of this systematic literature review is to
algorithms are
applied for the obtain a comprehensive understanding of the
models? published research on crop yield forecasting within
RQ4 - Which
evaluation
the field of machine learning and data mining. To
parameters and achieve this goal, various dimensions have been
which evaluation analyzed in the reviewed studies. The present study
approaches are used?
RQ5 - Which outlines five research questions (RQs) to guide the
algorithm performs SLR analysis.
the best?
RQ6 - What are the
challenges reported RQ1: When, where, and who have published studies?
in the identified
articles? RQ2: What are the algorithms that have been used so
56
Crop yield forecasting using machine learning techniques - a systematic literature review
far for prediction? plan included defining the search space, including
RQ3: What are the features/factors used in the electronic databases and printed processes, as
literature to predict crop yields using machine provided in Table 2. The studies were first taken from
learning? the above electronic databases and then analyzed to
RQ4: What are the evaluation parameters and identify other meaningful studies through
evaluation approaches used in the literature to investigative search (snowballing).
predict crop yields?
RQ5: What are the challenges in the field? The inclusion and exclusion criteria were then
applied to studies obtained, involving a different
Table 2: Search sources number of research as described in section 2.1.4
57
Crop yield forecasting using machine learning techniques - a systematic literature review
and December 2021. To answer the five research questions, data from the
selected studies were extracted and synthesized. The
Exclusion Criteria: selected studies that passed the inclusion and
exclusion criteria are presented in Table 6. During the
EC1: Publication is not written in English. data synthesis, all the extracted data were
EC2: Publication that is a duplicate or already consolidated and synthesized and the research
retrieved from another database. questions were answered accordingly. The results are
EC3: Full text of the publication is not available. set out in Section 3.
EC4: Studies that do not meet inclusion criteria.
EC5: Publication is a review/survey paper. Table 6: Selected publications
ID Retrieved Reference Title Year
from
2.2 Conducting the review
1 IEEE Xplore #12 Rice crop yield 2016
prediction in
In this section, we obtain information from our search India using
findings and related sources and databases. support vector
machines
2 IEEE Xplore #13 Rice crop yield 2016
2.2.1 Study Search and Selection prediction using
artificial neural
networks
By following the search strategy (previously 3 SpringerLink #14 Predicting Early 2017
Crop
explained in Section 2.12), the selected electronic Production by
databases were searched and the studies were Analyzing Prior
retrieved. In this original search, we retrieved 261 Environment
Factors
studies as shown in Table 5. After applying three
4 IEEE Xplore #15 Rice yield 2017
Inclusion criteria, only 72 studies remained for prediction
further analysis, after which fourteen studies were model using
data mining
selected for further analysis by applying all five 5 IEEE Xplore #16 Prediction of 2018
criteria for exclusion. In Table 5, we show the Crop
Production in
number of papers obtained initially and the India Using
number of papers remaining after applying the Data Mining
Techniques
selection criteria. 6 SpringerLink #17 Yield 2018
Forecasting of
Table 5: Distribution of papers based on the Spring Maize
databases Using Remote
Sensing and
Database # of # of papers Percenta Crop Modeling
initially after ge of in Faisalabad-
retrieved inclusion and Papers Punjab Pakistan
papers exclusion (%) 7 ScienceDirect #18 Artificial 2018
criteria intelligence
approach for the
ACM Digital prediction of
22 1 7
library Robusta coffee
yield using soil
IEEE Xplore 88 4 29 fertility
properties
ResearchGate 57 3 14
8 SpringerLink #19 Smart Farming 2018
System: Crop
SpringerLink 54 4 29
Yield Prediction
ScienceDirect 40 3 21 Using
Regression
Techniques
Total 261 15 100
9 ScienceDirect #20 Design of an 2019
integrated
58
Crop yield forecasting using machine learning techniques - a systematic literature review
climatic
assessment
indicator (ICAI) 2.2.3. Quality Assessment Criteria and Screening
for wheat Procedures
production: A
case study in In order to assess the systematic review at hand, the
Jiangsu
Province, China following quality criteria were employed:
10 SpringerLink #21 Sugarcane 2019
Yield Grade Examination of the number of citations
Prediction incorporated in each study sourced from existing
Using Random
Forest with research.
Forward Verification of publication venue, ensuring the
Feature
Selection and inclusion of papers published in recognized
Hyper- conferences or journals.
parameter
Tuning Analysis of the research inquiry that the authors
11 ResearchGate #22 Prediction of 2020 have concentrated on.
Rice Yield via Analysis of the conclusions drawn by the
Stacked LSTM
researchers in response to the research questions.
12 ACM Digital #23 Prediction of 2020
library Soybean Yield Scrutiny of adherence to standard reference
using Self- formatting.
normalizing
Neural Consideration of studies published between
Networks 2016 and 2021, prioritizing the most recent
13 ResearchGate #24 Prediction of 2021 research.
rice yield based
on LSTM long Prioritization of publications that possess well-
Short and defined discussions, results, and conclusions.
long memory
network 3. RESULTS
14 ScienceDirect #25 Crop Yield 2021
Forecasting
using Data
In this section, we describe the findings of our review
Mining of our research questions.
15 ResearchGate #26 Review on Crop 2021
Prediction 3.1. Overview of studies
Using Deep
Learning
Techniques Selected publications are shown in Table 6. The table
shows the publication year, title, and other details of
these publications.
2.2.2. Data extraction and synthesis
Number of publications
Based on the guidelines provided by Kitchenham et
al., (2007), we defined a data extraction process to 5
identify relevant information from 15 included 4
preliminary studies related to our research questions. 3
Our data extraction process includes the following:
First, we set up a form to report the ideas, concepts, 2
contributions, and findings of each of the 15 studies. 1
0
The following data were extracted from each 2016 2017 2018 2019 2020 2021
publication: (i) Research Topic; (ii) Database; (iii)
Year of publication; (iv) DOI; (v) Keywords; (vi) Figure 1: Distribution of the selected publications
Research Objectives Defined; (vii) Novelty of the per year.
study; (viii) Data Sample Details; (ix) Methodology;
(x) Country/location of the analysis; Figure 1 shows the number of publications that have
59
Crop yield forecasting using machine learning techniques - a systematic literature review
been published during the last six years from among All features we were able to extract are shown in
the selected publications. Table 8.
Table 8: All features/factors used. RMSE (Root mean square error), R2 (R-squared),
Feature/Factor # of times used MSE (Mean square error) and MAE (Mean absolute
Precipitation 10 error) appear to be the most commonly used
evaluation parameters.
Temperature 9
3.6. (RQ5) What are the challenges in the field? evapotranspiration, soil type, precipitation, and
temperature, as shown in Table 8. Additionally, some
To address the fifth research question (RQ5), studies have explored the use of trace elements such
publications were read to see if any issues or as magnesium, potassium, sulfur, and calcium as
improvements were stated for future models. In features. Notably, the types of features utilized in
several studies, data inadequacy (undersized data) these studies differ, with temperature being measured
has been cited as a problem. Data plays an important as an average in some experiments and as maximum
role in the machine learning process. One of the or minimum values in others.
important issues faced here is the lack of good quality
data. Dirty and useless data can make the whole In relation to RQ4, an investigation was conducted
process extremely tedious. A proposed improvement into the evaluation parameters utilized in several
would be to integrate more data sources. selected papers, as outlined in Table 9. The
evaluation parameters included RMSE (Root Mean
4. DISCUSSION Square Error), R2 (R-squared), MAE (Mean
Absolute Error), MSE (Mean Square Error), and
The analysis of our Selected 15 Studies underscores MAE (Mean Absolute Error). The majority of the
the significance of the authors' geographical location, models achieved high accuracy values for their
as they represent various regions around the world. evaluation parameters, indicating that these models
Despite our efforts to conduct an extensive search, it were capable of producing accurate predictions.
is possible that some valuable publications may have
been missed, and the use of additional similar terms Regarding RQ5, the identified articles provided clear
could have yielded new studies. Nonetheless, the statements on the challenges encountered.
search engine's ability to generate a considerable
number of publications indicates a sufficiently One of the studies focused on rice crop yield
comprehensive search. prediction in India using support vector machines
(SVMs) (Gandhi et al., 2016). The study faced the
In the context of our research question RQ1, which challenge of selecting appropriate input variables for
pertains to the temporal, geographical, and author- the SVM model. To overcome this challenge, the
related aspects of the selected crop forecasting papers authors used a feature selection method to identify the
related to Machine Learning, we have chosen papers most relevant input variables for the model.
published between 2016 and 2021 from various
reputable sources. Our systematic literature review Another study on rice crop yield prediction used
(SLR) specifically focuses on the 15 selected papers, artificial neural networks (ANNs) (Gandhi et al.,
which we have found to be relevant and informative 2016). The study encountered the challenge of over
for our study. fitting, which occurs when the model performs well
on the training data but poorly on the test data. To
In reference to RQ2, Table 7 demonstrates that a overcome this challenge, the authors used a
majority of experiments utilize a standard algorithm regularization technique, which randomly drops out
as a benchmark to evaluate the effectiveness of a some neurons during training to prevent over fitting.
proposed algorithm. Furthermore, recent studies have
incorporated Deep Learning (DL) techniques, a In the study on rice yield prediction using data mining
subset of Machine Learning, for predicting crop (Dey et al., 2017), the authors faced the challenge of
yield. Long Short-Term Memory (LSTM) is one such dealing with missing data. To overcome this
DL technique that has shown promising results. challenge, they used the k-nearest neighbour (k-NN)
imputation method to fill in the missing values.
Regarding RQ3, the grouping of key features/factors
can aid in visualization. The commonly employed Jambekar et al. aimed to predict crop production in
factors for crop yield prediction include India using data mining techniques. The authors
61
Crop yield forecasting using machine learning techniques - a systematic literature review
encountered the challenge of dealing with a large the challenge of limited training data. To address this
number of input variables, which can lead to poor challenge, they applied data augmentation techniques
model performance. To overcome this challenge, to increase the size of the training dataset.
they used a wrapper feature selection method, which
selects the best subset of input variables based on Kamath et al. (2021) used data mining techniques to
their performance on a validation dataset (Jambekar predict crop yields, and their approach relied on the
et al., 2018). availability of historical data. They collected and
processed data on weather conditions, soil quality,
Kouadio et al. aimed to predict the yield of robusta and crop yields to build decision trees that can be
coffee using soil fertility properties. The authors used to predict future yields.
faced the challenge of dealing with non-linear
relationships between the input variables and the However, it is possible that there may be additional
target variable. To overcome this challenge, they challenges that were not mentioned in the literature.
used a non-linear machine learning technique called
random forest, which can capture non-linear 5. CONCLUSIONS
relationships between variables (Kouadio et al.,
2018). This article presents a formal and original systematic
review of the literature on crop yield forecasting
Shah et al. focused on crop yield prediction using using machine learning methods. The review
regression techniques. The authors encountered the followed established guidelines for conducting
challenge of dealing with missing data and outliers in systematic literature reviews (Kitchenham et al.,
the dataset. To overcome this challenge, they used a (2007)) to identify and categorize all relevant
data preprocessing method to fill in the missing literature on the topic. After conducting a thorough
values and remove the outliers from the dataset (Shah search of a well-known electronic research database,
et al., 2018). a total of 261 original papers were identified.
Through a rigorous filtration process, 15 relevant
Xu et al. (2019) addressed the challenge “lack of papers were selected for analysis and evaluation.
accurate and reliable data on weather conditions, soil
quality, and crop growth stages” by designing an The selected papers were evaluated for their quality
integrated climatic assessment indicator (ICAI) that and analyzed based on the following research
integrates various climatic factors, such as questions: (i) Who are the authors, when and where
temperature, precipitation, and sunshine duration, to were the studies published? (ii) What are the
assess the impact of weather conditions on wheat algorithms that have been used for prediction? (iii)
production. What are the features/factors used in the literature to
predict crop yields using machine learning? (iv) What
Another challenge is the complexity of crop growth are the evaluation parameters and approaches used in
processes, which require models that can capture the the literature to predict crop yields? (v) What are the
nonlinear relationships between input factors and challenges in the field?
crop yields. Meng et al. (2020) used stacked Long
Short-Term Memory (LSTM) networks to predict The findings of this research provide valuable
rice yields. The stacked LSTM model was able to insights and future directions for industry and
capture the nonlinear relationships between the research professionals in the field of crop yield
weather factors and rice yield. forecasting using machine learning. The study
reveals that various features are used in different
Furthermore, the performance of machine learning research papers, depending on the scope and data
models depends on the quality and quantity of data availability of the selected publications. The choice
available for training and testing. Mo et al. (2021) of features is influenced by the availability of data
used LSTM networks to predict rice yields but faced and the purpose of the research. It was also observed
62
Crop yield forecasting using machine learning techniques - a systematic literature review
that models with more features do not necessarily machine learning models, and the availability of
provide better performance for yield forecasts. historical data for data mining approaches. However,
Several algorithms have been used in different there may be additional challenges not covered in the
studies, and while no definitive conclusion can be literature.
drawn as to the best model, some machine learning
models have been found to perform better than The findings of this study provide valuable insights
others. and future directions for industry and research
professionals, and pave the way for further research
The results of this review indicate that neural in the field. The authors recommend the exploration
networks and support vector machines are the most of deep learning algorithms, specifically LSTM-
commonly applied algorithms in the existing based models, for improving crop yield forecasting.
literature. Therefore, the authors recommend that
future studies explore the potential of deep learning 6. REFERENCES
algorithms, specifically LSTM-based models, for
predicting crop yields. The study concludes by Ahmad, I., Saeed, U., Fahad, M., Ullah, A., Habib ur
proposing further research on the development of the Rahman, M., Ahmad, A., & Judge, J. (2018). Yield
problem of crop yield forecasting. Forecasting of Spring Maize Using Remote Sensing
and Crop Modeling in Faisalabad-Punjab Pakistan. J.
In conclusion, our systematic literature review on of the Indian Society of Remote Sensing.
crop forecasting papers related to Machine Learning [Link]
has shed light on several key aspects, including the
temporal, geographical, and author-related aspects of Al-Zuabi, I. M., Jafar, A., & Aljoumaa, K. (2019).
the selected papers. We have found that the use of Predicting customer’s gender and age depending on
Deep Learning techniques such as Long Short-Term mobile phone data. J. of Big Data, 6(1).
Memory (LSTM) has shown promising results in [Link]
predicting crop yield. Additionally, the commonly
employed factors for crop yield prediction include Beulah, R., 2019. A survey on different data mining
evapotranspiration, soil type, precipitation, and techniques for crop yield prediction. Int. J. Comput.
temperature. Sci. Eng. 7 (1), pp.738–744
[Link]
Evaluation parameters such as RMSE, R2, and MAE
were used to assess the accuracy of the models, which Charoen-Ung, P., & Mittrapiyanuruk, P. (2018).
produced high accuracy values. Despite the Sugarcane Yield Grade Prediction Using Random
challenges encountered in crop forecasting, further Forest with Forward Feature Selection and Hyper-
data collection and exploration may lead to even Parameter Tuning. Advances in Intelligent Systems
better accuracy in predicting crop yield. and Computing, pp.33–42.
[Link]
The article discusses various challenges and solutions
encountered in crop yield prediction using machine Conway, J. A., Brown, L. M. J., Veck, N. J.,
learning techniques. Challenges include selecting Wielogorski, A., & Borgeaud, M. (1991). A model-
appropriate input variables, dealing with missing data based system for crop classification from radar
and outliers, and capturing non-linear relationships imagery. [Proceedings] IGARSS’91 Remote
between variables. Solutions include using feature Sensing: Global Monitoring for Earth Management.
selection and regularization techniques, imputation [Link]
methods, non-linear machine learning techniques,
data preprocessing, and data augmentation. The Dey, U. K., Masud, A. H., & Uddin, M. N. (2017).
article also emphasizes the importance of having Rice yield prediction model using data mining. 2017
accurate and reliable data for training and testing Int. Conf. on Electrical, Computer and
63
Crop yield forecasting using machine learning techniques - a systematic literature review
64
Crop yield forecasting using machine learning techniques - a systematic literature review
Technologies.
[Link]
Xu, X., Gao, P., Zhu, X., Guo, W., Ding, J., Li, C.,
… Wu, X. (2019). Design of an integrated climatic
assessment indicator (ICAI) for wheat production: A
case study in Jiangsu Province, China. Ecological
Indicators, 101, pp.943–953.
[Link]
65