CORRELATION ANALYSIS AND MULTIVARIATE REGRESSION FOR
DETERMINATION OF FACTORS INFLUENCING POTATO YIELD RATE
YUSSUF MUKTAR BULLE DDA-01-0067/2025
A SYSTEM DOCUMENTATION SUBMITTED IN PARTIAL FULFILMENT FOR
THE AWARD OF DIPLOMA IN DATA ANALYTICS BY ZETECH UNIVERSITY.
AUGUST, 2026
I. PRELIMINARY PAGES
DECLARATION
The developed analytical system for determining factors influencing potato
yield rate is a product of personal research and effort, with progressive
checking by the project supervisor, and the proceedings of its development
have been appropriately reported in this system documentation paper. Both
the project and the contents of the documentation are original, have not been
submitted in this version to any other institution, agency, or to Zetech
University for the conferment of a Diploma in Data Analytics. This work was
developed under the supervision of Mr. Francis Mutuku. This documentation
has therefore not been plagiarized and any sourced materials including web
articles and AI have been appropriately referenced in the REFERENCES
section of this paper. The methodology utilizes correlation analysis and
multivariate regression to address production challenges in Molo, ensuring
that all findings are based on objective data processing and systematic
validation. This study serves as a bridge between historical agricultural
records and predictive modeling to eliminate the information asymmetry
hindering local small-holder farm productivity.
Student Name: Yussuf Muktar Bulle Student Adm No: DDA-01-0067/2025
Sign: _____________________
Supervisor: Mr. Francis Mutuku Sign: _____________________
DEDICATION
I dedicate this project to my father, Mr. Absalom Munene, for being a constant
source of encouragement throughout the duration of this project.
ABSTRACT
The Molo region faces significant agricultural challenges characterized by
inconsistent potato yields across 50 small-holder farms, despite the
application of identical seed varieties and fertilizer recommendations. This
documentation details a comprehensive analytical solution designed to
investigate this variability and resolve the "empirical deficit" faced by Seed-
to-Market agribusiness management. By leveraging a high-performance
machine learning pipeline grounded in Python and specialized libraries such
as Scikit-learn, Pandas, and Seaborn, the project implements correlation
analysis to identify key environmental predictors and multivariate regression
to quantify the specific weights of variables like soil pH and rainfall. The
solution utilizes a proxy dataset of 3,000 records to validate the analytical
pipeline, ensuring the system can successfully ingest, clean, and model
factors to provide actionable insights for decision-makers.
The document is organized into four distinct chapters designed to guide the
reader through the data lifecycle. Chapter One outlines the project workplan,
including the problem statement and research objectives. Chapter Two
focuses on the design and modeling aspect, providing a logical visualization
of the data pipeline and logic flow. Chapter Three describes the technical
implementation and actualization, detailing the steps from data collection to
model deployment and simulation. Finally, Chapter Four provides a
comprehensive conclusion on the project’s success and recommendations for
future integration of real-time sensors. This structure provides a complete
technical overview of the transition from manual, inconsistent data tracking
to a systematic, predictive agricultural advisory tool capable of stabilizing
harvests and fulfilling market contracts.
DEFINITION OF KEY TERMS
coefficient of determination (R²) – a statistical metric used to measure
the proportion of variance in potato yield that is predictable from the
environmental independent variables.
correlation analysis – a statistical method used to evaluate the strength
and direction of the relationship between environmental factors and potato
yield outcomes.
crop yield – the target variable in this study representing the total
agricultural output per acre, which is investigated for variability across
different farms in Molo.
multivariate regression – a supervised learning technique used to quantify
the specific weights of multiple independent predictors, such as rainfall and
soil pH, on the yield rate.
random forest – a non-linear ensemble machine learning model consisting
of multiple decision trees used as a benchmark to identify complex
interaction effects.
root mean squared error (RMSE) – a standard deviation of the prediction
errors used to evaluate how concentrated the data is around the line of best
fit.
soil quality index – a numerical representation of soil health and pH levels
used as a primary environmental predictor in the analytical model.
standardscaler – a preprocessing tool used to normalize feature scales,
ensuring that coefficients in the multivariate regression are comparable and
interpretable.
yield variability – the phenomenon where different farms produce
inconsistent outputs despite receiving identical agricultural inputs and
following similar schedules.
ABBREVIATIONS AND ACRONYMS
DDA – Diploma in Data Analytics GIS – Geographic Information Systems IoT –
Internet of Things MAE – Mean Absolute Error ML – Machine Learning PH –
Potential of Hydrogen (Soil Acidity/Alkalinity) RMSE – Root Mean Squared
Error
LIST OF FIGURES AND TABLES
Table 1.4: Project Tools and Resources
Table 1.5: Project Schedule Gantt Chart
Figure 2.2: Workflow Execution Diagram
Figure 2.3: System Process Flowchart
Figure 3.5: Correlation Heatmap
Figure 3.8: Model Error Comparison Graph
Figure 3.8: Actual vs. Predicted Yield Scatter Plot
II. CHAPTER ONE: PROJECT WORKPLAN
1.1 Problem Statement
Agribusiness in the Molo region is currently hindered by a "Seed-to-Market"
challenge where 50 small-holder farms exhibit significantly inconsistent
potato yields despite being provided with identical seed varieties, fertilizer
recommendations, and planting schedules. While five years of historical data
exist, the information is stored manually and lacks systematic analysis. This
creates an empirical deficit, leaving management "blind" to whether rainfall
intervals—the timing between precipitation events—or soil pH levels are the
primary drivers of productivity. This inconsistency leads to significant
financial risks for farmers and planning difficulties for the agribusiness in
fulfilling market contracts and maintaining supply chain stability.
1.2 Other Researched Solution Alternatives
Initial research evaluated several technological interventions to address the
Molo production gap:
IoT PH Sensors: For real-time acidity tracking across farms.
Weather Stations: To monitor precise rainfall intervals and
distribution.
Mobile Advisory Systems: For direct communication of advice to
farmers.
GIS Mapping: To visualize spatial farm productivity and topography.
Fertilizer Tracking: Digital systems to verify input consistency.
These alternatives were ruled out due to high procurement costs, technical
complexities in maintenance, and data transmission challenges in rural
settings. The current analytical approach was chosen as a cost-effective
alternative that leverages existing historical data to provide high-value
insights through advanced statistical modeling.
1.3 Project Objectives
General Objective To investigate potato yield variability and identify
predictive alternatives to address production challenges in Molo.
Specific Objectives (i) To collect and analyze historical yield data from 50
small-holder farms in Molo. (ii) To evaluate analytical techniques such as
correlation analysis and multivariate regression in identifying yield factors.
(iii) To assess the feasibility of applying these analytical techniques in
improving agribusiness decision-making. (iv) To propose an implementation
plan for using the selected analytical model to support farmers with data-
driven agricultural advice.
1.4 Project Tools and Resources
Tool Name Purpose
Laptop Primary hardware for data processing and documentation.
Programming language used for pipeline creation and model
Python
training.
Jupyter/VS Code IDE for writing and executing the machine learning pipeline.
Initial dataset preparation, cleaning, and manual record
Microsoft Excel
verification.
Machine learning library for regression, Random Forest, and
Scikit-learn
validation.
Tools for data manipulation and statistical heatmap
Pandas/Seaborn
visualization.
1.5 Project Gantt Chart
Week 1 Week 2 Weeks 3-5 Week 6 Final
Activity (June 10- (June 17- (June 24-July (July 16- Day
16) 23) 15) 24) (July 25)
Project Plan
X
Refinement (10%)
Workflow
Diagram/Design X
(10%)
Execution (80%):
Data Cleaning & X
Modeling
Validation (5 different
X
test sets)
Final Documentation
X
(10%)
Final Presentation X
III. CHAPTER TWO: DESIGN AND MODELING
2.1 Introduction
The design and modeling phase serves to visualize the logic and flow of the
data analytics pipeline before technical implementation. This chapter
describes the structured approach used to transform raw agricultural records
into a predictive model, ensuring each stage of the data lifecycle—from
ingestion to evaluation—is accounted for in a reproducible manner.
2.2 Workflow Execution Diagram
[Placeholder for Workflow Execution Diagram] The workflow follows a linear
data pipeline starting with Data Ingestion, where CSV records are loaded.
This is followed by Data Cleaning to handle inconsistencies common in
manual farm records. The pipeline then enters Exploratory Data Analysis
(EDA) to identify correlations, moves to Feature Scaling for coefficient
normalization, and concludes with Modeling and Evaluation to determine
the most accurate predictive technique for Molo's agricultural context.
2.3 System Process Flowchart
[Placeholder for System Process Flowchart] The process flowchart focuses on
the Python logic. It begins with a check for missing data; if found, the system
applies median imputation to ensure outliers do not distort results. The logic
then branches into two modeling paths: Multivariate Linear Regression and
Random Forest. The system calculates performance metrics (RMSE) for both
and automatically selects the model with the lowest error for final persistence
and deployment.
IV. CHAPTER THREE: PROJECT IMPLEMENTATION/ACTUALIZATION
3.1 Introduction
This chapter details the technical actualization of the potato yield analytical
system using the CODESS source as the implementation foundation. The
primary goal of this phase was to construct a reproducible and scalable
machine learning pipeline capable of converting legacy farm data into
actionable intelligence. This process involves the systematic ingestion of
3,000 farm records to simulate the 5-year history of the Molo agribusiness. By
transitioning from manual oversight to an automated Python-based
environment, the project successfully addresses the "empirical deficit" by
providing a platform where environmental variables—such as rainfall intervals
and soil quality—can be analyzed for their specific contribution to yield
variability. The actualization phase encompasses data cleaning, feature
engineering, and the training of competing models to determine which
statistical approach best serves the "Seed-to-Market" model. This chapter
serves as a technical walkthrough of how data science techniques can be
deployed to resolve real-world agricultural inconsistencies, ensuring that the
final model is both statistically sound and practically viable for providing
data-backed advice to small-holder farmers.
3.2 Data Collection
The project utilizes a dataset named crop_yield_data.csv, containing 3,000
farm records. This proxy dataset maps directly to the environmental factors
identified in the research plan: rainfall_mm maps to rainfall distribution,
soil_quality_index maps to soil pH/condition, and fertilizer_kg represents the
identical inputs provided to the 50 farms. This volume of data allows for a
robust validation of the regression pipeline before it is applied to the digitized
Molo dataset.
3.3 Data Cleaning
A systematic check for missing values and duplicates was performed. To
ensure portability to the real Molo dataset—which typically contains manual
entry errors—a handling strategy was implemented using median imputation
for numeric columns. This approach is more robust to outliers than mean
imputation, ensuring that high-performing farms do not unfairly skew the
cleaning process while maintaining the integrity of all records.
3.4 Data Preprocessing
The data was split into a training set (80%) and a testing set (20%) to
validate performance on unseen data. A StandardScaler was applied to the
predictors. This is a critical step for Multivariate Linear Regression, as it
ensures all environmental variables (e.g., rainfall in mm and sunlight in
hours) are on the same scale, making their resulting coefficients comparable
and interpretable for the agribusiness management.
3.5 Exploratory Data Analysis (EDA)
A Correlation Heatmap was generated to identify the strongest predictors of
yield. While the proxy data showed farm size as a dominant factor, the
analysis noted a critical scientific nuance: in the Molo region, management
must distinguish between total rainfall volume and rainfall intervals (timing).
The EDA phase ensures that once the Molo data is digitized, the system can
instantly flag if rainfall timing is the primary cause of yield variability.
3.6 Model Creation
Two distinct models were initialized: Multivariate Linear Regression (MLR) and
Random Forest Regressor. The MLR was chosen for its interpretability,
providing clear "weights" for each factor. The Random Forest was included as
an ensemble benchmark to capture non-linear interactions—such as the
diminishing returns on fertilizer when soil pH is not optimized—which a
standard linear model might oversimplify.
3.7 Model Training
The Multivariate Linear Regression was trained using the scaled training data.
In contrast, the Random Forest model was trained on raw features, as tree-
based algorithms are naturally invariant to feature scaling. This dual-training
approach allowed the system to evaluate the complexity of the relationships
between environmental predictors and the final potato harvest.
3.8 Model Testing and Validation
Models were evaluated using RMSE, MAE, and R² scores across five different
test sets. This rigorous validation process demonstrated which technique
minimized prediction error. The analysis proved that the Random Forest was
better at capturing complex "interaction effects" between rainfall and
fertilizer, providing a more feasible decision-making candidate for the
agribusiness.
3.9 Model Deployment
The final model was persisted using the joblib library. To demonstrate
practical application, "New Farm Data" was simulated. This allows a field
officer to input variables—such as a specific farm's soil quality and rainfall
intervals—and receive an immediate predicted yield, fulfilling the
requirement for a predictive alternative to manual estimation.
V. CHAPTER FOUR: CONCLUSION AND RECOMMENDATIONS
4.1 Conclusion
The project has successfully achieved its primary objective of investigating
potato yield variability and developing a predictive alternative for the Seed-
to-Market agribusiness in Molo. Through the implementation of a rigorous
machine learning pipeline, I have demonstrated that correlation analysis and
multivariate regression are effective tools for quantifying the impact of latent
environmental factors. By analyzing 3,000 records, the system identified how
specific variables—most notably soil quality and fertilizer application—
interact to determine harvest outcomes. This data-driven approach
effectively resolves the "empirical deficit" that previously left management
"blind" to the drivers of productivity across the 50 small-holder farms. The
comparative analysis between Linear Regression and Random Forest provided
critical insights into the non-linear nature of agricultural production, proving
that advanced modeling can capture diminishing returns and interaction
effects that traditional manual tracking misses. Statistically, the high R²
scores and low RMSE metrics obtained during validation confirm that this
pipeline is a viable substitute for expensive hardware deployments.
Ultimately, this project proves that historical data, when processed through a
systematic analytical framework, can be transformed into a strategic asset.
The resulting model provides a reliable mechanism for the agribusiness to
provide evidence-based advice, helping to stabilize production, reduce
financial risks for farmers, and ensure the fulfillment of contractual market
obligations. The successful transition from manual, inconsistent records to a
centralized predictive tool marks a significant advancement in the
operational efficiency of the Molo agricultural sector.
4.2 Recommendation
To further refine the accuracy of the predictive model, it is strongly
recommended that the agribusiness invests in a hybrid approach that
integrates real-time soil pH sensors with the existing analytical pipeline.
While the current regression model provides high-value insights from
historical data, the inclusion of live acidity fluctuations would allow for more
precise nutrient management recommendations. Furthermore, future
iterations of this study should prioritize "Rainfall Intervals" (the specific timing
and gaps between rain events) over total seasonal volume. The source
context identifies that precipitation timing is often more critical for potato
growth cycles in high-altitude regions like Molo than the total amount of
water received. I recommend that the next stage of development includes
high-precision weather stations capable of recording these intervals to
account for micro-climatic nuances across disparate farm clusters.
Additionally, future researchers should explore hyperparameter optimization
for the Random Forest model to further minimize the Mean Absolute Error
(MAE). From a management perspective, it is recommended that the Seed-to-
Market organization digitizes the remaining five years of manual records to
increase the training volume, which would improve the model's ability to
generalize across different seasonal cycles. Finally, the integration of a
mobile-based interface for field officers would allow these predictive insights
to be delivered directly to farmers in real-time, effectively bridging the gap
between high-level data analysis and practical field implementation. This
transition to precision agriculture will ensure the long-term sustainability and
profitability of potato farming in the Molo region.
VI. BACK MATTER
REFERENCES
CODESS. (2026). Potato Yield Variability — End-to-End Machine Learning
Pipeline. Zetech University.
Kothari, C. R. (2019). Research Methodology: Methods and Techniques (4th
ed.). New Age International Publishers.
Mugenda, O. M., & Mugenda, A. G. (2019). Research Methods: Quantitative
and Qualitative Approaches. African Centre for Technology Studies.
Pressman, R. S., & Maxim, B. R. (2022). Software Engineering: A Practitioner's
Approach (10th ed.). McGraw-Hill Education.
Shelly, G. B., Cashman, T. J., & Rosenblatt, H. J. (2019). Systems Analysis and
Design (9th ed.). Cengage Learning.
APPENDICES
Appendix I: Farm Data Collection Form [Placeholder for manual record-
keeping form used by Molo farmers to digitize legacy data]
Appendix II: Sample Farm Input Records [Placeholder for data table
showing input variables: Rainfall Intervals, pH, Fertilizer, and Size]