INDIAN JOURNAL OF SCIENCE AND TECHNOLOGY

RESEARCH ARTICLE

Deep Learning for the Classification

and Recognition of Medicinal Plant
Species
OPEN ACCESS Manjusha Deshmukh1 ∗
Received: 08-12-2023 1 Saraswati College of Engineering, Mumbai, Maharashtra, India
Accepted: 14-02-2024
Published: 06-03-2024
Abstract
Objective: The Indian Forest is the primary source of many medicinal
Citation: Deshmukh M (2024) Deep herbs. Medicinal plants have long been the subject of extensive inquiry
Learning for the Classification and
Recognition of Medicinal Plant
and contemplation due to their significance in human survival. Botanist
Species. Indian Journal of Science professionals must classify and identify these plants, which is a hard and time-
and Technology 17(11): 1070-1077. consuming process. As a result, a vision-based technique may help scientists
[Link]
10.17485/IJST/v17i11.3099 and the general public to identify herb plants more quickly and accurately. As a
∗ result, this study proposes a vision-based smart strategy for recognizing herb
Corresponding author.
plants that involve developing a deep learning (DL) model. Although there is a
manjushad810@[Link]
range of helpful plants, we limit ourselves to only 15 from the Kaggle database.
Funding: None Methods: Despite numerous studies, accurately identifying plant species using
Competing Interests: None automation remains a challenge. In this system, we used 15 distinct Indian
Copyright: © 2024 Deshmukh. This
plants for experiment purposes. We use a dataset of 82,500 images, with
is an open access article distributed around 5500 images of each species. We use leaf shape, texture, and color
under the terms of the Creative as the features, whether physiological or morphological. Four deep learning
Commons Attribution License, which
permits unrestricted use, classifiers named Convolutional Neural Networks (CNNs), Recurrent Neural
distribution, and reproduction in Networks (RNNs), Generative Adversarial Networks (GANs) and Multilayer
any medium, provided the original Perceptrons (MLPs) are deployed on an optimized medicinal plant leaves
author and source are credited.
dataset. Findings: The web based system can identify species by simply
Published By Indian Society for uploading an existing image. The multi-layer perceptron classifier performs
Education and Environment (iSee)
well with an accuracy of 99.01%. Convolutional neural networks have an
ISSN accuracy of 98.3%. Novelty: The novelty of our study lies in the intersection of
Print: 0974-6846
Electronic: 0974-5645 comparison of advanced deep learning techniques. While earlier research has
focused on plant identification issues, our approach contributes to the larger
fields of explainable AI, and the construction of reliable models. In real-world
applications, the background of leaves may change hence photographs with
various backgrounds are employed for instructional purposes. Models trained
on various backgrounds and can withstand alterations in the background.
Keywords: Deep learning; Generative Adversarial Networks; Multilayer
Perceptrons; Recurrent Neural Networks; Convolutional Neural Network;
Machine Vision

[Link] 1070
Deshmukh / Indian Journal of Science and Technology 2024;17(11):1070–1077

1 Introduction
A ”deep learning ” uses multi-layered artificial neural networks. These sophisticated networks are capable of changing a variety
of sectors through their predictive skills. They can independently learn complex relationships and patterns in large datasets,
excelling in tasks like the detection of images, classification, and complicated problem-solving (1,2) . Rotations flips, or color
modifications are just a few of the transformations that data augmentation algorithms employ to enrich datasets (3) .
Rapid and precise identification and classification of medicinal plant species are essential for efficient biodiversity
management and study. The focus of deep learning is on teaching multi-layered artificial neural networks to extract data
representations on their own (4) . Applications for it include the classification of species of medicinal plants. In the field of
machine vision, deep learning techniques have shown impressive results. Applications like image enhancement and detection
are widely used in industries, such as but not limited to industry, healthcare, agriculture, education etc. (5,6) . A unique strategy
based on deep learning techniques is the best answer for detecting medicinal plants in the Ardabil region. The Transfer Learning
approach uses a well-known pre-trained CNN architecture called mobile net v2 and obtained an accuracy of 98%. (7) . The Deep
Learning (DL) model consists of a CNN block for feature extraction and a classification block for classifying extracted features.
The classifier block consists of four layers: Global Average Pooling (GAP), dropout, dense, and softmax. As a result, the vision-
based system achieved more than 99.3% accuracy across all picture definitions (8) .
Researchers demonstrated how to identify Ayurvedic botanicals using machine learning as well as deep learning technology.
The project intends to improve the precision and effectiveness in recognizing medicinal herbs applied to Ayurvedic medicine
by utilizing these cutting-edge technologies, helping to preserve and apply traditional knowledge (9) . Narrowing the study’s
scope to six frequently utilized medicinal herbs enhances its focus, facilitating a comprehensive evaluation of specific species.
Employing MobileNet as the preferred deep learning model is deemed suitable, especially for real-time and mobile applications.
To augment the study’s comprehensiveness and practicality, discussing potential challenges or constraints in deploying the
model across various plants or environmental conditions would be beneficial (10) . The research suggests using a convolutional
neural network (CNN) based on the VGG-16 model with deep learning to identify therapeutic plants. The study emphasizes
the drawbacks of conventional methods due to the intricate appearance of medicinal plants. A sizable dataset, an astounding
98% detection percentage, and an emphasis on real-world utility for medical professionals are among its strong points (11) . The
literature review offers an analysis of earlier studies on the classification of medicinal plant species using deep learning. The
application of PRISMA principles and a careful selection procedure improved the study’s rigor. One important conclusion is
a worldwide data shortage (12) . With a focus on medicinal plants found in the Borneo region, the work addresses the need
for efficient plant-type recognition using computer vision and deep learning techniques. The suggested approach consists of
a smartphone application for instant detection and feedback, a knowledge library, and a deep learning model. Using a model
based on EfficientNet-B1 outperforms the baseline, achieving 87% and 84% Top-1 accuracies on public and private datasets,
respectively. Using the smartphone application, real-time testing on samples had a somewhat lower accuracy. The unique
combination of geo-mapping and crowdsourcing feedback adds to the study’s practical significance. The findings indicate a
workable path toward a field-problem-solving real-time plant-type detection system (13) . In light of the difficulties caused by
similar-looking plants and their restricted availability, this research investigates the critical role that precise plant identification
plays in Ayurvedic treatment. Deep learning techniques are becoming increasingly popular to get more accurate classification
because traditional machine learning methods have produced inconsistent results. Using a dataset of thirty species of medicinal
plants from Mendeley Data, the study uses hybrid transfer learning methods. Remarkably, the model outperforms other popular
transfer learning techniques with a test accuracy of 95.25%. In an attempt to provide more accurate and successful treatment
outcomes, the research attempts to close the knowledge gap between machine-driven identification and the experience of
Ayurvedic practitioners (14) . In a groundbreaking approach, OTAMNet is developed by incorporating Log-Gabor filters into
the DenseNet201 architecture. The model’s remarkable accuracy of 98% on the MyDataset (14) underscores its significance and
efficacy (15) . Images of medicinal plants are used from the Indian Medicinal Plant Datasets collection on the Kaggle dataset.
Without regard to environmental restrictions, the photographs are captured against a variety of backgrounds (16) .
A novel plant leaf classification model using optimal feature selection and better segmentation is presented in this study (17) .
Its preprocessing steps include RGB to greyscale conversion, median filtering, and histogram equalization. An optimized U-Net
model is for leaf segmentation. The Crow-Electric Fish Optimization (C-EFO) algorithm is used to extract and decrease features
such as shape, colour, and texture. The Enhanced Recurrent Neural Network (E-RNN) is used for classification, and it produces
better results than traditional models like k-NN, VGG16, LSTM, and RNN. The suggested approach is feasible and practical, as
demonstrated by the experimental findings on two plant leaf databases. The study offers an analysis of current studies on deep
learning-based picture classification. It includes recent developments intended to improve deep learning models’ performance
in picture categorization tasks. The review also explores the detection of potential issues and difficulties related to deep learning
technology (18) .

[Link] 1071
Deshmukh / Indian Journal of Science and Technology 2024;17(11):1070–1077

2 Methodology
2.1 Dataset Collection
Medicinal Plants dataset is available for use (16) . Furthermore, we have individually taken and pre-processed pictures of the
medicinal leaves to include in the dataset. The mobile camera is used to take pictures of the leaves, and the direction of the
leaves in the photos is not restricted. A total of 82500 photos representing 15 different types have been taken, with 5500 images
of each type.

2.2 Proposed system 1: Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) are incredibly efficient at image classification. Multiple layers in CNN models convert
input data into output. Fully Connected, Pooling, and Convolutional layers are the main parts utilized in the processing of
RGB images. Convolutional layers use local correlation in pictures to extract low-level characteristics such as corners, edges,
and patterns. Feature selection and down sampling from higher-level feature maps are achieved via pooling layers. As a result,
overfitting is avoided and network parameters and training time are reduced. Maximum and Average pooling are the two
most used varieties of pooling layers. After applying Convolutional and Pooling layers, Fully Connected layers take over. These
layers link each neuron to each other using neurons, biases, and weights. As seen in Figure 1, the architecture of the image
recognition model consists of five convolutional blocks and a classifier block. In Figure 2, an architecture diagram is displayed.
Each of the five convolutional blocks uses the output of the previous block as an input for the subsequent one. Form, colour,
and texture attributes are extracted using two convolutional layers with 3x3 kernels and a stride of 1 pixel in each convolutional
block. Activation functions like ReLU layers come after every convolutional layer. Batch normalization layers are applied after
convolutional layers to enable deeper CNNs and minimize the amount of training iterations needed. Feature maps are made
smaller by employing max-pooling layers with a stride of two and a size of 2 by 2. To avoid overfitting, a 0.1 dropout layer is
incorporated throughout every block.

Fig 1. The architecture of the CNN model for image recognition (2)

2.3 Proposed system 2: Recurrent Neural Networks (RNNs)

Recurrent neural networks are potent neural network applications utilized in image processing and speech synthesis
applications. The suggested approach presents a method for identifying leaves of plants using edge morphology and Recurrent
Neural Networks (RNNs), taking advantage of the distinctive shapes of plant leaves. This method incorporates Edge feature
estimates to increase precision and effectiveness. A kind of neural network that operates sequentially is called a recurrent neural

[Link] 1072
Deshmukh / Indian Journal of Science and Technology 2024;17(11):1070–1077

Fig 2. Architecture diagram

network. This would imply that the recurrent neural network can learn the challenges and offer a more thorough comprehension
of the issues. The reason for this is that a sequential algorithm needs memory to retain the input that has already been sent to
the machine.

Fig 3. Overview of the Proposed Methodology

2.4 Proposed system 3: Multilayer Perceptrons (MLPs)

Multilayer Perceptrons (MLPs) belong to the family of feedforward neural networks, which are made up of activation function-
equipped layers of perceptrons. Fully interconnected input and output layers, sometimes with several hidden layers, are
the defining features of MLPs. Their adaptability reaches to software creation for machine translation, picture and speech
recognition, and other fields.
In current research, a multiple-layer feed-forward network technique is used to identify medicinal leaves. Form, color,
and vein properties are taken into account. The leaf images are preprocessed and segmented, and the features—like area,
diameter, length, width, perimeter, eccentricities, solidity, major and minor axis length—are extracted from the segmented
leaf image. Furthermore, the mean, variance, skewness, and curves of the RGB spectrum characteristics are calculated. Canny
edges detection and Wiener filtering are used at different thresholds to identify vein features such as Vn1, Vn2, Vn3, and Vn4.
The identification procedure entails calculating grey-level values from the grey-level histogram. MLP has numerous layers of
nodes in a directed graph, with each layer fully connected to the next. Except for the input nodes, each node is classified as
a processing element or neuron with a nonlinear activation function. MLPs use backpropagation to train their networks. The
perceptron learns by adjusting connection weights depending on output errors vs expected results.

[Link] 1073
Deshmukh / Indian Journal of Science and Technology 2024;17(11):1070–1077

Table 1. P arameters for MLP classifier

Parameter Value
Input Layers 1
Hidden Layers 28
Neurons 18
Learning Rate 0.7
Momentum 0.5
Validation Threshold 18
Epoch 400

2.5 Proposed system 4: Generative Adversarial Networks (GANs)

The potential of GANs is to address issues associated with tiny datasets. GAN, a generative model, consists of two neural
networks—the generator and the discriminator—that are trained simultaneously. The generator creates realistic images, whereas
the discriminator distinguishes between synthetic and real images. GANs are used to generate synthetic images that resemble
authentic photographs for datasets with limited training size.

Fig 4. Generative Adversarial Networks

To generate synthetic images, we used WGAN GAN models to train distinct groups with varying amounts of medicinal plant
species. We examine classification performance between actual, fake, and mixed images produced by GANs trained on several
medicinal plant classes. The Wasserstein GAN (WGAN) stands out as a prominent model for image generation, employing a
cost function rooted in the Wasserstein distance between authentic and generated images. This innovative approach addresses
challenges such as vanishing gradient and mode collapse, making WGAN a widely acclaimed solution in the field of image
generation. WGAN applies discriminator weights within a range defined by hyperparameter c, resulting in a loss function that
accurately reflects image quality. If hyperparameter c is not properly set, the model will output low-quality images and fail to
converge. We used a model called VGG16 for the extraction of features via transfer learning. Based on the prediction provided
by the VGG16 model, we gave the accuracy, recall, precision, and F1-score.

2.6 Evaluation Measurement

A confusion matrix was utilized to demonstrate the relationship between the effectiveness of the algorithm and its true values.
A successful recognition system should produce a small number of false positives and negatives. The proposed algorithm’s
effectiveness was then assessed utilizing the accuracy evaluation, based on the ratio of correctly anticipated observations
compared to all observations, as indicated in the equation below.
True Negatives (TN) are cases in which the algorithm accurately predicts the negative class. False Positives (FP) occur when
the algorithm predicts a positive class while the true class is negative. False Negatives (FN) occur when the algorithm predicts
a negative class while the true class is positive.
TP+TN
Accuracy =
T P + FP + T P + FN

TP
Precision =
T P + FP
[Link] 1074
Deshmukh / Indian Journal of Science and Technology 2024;17(11):1070–1077

TP
Recall =
T P + FN

Precision X Recall
F1 Score = 2 X
Precision + Recall
Cohen’s Kappa is a helpful indicator of classification model performance that exceeds what would be predicted by chance.
Po − Pe
Kappa =
1 − Pe

TP+TN
Po = Observed agreement =
T P + FP + FN + T N
Pe denotes the hypothetical probability of random agreement.

Expected agreement = Pe =
(Total positive predictionns)X (Total positive actual instances)+(Total negative predictionns)X (Total negative actual instances)
(Total instances)2

3 Results and discussion

We made use of 256 × 256-pixel photos. The proposed method 1 develops a convolutional neural network-based categorization
method for medicinal plant leaves. Figures 4 and 5 illustrate the outcomes of an end-to-end computer vision system that uses a
convolutional neural network (CNN) model to identify different kinds of medicinal plants based on an image. When it comes
to handling images and videos, convolutional neural networks (CNNs) perform better than ordinary neural networks. Standard
feedforward neural networks are unable to utilize the intrinsic properties of various visual input modalities, which they may be
able to exploit.
Typical neural networks do not have an intrinsic understanding of the structure or order of the incoming input. CNNs, on
the other hand, take advantage of pictures’ inherent spatial coherence. They can significantly reduce the computing complexity
needed for image processing because of this crucial feature. CNNs can recognize and depict regional structures and trends.
We have proposed a technique that is more effective and efficient than the method that was set out. The model is tested with
different image backgrounds and in a variety of environmental settings. Different image backgrounds have an impact on how
this system performs.

Fig 5. Upload Window with Leaf Image and Plant identification (below)

The proposed method 2, leaf categorization algorithm based on Recurrent Neural Networks (RNNs) analyzed more than
5500 leaf images from 15 species in real-time. After transforming each leaf image into an edge image and creating a morphology
vector using these edges it is then saved in a file. All leaf images are painstakingly indexed. The image’s shape is concisely
represented along a single axis by this morphology array. For the input leaf image, a comparable morphology array is generated
during the search process. Next, the degree of correlation between the saved vector with the input leaves vector is measured

[Link] 1075
Deshmukh / Indian Journal of Science and Technology 2024;17(11):1070–1077

by calculating their Pearson correlation value. The system compares the input image to the stored image that has the highest
correlation value.
Proposed method 3, MLP classifier combined with the K-Means clustering technique demonstrated a method accuracy
of 99.01%. Proposed method 4, GANs lack inherent assessment metrics due to their two-network structure (generator and
discriminator). GAN models include multiple hyperparameters based on data type and selected features during training,
making performance evaluation difficult. Image augmentation with WGAN achieves high accuracy (94% and 96.77%)
throughout training.

Table 2. Classification of medicinal plant leaves Recall, ROC, Time, precision and F1 scores, TP, FP and Kappa Statistics
Classifiers Kappa True Posi- False Posi- Recall F1-Score Receiver Time (Sec) Precision
Statistics tive Rate tive Rate Operating
Characteris-
tic
MLP 0.96 0.95 0.009 0.95 0.95 0.99 0.2 0.99
CNN 0.95 0.94 0.011 0.96 0.95 0.98 0.12 0.983
RNN 0.92 0.93 0.02 0.93 0.94 0.97 0.3 0.97
WGANs 0.91 0.92 0.03 0.92 0.92 0.96 0.14 0.96

4 Conclusion
Through the use of custom features, a deep learning-based approach performs better. An improved CNN network with layers for
global average pooling, dense learning, dropout learning, and softmax learning for classification has been integrated into a real-
time vision-based system. This creative technique lowers the number of parameters while simultaneously increasing accuracy
and processing speed. This cutting-edge algorithm can classify medicinal plant photos with different degrees of detail, which
will help meet the growing need for medicinal plants in many industries. It might be better with more pictures and layers.
The mean absolute error (MAE) is pre-owned to evaluate the efficacy of the Recurrent Neural Networks-based system.
The system’s exceptional MAE of 0.08 and accuracy of 98.9 % are demonstrated by the experimental results. The system
displays excellent performance in comparison with different leaf classification models. These outcomes highlight the system’s
effectiveness and precision in categorizing plant leaves according to their morphology.

5 Future scope
Deep learning algorithms improve, and the future vision includes the creation of progressively powerful models capable
of accurately distinguishing various species, resulting in advancements in healthcare, preservation of the environment, and
pharmacology. Future research can concentrate on expanding the dataset and evaluating the model in more difficult real-world
scenarios. This may entail gathering data from various geographic regions, ecosystems, and conditions in the environment to
guarantee the model’s flexibility in multiple scenarios. Developing models to work in changing and unexpected environments,
such as the wild, is important to their effectiveness.

6 Data availability
1. [Link]
2. [Link]

References
1) Yu Y. Deep Learning Approaches for Image Classification. In: Proceedings of the 2022 6th International Conference on Electronic Information Technology
and Computer Engineering. New York, NY, USA. ACM. 2022;p. 1494–1498. Available from: [Link]
2) Xu M, Kim H, Yang J, Fuentes A, Meng Y, Yoon S, et al. Embracing limited and imperfect training datasets: opportunities and challenges in plant disease
recognition using deep learning. Frontiers in Plant Science. 2023;14:1–14. Available from: [Link]
3) Yan X, Zang Z, Luo N, Jiang Y, Li Z. New interpretable deep learning model to monitor real-time PM2.5 concentrations from satellite data. Environment
International. 2020;144:1–14. Available from: [Link]
4) Chanyal H, Yadav RK, Saini DKJ. Classification of Medicinal Plants Leaves Using Deep Learning Technique: A Review. International Journal of Intelligent
Systems and Applications in Engineering. 2022;10(4):78–87. Available from: [Link]

[Link] 1076
Deshmukh / Indian Journal of Science and Technology 2024;17(11):1070–1077

5) Zhou J, Wu Z, Jiang Z, Huang K, Guo K, Zhao S. Background selection schema on deep learning-based classification of dermatological disease. Computers
in Biology and Medicine. 2022;149:105966. Available from: [Link]
6) Su Q, Wang F, Chen D, Chen G, Li C, Wei L. Deep convolutional neural networks with ensemble learning and transfer learning for automated detection
of gastrointestinal diseases. Computers in Biology and Medicine. 2022;150:106054. Available from: [Link]
7) Abdollahi J. Identification of Medicinal Plants in Ardabil Using Deep learning : Identification of Medicinal Plants using Deep learning. In: 2022 27th
International Computer Conference, Computer Society of Iran (CSICC). IEEE. 2022;p. 1–6. Available from: [Link]
9780493.
8) Azadnia R, Al-Amidi MM, Mohammadi H, Cifci MA, Daryab A, Cavallo E. An AI Based Approach for Medicinal Plant Identification Using Deep CNN
Based on Global Average Pooling. Agronomy. 2022;12(11):1–16. Available from: [Link]
9) Rajani S, Veena MN. Ayurvedic Plants Identification based on Machine Learning and Deep Learning Technologies. In: 2022 Fourth International
Conference on Emerging Research in Electronics, Computer Science and Technology (ICERECT). IEEE. 2023;p. 1–6. Available from: [Link]
1109/ICERECT56837.2022.10060533.
10) Kavitha S, Kumar TS, Naresh E, Kalmani VH, Bamane KD, Pareek PK. Medicinal Plant Identification in Real-Time Using Deep Learning Model. SN
Computer Science. 2024;5(1):1–11. Available from: [Link]
11) Sharrab Y, Al-Fraihat D, Tarawneh M, Sharieh A. Medicinal Plants Recognition Using Deep Learning. In: 2023 International Conference on Multimedia
Computing, Networking and Applications (MCNA). IEEE. 2023;p. 116–122. Available from: [Link]
12) Mulugeta AK, Sharma DP, Mesfin AH. Deep learning for medicinal plant species classification and recognition: a systematic review. Frontiers in Plant
Science. 2023;14:1–18. Available from: [Link]
13) Malik OA, Ismail N, Hussein BR, Yahya U. Automated Real-Time Identification of Medicinal Plants Species in Natural Environment Using Deep Learning
Models—A Case Study from Borneo Region. Plants. 2022;11(15):1–17. Available from: [Link]
14) Ghosh S, Singh A, Kumar S. Identification of medicinal plant using hybrid transfer learning technique. Indonesian Journal of Electrical Engineering and
Computer Science. 2023;31(3):1605–1615. Available from: [Link]
15) Oppong SO, Twum F, Hayfron-Acquah JB, Missah YM. A Novel Computer Vision Model for Medicinal Plant Identification Using Log-Gabor Filters and
Deep Learning Algorithms. Computational Intelligence and Neuroscience. 2022;2022:1–21. Available from: [Link]
16) Pushpa BR, Rani S. Indian Medicinal Leaves Image Datasets. 2023. Available from: [Link]
17) Dudi B, Rajesh V. A computer aided plant leaf classification based on optimal feature selection and enhanced recurrent neural network. Journal of
Experimental & Theoretical Artificial Intelligence. 2023;35(7):1001–1035. Available from: [Link]
18) Yu Y. Deep Learning Approaches for Image Classification. In: Proceedings of the 2022 6th International Conference on Electronic Information Technology
and Computer Engineering. New York, NY, USA. ACM. 2022;p. 1494–1498. Available from: [Link]

[Link] 1077