Joint Plant Species-Disease Prediction: A Comparative Evaluation of Classification Approaches and Novel Conditional Multi-Task Learning Frameworks

Abstract

This study investigates automated plant species and disease classification using image-based machine learning models. Specifically, our analysis is based on the idea that disease prediction is inherently dependent on plant species. We propose two novel conditional multi-task learning (CMTL) frameworks, while comparing them to one-stage classification, two-stage classification, and prior fusion-based CMTL approaches. We utilize a diverse dataset of 40,000 images spanning 9 plant species, 20 unique disease categories, and 33 joint species-disease classes. All frameworks were trained across multiple random seeds under a unified protocol using fine-tuned transfer learning backbones and were evaluated using species-level, disease-level, and joint prediction metrics. The best one-stage and two-stage frameworks achieved joint Top-1 accuracies of 94.35% and 93.38%, respectively. Among the reimplementations of previously proposed CMTL frameworks, the highest joint Top-1 accuracy was 93.09%. Our proposed Conditional Gating (Mixture-of-Experts) and Species-Conditioned Attention frameworks both achieved a higher joint Top-1 accuracy of 97.98%, while also achieving higher true conjoint F1-scores commonly underperforming classes. These results suggest that explicitly modeling the dependency between plant species and diseases can improve joint prediction performance within the evaluated dataset setting. Despite high predictive performance, the proposed frameworks come with the trade-off of increased computational cost compared to other evaluated frameworks. Furthermore, the findings should be interpreted in light of several limitations. Evaluation was conducted on curated, publicly available data with moderate class imbalance mitigation. Future work should focus on external validation on additional datasets investigating causes behind certain underperforming classes.

Keywords: Plant, Species, Disease, Joint Species-Disease Prediction, Machine Learning (ML), Transfer Learning, One-Stage, Two-Stage, Multi-Task Learning (MTL), Conditional Multi-Task Learning (CMTL), Computer Vision

Introduction

Plant diseases pose a significant threat to food security and economic stability, causing an estimated annual loss in global food production of 10-16% and $220 billion worth of economic damage¹. Detecting plant diseases earlier could minimize spread and thus prevent such large-scale losses. Traditional disease identification methods relied on manual disease diagnosis by experienced farmers or plant pathologists, but this is time-consuming and often unavailable in remote regions². Technological advancements have made machine learning and computer vision techniques feasible and practical alternatives for plant disease detection by shortening the diagnosis period³. Due to the widespread availability of digital cameras, smartphones, and drones, plant images can be readily captured at large scales and leveraged for automated disease classification using machine learning models⁴.

Many existing machine learning approaches can be grouped into one-stage, two-stage, and multi-task learning (MTL) frameworks. One-stage methods treat species-disease combinations as single classes, two-stage methods perform the two tasks sequentially, and MTL jointly predicts both outputs. However, MTL treats the outputs as parallel tasks, where species and disease predictions are separate. This motivates conditional multi-task learning (CMTL).

In this study, joint plant species-disease prediction is treated as a multi-output classification problem. Given an input image , the objective is to predict both a species label () and a disease label (), where belongs to the set of plant species labels and belongs to the set of disease labels. This dependency can be expressed as:

   

where denotes the shared feature representation extracted from the input image. This formulation reflects the hierarchical relationship between the tasks, where disease prediction is conditioned on species information. Accordingly, the learning objective is to jointly learn species and disease predictions while leveraging dependencies to improve overall prediction performance.

Although prior studies have compared one-stage, two-stage, and CMTL approaches, most have focused on a small set of fusion-based conditioning mechanisms. As a result, there is limited understanding of how alternative conditioning strategies perform under a unified dataset and training protocol. To address this gap, we propose two novel CMTL architectures that model species-disease dependencies using Conditional Gating and Species-Conditioned Attention. We then evaluate them against one-stage, two-stage, and existing fusion-based CMTL approaches.

Related Work

Research on automated crop disease detection has evolved significantly with advances in deep learning, transitioning from single-task classification models to more structured multi-task learning (MTL) frameworks. Table 1 summarizes representative studies that are most relevant to the methodological progression from single-task classification to conditional multi-task learning (CMTL). This overview highlights that prior work largely focuses on either single-task classification, shared multi-task prediction, or fixed hierarchical pipelines, without explicitly modeling conditional dependencies between plant species and disease classes. This gap motivates the approach proposed in this study.

Study	Model Type / Approach	Primary Dataset Images	Number of Species	Number of Diseases
Agarwal et al. (2020)	Lightweight, 8-layer CNN	18,000	1	9
Paymode and Malode (2022)	Transfer Learning VGG16 CNN (Implemented twice for 2 species)	18,483 (4,062 for Grape and 14,421 for Tomato)	2	12
Fu et al. (2025)	PMJDM (Multi-Task Learning Framework)	26,073 (with annotations for each)	5	28
Mohanty et al. (2016)	One-Stage Joint Classification	54,306	14	26
Costa et al. (2019)	Two-Stage Hierarchical CNN	23,988	3	13
Lee et al. (2021)	CMTL (FFS, FFC, and FiLM)	54,305	311	289

Early work focused mainly on applying convolutional neural networks (CNNs) or transfer learning techniques on leaf images for plant disease classification. For example, an 8-layer CNN on tomato disease images and a pre-trained backbone on grapes both achieved high accuracy of 98.40%^5,6. However, these approaches are limited to single-task disease classification, requiring species-specific datasets and separately trained models that increase computational complexity.

To address this, MTL frameworks have been introduced, in which a single model jointly predicts the species and disease. The PlantDisease Multi-task Joint Detection Model (PMJDM), for example, introduced a dual-branch architecture with shared feature extraction and task-specific heads, achieving competitive performance (61.83% mAP50) on multi-label agricultural datasets⁷. However, this approach relies on detailed multi-task annotations for each image, increasing dataset preparation costs. Other MTL-based studies have also explored attention mechanisms and Vision Transformer backbones for joint classification and localization tasks^8,9. While these methods improve feature sharing, they do not explicitly condition disease predictions on species.

In parallel, hierarchical and structured prediction methods have been explored. One landmark work applied a one-stage classification approach on the PlantVillage dataset for 38 joint species-disease classes and achieved a high accuracy of 99.35% across multiple crops¹⁰. A two-stage classification approach decomposes the problem into sequential predictions, where species is predicted first, followed by disease classification, and this has achieved high results such as an average accuracy of 98.33%^11,12. While they reduce disease prediction complexity, these approaches require multiple models and separate training pipelines, which increases computational complexity by requiring multiple trained components. Importantly, they also do not enable end-to-end learning of species-disease interactions within a single unified model.

More general hierarchical learning techniques such as hierarchical softmax and label embedding have also been proposed in computer vision and large-scale classification tasks^13,14. These methods improve efficiency or capture semantic relationships between labels, but they primarily model static label structures rather than conditional dependencies between related prediction tasks. As a result, their direct application to plant species-disease prediction remains limited, particularly in tasks requiring explicit conditioning of disease prediction on species identity.

Recent agricultural AI research has explored transformer-based architectures, self-supervised learning (SSL), and federated learning. Vision Transformers and attention-based models improve feature representation for plant disease recognition by capturing long-range dependencies in images¹⁵. Similarly, SSL approaches leverage large-scale unlabeled agricultural datasets to learn robust feature embeddings prior to fine-tuning¹⁶. Federated learning enables distributed training across decentralized agricultural data sources while preserving privacy¹⁷. Although beneficial for representation learning, data governance, and distributed model training, these methods are complementary to CMTL frameworks, which is a key focus of this study.

Finally, CMTL has emerged as a promising direction that explicitly models the dependency between the two outputs. Unlike standard MTL, CMTL introduces conditional mechanisms that allow disease prediction to be influenced by species information, thereby explicitly modeling dependencies between the two prediction tasks. Lee et al. compared one-stage, two-stage, and CMTL approaches using fusion-based conditioning mechanisms (FFS, FFC, and FiLM)¹⁸. Rather than introducing CMTL itself, the present study extends that line of work by proposing and evaluating alternative conditioning mechanisms (gating- and attention-based) under a different dataset configuration and unified training protocol.

Research Scope and Contributions

The scope of this study is limited to image-based plant species and disease classification using a subset of the PlantVillage¹⁰ dataset containing 9 plant species and 20 disease categories. We limit class imbalance mitigation to only class-weighted loss functions, and we do not incorporate temporal, environmental, or genomic information. Although per-class results are provided, investigation of the causes of underperforming classes is left for future work. The study focuses on controlled dataset-based evaluation of MTL frameworks for joint plant species-disease prediction.

In this study, we propose two novel CMTL frameworks: Conditional Gating (Mixture-of-Experts) and Species-Conditioned Attention. We present a comparative evaluation of these frameworks alongside one-stage, two-stage, and reimplemented prior CMTL approaches using various pre-trained transfer learning backbones on a unified dataset.

Methods

In this study, multiple machine learning frameworks are trained and evaluated on the same dataset to enable comparison of model performance across different classification approaches.

Dataset

This study utilizes the PlantVillage dataset, a publicly available collection of labeled plant images for plant species and disease classification originally introduced by Mohanty et al.¹⁰. PlantVillage is a widely used benchmark dataset with expert-verified disease annotations. The dataset was downloaded on March 1, 2026 under the CC BY-SA 3.0 license from the official GitHub repository. Only the colored image subset was used in this work.

Because this study focuses on joint plant species-disease prediction, which inherently relies on intra-species class variability, species containing only a single disease class or only a “Healthy” class were excluded; this includes Orange, Blueberry, Raspberry, Soybean, and Squash. After this filtering, the final dataset consists of 40,000 images spanning 9 plant species, 20 unique disease categories (including “Healthy”), and 33 joint species-disease classes. Figures 1 and 2 show image distributions across species and joint species-disease classes, respectively. Our dataset exhibits moderate class imbalance, with class sizes ranging from 152 images to 5,357 and a median class size of 1,052 images. Also, although acquisition metadata such as device type, geographic origin, and specific lighting conditions are not provided in the original dataset, all images were captured under varying conditions, resulting in differences in lighting and background. They originally varied in resolution, but Figure 3 shows representative sample images after resizing to 224×224.

Data Preprocessing and Partitioning

We designed our preprocessing steps to ensure compatibility with our selected pre-trained backbones. For most experiments, images were resized to a consistent 224×224, while alternative resolutions were used when required by specific architectures. Pixel intensities were subsequently normalized using backbone-specific preprocessing procedures. Furthermore, no data augmentation was applied during model training to isolate performance to model design rather than augmentation strategies. Also, our dataset contains 40,000 images spanning 33 species-disease classes, providing a sufficiently large dataset.

To reduce potential train-test leakage, a perceptual hash (pHash) analysis was performed. This identified 39,978 unique image groups, meaning there are 22 duplicate images (0.055% of the data). Dataset spitting was therefore performed at the duplicate-group level rather than at the individual-image level. Duplicate groups were stratified according to their joint species-disease labels and split into training (70%), validation (15%), and testing (15%) subsets using a random seed (42). This method ensures that visually identical images do not span multiple splits while maintaining joint class distributions for valid training and evaluation.

Descriptions of the Classification Approaches

In this study, we evaluate multiple approaches for joint plant species-disease prediction: one-stage, two-stage, and various conditional multi-task learning (CMTL) frameworks.

One-Stage Classification Approach

First, we used the one-stage classification approach shown in Figure 4, in which one model was trained on all 33 joint species-disease classes. We leveraged VGG16, ResNet50, EfficientNetB0, and MobileNetV2 to fine-tune their learned features for species and disease classification^19,20,21,22. These models were selected due to their widespread use in plant disease detection literature^23,24.

For each backbone, the original ImageNet²⁵ classification layer was replaced with a new head comprising global average pooling, dropout, and a 33-neuron softmax output layer.

Two-Stage Classification Approach

Next, as depicted in Figure 5, we used a two-stage classification approach in which plant species and disease classification are performed separately and sequentially. The same backbones and fine-tuning procedure used in the one-stage approach were applied to all models in this approach.

The first stage performs species classification using transfer learning. The second stage uses one of nine species-specific models, each trained only on diseases valid for that species. For inference, the first-stage species prediction determines which second-stage model is deployed.

Conditional Multi-Task Learning (CMTL) Approaches

To explicitly model relationships between plant species and diseases, we implemented several conditional multi-task learning (CMTL) approaches. Unlike the one-stage and two-stage approaches, CMTL jointly learns species and disease classification within a shared network while allowing species-related information to condition disease prediction.

Three frameworks were reimplemented from prior work by Lee et al.¹⁸ and fine-tuned to our dataset: Feature Fusion Sum (FFS), Feature Fusion Cascade (FFC), and Feature-wise Linear Modulation (FiLM). These architectures used the pre-trained InceptionV3 backbone²⁶ to remain consistent with the original framework. In addition, two novel CMTL architectures were proposed in this study: a Conditional Gating Mixture-of-Experts framework and a Species-Conditioned Attention framework. Both proposed models employed a pre-trained VGG16 backbone¹⁹ due to its strong overall performance among the one-stage and two-stage approaches.

All CMTL models consisted of a shared feature extraction backbone followed by two task-specific branches: a species classification branch and a disease classification branch. The disease branch was conditioned using species-related information through various mechanisms.

Let denote the backbone feature representation extracted from an input image . The shared representation was projected into task-specific embeddings:

   

   

where represents the species embedding and represents the disease embedding, each containing 512 hidden units. Species classification was performed directly from , whereas disease classification depended on the framework-specific conditioning mechanism.

Reimplementation of Existing CMTL Architectures

Figure 6 shows the three CMTL fusion mechanisms proposed by Lee et al.¹⁸: Feature Fusion Sum (FFS), Feature Fusion Cascade (FFC), and Feature-wise Linear Modulation (FiLM).

In all cases, species and disease embeddings are combined to produce a species-conditioned representation that is subsequently used for disease classification. All three architectures were reimplemented using the same progressive fine-tuning and hyperparameter selection strategy used throughout this study to ensure consistency. Based on the formulations proposed by Lee et al.¹⁸, the conditional fusion mechanisms are defined as follows.

The FFS architecture performs conditioning through additive feature fusion. Species and disease embeddings are first projected into a common latent space:

   

   

A conditioned representation is then computed using element-wise addition and linear activation:

   

The second reimplemented architecture utilizes cascade fusion (FFC), in which disease prediction is conditioned through feature concatenation rather than direct addition. The species and disease embeddings are first projected into a common feature space and concatenated:

   

where [,] denotes vector concatenation and is a learned projection layer. Unlike FFS, the FFC model learns species-disease relationships through the concatenated representation.

The FiLM model employs a slightly different form of conditioning between the species and disease tasks, in which species-conditioned scaling () and shifting () parameters are learned from the species embedding:

   

   

and applied to a disease representation, which is computed as:

   

The resulting conditioned representation is then computed as

   

where denotes element-wise multiplication¹⁸. Thus, disease features can be selectively amplified or suppressed based on species information^18,27.

For all three proposed architectures, Lee et al.¹⁸ also explored the addition of a residual connection (-res) in their analysis. Residual connections often improve gradient flow and stabilize trainingin deep neural networks²⁰. Because the three highest-performing architectures were FFS-res, FiLM-no-res (without a residual connection), and FFC-res, those same variants were reimplemented in this study. For the FFS and FFC models, the conditioned representation was combined with the original disease embedding through an identical mapping¹⁸:

   

where the residual connection reintroduces the original disease representation prior to disease classification. Consistent with the prior study’s results, the FiLM framework in this study does not have a residual connection applied.

Proposed CMTL Framework 1: Conditional Gating (MoE)

Our first proposed CMTL framework is the Conditional Gating model incorporating a Mixture-of-Experts (MoE) architecture, as shown in Figure 7.

Rather than relying on a single disease classifier, we train multiple expert subnetworks to learn complementary disease representations. The motivation for this design originates from prior MoE research showing that expert specialization can improve modeling of heterogeneous prediction tasks^28,29,30. The gating network receives both the species embedding and species prediction probabilities. First, the gating logits () are computed as:

   

where denotes the species prediction probabilities. A temperature-scaled softmax is then applied to obtain the expert-selection weights:

   

where T denotes the temperature parameter. Temperature scaling moderates the sharpness of the expert-selection probabilities, with larger values producing smoother routing distributions^29,31.

The final disease representation is computed as a weighted combination of expert outputs:

   

where denotes the number of experts. During development, multiple expert counts were evaluated: . We evaluated a logarithmic scale of powers of two to efficiently span a wide range of model capacities while minimizing hyperparameter optimization overhead. Based on validation disease loss, the final configuration of four experts was selected.

An entropy regularization term is also incorporated into the gating network and is minimized during optimization^32,33,34:

   

which encourages lower-entropy gating distributions and more selective expert routing. This promotes stronger expert specialization during training by penalizing high-entropy gate outputs.

Proposed CMTL Framework 2: Species-Conditioned Attention (SCA)

Our second proposed CMTL architecture is the Species-Conditioned Attention (SCA) model shown in Figure 8. It aims to allow species information to dynamically determine which disease features should receive greater emphasis during classification, motivated by the observation that certain visual disease symptoms may be more informative for specific plant species.

The species and disease embeddings were first projected into a shared latent space and represented as two feature tokens:

   

   

These tokens were concatenated into a two-token sequence,

   

which was then processed using multi-head attention (MHA) with four attention heads³⁵:

   

A residual connection and layer normalization were subsequently applied to preserve disease information that may otherwise be weakened during attention-based modulation^20,36:

   

The final conditioned disease representation was obtained from the updated disease token:

   

where denotes the disease-token component of the normalized attention output.

Model Training and Fine-Tuning

Certain training settings were held constant across all frameworks. All models were initialized using backbones pre-trained on ImageNet²⁵ and subsequently fine-tuned on our dataset. Training was performed using the Adam optimizer with early stopping based on validation loss and ReduceLROnPlateau learning-rate scheduling. All experiments were trained for a maximum of 20 epochs on an NVIDIA GeForce RTX 4060 Laptop GPU. To quantify variability resulting from random initialization and stochastic optimization, every framework was trained using five random seeds (0, 1, 42, 123, and 492). Dropout was applied within the classification heads of the networks, and early stopping (with a patience of 3 epochs) served as an additional regularization mechanism. Weight decay was not employed.

Hyperparameters were selected through a framework-specific grid search. We evaluated learning rates of 10^-3, 10^-4, and 10^-5, batch sizes of 16 and 32, and dropout rates of 0.1, 0.2, and 0.3. To balance feature preservation and domain adaptation, we evaluated fine-tuning configurations with 2, 10, 20, or 40 trainable backbone layers depending on the architecture. Pre-trained backbones were also varied for the one-stage and two-stage approaches. Final model configurations were selected based on the lowest validation disease loss (for CMTL) or lowest overall validation loss (for one-stage and two-stage), maintaining consistency with the training optimization objectives.

Table 2 summarizes the final selected hyperparameter configurations and backbone choices for the best-performing version of each framework. These configurations were subsequently retrained across the five random seeds and were used for all reported results.

Framework	Backbone	Learning Rate	Trainable Layers	Batch Size	Dropout
One-Stage	VGG16	10-5	2	16	0.3
Two-Stage	VGG16	10-5	2	16	0.3
CMTL – FFS	InceptionV3	10-5	40	16	0.3
CMTL – FFC	InceptionV3	10-5	40	16	0.3
CMTL – FiLM	InceptionV3	10-5	40	16	0.3
CMTL – Gating	VGG16	10-5	20	16	0.3
CMTL – SCA	VGG16	10-5	20	16	0.3

For the Conditional Gating (MoE) architecture, additional framework-specific hyperparameters were explored. We evaluated expert counts of 2, 4, 8, 16, and 32, entropy regularization coefficients (or entropy weights) of 0.001, 0.005, and 0.01, and gating temperatures of 0.5, 1, and 1.5. Based on validation disease loss, the final configuration used four experts, an entropy regularization coefficient of 0.01, and a temperature of 1.5. A validation summary across expert-count configurations is provided as supplementary material.

Class imbalance was addressed through class-weighted loss functions. For one-stage classification, class weights were computed using the 33 joint species-disease classes. For two-stage classification, class weights were computed separately for the species classifier and for each species-specific disease classifier. For the conditional multi-task learning (CMTL) models, separate class-weight vectors were applied to the species and disease prediction tasks.

For all CMTL architectures, species prediction and disease prediction were learned jointly using a multi-task objective:

   

where and are sparse categorical cross-entropy losses for species and disease classification, and and are the loss weights controlling the tasks’ relative importance. We set = =1 in order to be consistent with Lee et al.¹⁸ and treat both tasks as primary objectives.

For the proposed Conditional Gating (MoE) architecture, an additional entropy regularization term was incorporated:

   

where denotes the entropy regularization coefficient (also referred to as the entropy weight). Because the entropy term is added to the training objective and is minimized during optimization, it encourages lower-entropy gating distributions. This promotes more selective expert routing and increased expert specialization.

where γ denotes the entropy regularization coefficient (also referred to as the entropy weight). Because the entropy term is added to the training objective and is minimized during optimization, it encourages lower-entropy gating distributions. This promotes more selective expert routing and increased expert specialization.

Model Evaluation

To enable direct comparison across approaches, all models were evaluated using the same species-level, disease-level, and joint prediction metrics. Per-class precision, recall, and F1-scores were computed for all species, diseases, and joint species-disease classes. Additionally, we have computed the per-joint-label F1-scores, disease-only F1-scores conditional on correct species, disease-only F1-scores regardless of species, and true conjoint F1-scores. This is provided as supplementary material.

For species- and disease-level classification, precision, recall, weighted F1-score, macro F1-score, balanced accuracy, and Top-1 accuracy were computed. Weighted F1-score accounts for class frequency and provides an overall estimate of performance across the dataset. Macro F1-score assigns equal importance to each class and is therefore more sensitive to performance on classes with fewer samples. Balanced accuracy was additionally included to account for unequal class distributions. Because weighted averages can obscure poor performance on these underrepresented classes, both weighted and macro F1-scores are reported for all approaches.

For joint plant species-disease prediction, performance was evaluated using joint Top-1 accuracy. Joint Top-1 accuracy (or conjoint Top-1 accuracy) requires a model to correctly predict both the plant species and the disease label simultaneously for a given sample; an individual prediction is counted as correct only if both target labels match the ground truth. Additionally, joint-label performance was evaluated using weighted and macro F1-scores computed across the 33 joint species-disease classes. This ensured that all approaches were evaluated using a common prediction space.

To quantify uncertainty arising from random initialization, metrics are reported as mean ± 95% confidence interval across five independent training runs. Detailed per-seed and per-class results, as well as statistical testing outputs, are provided as supplementary material. Statistical significance between the five CMTL frameworks was assessed using pairwise McNemar tests on conjoint Top-1 predictions³⁷. For each model pair, accuracy differences ( =A-B) were averaged across the five random seeds, and p-values were combined. Holm-Bonferroni correction³⁸ was applied as control for multiple comparisons within each task.

To compare computational costs, we recorded total trainable parameters, average training time (across the seeds), and average inference time per image. For the two-stage framework, parameters and training times are summed across its ten models. Its inference time equals the species classifier’s inference time plus the mean inference time of the nine disease classifiers, as only the predicted species’ disease classifier model is used.

Ethical Considerations

This study does not involve human participants or personal data. All images are from a publicly available dataset and were leveraged in accordance with the licensing and usage guidelines.

Results

This section evaluates all classification approaches explored in this study. These include a one-stage approach, a two-stage approach, three reimplemented CMTL frameworks (Feature Fusion Sum, FFS; Feature Fusion Cascade, FFC; and Feature-wise Linear Modulation, FiLM), and two novel CMTL frameworks (Conditional Gating and Species-Conditioned Attention, SCA).

Overall Predictive Performance Across Frameworks

Table 3 summarizes the predictive performance of all evaluated frameworks. This allows clear comparison at the joint-, species-, and disease- levels.

Framework	One-Stage	Two-Stage	FFS	FFC	FiLM	Gating	SCA
Joint Top-1 Accuracy	0.9435 ± 0.0046	0.9338 ± 0.0055	0.9309 ± 0.0075	0.9281 ± 0.0086	0.9244 ± 0.0100	0.9798 ± 0.0066	0.9798 ± 0.0074
Joint Balanced Accuracy	0.9423 ± 0.0046	0.9153 ± 0.0060	0.9111 ± 0.0118	0.9098 ± 0.0107	0.9040 ± 0.0128	0.9728 ± 0.0083	0.9740 ± 0.0099
Joint Macro F1	0.9408 ± 0.0052	0.6075 ± 0.0210	0.4880 ± 0.0317	0.4749 ± 0.0317	0.4868 ± 0.0092	0.6821 ± 0.1287	0.6623 ± 0.0607
Joint Weighted F1	0.9439 ± 0.0043	0.9407 ± 0.0047	0.9384 ± 0.0074	0.9358 ± 0.0077	0.9325 ± 0.0104	0.9819 ± 0.0053	0.9817 ± 0.0069
Species Macro F1	0.9853 ± 0.0016	0.9671 ± 0.0079	0.9782 ± 0.0093	0.9794 ± 0.0026	0.9755 ± 0.0069	0.9907 ± 0.0063	0.9899 ± 0.0074
Species Weighted F1	0.9884 ± 0.0014	0.9705 ± 0.0068	0.9814 ± 0.0087	0.9828 ± 0.0022	0.9792 ± 0.0067	0.9924 ± 0.0052	0.9913 ± 0.0067
Disease Macro F1	0.9334 ± 0.0044	0.9491 ± 0.0043	0.9384 ± 0.0051	0.9352 ± 0.0081	0.9333 ± 0.0054	0.9826 ± 0.0018	0.9840 ± 0.0020
Disease Weighted F1	0.9479 ± 0.0039	0.9620 ± 0.0022	0.9471 ± 0.0046	0.9430 ± 0.0086	0.9417 ± 0.0051	0.9856 ± 0.0024	0.9874 ± 0.0018
ECE	0.0083 ± 0.0030	0.0115 ± 0.0056	0.0136 ± 0.0137	0.0110 ± 0.0054	0.0110 ± 0.0031	0.0051 ± 0.0024	0.0055 ± 0.0032

The one-stage framework outperformed the two-stage framework in joint Top-1 and balanced accuracy, but the two-stage approach showed better disease-level performance, as seen in the disease weighted F1-score. Among the three reimplementations of prior CMTL architectures, FFS achieved the highest conjoint Top-1 accuracy, followed by FFC. Species-level and disease-level metrics were generally similar across the three fusion-based architectures, although FFS consistently achieved the strongest overall performance within this group. The proposed Conditional Gating and SCA architectures achieved the strongest overall results across nearly all evaluation metrics. Conditional Gating achieved a conjoint Top-1 accuracy of 97.98% ± 0.66%, while SCA achieved 97.98% ± 0.74%. These values exceeded those of the one-stage, two-stage, and fusion-based CMTL approaches. Similar trends were observed for joint balanced accuracy and joint weighted F1-scores. Additionally, the gap between the joint macro F1-scores and joint weighted F1-scores across all frameworks suggests the presence of class imbalance despite our measures taken to mitigate it.

At the species level, both proposed architectures achieved weighted F1-scores above 0.99, exceeding all other frameworks. Disease-level performance followed a similar trend; SCA achieved the highest disease weighted F1-score (0.9874 ± 0.0018) and disease macro F1-score (0.9840 ± 0.0020), while Conditional Gating achieved nearly identical results. Regarding calibration performance, the Expected Calibration Error (ECE), was also strong across all frameworks, but the Conditional Gating and SCA frameworks achieved the lowest ECE values (0.0051 ± 0.0024 and 0.0055 ± 0.0032, respectively). This indicates that their predicted probabilities were generally well aligned with observed outcomes in comparison to the other frameworks. Overall, the proposed CMTL architectures achieved the strongest predictive performance and the best calibration characteristics.

Computational Cost Analysis

Table 4 compares the computational characteristics of all frameworks for analyzing complexity.

Framework	One-Stage	Two-Stage	FFS	FFC	FiLM	Gating	SCA
Backbone	VGG16	VGG16	InceptionV3	InceptionV3	InceptionV3	VGG16	VGG16
Total Trainable Parameters	2,376,737	23,619,626	8,712,093	9,236,893	8,974,749	17,376,645	16,831,837
Per-Image Inference Time (ms)	29.03	33.24	48.28	23.1	23.06	8.13	46.91
Total Training Time (hrs)	0.7	1.44	0.87	0.79	0.77	3.92	4.24

The one-stage framework required the fewest trainable parameters and the shortest average training time, while the two-stage framework reflected much higher computational cost due to its reliance on 10 separately trained models. Among the reimplemented CMTL frameworks, the trainable parameter counts ranged from about 8.7 to 9.2 million. FFC and FiLM had the lowest inference times among all evaluated frameworks, requiring about 23 milliseconds per image, while FFS required 48.28. The proposed Conditional Gating and SCA frameworks required substantially more trainable parameters and longer training times than the other CMTL approaches. This introduces a trade-off for the proposed frameworks: despite high predictive performance across nearly all metrics, they also come with increased computational cost.

Statistical Significance Test

Figure 9 presents pairwise comparisons among the five CMTL frameworks using McNemar’s test³⁷. Asterisks indicate statistical significance after correction (=0.05).

Among the reimplemented architectures, FFS generally achieved slightly higher performance than FFC and FiLM, although the magnitude of these differences was substantially smaller than the improvements associated with the proposed architectures. Across joint species-disease prediction, both proposed frameworks demonstrated accuracy improvements of approximately 0.05 over FFS, FFC, and FiLM. These differences were consistently identified as statistically significant across all five seed runs. In contrast, the comparison between Conditional Gating and SCA revealed only negligible performance differences. Although some pairwise comparisons reached statistical significance, the observed accuracy differences were extremely small relative to the differences observed between the proposed and reimplemented frameworks.

Commonly Underperforming Classes

To better understand performance at the disease and joint levels, we have additionally computed the per-joint-label F1-scores, disease-only F1-scores conditional on correct species, disease-only F1-scores regardless of species, and true conjoint F1-scores. This is provided as supplementary material. Here, Figure 10 presents true conjoint F1-scores for the lowest-performing joint species-disease classes after averaging across the five random seeds.

Several Tomato disease categories consistently appeared as the most challenging classes for all evaluated frameworks. This could be attributed to the large number of disease classes within Tomato species or subtle visual differences that are difficult for the models to distinguish between. The one-stage and two-stage frameworks generally exhibited the lowest performance on these classes, and the reimplemented CMTL frameworks often produced moderate improvements. On the other hand, the proposed Conditional Gating and SCA frameworks achieved the highest F1-scores across nearly all underperforming joint classes. For example, although these classes still performed worse than others, both approaches improved performance on Tomato’s Target Spot and Tomato Mosaic Virus relative to the other frameworks.

Discussion

This study evaluated various classification approaches for joint plant species-disease prediction under a unified experimental protocol, including two proposed conditional multi-task learning (CMTL) frameworks. The results provide insight into the advantages and trade-offs of explicitly modeling species-disease dependencies.

Key Findings and Interpretation

The results indicate that explicitly modeling species–disease dependencies can improve joint prediction performance relative to both one-stage and two-stage classification approaches. While the one-stage framework achieved strong overall performance and the two-stage framework produced competitive disease-level results, both approaches treat the relationship between species and disease either implicitly or through a fixed prediction pipeline. In contrast, the conditional multi-task learning frameworks incorporate species information directly into disease prediction, allowing the model to learn task interactions within a unified architecture.

Among the evaluated CMTL approaches, the proposed Conditional Gating and Species-Conditioned Attention models consistently achieved the strongest overall results, reaching conjoint Top-1 accuracies of approximately 97.98%, compared with 94.35% for the best one-stage model and 93.38% for the best two-stage model. They also achieved the highest disease-level F1-scores and the lowest calibration errors. These findings suggest that conditioning mechanisms based on dynamic expert selection or attention may capture species-specific disease characteristics more effectively than previously proposed fusion-based approaches.

Performance differences were particularly evident among the most challenging joint species–disease classes, many of which involved visually similar Tomato diseases. Although all frameworks exhibited worse performance on these classes, the proposed architectures generally showed greater robustness. This observation supports the hypothesis that explicit conditioning can help reduce ambiguity when disease symptoms share overlapping visual characteristics.

The computational analysis highlights an important trade-off. The proposed architectures achieved the highest predictive performance but required substantially more parameters and longer training times than simpler approaches. Therefore, the choice of framework may depend on whether predictive performance or computational efficiency is the priority.

Limitations and Future Work

Although this study is a comprehensive evaluation of several classification approaches for joint plant species-disease prediction, several limitations should be considered when interpreting the results. First, the models were evaluated only using stratified splits derived from a widely used, public dataset. Although the dataset includes 40,000 images captured under varying conditions, external validation using more test sets and a wider variety of species (and corresponding diseases) in future studies would allow for assessment on how well the models generalize to unseen acquisition settings, geographic regions, and field environments in the real world.

Second, the causes of poor performance for certain disease classes remain only partially understood. Future work may investigate feature-level confusion between visually similar diseases, quantify inter-class similarity, and analyze attention patterns or learned representations to better understand whether errors arise from dataset characteristics, label definitions, or limitations of the model architectures themselves.

Additionally, moderate class imbalance remained present despite the use of class-weighted losses, as reflected by the gap between joint macro and weighted F1-scores. Future studies could explore alternative mitigation strategies. It may also consider incorporation of environmental, temporal, or genomic data to provide additional context for disease prediction. Finally, this study focused on comparing complete classification frameworks rather than isolating the contribution of individual conditioning components. Future work may include targeted ablation studies to further quantify the contribution of specific mechanisms.

Conclusions

This study compared one-stage classification, two-stage classification, and conditional multi-task learning (CMTL) approaches for joint plant species-disease prediction using a filtered subset of PlantVillage¹⁰ containing 40,000 images, 9 plant species, 20 disease categories, and 33 joint species–disease classes. All frameworks were evaluated under a common experimental protocol. The results show that explicitly modeling the conditional relationship between species and disease improves predictive performance. The proposed Conditional Gating and Species-Conditioned Attention frameworks achieved the strongest overall results, outperforming both conventional classification approaches and previously proposed fusion-based CMTL frameworks across joint, species-level, and disease-level evaluation metrics.

More broadly, the findings suggest that conditional learning can be beneficial in multi-task problems where one prediction task naturally constrains another. However, the results were obtained on a curated benchmark dataset without external validation and should therefore be interpreted within that context. Future work should evaluate these architectures on larger and more diverse field-collected datasets and further investigate the sources of error among visually similar disease classes.

Overall, this study demonstrates the potential of CMTL for joint plant species-disease prediction and provides preliminary evidence that gating- and attention-based conditioning mechanisms are promising alternatives to traditional fusion-based approaches.

Code Availability and Supplemental Reports

The accompanying supplementary repository provides training scripts for each framework, dataset metadata, split mappings, per-seed and per-class evaluation metrics, a validation summary across different Conditional Gating models (varying expert counts), and the full McNemar test outputs. This is located at https://github.com/ameya-kella/JointPlantSpeciesDiseasePredCMTL.

Acknowledgements

Although this paper uses the “authorial we” common in academia, all of the research, analysis, and writing were done solely by the author. I would like to thank Abdulla Kerimov (PhD in Geophysics from Stanford, Applied Scientist at BP) for guiding and supporting me throughout the process.

References

1. S. Chakraborty and A. C. Newton, “Climate change, plant diseases and food security: an overview,” Plant Pathol., vol. 60, no. 1, pp. 2–14, Feb. 2011, doi: 10.1111/j.1365-3059.2010.02411.x. [↩]
2. B. Sambana et al., “An efficient plant disease detection using transfer learning approach,” Sci. Rep., vol. 15, no. 1, p. 19082, May 2025, doi: 10.1038/s41598-025-02271-w. [↩]
3. T. Nyawose, R. C. Maswanganyi, and P. Khumalo, “A review on the detection of plant disease using machine learning and deep learning approaches,” J. Imaging, vol. 11, no. 10, p. 326, Sep. 2025, doi: 10.3390/jimaging11100326. [↩]
4. N. Raji, “Plant disease classification using mobile-captured images: A deep learning approach,” Int. J. Intell. Syst. Appl. Eng., vol. 12, no. 3, pp. 3575–3581, Mar. 2024. [↩]
5. M. Agarwal, S. Kr. Gupta, and K. K. Biswas, “Development of Efficient CNN model for Tomato crop disease identification,” Sustain. Comput. Inform. Syst., vol. 28, p. 100407, Dec. 2020, doi: 10.1016/j.suscom.2020.100407. [↩]
6. A. S. Paymode and V. B. Malode, “Transfer learning for multi-crop leaf disease image classification using convolutional neural network VGG,” Artif. Intell. Agric., vol. 6, pp. 23–33, 2022, doi: 10.1016/j.aiia.2021.12.002. [↩]
7. R. Fu, X. Wang, S. Wang, and H. Sun, “PMJDM: a multi-task joint detection model for plant disease identification,” Front. Plant Sci., vol. 16, p. 1599671, May 2025, doi: 10.3389/fpls.2025.1599671. [↩]
8. D. Dai, P. Xia, Z. Zhu, and H. Che, “MTDL-EPDCLD: A multi-task deep-learning-based system for enhanced precision detection and diagnosis of corn leaf diseases,” Plants, vol. 12, no. 13, p. 2433, Jun. 2023, doi: 10.3390/plants12132433. [↩]
9. S. Hemalatha and J. J. B. Jayachandran, “A multitask learning-based vision transformer for plant disease localization and classification,” Int. J. Comput. Intell. Syst., vol. 17, no. 1, p. 188, Jul. 2024, doi: 10.1007/s44196-024-00597-3. [↩]
10. S. P. Mohanty, D. P. Hughes, and M. Salathé, “Using deep learning for image-based plant disease detection,” Front. Plant Sci., vol. 7, p. 1419, Sep. 2016, doi: 10.3389/fpls.2016.01419. [↩] [↩] [↩] [↩]
11. J. Costa, C. Silva, and B. Ribeiro, “Hierarchical deep learning approach for plant disease detection,” in Pattern Recognition and Image Analysis (Lecture Notes in Computer Science), vol. 11868, A. Morales, J. Fierrez, J. S. Sánchez, and B. Ribeiro, Eds. Cham, Switzerland: Springer, 2019, pp. 383–393, doi: 10.1007/978-3-030-31321-0_33. [↩]
12. T. T. Showrav, S. Bain, M. Hossain, K. I. Ahmed, S. A. Fattah, and S. Ahmed, “A two-stage approach for plant disease classification based on deep neural networks and transfer learning,” in 2022 12th International Conference on Electrical and Computer Engineering (ICECE), Dhaka, Bangladesh, Dec. 2022, pp. 469–472, doi: 10.1109/ICECE57408.2022.10088587. [↩]
13. A. A. Mohammed and V. Umaashankar, “Effectiveness of hierarchical softmax in large scale classification tasks,” in 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Sep. 2018, pp. 1090–1094, doi: 10.1109/ICACCI.2018.8554637. [↩]
14. Z. Akata, F. Perronnin, Z. Harchaoui, and C. Schmid, “Label-embedding for image classification,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 7, pp. 1425–1438, Jul. 2016, doi: 10.1109/TPAMI.2015.2487986. [↩]
15. S. Barath and S. M, “Federated transformer–blockchain framework for secure and generalized crop disease detection in smart agriculture,” J. Saudi Soc. Agric. Sci., vol. 25, no. 4, p. 59, Apr. 2026, doi: 10.1007/s44447-026-00153-9. [↩]
16. A. A. Mamun, M. Zhang, D. Ahmedt-Aristizabal, Z. Hayder, and M. Awrangjeb, “StateSpace-SSL: Linear-time self-supervised learning for plant disease detection,” 2025, arXiv:2512.09492. [↩]
17. D. Mamba Kabala, A. Hafiane, L. Bobelin, and R. Canals, “Image-based crop disease detection with federated learning,” Sci. Rep., vol. 13, no. 1, p. 19220, Nov. 2023, doi: 10.1038/s41598-023-46218-5. [↩]
18. S. H. Lee, H. Goeau, P. Bonnet, and A. Joly, “Conditional multi-task learning for plant disease identification,” in 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, Jan. 2021, pp. 3320–3327, doi: 10.1109/ICPR48806.2021.9412643. [↩] [↩] [↩] [↩] [↩] [↩] [↩] [↩] [↩]
19. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014, arXiv:1409.1556. [↩] [↩]
20. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015, arXiv:1512.03385. [↩] [↩] [↩]
21. M. Tan and Q. V. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” 2019, arXiv:1905.11946. [↩]
22. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” 2018, arXiv:1801.04381. [↩]
23. G. S. Hukkeri, B. C. Soundarya, H. L. Gururaj, and V. Ravi, “Classification of various plant leaf disease using pretrained convolutional neural network on ImageNet,” Open Agric. J., vol. 18, no. 1, p. e18743315305194, May 2024, doi: 10.2174/0118743315305194240408034912. [↩]
24. T. S. Alam, C. B. Jowthi, and A. Pathak, “Comparing pre-trained models for efficient leaf disease detection: a study on custom CNN,” J. Electr. Syst. Inf. Technol., vol. 11, no. 1, p. 12, Feb. 2024, doi: 10.1186/s43067-024-00137-1. [↩]
25. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, Jun. 2009, pp. 248–255, doi: 10.1109/CVPR.2009.5206848. [↩] [↩]
26. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” 2015, arXiv:1512.00567. [↩]
27. E. Perez, F. Strub, H. de Vries, V. Dumoulin, and A. Courville, “FiLM: Visual reasoning with a general conditioning layer,” 2017, arXiv:1709.07871. [↩]
28. R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive mixtures of local experts,” Neural Comput., vol. 3, no. 1, pp. 79–87, Mar. 1991, doi: 10.1162/neco.1991.3.1.79. [↩]
29. N. Shazeer et al., “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” 2017, arXiv:1701.06538. [↩] [↩]
30. J. Ma, Z. Zhao, X. Yi, J. Chen, L. Hong, and E. H. Chi, “Modeling task relationships in multi-task learning with multi-gate mixture-of-experts,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK, Jul. 2018, pp. 1930–1939, doi: 10.1145/3219819.3220007. [↩]
31. C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” 2017, arXiv:1706.04599. [↩]
32. G. Pereyra, G. Tucker, J. Chorowski, Ł. Kaiser, and G. Hinton, “Regularizing neural networks by penalizing confident output distributions,” 2017, arXiv:1701.06548. [↩]
33. B. Peralta, A. Saavedra, L. Caro, and A. Soto, “Mixture of experts with entropic regularization for data classification,” Entropy, vol. 21, no. 2, p. 190, Feb. 2019, doi: 10.3390/e21020190. [↩]
34. X. Chen, J. Zhou, Y. Li, J. Chen, S. Yu, and Q. Xuan, “Mixture of message passing experts with routing entropy regularization for node classification,” 2025, arXiv:2502.08083. [↩]
35. A. Vaswani et al., “Attention is all you need,” 2017, arXiv:1706.03762. [↩]
36. J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” 2016, arXiv:1607.06450. [↩]
37. Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages,” Psychometrika, vol. 12, no. 2, pp. 153–157, Jun. 1947, doi: 10.1007/BF02295996. [↩] [↩]
38. S. Holm, “A simple sequentially rejective multiple test procedure,” Scand. J. Statist., vol. 6, no. 2, pp. 65–70, 1979. [↩]