Navigating Data Scarcity in Medical Image Analysis: A Controlled Study on Transfer Learning, Parameter Efficiency, and Evaluation Metric Pitfalls: A Multi-Seed Robustness Analysis

Diabetic retinopathy (DR) is a leading cause of preventable vision loss worldwide, and automated screening tools based on retinal fundus imaging have become an active area of research. Transfer-learning architectures are frequently applied to DR classification, but the relative performance of different pretrained families under identical experimental conditions remains incompletely characterised. This study presents a controlled comparative evaluation of five CNN architectures for binary DR classification on the Messidor dataset: a lightweight baseline CNN trained from random initialization, and four ImageNet-pretrained models with partial fine-tuning (EfficientNet-B0, MobileNetV2, DenseNet121, and ResNet50). All models were trained and evaluated under identical conditions, including the same preprocessing pipeline, train–validation split, augmentation policy, class-weighting scheme, optimizer, and evaluation metrics. To account for sampling variation, each model was trained with three independent random seeds (42, 123, 2024), and results are reported as mean ± standard deviation. The binary target mapped Messidor retinopathy grade 0 to “no DR” and grades 1–3 to “DR present.” On the held-out 240-image test set, EfficientNet-B0 achieved the highest mean area under the receiver operating characteristic curve (AUC = 0.849 ± 0.006) with only 0.90M trainable parameters. DenseNet121 achieved the highest mean accuracy (76.81% ± 1.27) with the fewest trainable parameters of any pretrained model (0.32M). MobileNetV2 (AUC = 0.814 ± 0.009) and ResNet50 (AUC = 0.816 ± 0.003) were statistically indistinguishable from each other. The baseline CNN failed to converge during training and produced near-chance discrimination (AUC = 0.663 ± 0.015) despite an apparently competitive point-estimate accuracy, which is shown to reflect a positive-prediction bias rather than genuine learning. The gap between the baseline and every pretrained model exceeded ten times the standard deviation of either model, confirming that the transfer-learning advantage is robust to seed variation. A secondary analysis conducted on two of the five models (baseline CNN and EfficientNet-B0) found that CLAHE preprocessing did not improve validation accuracy and modestly reduced it for those two models. Because this analysis covers only a subset of the models and is based on validation performance, it is reported as a preliminary observation rather than a general claim about CLAHE in retinal classification.