DermaVision AI: A Region-Based Multi-Model Approach to Skin Disease Classification with AI-Generated Patient Guidance

Project Title: DermaVision AI: A Region-Based Multi-Model Approach to Skin Disease Classification with AI-Generated Patient Guidance Executive Summary: DermaVision AI is an end-to-end, privacy-preserving clinical decision-support framework engineered to address the critical failure modes of conventional computer vision models when deployed in unconstrained everyday smartphone environments. Standard closed-set deep learning classifiers are typically trained on sterile, tightly cropped dermatological archives (such as ISIC) and lack the capability to reject non-cutaneous artifacts or recognize spatial body context, frequently resulting in confident false-positive hallucinations on everyday fabrics, packaging, or arbitrary surfaces. DermaVision systematically eliminates these operational risks by integrating a five-stage safety-gated pipeline running completely offline on commodity CPU hardware without cloud latency, recurring API costs, or patient privacy violations. System Architecture and Technical Methodology: The pipeline operates across five sequential stages: 1. Anatomical Input Verification Guard: Employs discrete Laplacian variance (threshold >= 100) to reject motion blur, dual-space (YCbCr and HSV) skin chrominance segmentation (requiring >= 25% cutaneous surface area), MediaPipe 3D Mesh / BlazeFace facial landmark detection to block cross-anatomical mismatches, and high- frequency edge entropy filters to weed out textile patterns and packaging. 2. Dual-Vision Classification Engine: Constrains diagnostic entropy by partitioning the human body into seven anatomical zones (Face & Neck, Chest & Abdomen, Back & Spine, Arms & Forearms, Hands & Nails, Legs & Thighs, Feet & Soles), encompassing 23 candidate conditions per region (161 region-condition pairings total). Features are extracted using a fine-tuned Vision Transformer (ViT-Base/16) tokenized across 196 image patches (16x16 pixels) combined with zero-shot CLIP (ViT-B/32) visual- semantic cosine alignment. 3. Optical Pigmentation Proxies: Extracts calibrated Erythema Index [EI = 100 * log10(R/G)] and Melanin Index [MI = -100 * log10(R)] directly from normalized camera RGB channels to track vascular inflammation and distinguish natural pigmentation from artificial cosmetic confounders. 4. Autonomous Geospatial Navigator: Coordinates real-time OpenStreetMap Overpass API queries within a 15 km bounding perimeter to locate accredited dermatological centers, sorted by great-circle Haversine distance with turn-by-turn navigation. 5. Privacy-Preserving On-Device RAG Assistant: Indexes 48 verified clinical guidelines into 1024-dimensional dense vectors using mxbai-embed-large. Top-3 retrieved guideline excerpts are fed into a 4-bit quantized Llama 3.2 1B model running locally via Ollama on CPU to generate empathetic, plain-language patient education and clinical follow-up advice. Experimental Benchmarks and Performance: - Front-End Guard Screening Accuracy: 98.7% across 237 stress test samples (successfully blocked 112 out-of-distribution packaging, textile, and blur artifacts with 0% false rejections of valid lesions). - Regional Classification Accuracy: 92.4% mean multi-class accuracy across all 7 anatomical zones (161 condition classes). Hands and Nails achieved 94.8% accuracy (F1: 0.947), Face and Neck achieved 93.4% accuracy (F1: 0.934), and Arms and Forearms achieved 92.9% accuracy (F1: 0.926). - Computational Latency: Executes end-to-end in 3.88 seconds on a standard quad-core Intel CPU without requiring dedicated GPU acceleration (Guard: 0.18 s, ViT: 0. 52 s, CLIP: 0.28 s, Optical: 0.04 s, Geospatial: 0.42 s, Local RAG: 2.44 s). Authors & Institutional Affiliation: Mohammad Usman M, Kishore S, Anthony Benny A, Liwi Nishanth A, and Mrs. P. Shanthi (Project Supervisor). Department of Artificial Intelligence and Machine Learning, School of Engineering, Vels Institute of Science, Technology and Advanced Studies (VISTAS), Chennai, India. Keywords: Dermatological Image Analysis, Anatomical Safety Guard, Vision Transformer (ViT-Base/16), Zero-Shot CLIP Semantic Alignment, Optical Pigmentation Proxies (EI/MI), Local Retrieval-Augmented Generation (RAG), Privacy-Preserving Healthcare AI, Edge Computing.