AI-driven framework for drug safety enhancement and dermatological diagnostics using XAI

Abstract

MedScan is an AI-based mobile application designed to enhance medication safety and dermatological diagnostics in resource-constrained contexts, particularly in Sri Lanka. It addresses two critical healthcare challenges: medication errors, mitigated through Optical Character Recognition (OCR) of drug labels, and timely skin condition detection using Convolutional Neural Networks (CNNs) with Explainable AI (XAI) techniques, including Grad-CAM. The application supports multilingual drug information (English, Sinhala, Tamil) and distinguishes between drug-induced skin reactions and skin cancers. MedScan achieves 93.8% OCR accuracy and 97.4% classification accuracy via EasyOCR and a fine-tuned DenseNet121. Offline availability and a user-friendly interface ensure accessibility for elderly and low-literacy users. User trials showed a 30% reduction in medication errors, 92% user satisfaction, and a 25% increase in clinician confidence through XAI visualisations. In addition, MedScan is designed with data privacy in mind; sensitive patient information is processed locally, and any necessary transmissions are anonymised and encrypted in compliance with GDPR and HIPAA standards. Despite strong performance across skin tones, initial testing revealed reduced accuracy in darker skin types. To address this, the dataset was expanded with 500 curated images from Fitzpatrick IV–VI, resulting in a 4.1% improvement in recall, with ongoing efforts to prioritise skin tone diversity and minimise algorithmic bias. To the best of our knowledge, MedScan is among the first multimodal mHealth frameworks that unify OCR for drug safety and CNN-based dermatology diagnostics within a single application, supported by XAI for transparency. With an inference time of 0.2 seconds on mobile devices, MedScan provides a scalable, ethical, and inclusive solution that enhances patient safety, diagnostic equity, and healthcare workflow efficiency.