Triple-Stream Transformer Architecture for Multi-Class Skin Cancer Classification in Dermoscopic Images

Abstract

The most often occurring kind of cancer globally is skin cancer; melanoma is the deadliest kind. Good treatment results depend on early and correct diagnosis. Dermatologists visually inspect skin cancers as part of traditional diagnostic methods, which can be arbitrary and unreliable. However, recent advances in deep learning show that automated skin cancer identification has a lot of potential. This work presents a new hybrid model for dermoscopic image-based multi-class skin cancer classification. The five steps in our method are: using DeepLabV3+ with a ResNet50 backbone to separate skin lesions; extracting features using a triple-stream transformer-based architecture (Derm-ViT, Swin Transformer V2, and ConvNeXt V2); joining features together; choosing features using the ReliefF algorithm; and classifying with k-nearest neighbors (kNN). Each transformer branch collects a number of different but related parts of skin lesions, such as fine-grained texture information, multiscale characteristics, and patterns that are specific to dermatology. On the ISIC-2019 dataset, which has eight diagnostic categories, our suggested method has 94.42% accuracy, 94.13% precision, 92.99% sensitivity, and 98.96% specificity compared to individual transformer models and state-of-the-art approaches. This result shows how well our hybrid method addresses the difficulties of multi-class skin cancer classification and provides a consistent instrument to support dermatologists in daily clinical work.