Low-Cost Fine-Tuning of Data-Efficient Image Transformers on Knee X-Ray Imaging for Osteoarthritis Detection

Abstract

Introduction: Technological advancements in artificial intelligence within the field of medicine, specifically skeletal pathology, have witnessed exponential growth in recent years. Researchers have trained deep learning models on radiographs to improve the detection of diseases. There is precedence for low data training on musculoskeletal imaging tasks, and “specialized” medical imaging-related tasks in general have exhibited high performance on low amounts of data. Thus, the Data-efficient Image Transformer (DeiT) model has potential to surpass conventional convolutional models in detecting musculoskeletal diseases due to its ability to extract relevant features with scarce data.

Methods: This study utilizes a DeiT model pre-trained on ImageNet 2012. The model was fine-tuned on labelled knee X-rays of patients with and without osteoarthritis using the PyTorch library. Fine-tuning and testing were done on a Google Colab notebook using a T4 GPU. A hyperparameter sweep cycling through different dropout values, optimizers, and image input sizes was tested. Results were recorded using multiple accuracy metrics.

Results: The DeiT-B 384 model with unspecified dropout had the highest scores out of all the model variations. The DeiT’s composite performance in diagnosing knee osteoarthritis exceeded that of convolutional models. Fine-tuning resulted in accuracies within the standard established by current literature for the 384 model, but not for the 224 model. This suggests that larger input models trained on lower quality images performed better than smaller input models trained on higher quality images.

Implications: Fine-tuning can be an alternative to training from scratch for medical imaging. Future studies should expand their scope by taking additional patient details into account to increase diagnostic accuracy and provide local and individualized patient care. The absence of these variables in our model’s training potentially limited its accuracy. Current real-time diagnostic errors are less than even the best performing computer vision models, so accuracy must significantly improve before there is incentive to adopt these methods.