Lung Cancer Detection using Deep Learning with Hybrid Preprocessing Pipeline

Abstract

Lung cancer remains one of the leading causes of death globally, with millions of lives lost each year. It is one of the most prevalent non-communicable diseases, responsible for approximately 6% of all deaths. Symptoms of lung cancer are diverse and may include chest and bone pain, wheezing, persistent coughs, unexplained weight loss, fatigue, shortness of breath, and hemoptysis, among others. Risk factors include long-term smoking, exposure to secondhand smoke, asbestos, radon gas, radiation therapy to the chest, and a family history of lung cancer. While CT scans are commonly used for detection, they have limitations, particularly in early-stage diagnosis, due to high false positive rates, and can be uncomfortable for patients. An alternative approach, incorporating machine learning and deep learning, offers the potential for improved early detection, increasing survival rates and reducing unnecessary follow-up tests and treatments. This study focuses on detecting lung cancer using CT scan images, applying a multiclass classification system to differentiate between malignant, benign and normal images. The proposed system has been designed for use in hospitals to aid in the diagnosis and treatment of lung cancer. To obtain accurate results after applying our model, we use an online dataset, for which high and low quality CT images are presented here for this online dataset. In this study, to process the quality CT images we use a hybrid preprocessing pipeline where we use stratified sampling and SMOTE oversampling method to do sampling and to increase image quality we use gaussian blur method. Experimental results show that while not all models achieved high accuracy, most surpassed 96%. Notably, models such as CNN (96%), ResNet50 (91%) and VGG16 (88%) demonstrated superior performance in accurately identifying lung cancer in CT scans.