Deep Neural Network
Detection & Clinical Staging
of COVID-19 Chest X-rays

Peter Riley, Deakin University

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

AIM

Using Mathematica & modest computational resources, 1-10 TFlops FP32 :

	1. Differentiate COVID-19 from other viral pneumonia & normal chest x-rays
	2. Stage the infection into clinically early & late phases

Introduction

Deakin University’s Faculty of Health mandates:
	1. Cross curriculum Medical Informatics (MI)
	2. Now includes Artificial Intelligence/Machine Learning (AI/ML)
		-Follows FDA approval for AI deployment1 (≈135 in Radiology)

1.The state of artificial intelligence-based FDA-approved medical devices and algorithms. Benjamens, S. et al., Nature, npj 3:118 (2020)

For current listing of algorithms see also https://models.acrdsi.org
Data Science Institute, American College of Radiology

Introduction 2

Medical Imaging Dept has extensive MI coverage, but not AI/ML:
	1. Proposal to implement 4th Year research project option
               -Based upon previous R&D experience((.)^2)
	2. Requiring common GPU/CUDA speeds-
		-Limits to Transfer Learning adaptation of advanced DNN (ResNet etc)
		-Limits to small image size e.g. CT & MRI  (256^2),(512^2)

3. Requiring easy Interface-
		-Students do not have the background for Python based platforms
		-Mathematica provides an easy syntax & instruction
		-Wolfram provides easy access to DNN libraries

2. Prospects for Artificial Intelligence in Medicine. Riley,P., ASMIRT conf. guest speaker, Australian Society of Medical Imaging & Radiation     	Therapy. 28 Nov 2020.

COVID-19 Chest X-rays

Didn’t we say CT & MRI?:
	1. Yes, chest x-rays are typically 2kpx x 2.5kpx x 14 bit
               -Requires PetaFlops+ processing power.
	2. But, scaled chest images have been successfully processed
		-Differentiation of disease on (256^2)to (512^2) chest x-rays((.)^3)

3. And, there is a large public database of COVID-19 images((.)^4)
		-One of the main stumbling blocks to short (1 semester) research projects is compiling 		 the raw images required.
		-Specific disease studies require collaboration with local clinical stakeholders which 			  entails a lengthy Ethics approval process.

3. Using Machine Learning to Diagnose Pneumonia from Chest X Rays, Rohit Panse, Wolfram Summer Camp (2018)
4. Thanks to  https://www.kaggle.com/datasets, https://github.com/ieee8023/covid-chestxray-dataset and local clinical stakeholders

COVID-19 Detection

1. Data selection & preparation from over 5k images:
	      -Images filtered for PA (posterior-anterior); AP & Lateral not selected
	      -Images with intubation & other artefacts were rejected	      
                -1200 images chosen for each of the Covid, Normal and Other-Viral datasets
                -100 images chosen randomly from each set for the Test images

Import the images into the COVID, NORMAL, VIRAL datasets & label them accordingly:

trainCOV=Map[#  -> "Covid" & , COVID];
trainNORM = Map[#  -> "Normal" & , NORMAL];
trainVIR = Map[#  -> "Viral" & , VIRAL];

COVID-19 Detection

2. Transfer Training
		-ResNet: net=NetModel[“ResNet-50 Trained on ImageNet Competition Data”]
		-WIIN: net=NetModel[“Wolfram ImageIdentify Net V1”]

Repurpose the DNN by replacing the last 2 layers and specify the 3 required classes:

changedNet=NetDrop[net,-2]
(CopyTag[NetChain[]])
Join with a chain of two new layers:
newNet=NetJoin[changedNet,NetChain[Association["classifier"->LinearLayer[],"probabilities"->SoftmaxLayer[]],"Output"->NetDecoder[{"Class",{"Covid","Normal","Viral"}}]]]
(CopyTag[NetChain[]])

COVID-19 Detection

2. Transfer Training contd.
		-ResNet: net=NetModel[“ResNet-50 Trained on ImageNet Competition Data”]
		-WIIN: net=NetModel[“Wolfram ImageIdentify Net V1”]

Only the Classifier layer undergoes further training all other layers are frozen, LearningRateMultipliers -> 1 or 0 resp :

trainedNet=NetTrain[newNet,training,LearningRateMultipliers->{"classifier"->1,_->0},TargetDevice->"GPU"]

GPU: 1920 CUDA, 240 Tensor cores, 6GB GDDR6 @ 14 Gbps, 960/1200 MHz, 90W; 4.6 TFlops FP32

COVID-19 Detection

3.  Training Results
		-ResNet: net=NetModel[“ResNet-50 Trained on ImageNet Competition Data”]
		-COVID-19: SENSITIVITY = 100%, 
				   SPECIFICITY = 99%
		-WIIN: net=NetModel[“Wolfram ImageIdentify Net V1”] gave 97% & 97% resp.

Showing the ResNet Confusion Matrix & WIIN Receiver Operating Characteristic curve:

({{"actual class", }, {, "predicted class"}});({{"recall", }, {, "false positive rate"}})

COVID-19 Detection Assessment

1.  Time Constraints

		-Processing time was remarkably fast:
			-Seeing x10 improvement with current laptop GPUs over previous Python based Nano accelerators
		-Possible to demo tweaking within the limits of a lecture session

2.  Student Outcomes

		-The hands-on experience will make students more aware of the possibilities & potential hazards of AI
		-Should make them more tolerant of advanced training which is likely to entail with the proliferation of AI

2.  Clinical Outcomes

		-The demo process shows what may be possible
		-But requires a Clinical Visualisation platform able to embed developed DNN into the daily workflow
		-May be implemented in a research capacity but cannot be used routinely without Federal compliance testing

COVID-19 Staging (WIP)

1. Data validation
		-Surprisingly sparse data with unbiased images available
		-Late stage problem:
			-most late stage images are acquired in ICU
			-AP orientation
			-intubation & other interventional artefacts

COVID-19 Staging (WIP)

2. Final Dataset
		-Only 50 images for each category of Early & Late
			-40 x2 training images
			-10 x2 test images
		-Classification stages((:)^5)
			-Early:  Clinical Stage II/1-3 days since COVID-19 onset
			- Late: Clinical Stage III/7-10 days since COVID-19 onset
			-PA without artefacts

Output Classes defined as Early or Late:

newNet=NetJoin[changedNet,NetChain[Association["classifier"->LinearLayer[],"probabilities"->SoftmaxLayer[]],"Output"->NetDecoder[{"Class",{"Early","Late"}}]]]

5. Covid - 19 : Staging of a New Disease, Cordon-Cardo, C., et al., (2020) Cancer Cell 38, p.594-7.

COVID-19 Staging (WIP) Results

1. Accuracy: 80%
		-Weak but encouraging

2. Ongoing work in progress
		-More validated data required

QUALITY ASSURANCE

1. Data validation
		-Must ensure there is no biasing data present
			-e.g. detecting PA from AP images rather than Early stage from Late
		-Essential to work with practitioners to identify such biases

2. Data quality
		-The presentation quality in the 5k+ datasets is quite variable.
			-under & over exposed images
			-typical errors are rotation of the torso in any of 3 directions:
				-rotated spine (around AP axis)
				-rotated shoulders (around spinal axis)
				-leaning (rotation around LAT axis)
				-ALL of these can obscure anatomy

QUALITY ASSURANCE

3. Data size
		-The larger the dataset (the greater the number of examples), the more robust the 			  NeuralNet will be:
			-in the COVID-19 DETECTION DNN the presence of rotation & exposure artefacts 			 was well sampled & incorporated into the DNN’s discrimination.
		-Whereas the number of artefacts in the STAGING DNN was insufficient to form a 			  generalisation
			-a 1st attempt with 300 samples gave an Accuracy of 45%; there was too much 				 diversity in the images
			-reducing the dataset to 80 by discarding worst case artefacts improved the DNN

AI QUALITY ASSURANCE

Data validation
		-Validating data is a time consuming process
		-Suggests that DNN might be useful in automating this task?
		-Work In Progress for some lucky students

THANK YOU!

References:
0.Wolfram Mathematica v.12.3.1.0, Wolfram Research, Inc., Champaign, IL, (1988-2021).

1.The state of artificial intelligence-based FDA-approved medical devices and algorithms. Benjamens, S. et al., Nature, npj 3:118 (2020)

2. Prospects for Artificial Intelligence in Medicine. Riley,P., ASMIRT conf. guest speaker, Australian Society of Medical Imaging & Radiation     	Therapy, 28 Nov 2020.

3. Using Machine Learning to Diagnose Pneumonia from Chest X Rays, Rohit Panse, Wolfram Summer Camp (2018)

4. Thanks to  https://www.kaggle.com/datasets, https://github.com/ieee8023/covid-chestxray-dataset and local clinical stakeholders

5. Covid - 19 : Staging of a New Disease, Cordon-Cardo, C., et al., (2020) Cancer Cell 38, p.594-7.