On Boosting the Accuracy for the Classification of Histopathological Slides

Catalin Stoean
University of Craiova, Romania
catalin.stoean@inf.ucv.ro

Daniel Lichtblau
Wolfram Research
danl@wolfram.com

Wolfram Technology Conference
October 19, 2018

The problem under consideration

Goal: Automate cancer detection and grading

classification of tissue samples

Works with H&E stained slides

We show two slides from each of the four categories in the main data set

There is considerable variation even within a given classification

Data set descriptions

Three data sets, of which the first was the “main” set on which we constructed the ensemble

IMEDIATREAT H&E stained colorectal tissue

Magnification of 10x

Four grades (normal, and three levels of cancerous)

Approximately 60 normal and 100 in each grade of malignant images

2/3 training and 1/3 test splits

GlaS H&E stained colorectal tissue

Magnification of 20x

Two grades (74 normal and 91 malignant)

Roughly evenly split into training vs testing

BreaKHis H&E stained breast tumor tissue

Four separate magnification levels (40x, 100x, 200x, 400x)

Two grades (benign tumor and malignant tumor)

Each magnification level has 590-650 benign and 1230-1440 malignant images

Five different splits of train vs test images (all images from a given patient all go into the same set)

Current methods

One family of automated methods tries to compute various features

Heavy reliance on image processing

Segmentation, feature extraction, classification

Extracted 76 morphological and topological features for glands & nuclei

Used feature selection and SVM

Current methods...

Another approach uses Deep learning (via Convolutional Neural Networks)

Using transfer learning from ResNet50 further improved the results

Our approach

We create a one-step classifier.

Take many images with known classifications

Feed them to several machine learning classifiers

Use an optimization to determine weights for each classifier

The weighted scores give an ensemble classifier

Results will show this approach to be competitive with the best methods presented in prior literature.

Machine learning methods

The ensemble is comprised of seven methods.

Six are standard in ML and are Method options for Classify.

Random forest

Naive Bayes

K-nearest neighbors

Support vector machine

Logistic regression

Neural network

FTT+PCA (another nearest neighbors approach)

FTT + PCA classifier

This method was shown at SYNASC 2016 (by DL)

The classifier is constructed as follows

Take a Fourier trig transform of images

sine or cosine depending on need

Retain only low frequency components

This reduces dimension

Flatten these numeric arrays into a set of vectors

Compute SVD of the resulting matrix

Retain only a few singular values

This further reduces dimension

Also retain the right side conversion matrix for later lookup purposes

Place resulting vectors into a nearest neighbors lookup

Uses Nearest in the Wolfram Language

FTT + PCA predictor

Predictions are done as follows

Preprocess images with the same Fourier trig transform

Retain the same low frequency components as before

Use the right multiplication matrix from the SVD step to create a vector of same dimension as those in the lookup function

Find several nearest neighbors from the training set

Use their known classifications to predict the class of the test image

Weight these predictions using a distance measure between test vector and the nearest neighbors from the training set

Individual method results for IMEDIATREAT

We ran 40 trials each using a random split of 2/3 per class for training and 1/3 for testing

Results range from 87-94 % correct identification

Ensemble optimization step

Next goal: combine methods to get an ensemble that gives better results?

Use the classifiers on the 40 individual training runs for all seven classifiers

Select half the test images in each run for purposes of tuning weights

Assign and adjust weights to each classifier

Optimize these weights so as to get the best overall result on the half of the test images selected

Assess how good the optimized weights are by their performance on the half of the test set that was not used in the optimization step

Ensemble optimization...

This may sound complicated. It is really not so bad

The code snippets below should give an idea

The objective is the sum of squares of discrepancies in grade between classification and actual category

Example: an image is predicted at grade 1 but is actually grade 3 so the function evalautes to (((1-3))^2) or 4

obj[vars:{_?NumberQ..},sample_]:=
With[{av=allVals[[sample]],tv=testValues},
Module[{plists,fr,diffs,misses},
plists=Map[vars.#&,av];
fr=Map[First[Ordering[#,1,Greater]-1]&,plists,{2}];
diffs=Map[#-tv&,fr];
misses=Map[Total,Abs[diffs]^2];
Total[misses]
]]

bestDE=NMinimize[{obj[vars,sample],constraints},initBounds,Method->{"DifferentialEvolution","SearchPoints"->20,"PostProcess"->False,RandomSeed->RandomInteger[10^100]},MaxIterations->400]

We did 20 such runs and took the weights that were best overall.

Ensemble optimization...

We did 20 such runs and took the weights that seemed best overall

FTT+PCA was a middle-of-the-pack performer

So why does it get such a relatively large weight?

Ensemble optimization...

The Cohen’s Kappa values for categorical data will help to explain this phenomenon

Green = smallest in row

In short, FTT+PCA tends to make mistakes in different places from other methods

When it is wrong, other methods are likely to outvote it

When some others are wrong, the big weight for FTT+PCA tends to offset them

Overall results for IMEDIATREAT

The confusion matrix shows how we fared with IMEDIATREAT using the best ensemble weighting, using individual classifier scores in each of the 40 trials.

Good news: only missed 93 out of 4,760 test cases (98%)

Most misses were “off by one” mistakes in grade of malignancy

Bad news: We told a perfectly healthy patient that we found a grade 3 malignancy

Results for GlaS and BreaKHis

GlaS results

Test set A: 95 %

Test set B: 90 %

BreaKHis results

40x magnification test set: 82.0 %

100x magnification test set: 83.1 %

200x magnification test set: 87.1 %

400x magnification test set: 84.4 %

Next goal: Do better on BreaKHis

Summary

We created a one-step classifier

No image segmentation or other complicated measurements

Just based on machine learning

It was optimized for a particular data set, but does tolerably well on others using the same settings

So no tuning necessary

Room for improvement:

Use more refined ML methods and settings

In particular, use a neural net that is well suited for H&E stained images

Other directions to consider

Use color deconvolution (so varying levels do not confuse the classifier)

Use rotations of training images (and possibly also test images) to get a better training set

Perhaps find better-suited dimension reduction scheme for images of this type