Automated ML
Past, present, future

Giulio Alessandrini, Wolfram Research

ABSTRACT

Since their first introduction in Version 10 of the Wolfram Language, machine learning superfunctions like Predict and Classify have tried to bridge the gap between beginner and specialist users by offering a fully automated yet customizable pipeline. Over time, new tools like feature extraction, distribution fitting and anomaly detection have been added. This talk will review the current set of our machine learning capabilities and look at what might come in the future.

Story

The ML framework went a long way since it’s introduction in version 10.0 (July 2014)

versionDate = ;
versions=;
TimelinePlot[Values@GroupBy[versions,Identity]/.versionDate,ImageSize->Large,PlotLegends->SwatchLegend[DeleteDuplicates[Values[versions]],LegendLabel->"Version",LegendFunction->(Framed[#,RoundingRadius->5]&),LegendMargins->5],PlotStyle->ColorData[103],BaseStyle->{FontSize->12},AspectRatio->2/3
]

Simple pipeline

data import

cleaning

feature selection

training

analysis

Simple pipeline

Heart Disease Databases

https://archive.ics.uci.edu/ml/datasets/Heart+Disease

Creators:

1. Hungarian Institute of Cardiology. Budapest: Andras Janosi, M.D.
2. University Hospital, Zurich, Switzerland: William Steinbrunn, M.D.
3. University Hospital, Basel, Switzerland: Matthias Pfisterer, M.D.
4. V.A. Medical Center, Long Beach and Cleveland Clinic Foundation: Robert Detrano, M.D., Ph.D.

Simple pipeline

Data import

https://www.kaggle.com/ronitf/heart-disease-uci

SetDirectory[NotebookDirectory[]]

ExtractArchive["HeartDiseaseData/archive.zip",OverwriteTarget->True]

Short[Import@First@%,4]

kaggle=Import[First@%%,"Dataset","HeaderLines"->1]

keys=Normal@Keys[kaggle[[1]]]

Simple pipeline

Cleaning

rawdata=Join[
Import["HeartDiseaseData/processed.switzerland.data","CSV"],
Import["HeartDiseaseData/processed.cleveland.data","CSV"],
Import["HeartDiseaseData/processed.hungarian.data","CSV"],
Import["HeartDiseaseData/processed.va.data","CSV"]
];

SeedRandom[123];
RandomSample[rawdata,5]//Column

rawdata=MapAt[Round,rawdata,Thread[{All,Delete[Range[14],10]}]]/.Round["?"]->Missing[];

SeedRandom[123];
RandomSample[rawdata,5]//Column

dataset=RandomSample@Dataset[Transpose[AssociationThread[keys,Transpose[rawdata]],AllowedHeads->All]]

Simple pipeline

Visualization

twoclasses=MapAt[Sign,{All,"target"}]@dataset;

features=Values@Normal@KeyDrop[twoclasses,"target"];

target=First/@Normal@KeyTake[twoclasses,"target"];

features2D=AssociationMap[DimensionReduce[features,2,Method->#]&,{"PrincipalComponentsAnalysis","Autoencoder","UMAP","TSNE"}];

KeyValueMap[ListPlot[GroupBy[Thread[{#2,ToString/@target}],Last->First],Frame->True,Axes->False,ImageSize->Scaled[.3],PlotLabel->#1,PlotStyle->Directive[PointSize[.01],Opacity[.9]]]&,features2D]//Multicolumn

Simple pipeline

Feature Selection

input=Values@Normal@KeyDrop[dataset,"target"];

target=First/@Normal@KeyTake[dataset,"target"];

{fe,features}=FeatureExtraction[KeyDrop[dataset,"target"],Automatic,{"ExtractorFunction","ExtractedFeatures"}];

fe

Information[fe,"FeatureTypes"]

fe[[1,"Processor"]]

Simple pipeline

Training

SeedRandom[1234];
{test,train}=TakeDrop[RandomSample@twoclasses,300];

cf=Classify[
train->"target",
Method->{"LogisticRegression","L1Regularization"->0.001,"L2Regularization"->0.1,"OptimizationMethod"->"Newton"},
FeatureExtractor->fe
]

Information[cf]

cf[Take[test,10]]

Simple pipeline

Analysis

cm=ClassifierMeasurements[cf,test]

cm["Properties"]//Multicolumn[#,3]&

cm["ROCCurve"]

cm["LinearCalibrationCurve"]

cm["SHAPPlots"]

Automation

autoCF=Classify[train->"target"]

Information[autoCF]

autoCF[[1,"Input","Processor"]]

autoCF[[1,"Model"]]

Current directions

Explainability

Human in the Loop: Interpretable Machine Learning by Jesse Galef

https://wtc2021.pathable.co/meetings/virtual/eEmfXpZeSDzv889zv

Individual Conditional Expectation

wine=RandomSample[ResourceData["Sample Data: Wine Quality"]];
p=Predict[wine->"WineQuality",Method->"GradientBoostedTrees",PerformanceGoal->"Speed"];

pm=PredictorMeasurements[p,RandomSample[wine,100]];

pm["ICEPlots"]//Values

SHAPs

Current directions

Content detection

Train a custom text tagging function (trainable TextCases)

detector=TrainTextContentDetector[{
"I like banana"->{{8,13}->"Fruit"},
"I eat apples watching TV"->{{7,12}->"Fruit"},
"I am enjoying raspberries"->{{15,25}->"Fruit"},
"I play soccer"->{{8,13}->"Sport"},
"I watch TV"->{}
}]

Apply the detector to new text

detector[{"I ate cranberries","I like basketball"}]

Query for specific properties

detector[{"I ate cranberries","I like basketball"},{"Position","Class","Probability"}]

Train a custom object detector function (trainable ImageCases)

df=TrainImageContentDetector[
{->{Rectangle[{48, 25},{160, 144}]->"apple",Rectangle[{144, 24}, {279, 144}]->"apple"},
->{Rectangle[{84, 43},{227, 155}]->"strawberry"},
->{Rectangle[{67, 60},{140, 220}]->"banana"},
->{Rectangle[{60,70},{180,212}]->"strawberry"}}
]

df=Import["fruit detector [2021-10-14T16:20:17].wmlf"]

Apply the detector on a new image:

df[testImage=]

df[testImage,All]

Highlight the detection on the input image::

HighlightImage[testImage,Legended[#BoundingBox,#Class]&/@df[testImage,{"BoundingBox","Class"}]]

Future

More automation

Sequence prediction

Unsupervised learning

Data preprocessing

Probabilistic programming

Reinforcement learning

Graph NN

Thank you!