Advanced Use of Automatic Machine Learning Functions

Etienne Bernard, Wolfram Research

Join the Conversation #WolframTechConf

Automatic Machine Learning Functions

Classify

Predict

LearnDistribution

SynthesizeMissingValues

AnomalyDetection

FindClusters

DimensionReduction

Classify & Predict

c=Classify[{1->"A",2->"A",3.5->"B",4->"B"}]

p=Predict[{1->1.2,2->3.2,3.5->5.6,4->7.6}]

mnist=RandomSample[ResourceData["MNIST"],20000];

mnist[[;;5]]

c=Classify[mnist]

High-level Training Specification

TimeGoal

c=Classify[mnist,TimeGoal->30]

PerformanceGoal

”Memory” =  small model

“Speed” = fast model

“TrainingSpeed”  = fast training

“Quality” = best predictions

Automatic = tradeoff

Low-level Training Specification

Method

c=Classify[mnist,Method->"LogisticRegression"]

c=Classify[mnist,Method->{"LogisticRegression", "L2Regularization"->1}]

FeatureExtractor

Preprocessor

c=Classify[{"The cat is grey."->,"My cat is fast."->,"This dog is scary."-> ,"The big dog."->},FeatureExtractor->"SegmentedWords"]

c=Classify[{"The cat is grey."->,"My cat is fast."->,"This dog is scary."-> ,"The big dog."->},FeatureExtractor->ToUpperCase]

c=Classify[{"The cat is grey."->,"My cat is fast."->,"This dog is scary."-> ,"The big dog."->},FeatureExtractor->{ToUpperCase, RemoveDiacritics,"SegmentedWords"}]

Feature extractors can also be added to the ClassifierFunction[...] after training

fe=FeatureExtraction[{"The cat is grey.","My cat is fast.","This dog is scary.","The big dog."},"SegmentedWords"]

c=Classify[fe[{"The cat is grey.","My cat is fast.","This dog is scary.","The big dog."}]->{,,,}]

c2=Classify[c,FeatureExtractor->fe]

How to control (almost) everything

Fully specified Method option

c=Classify[mnist,Method->{"LogisticRegression", "L1Regularization"->0,"L2Regularization"->1,"OptimizationMethod"->"StochasticGradientDescent"}]

FeatureExtractor ->  ”Minimal”

Removes non-necessary processing

c=Classify[mnist,FeatureExtractor->"Minimal"]

PerformanceGoal ->  ”DirectTraining”

Disable automation

c=Classify[mnist,PerformanceGoal->"DirectTraining"]

Everything at once: full control

c=Classify[mnist,FeatureExtractor->"Minimal",PerformanceGoal->"DirectTraining",Method->{"LogisticRegression", "L1Regularization"->0,"L2Regularization"->1,"OptimizationMethod"->"LBFGS"}]

Information

Information[c]

Information[c,"MeanCrossEntropy"]

Information[c,"Properties"]

Information[c,"MethodOption"]

Internal Information...

c[[1]]

Internal Processing

?MachineLearning`ToMLDataset

?MachineLearning`CreateProcessor

?MachineLearning`MLProcessor

MachineLearning`$MLProcessors

Measurements

test=ResourceData["MNIST","TestData"];

cm=ClassifierMeasurements[c,test]

cm["Properties"]

cm["Report"]

cm["ConfusionMatrixPlot"->{0,1,7}]

cm["ConfusionMatrixPlot"->2]

cm["TopConfusions"]

cm["WorstClassifiedExamples"]

cm["WorstClassifiedExamples"->20]

Post-training model modification

Probabilistic models

Can change prior probabilities or decision utility after training

ClassPriors

c2=Classify[c,ClassPriors-><|9->0.5|>]

Information[c2,ClassPriors]

IndeterminateThreshold

c2=Classify[c,IndeterminateThreshold->0.3]

UtilityFunction

Decision is done by maximizing the expected utility

Information[c,"UtilityFunction"]

Storing & deploying model

CloudDeploy

Export / Import WMLF

Wolfram Machine Learning Format

Export["~/test.wmlf",c]

ByteCount[c]

FileByteCount["~/test.wmlf"]

Import["~/test.wmlf"] // AbsoluteTiming

Works for neural networks as well (and any other expression)

MX format can also be used