Simplified 
Machine Learning 
Workflows
Overview

Anton Antonov, Accendo Data LLC

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

In this presentation we overview the main Machine Learning (ML) workflows and their rapid specification through monadic packages. Additional facilitation is given by a Natural Language Processing (NLP) template engine and a Domain Specific Language (DSL) translator

We demonstrate that the ML workflows we consider are universal -- they have multi-language implementations in Python, R, WL, and others.

The overview covers Data wrangling, Classification, Regression, Latent Semantic Analysis, and Recommendations. Extensive explanations and examples are given in the book "Simplified Machine Learning Workflows."

Presentation structure

The "main message" in 5 minutes

Reviewing each ML workflow

Focused, concise

More dedicated, detailed examples

Real-life, and / or interesting

General discussion topics

If there is time left

This is an "advertisement" presentation

The workflows are discussed in depth elsewhere

See Wolfram U lectures labeled "Simplified Machine Workflows"

Main ML workflows

We can say that between 60%  and 85% of the ML workflows are in those ML sub-domains:

Classification

Regression

Latent Semantic Analysis

Recommendations

Additional sub-domains are:

Data wrangling

Event records transformation

Data wrangling does is not belong to ML per se, but some people convinced me it has to be included.

First examples

Consider the following -- intentionally short and non-specific -- computational workflow specifications:

lsCommands={
"Make a classifier over dsTitanic.", 
"Use the document collection aAbstracts; extract 32 topics",
"Do quantile regression over dfFinData."
};

Here we generate the code -- note the only the list of commands is given to the function Concretize, [AAp1]:

aRes=Concretize[lsCommands];

Here we tabulate the code generation results (i.e. computational templates that are filled-in):

ResourceFunction["GridTableForm"][aRes,Background->White]//Magnify[#,1.2]&

Grid[List/@Flatten[List@@@Normal[KeyMap[Style["\n"<>#,Blue,FontFamily->"Times",FontSize->16]&,aRes]]],Dividers->All,FrameStyle->Directive[Gray],Alignment->Left]

Pareto within Pareto

We can go even further and say that within each ML workflow 75% of the time we use the same workflow.

How the simplification is done?

The simplification of the workflows is done from two points of view:

Extracted invariant workflows

Flowcharts

Monadic packages for each workflow

Code generation via natural language commands

Python, R, WL

Sometimes, also, Julia, Raku, Swift

I can claim that the abstracted workflows are "both operational" and theoretic" because of monadic implementations.

Variety of interfaces

Using dedicated DSL mode for notebooks

Dashboard for the web service

Chat-bot for Discord or Slack

Resource function (WL)

Dedicated web service API utilization packages in Python, R, etc.

Using Command Line Interface (CLI)

Using Siri Shortcuts

CLI

dsl-translation WL "make document term matrix"

dsl-web-translation --to-language=WL "make document term matrix"

Special notebook DSL cells

DSLMode[]

Development work

(*Import["https://github.com/antononcube/ConversationalAgents/raw/master/ConceptualDiagrams/Monadic-making-of-ML-conversational-agents.pdf","PageGraphics"][[1]]*)
Import["https://github.com/antononcube/ConversationalAgents/raw/master/ConceptualDiagrams/Monadic-making-of-ML-conversational-agents.jpg"]

NLP Template Engine

The examples above were generated NLP template engine generates codes in Python, R, Raku, Swift, WL.

plMindMap=Import["https://github.com/antononcube/NLP-Template-Engine/raw/main/Documents/Diagrams/General/Conversational-agents-with-QAS-demo-mind-map.pdf","PageGraphics"][[1]]

This is the subject of my presentation Friday 10:00 AM.

Note, that this is not the only way the workflows code is generated: I also use parsers and interpreters.

DSL translation

Here is Domain Specific Language (DSL) translation via grammar-based parsers and interpreters:

command="
use dsTitanic;
split the data with ratio 0.76;
create a random forest classifier;
show accuracy and precision;
show ROC plots";
ResourceFunction["DSLTranslation"][command]

ClConUnit[dsTitanic]⟹ClConSplitData["TrainingFraction"->0.76`]⟹ClConMakeClassifier["RandomForest"]⟹ClConClassifierMeasurements[{"Accuracy","Precision"}]⟹ClConEchoValue[]⟹ClConROCPlot[];

Of course we should be able to also translate to natural languages:

ResourceFunction["DSLTranslation"][command,"English"->"Russian"]

DSL translation 2

This is a multi language system -- here we translated Bulgarian into R:

command="
DSL MODULE LSAMon;
използвай aAbstracts;
направи документ терм матрицата;
извади 30 теми;
покажи таблицата на темите;
покажи статистически речник за думите notebook, function, neural
";
ResourceFunction["DSLTranslation"][command, "Bulgarian"->"R","Format"->"Code"]

If for some reason you do not know Bulgarian here is the translation to English:

ResourceFunction["DSLTranslation"][command, "Bulgarian"->"English"]

...and Russian:

ResourceFunction["DSLTranslation"][command, "Bulgarian"->"Russian"]

DSL translation 3

Here is a component diagram that corresponds to the execution above:

(*Import["https://raw.githubusercontent.com/antononcube/RakuForPrediction-book/main/Diagrams/DSL-Web-Service-via-Cro-with-WE-QAS-Shortcuts.pdf","PageGraphics"][[1]]*)
ImageCrop@Import["https://raw.githubusercontent.com/antononcube/RakuForPrediction-book/main/Diagrams/DSL-Web-Service-via-Cro-with-WE-QAS-Shortcuts.jpg"]

Not just ML

Here is a demonstration of the same technology applied to predictions of different kind:

ResourceFunction["DSLTranslation"]["
DSL MODULE ECMMon;
create with the model susceptible exposed infected two hospitalized recovered;
assign 100000 to the susceptible population;
set infected normally symptomatic population to be 0;
set infected severely symptomatic population to be 1;
assign 0.56 to contact rate of infected normally symptomatic population;
assign 0.58 to contact rate of infected severely symptomatic population;
assign 0.1 to contact rate of the hospitalized population;
simulate for 240 days;
plot populations results;
"]

ECMMonUnit[SEI2HRModel[t]]⟹
ECMMonAssignInitialConditions[Association[SP[0]->100000]]⟹
ECMMonAssignInitialConditions[Association[INSP[0]->0]]⟹
ECMMonAssignInitialConditions[Association[ISSP[0]->1]]⟹
ECMMonAssignRateRules[Association[β[INSP]->0.56`]]⟹
ECMMonAssignRateRules[Association[β[ISSP]->0.58`]]⟹
ECMMonAssignRateRules[Association[β[HP]->0.1`]]⟹
ECMMonSimulate["MaxTime"->240]⟹
ECMMonPlotSolutions["Stocks"->__~~"Population",ImageSize->Large];

Upgrade(s)

Osipov-Lanchester equations with "will to fight" extensions

The easiest way to include the will to fight with via NDSolve's WhenEvent:

eqs2={
A'[t]==-β*B[t],
B'[t]==-α*A[t],
A[0]==A0,
B[0]==B0,
WhenEvent[A[t]<A0*(1-ωA)||B[t]<B0*(1-ωB),end=t;"StopIntegration"]
}

Here we make a model object:

model2=
<|"Rates"-><|
α->"Arms quality of A",
β->"Arms quality of B",
ωA->"Will to fight of A soldiers",
ωB->"Will to fight of B soldiers"|>,
"Stocks"-><|A[t]->"Soldiers of A",
B[r]->"Soldiers of B"|>,
"Equations"->eqs2,
"RateRules"-><|α->1.5,β->1.9|>,
"InitialConditions"->{A[0]==10^5,B[0]==10^5}|>;

Here we plug-in the model into the monadic pipeline, simulate, and plot solutions:

ECMMonUnit[]⟹
ECMMonSetSingleSiteModel[model2]⟹
ECMMonAssignInitialConditions[<|A[0]->2*10^4,B[0]->10^4|>]⟹
ECMMonAssignRateRules[<|α->0.04,β->0.12,ωA->0.3,ωB->0.7|>]⟹
ECMMonEchoModelGridTableForm⟹
ECMMonSimulate[100]⟹
ECMMonPlotSolutions[PlotRange->{0,2.1*10^4}];

Simulation end time:

end

Third choice for a talk

This is my third choice for a talk.

The first one is titled "Becoming the best Python programmer I know". (Which was rejected within 25 min of submission.)

The related software technology work -- the monads and code generators -- have R and WL as primary targets. Python is secondary target. (Everything else is ternary.)

Goals of the project

Speeding up the coming of the next AI winter

Making the future more evenly distributed

Replacing expertise with enthusiasm

Facilitating the professional life of code-baristas

Rapid specification of executable code

Knowledge transfer

Clothes have no emperor

Book structure

I am writing a book titled "Simplified Machine Learning Workflows."

Here is the structure of the book:

Import["https://github.com/antononcube/SimplifiedMachineLearningWorkflows-book/raw/master/org/Simplified-Machine-Learning-Workflows-mind-map.pdf","PageGraphics"][[1]]

Big-picture book-wise

Here are dependencies with other books and related GitHub repositories:

ImageCrop@Import["https://raw.githubusercontent.com/antononcube/HowToBeADataScientistImpostor-book/master/Part-1-General-directions-and-orientation/Diagrams/Books-dependencies.jpg"]

Classification workflows

First we are going to consider classification workflows.

This is the first non-trivial monad I implemented.

Making competitions classifiers

Import["https://github.com/antononcube/SimplifiedMachineLearningWorkflows-book/raw/master/Diagrams/Making-competitions-classifiers-mind-map.pdf","PageGraphics"][[1]]

Classification workflows diagram

ImageCrop@Import["https://github.com/antononcube/SimplifiedMachineLearningWorkflows-book/raw/master/Diagrams/Classification-workflow-extended.jpg"]

Example: Classification (ClCon)

Here is a classification workflow specification and the corresponding template filled-in:

clCommand="Make a classifier for the data dsTitanic and using the method RandomForest. Show TruePositve rate and FalsePositiveRate.";

Here we generate code for the software monad ClMon, [AAp3]:

clRes=Concretize["ClCon",clCommand]

Here the code generated above is evaluated:

clRes//ReleaseHold

Remark: Note that the correct dataset name and classifier method are filled-in in the ClCon template.

Remark: Due to some wrong answers for the classifier measures different measures are given.

Imperative WL code (instead of monadic)

After a few demos of this approach I am often told:

I don't care about your monads -- can I get code with built-in commands or standard packages?

Yes:

Concretize["Classification",clCommand]

Special features

These ClCon features that are not built-in:

Ensemble classifiers

Direct access to ROC functions

Classification by a threshold

Classifier ensembles

Here is an example of classifier ensemble:

SeedRandom[12];
res=
ClConUnit[dsTitanic]⟹
ClConSplitData[0.75,Method->"LabelsProportional"]⟹
ClConSummarizeData["MaxTallies"->8]⟹
ClConEchoFunctionValue["long form summary:",ColumnForm]⟹
ClConMakeClassifier[{{"LogisticRegression",0.87},{"LogisticRegression",0.93},{"SupportVectorMachine",0.94}}]⟹
ClConClassifierMeasurements[{"Accuracy","Precision","Recall"}]⟹
ClConEchoValue;

Latent Semantic Analysis workflows

Next we consider Latent Semantic Analysis (LSA) workflows.

At this point I have extensively published about LSA applications within the WL's ecosystem. For example, my winning computational art contest 2022 entry.

Remark: LSA is closely related to dimension reduction. Except the built-in SingularValueDecomposion there are Wolfram Function Repository (WFR) functions: NonNegativeMatrixFactorization and IndependentComponentAnalysis.

Latent Semantic Analysis workflows diagram

Import["https://github.com/antononcube/SimplifiedMachineLearningWorkflows-book/raw/master/Diagrams/LSA-workflows.jpg"]

Example: Latent Semantic Analysis (LSAMon)

Here is a LSA workflow specification and the corresponding template filled-in:

lsaCommand="Extract 20 topics from the text corpus aAbstracts using the method NNMF. Show statistical thesaurus with the words neural, function, and notebook";

Here we generate code for the software monad LSAMon, [AAp4]:

lsaRes=Concretize["LSAMon",lsaCommand]

Here the code generated above is evaluated:

lsaRes//ReleaseHold

Remark: Note that the correct dataset name, dimension reduction method, number of topics, and statistical thesaurus are filled-in in the LSAMon template.

Not just text collections

The LSA monad can be applied to images collections. See

"LSA methods comparison over random mandalas deconstruction -- Python"

"LSA methods comparison over random mandalas deconstruction -- WL"

These two MNIST recognition bases can be used to summarize differences between Statistics and Machine Leaning.

Not just text collections -- MNIST

For more details see "Handwritten digits recognition by matrix factorization".

SVD

Quick algorithm, theoretically well justified. We have a mean image that is modified with difference images into an unknown image. (Kind of like regression.)

NNMF

Using an optimization algorithm over a landscape of lots of local minima. Easy to interpret results. An unknown digit is approximated as most likely combination of several known ways to write “5”. We also see “multiplicity of good models.”

Quantile regression workflows

The presented Quantile Regression (QR) workflows include Linear Regression and regression view Neural networks.

At this point I have made many posts about using QR in WL's ecosystem. There is WFR function "QuantileRegression".

Quantile regression workflows diagram

ImageCrop@Import["https://github.com/antononcube/SimplifiedMachineLearningWorkflows-book/raw/master/Diagrams/Quantile-regression-workflow.jpg"]

Example: Quantile Regression

Here is QR workflow specification:

qrCommand="Do quantile regression with 12 knots and probabilities 0.1, 0.5 and 0.7 over the dataset dfTemperatureData. Use a datelist plot.";

Here we generate code for the a typical statistical regression workflow based on QR, [AAp5]:

qrRes=Concretize["QRMon",qrCommand,"TargetLanguage"->"WL"]

Here we evaluate the generated code:

qrRes//ReleaseHold

QR vs Neural nets

There is fairly close analogy between QR and regression with Neural networks:

ResourceFunction["GridTableForm"][{
{"Basis of functions\nB-spline, Chebyshev","ReLU, Ramp, LogisticSigmoid"},
{"Number of knots","Number of neurons"},
{"Probabilities","Loss function"},
{"Linear programming","Opmization algorithms"},
{"Over-training","Over-training"}
},TableHeadings->{"QR","Neural nets"}]

dfTemperatureData2=Map[If[AbsoluteTime[{2021,2,1}]≤#[[1]]≤AbsoluteTime[{2021,4,1}],{#[[1]],2*#[[2]]},{#[[1]],#[[2]]}]&,dfTemperatureData];
DynamicModule[{data,qrObj},
Manipulate[
data=If[whichData=="Changed",dfTemperatureData2,dfTemperatureData];
qrObj=
QRMonUnit[data]⟹
QRMonQuantileRegression[knots,0.5,InterpolationOrder->ord]⟹
QRMonSetRegressionFunctionsPlotOptions[{PlotStyle->Red}]⟹
QRMonSetDataPlotOptions[{PlotStyle->GrayLevel[0.6]}]⟹
QRMonDateListPlot[AspectRatio->1/3,ImageSize->1000,PlotRange->{0,35},"Echo"->False];
qrObj⟹QRMonTakeValue,
{{knots,20,"number of knots:"},1,200,1,Appearance->"Open"},
{{ord,2,"interpolation order:"},{0,1,2,3}},
{{whichData,"Original","Which data? :"}, {"Original","Changed"}}]
]

Other programming languages

As it was mentioned, the code for other programming languages can be generated:

Concretize["LSAMon",lsaCommand,"TargetLanguage"->"Python"]

Concretize["QRMon",qrCommand,"TargetLanguage"->"R"]

Recommendation workflows

I am not sure what is audience's interest in to Recommendation System (RS) building and usage as an ML sub-culture

Of course, at this point everyone has be exposed or been a subject of an RS

In the last 14 years at least 60% of the time I have worked on RS, or data science tasks supporting RS

I make RS objects very often for data exploratory / analysis purposes

RS object provide "1st level of regression"

See an "umbrella" project "Scalable Recommender Framework project":

Import["https://github.com/antononcube/Scalable-Recommender-Framework-project/raw/main/Diagrams/Scalable-Recommender-Framework-mind-map-Personal.pdf","PageGraphics"][[1]]

Recommendation workflows diagram

ImageCrop@Import["https://github.com/antononcube/SimplifiedMachineLearningWorkflows-book/raw/master/Diagrams/SMR-workflows.jpg"]

Example: Recommendations (SMRMon)

Here is QR workflow specification:

smrCommand="Make a recommender using dsTitanic. Give top 8 recommendations using the profile male and 1st.";

Here we generate code for the software monad QRMon, [AAp5]:

smrRes=Concretize[smrCommand]

Here we evaluate the generated code:

smrRes//ReleaseHold

Remark: Note that the correct dataset name, profile, and number of top results are filled-in in the SMRMon template.

Example: Random data generation

Here is a more elaborated example with RDG:

rtdCommand="Create a random dataset with 30 rows, 8 columns, and 60 values using column names generator RandomPetName.";
res=Concretize["RandomDataset",rtdCommand]

Here we define a few random data generation functions:

Here we evaluate the code produced above:

res//ReleaseHold

Remark: Note the correct number of rows, number of columns, column names generator, and min/max number of values are filled-in in the template of RandomTabularDataset.

Python or R code

res1=Concretize[rtdCommand,"TargetLanguage"->"Python"]

random_data_frame(n_rows = 30, columns_spec = 8, column_names_generator = random_pet_name, form =  "Wide", max_number_of_values = 60, min_number_of_values = 60, row_names = False)

RandomDataFrame(nrow = 30, ncol = 8, columnNamesGenerator = RandomPetName, form =  "Wide", rowNamesQ = FALSE)

Data transformations workflows

Data transformations workflows -- or Data Wrangling -- has been extensively discussed in previous WTC presentations and live coding sessions.

Multi-language Data-Wrangling Conversational Agent

Data Transformation Workflows with Anton Antonov, Session #1

Data Transformation Workflows with Anton Antonov, Session #2

Most of the time data scientists do data acquisition and data wrangling.

Data wrangling in general

Although data acquisition and data transformations are not formally part ML, most of the time data scientists and ML practitioners do various "data wrangling" tasks.

Explaining Data Science with trucks

lsPages=Import["https://github.com/antononcube/HowToBeADataScientistImpostor-book/raw/master/Presentations/OMLDS-Oct-2019/Illustrating-DS-approaches-with-trucks.pdf"];
TabView[lsPages]

Data wrangling workflows diagram

The target is tabular data or lists or associations of tabular data:

Import["https://github.com/antononcube/SimplifiedMachineLearningWorkflows-book/raw/master/Diagrams/Tabular-data-transformation-workflows.jpg"]

Example: Data Wrangling

Data wrangling requires more that just templates -- hence using grammars is the right way of addressing the generation of data wrangling workflows.

ResourceFunction["DSLTranslation"]["
DSL MODULE DataQueryWorkflows;
use dsTitanic;
show dimensions
group by passengerSex;
show counts;
ungroup;
cross tabulate passengerSex and passengerSurvival;
show pipeline value
"]

obj=dsTitanic;Echo[Dimensions[obj],"dimensions:"];obj=GroupBy[obj,#1["passengerSex"]&];Echo[Length/@obj,"counts:"];obj=Join@@Values[obj];obj=ResourceFunction["CrossTabulate"][({#1["passengerSex"],#1["passengerSurvival"]}&)/@obj];Echo[obj];

Data wrangling -- current status

The following diagram shows the current status of the data wrangling constellation of parsers, interpreters, and (execution-)packages:

plDataWranglingStatus=ImageCrop@Import["https://github.com/antononcube/RakuForPrediction-book/raw/main/Diagrams/DSLs-Interpreter-for-Data-Wrangling-August-2022-state.pdf","PageGraphics"][[1]]

Data wrangling -- notebooks in Python, R, Raku, WL

There is a GitHub repository with notebooks for different programming languages.

The notebooks are in Markdown, Mathematica, and Jupyter

Jupyter notebooks

Mathematica notebooks

Workflow combinations

Combining the workflows can be done in several ways. Here are few examples:

"Handwritten Arabic characters classifiers comparison"

Combines ClCon and LSAMon

(Beats Neural networks classifiers)

"Re-exploring the structure of Chinese character images"

Combines SMRMon and LSAMon

Of course, any data wrangling workflow combines with the others

Parallel construction

Consider the following Python projects:

Mind-map

ImageCrop@Import["https://github.com/antononcube/PythonForPrediction-blog/raw/main/MarkdownDocuments/Diagrams/Becoming-the-Python-programmer-I-know/Becoming-the-best-Python-programmer-I-know-mind-map.pdf","PageGraphics"][[1]]

Flowchart

Import["https://github.com/antononcube/PythonForPrediction-blog/raw/main/MarkdownDocuments/Diagrams/Becoming-the-Python-programmer-I-know/Becoming-the-best-Python-programmer-I-know.pdf","PageGraphics"][[1]]

Additional topics

Not owning ML implementations in a variety of ways

LSA is better than BERT ?!

Taylor-made ML algorithms for specific applications

Bringing it in: Web service, WFR, iOS app

Sparse matrix with named rows and columns is the linchpin

Same approach for Google Cloud Console (GCP), Amazon SageMaker (AWS), Oracle Cloud Infrastructure (OCI)

Some history of the project

The need for monad-type workflows functionalities become obvious at 2017 summer school in Oxford

ClCon monad first, then LSAMon and SMRMon

Parallel implementation of R monads

Grammar parsers and interpreters implementation

Started with WL

Moved to Raku in the fall of 2018

People who would be interested to learn about ML workflows do "not get" the monadic workflows

Several reasons for this

Pipelines for some reason seemed unnatural

Too much focus on "how you are doing that?"

Hence, implementing data wrangling grammars and interpreters in 2020

Implementation of Python OOP packages in the fall of 2021

Extending the technological solutions with data acquisition workflows 2021

Too many "real-life" obstacles (COVID-19, SMO/War, etc.)

Call for collaboration

Currently I translate to code commands in Bulgarian and English

I am interested to add other languages

A related much easier task is to add other language targets

Currently I translate to Bulgarian, English, and Russian

Sometimes to Korean and Spanish

"Just" using it

Installation

Mileage

Unit testing

Different languages

Different ML sub-cultures

Selling points

The work on this project is sort of absolute

The system is very modular and the modules are lightweight and independent

Future plans

Make Wolfram Paclet Repository (WPR) entries for all monads

Finishing the book

More streamlined parsers and interpreters access in WL

Moving many of the grammars to WL

Questions

Anticipated

Is this a product?

Can the modules be used by themselves?

Can you translate WL code (or Python, or R) into Python (or WL, or R, or English)?

Why use monads?

What is not covered?

References

Articles

[AA1] Anton Antonov, “ROC for classifier ensembles, bootstrapping, damaging, and interpolation”, (2016), MathematicaForPrediction at GitHub, folder Documents. WordPres blog post.

[AA2] Anton Antonov, “Distribution extraction from a Gaussian noised mixture”, (2016), MathematicaVsR at GitHub, project DistributionExtractionAFromGaussianNoisedMixture .

[AA3] Anton Antonov, “RSparseMatrix for sparse matrices with named rows and columns”, (2015), MathematicaForPrediction at WordPress.

[AA4] Anton Antonov, “Quantile regression through linear programming”, (2013), MathematicaForPrediction at WordPress.

[AA5] Anton Antonov, “Morphological analysis of data mining and prediction algorithms”, (2013), MathematicaForPrediction at WordPress.

[AA6] Anton Antonov, “Estimation of conditional density distributions”, (2014), MathematicaForPrediction at WordPress.

[JL1] Jérôme Louradour, "New in the Wolfram Language: FindTextualAnswer", (2018), blog.wolfram.com.

[Wk1] Wikipedia entry, Question answering.

[Wk2] Wikipedia entry, Template processor.

Books

[AA1] Anton Antonov, Simplified Machine Learning Workflows, (2018), GitHub/antononcube. 
           (Unfinished.)

Functions

[AAf1] Anton Antonov, GridTableForm, (2020), Wolfram Function Repository.

[AAf2] Anton Antonov, RandomTabularDataset, (2021), Wolfram Function Repository.

[WRI1] Wolfram Research, FindTextualAnswer, (2018), Wolfram Language function, https://reference.wolfram.com/language/ref/FindTextualAnswer.html (updated 2020).

Packages

[AAp1] Anton Antonov, Computational Spec Completion Mathematica package, (2021), MathematicaForPrediction at GitHub/antononcube.

[AAp2] Anton Antonov, Computational Spec Completion Mathematica package, (2021), MathematicaForPrediction at GitHub/antononcube.

[AAp3] Anton Antonov, ClCon, (2021), MathematicaForPrediction at GitHub/antononcube.

[AAp4] Anton Antonov, LSACon, (2019), MathematicaForPrediction at GitHub/antononcube.

[AAp5] Anton Antonov, QRMon, (2019), MathematicaForPrediction at GitHub/antononcube.

[AAp6] Anton Antonov, SMRMon, (2019), MathematicaForPrediction at GitHub/antononcube.

Videos

[AAv1] Anton Antonov, "NLP Template Engine, Part 1", (2021), Anton Antonov's channel at YouTube.

Load packages and data

Packages

Import["https://raw.githubusercontent.com/antononcube/NLP-Template-Engine/main/Packages/WL/NLPTemplateEngine.m"]
Import["https://raw.githubusercontent.com/antononcube/SystemModeling/master/Projects/Coronavirus-propagation-dynamics/WL/MonadicEpidemiologyCompartmentalModeling.m"]

Data

Quantile Regression

tsFinData=FinancialData["NYSE:GE",{{2020,1,1},{2022,1,1},"Day"}];
dfFinData=QuantityMagnitude@tsFinData["Path"];

tsTemperatureData=WeatherData[{"Orlando","Florida"},"Temperature",{{2020,1,1},Now,"Day"}];
dfTemperatureData=QuantityMagnitude@tsTemperatureData["Path"];

Latent Semantic Analysis

dsAbstracts=ResourceFunction["ImportCSVToDataset"]["https://raw.githubusercontent.com/antononcube/SimplifiedMachineLearningWorkflows-book/master/Data/Wolfram-Technology-Conference-2016-to-2019-abstracts.csv"];
aAbstracts=Normal@dsAbstracts[Association,#ID->#Abstract&];

Length[aAbstracts]

RandomSample[dsAbstracts,4]

Classification

dsTitanic=ResourceFunction["ImportCSVToDataset"]["https://raw.githubusercontent.com/antononcube/MathematicaVsR/master/Data/MathematicaVsR-Data-Titanic.csv"];

RandomSample[dsTitanic,6]

Random data generation

Clear[RandomString,RandomPetName];
RandomString[n_Integer,args___]:=Block[{},Table[ResourceFunction["RandomString"][Max[2,RandomVariate[PoissonDistribution[6]]]],n]];
RandomPetName[n_Integer,args___]:=ResourceFunction["RandomPetName"][n];

Python setup

from RandomDataGenerators import *
import pandas
from SparseMatrixRecommender import *
from LatentSemanticAnalyzer import *

R setup

Needs["RLink`"];
InstallR[
"RHomeLocation"->"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources",
"RVersion"->"4.2","NativeLibLocation"->$HomeDirectory<>"/Library/R/arm64/4.2/library/rJava/jri"]

{
library(magrittr)
library(Matrix)
library(RandomDataGenerators)
library(LSAMon)
library(QRMon)
library(SparseMatrixRecommender)
library(SMRMon)

library(MathematicaVsRData)
}

{
dfAbstracts <- read.csv("https://raw.githubusercontent.com/antononcube/SimplifiedMachineLearningWorkflows-book/master/Data/Wolfram-Technology-Conference-2016-to-2019-abstracts.csv")
aAbstracts <- setNames( dfAbstracts$Abstract, dfAbstracts$ID)
length(aAbstracts)
}

Shell

source ~/.zshrc