Unified Fitting and Modeling in Wolfram Language

Giulio Alessandrini, WRI

#WolframTechConf

ABSTRACT

The new unified model framework brings together statistical modeling, machine learning and symbolic computation in a single, consistent interface. We'll show how diverse model families—linear, nonlinear, generalized additive, tree-based and more—fit into the same symbolic pipeline. Built for efficiency with the new Tabular object, the framework supports high-performance fitting on large datasets while preserving Wolfram Language's advantages in probability, integration and symbolic analysis.

OUTLINE

A Fragmented Landscape

A New Unified Framework

Fitting and Model Selection

Advanced Reporting

Future Directions

A Fragmented Landscape

Today, fitting data requires navigating many specialized functions:

Framed[Grid[Append[{"FittedModel","special function","parameters/expression"}]@Transpose@PadRight[{
{Style["Statistical Models:",Bold],"- LinearModelFit","- GeneralizedLinearModelFit","- NonlinearModelFit","- LogitModelFit","- ProbitModelFit","- LocalModelFit"},
{Style["Machine Learning:",Bold],"- Classify","- Predict"},
{Style["Direct Fitting:",Bold],"- Fit","- FindFit"}
},Automatic,SpanFromAbove],Alignment->{Left,Top},BaseStyle->"Text",Spacings->{1, {1,1,{.5},1,1}},Dividers->{None,{2->True,-2->True}},FrameStyle->SetAlphaChannel[StandardGray,.3]],ImageMargins->10,FrameStyle->None]

Plus other specialized functions like TimeSeriesModelFit, EstimatedDistribution, EstimatedProcess, RegionFit.

Pain Points:

Different interfaces, different outputs

Can't easily compare LinearModelFit vs Classify

No unified way to do model selection across paradigms

Adding new model types requires new *ModelFit functions

A New Unified Framework

Design Goal: One interface. Any model. Extensible architecture

A New Unified Framework

Three Core Components:

MODEL OBJECTS - Symbolic, extensible, composable

PolynomialModel[3, {x, y}]
DecisionTreeModel[]
PowerModel[{a, b, c}, {x, y}]

SINGLE TRAINER - ModelFit works with everything

ModelFit[data, model]
ModelFit[data, {model1, model2, ...}]  (* automatic selection *)

SMART REPORTS - Model-specific diagnostics

Statistical models -> ANOVA, parameter inference, residuals
ML models -> feature importance, cross-validation, tree depth

Model Objects: The Building Blocks

All models follow a general structure

Model[]
Model[hyperparameters]
Model[hyperparameters, variables]
Model[hyperparameters, parameters, variables]

Model Objects: The Building Blocks

Polynomial with fitted parameters

PolynomialModel[2,{0.1,0.2,0.3},x]

Polynomial with symbolic parameters (+ just using input size)

PolynomialModel[2,Automatic,3]

Mixed: track parameters symbolically (+ giving custom variable names)

PolynomialModel[2,{a,b->0.2,c->0.3},myvar]

ML model - same pattern

DecisionTreeModel["MaxDepth"->10]

NeuralModel[<|"Width"->10,"Activation"->"GELU"|>]

Model Objects: The Building Blocks

Model expression can be evaluated as functions

model=PolynomialModel[2,{-0.4,1.3,0.45},x]

Evaluate at a point

model[2]

Vectorized evaluation

model[{1,2,3,4}]

Multivariate models work the same way

PolynomialModel[2,{x,y}][{0.5,0.8}]

Built for Extension

Model Hierarchy

Grid[Join[
{{Style["LinearModel", Bold],SpanFromLeft},
{"PolynomialModel","specialized: easy syntax for polynomials"},
{"FourierModel","specialized: periodic data"},
{"GeneralizedAdditiveModel","[future]"}
},
{{Style["NonlinearModel", Bold],SpanFromLeft},
{"PowerModel",TraditionalForm[a x^b]},
{"ExponentialModel",TraditionalForm[a Exp[b x]]},
{"LogModel",TraditionalForm[a+b Log[x]]}
},
{{Style["Trees", Bold],SpanFromLeft},
{"DecisionTreeModel","classification/regression tree"},
{"RandomForestModel","[future]"}
},
{{Style["", Bold],SpanFromLeft},
{"NeuralModel","neural networks"},
{"NearestModel","k-NN classification/regression"}
}
],Alignment->{Left,Top},BaseStyle->"Text",Spacings->{1, .8},Dividers->SetAlphaChannel[StandardGray,.3]]

Real Example: Noisy Nonlinear Surface

The Challenge:

200 noisy observations

Two input variables (x, y)

Unknown functional form

Which model fits best?

ListPlot3D[data=,PlotFit->"Cubic",PlotFitElements->"DataPoints",PlotRange->All]

Model Selection: The Candidates

Seven different model families - parametric, nonparametric, ML - in one list

models={
PolynomialModel[UpTo[5]],
PowerModel[],
ExponentialModel[],
LogModel[],
LinearModel[{x,y,x y},{x,y}],
DecisionTreeModel[],
NearestModel[5]
};

Statistical: Polynomial family (degree 1→5)
Nonlinear: Power, Exponential, Log
Custom: LinearModel with interaction term
ML: DecisionTree, k-NN (non-parametric)

Training: One Line

report=ModelFit[data,models,"Report"]

What happened behind the scenes:

Trained all 11 models

5-fold cross-validation (automatic)

Computed performance metrics

Selected best model

Packaged everything in one report object

Which Model Won? Why?

The ModelFitReport object can be used to build reports and visualizations.

report["CrossValidationChart"]

Exponential/Power perform poorly (wrong functional form)

Polynomial(2), (3) and (4) have the lowest, most stable loss. (3) has lower median

Polynomial(5) is beginning to overfitting

The other models are doing better but fail to capture the trend

Which Model Won? Why?

The ModelFitReport object can be used to build reports and visualizations.

Tabular[report[{"ShortName","ValidationLoss","TrainingLoss"}->All],{"ShortName","ValidationLoss","TrainingLoss"}]

Deep Dive: Best Model Analysis

Extract best model statistics

report[{"BIC","AIC","RSquared"}]

Parameter estimates

Tabular[report["ParameterEstimates"][All,<|"BasisFunction"->#BasisFunction,"Parameter"->Around[#BestFitParameter,#StandardError],"TStatistic"->#TStatistic,"PValue"->#PValue|>&],<|"KeyColumns"->{"BasisFunction"}|>]

ANOVA

Tabular[report["ANOVA"],<|"KeyColumns"->{"Term"}|>]

Cross-Validation: Under the Hood

Compare losses across folds

ListLinePlot[Callout@@@report[{"ValidationLossList","ShortName"}->All],PlotRange->All,ScalingFunctions->{"Linear","Log"},PlotTheme->"Detailed",FrameLabel->{"Cross validation fold","Validation Loss"},ImageSize->600]

Compare training time and performance

ListPlot[{Callout[{##2},#1]}&@@@report[{"ShortName","TrainingTime","Loss"}->All],Frame->True,FrameLabel->{"Training Time (s)","Validation Loss"},ScalingFunctions->{"Log","Log"},ImageSize->400,PlotTheme->"Detailed"]

Beyond Fitting

Models can be used as numerical functions

bestModel=report["BestModel"];

Predict new points

bestModel[{0.6,0.9}]

Visualize the function

Plot3D[bestModel[{x,y}],{x,0.5,1},{y,0.5,1}]

Beyond Fitting

Or symbolically

Model derivative

D[bestModel[{x,y}],x]

Model integral

Integrate[bestModel[{x,y}],{x,0,1}]//Simplify

Advanced Features

Meta-Models (Regularization)

Control model properties via meta-model wrappers

ModelFit[data,RegularizedModel[PolynomialModel[2],"L1"->1]]

Data Types

Built-in support for categorical data, units, dates

ModelFit[ResourceData["Sample Tabular Data: Titanic"]->"survived",DecisionTreeModel[],"Function"]

Model Families

Compact specification for different model hyperparameters

ModelFit[data,NearestModel["Neighbors"->3|5|11]]

ModelFit[data,PolynomialModel[UpTo[10]]]

Current Status & Roadmap

Available Now (in development)

✅ Core framework: ModelFit + model objects + ModelFitReport
✅ Statistical models: Linear, Polynomial
✅ Nonlinear models: Power, Exponential, Log
✅ ML models: DecisionTree, Nearest
✅ Specialized: Fourier, Neural
✅ Cross-validation and model selection
✅ Model families

Coming Soon

🔄 Generalized Linear
🔄 Feature parity with LinearModelFit (weighted data, categorical support)
🔄 Additional ML models (RandomForest, SVM, GaussianProcess)
🔄 Additive models
🔄 Advanced categorical handling
🔄 Advanced hyperparameter selection

Future Exploration

💡 Peak fitting
💡 Differential equation models
💡 Hierarchical/mixed-effects models

Key Takeaways

For the Users

One interface: ModelFit[data, model]
• Works with statistical, ML, and specialized models
• Automatic cross-validation and selection
• Rich diagnostics adapted to each model type

For Developers

Extensible architecture
• Add specialized models easily (inherit + customize)
• Each model declares its own report properties
• No need to modify core framework

For the Ecosystem

Unified framework = better science
• Fair comparison across paradigms
• Consistent interface reduces learning curve
• Leverage WL's symbolic power throughout

THANK YOU