Biological Data Insight & Modeling

Mark Kotanchek, Evolved Analytics LLC

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

Multiomics (genomics, proteomics, lipisomics, etc.) data is typically ill-conditioned with many (coupled) variables and relatively few data records. As a result, such data sets are very difficult to analyze with conventional statistical and machine learning techniques.

ParetoGP assumes we can develop simple algebraic models using just a few of the inputs via an evolutionary search rewarding model simplicity and accuracy. From the thousands of candidate models, we can garner insights on key variables, variable associations and combinations as well as generate concise explainable and human-interpretable models. 

In this presentation, we demonstrate the insight and modeling process using DataModeler applied to cancer therapeutics and bone regeneration data sets.

The DataModeler GUI

Search & Model Selection Nuances

GridTable[{{, 
GridTable[{
"Subsample for efficiency if big data",
"Interval Arithmetic for quantized data",
"Limit variables & bases considering data set size",
"ANOVATrim makes the statisticians happy",
"Ensembles give us Trustable Models"
}, Label-> "Guidelines", FontSize -> 20, Dividers -> None]
}}, Label -> None, FontSize -> 32]

Data Reality is Painful

Interesting Models are at the Knee of the Pareto Front

GridTable[{{(Labeled[({{}}), "Pareto Front Context LogPlot — 928 of 8953 unique models selected"]), 
GridTable[{
"Accuracy is better",
"Simplicity is better",
"Want to balance complexity & accuracy"
}, Label-> "Essential Assumptions", FontSize -> 20, Dividers -> None]
}}, Label -> None, FontSize -> 32]

52 x 20-minute searches

8,200 allowed variables

Model Dimensionality — How Many Variables?

GridTable[{{
(Labeled[({{"# Vars", "# Models ⟹ %", "VariableCombinationMap", "ParetoFrontPlot"}, {"2", "73 ⟹ \!\(\*FormBox[TagBox[InterpretationBox[StyleBox[\"\\\"5.3\\\"\", Rule[ShowStringCharacters, False]], 5.309090909090909`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[4, 1], Rule[ExponentFunction, Function[If[Less[-1, Slot[1], 4], Null, Slot[1]]]], Rule[NumberPadding, List[\"\", \"0\"]]]]], TraditionalForm]\) %", , ({{}})}, {"3", "134 ⟹ \!\(\*FormBox[TagBox[InterpretationBox[StyleBox[\"\\\"9.7\\\"\", Rule[ShowStringCharacters, False]], 9.745454545454544`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[4, 1], Rule[ExponentFunction, Function[If[Less[-1, Slot[1], 4], Null, Slot[1]]]], Rule[NumberPadding, List[\"\", \"0\"]]]]], TraditionalForm]\) %", , ({{}})}, {"4", "184 ⟹ \!\(\*FormBox[TagBox[InterpretationBox[StyleBox[\"\\\"13.4\\\"\", Rule[ShowStringCharacters, False]], 13.381818181818181`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[4, 1], Rule[ExponentFunction, Function[If[Less[-1, Slot[1], 4], Null, Slot[1]]]], Rule[NumberPadding, List[\"\", \"0\"]]]]], TraditionalForm]\) %", , ({{}})}, {"5", "456 ⟹ \!\(\*FormBox[TagBox[InterpretationBox[StyleBox[\"\\\"33.2\\\"\", Rule[ShowStringCharacters, False]], 33.163636363636364`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[4, 1], Rule[ExponentFunction, Function[If[Less[-1, Slot[1], 4], Null, Slot[1]]]], Rule[NumberPadding, List[\"\", \"0\"]]]]], TraditionalForm]\) %", , ({{}})}, {"6", "523 ⟹ \!\(\*FormBox[TagBox[InterpretationBox[StyleBox[\"\\\"38.0\\\"\", Rule[ShowStringCharacters, False]], 38.03636363636363`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[4, 1], Rule[ExponentFunction, Function[If[Less[-1, Slot[1], 4], Null, Slot[1]]]], Rule[NumberPadding, List[\"\", \"0\"]]]]], TraditionalForm]\) %", , ({{}})}}), "\!\(\*FormBox[StyleBox[\"\\\"Model Dimensionality of 1375 ratio•of•CD4•to•CD8•Memory•Cells Day 6 Models\\\"\", Rule[FontSize, 12], Rule[FontWeight, Bold], Rule[FontFamily, \"Helvetica\"], Rule[FontSlant, \"Plain\"], Rule[FontColor, GrayLevel[0]], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\)"]),
GridTable[{
"How many variables matter?",
"What is the implication of adding \ncomplexity and variables?",
"There are multiple paths\n(variable combinations) to success"
}, Label-> None, FontSize -> 20, Dividers -> None]
}}, Label -> None, FontSize -> 36]

What Variables Matter?

Variable Presence

GridTable[{{
({{}, {}, {}, {}}),
GridTable[{
"Started with 8,382 allowed metabolites & 20 data records",
"Restricted to less than 3 variables or bases",
"~200 generations ⟹ 4(x10^6) models considered",
"5(x10^11) possible triplets ⟹ needle in a haystack",
"m5098 in 11% of the models & 3.7% linear correlation to target",
"Use to focus subsequent modeling ⟹ Smaller Haystack"
}, Label-> None,Alignment -> Left,  FontSize -> 28, Dividers -> None]
}}, Label -> None, FontSize -> 36]

Variable Combinations

GridTable[{{
(Labeled[({{" ", "num ⟹ %", "Variables Used", "ParetoFrontPlot"}, {"1", "50 ⟹ \!\(\*FormBox[TagBox[InterpretationBox[StyleBox[\"\\\"5.4\\\"\", Rule[ShowStringCharacters, False]], 5.387931034482758`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[4, 1], Rule[ExponentFunction, Automatic], Rule[NumberPadding, List[\"\", \"0\"]]]]], TraditionalForm]\) %", ({{"\!\(\*FormBox[StyleBox[\"m4284\", Hue[Rational[4349, 8347], 1, Rational[5, 6]], Rule[StripOnInput, False]], TraditionalForm]\)"}, {"\!\(\*FormBox[StyleBox[\"m5477\", Hue[Rational[326, 491], 1, Rational[1, 2]], Rule[StripOnInput, False]], TraditionalForm]\)"}, {"\!\(\*FormBox[StyleBox[\"m6536\", Hue[Rational[6601, 8347], 1, Rational[5, 6]], Rule[StripOnInput, False]], TraditionalForm]\)"}}), ({{}})}, {"2", "46 ⟹ \!\(\*FormBox[TagBox[InterpretationBox[StyleBox[\"\\\"5.0\\\"\", Rule[ShowStringCharacters, False]], 4.956896551724138`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[4, 1], Rule[ExponentFunction, Automatic], Rule[NumberPadding, List[\"\", \"0\"]]]]], TraditionalForm]\) %", ({{"\!\(\*FormBox[StyleBox[\"m5889\", Hue[Rational[5954, 8347], 1, Rational[1, 2]], Rule[StripOnInput, False]], TraditionalForm]\)"}, {"\!\(\*FormBox[StyleBox[\"m6072\", Hue[Rational[361, 491], 1, Rational[5, 6]], Rule[StripOnInput, False]], TraditionalForm]\)"}, {"\!\(\*FormBox[StyleBox[\"m6162\", Hue[Rational[6227, 8347], 1, Rational[5, 6]], Rule[StripOnInput, False]], TraditionalForm]\)"}}), ({{}})}, {"3", "36 ⟹ \!\(\*FormBox[TagBox[InterpretationBox[StyleBox[\"\\\"3.9\\\"\", Rule[ShowStringCharacters, False]], 3.8793103448275863`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[4, 1], Rule[ExponentFunction, Automatic], Rule[NumberPadding, List[\"\", \"0\"]]]]], TraditionalForm]\) %", ({{"\!\(\*FormBox[StyleBox[\"m5477\", Hue[Rational[326, 491], 1, Rational[1, 2]], Rule[StripOnInput, False]], TraditionalForm]\)"}, {"\!\(\*FormBox[StyleBox[\"m6670\", Hue[Rational[6735, 8347], 1, Rational[5, 6]], Rule[StripOnInput, False]], TraditionalForm]\)"}, {"\!\(\*FormBox[StyleBox[\"m7431\", Hue[Rational[7496, 8347], 1, Rational[1, 2]], Rule[StripOnInput, False]], TraditionalForm]\)"}}), ({{}})}}), "\!\(\*FormBox[StyleBox[\"\\\"Variable Combination Table\\\"\", Rule[FontSize, 12], Rule[FontWeight, \"Bold\"], Rule[FontFamily, \"Helvetica\"], Rule[FontSlant, \"Plain\"], Rule[FontColor, GrayLevel[0]], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\)"]),
GridTable[{
"Variable combinations with proven performance \ncan be worth further investigation"
}, Label-> None, FontSize -> 20, Dividers -> None]
}}, Label -> None, FontSize -> 36]

Variable Associations

GridTable[{{
({{, }, {, (Labeled[({{"m1475", , , , }, {"m5357", , , , }, {"m6536", , , , }, {"Functional Score", , , , }}), "(d) Bivariate Plot"])}}),
GridTable[{
"Variables which work together well \n(aka in interesting models) can \nbe worth investigating",
"Different Variable Combinations \ncan be successful",
"Impactful variables need not be tightly \ncorrelated to the response"
}, Label-> None, FontSize -> 20, Dividers -> None, Alignment -> Left]
}}, Label -> None, FontSize -> 36]

MetaVariables & Basis Sets

MetaVariables can be thought of as linearizing transforms

They can provide insight into couplings & mechanisms

Ensembles

Training/Test/Validation data partitioning is needed in most methods because there is no notion of complexity control

Ensembles chosen from “good enough” models based upon residual diversity

Trust metric of ensembles means we can use all of the data for modeling

The human is responsible for providing properly diverse data

Bone Regeneration Ensembles

GridTable[{{
({{}, {(Labeled[({{, }, {, }}), "(b) Prediction Performance vs Variables Used for Bone Regeneration"])}}),
(Labeled[({{(Labeled[({{, , , , }}), "\!\(\*FormBox[StyleBox[\"\\\"\\\\!\\\\(\\\\*FormBox[StyleBox[\\\\\\\"\\\\\\\\\\\\\\\"bone•Volume•Wk20\\\\\\\\\\\\\\\"\\\\\\\", Rule[FontSize, 12], Rule[FontColor, RGBColor[0, 0, Rational[2, 3]]], Rule[FontFamily, \\\\\\\"Helvetica\\\\\\\"], Rule[FontWeight, \\\\\\\"Bold\\\\\\\"], Rule[FontSlant, \\\\\\\"Plain\\\\\\\"], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\\\\)  ⟹  2-var ensemble [ \\\\!\\\\(\\\\*FormBox[StyleBox[\\\\\\\"\\\\\\\\\\\\\\\"ref = \\\\\\\\\\\\\\\\!\\\\\\\\\\\\\\\\(\\\\\\\\\\\\\\\\*FormBox[TagBox[InterpretationBox[StyleBox[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"105.48\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", Rule[ShowStringCharacters, False]], 105.48037948193598`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[8, 2], Rule[ExponentFunction, Function[If[Less[-2, Slot[1], 8], Null, Slot[1]]]], Rule[NumberPadding, List[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"0\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"]]]]], TraditionalForm]\\\\\\\\\\\\\\\\)\\\\\\\\\\\\\\\"\\\\\\\", Rule[FontSize, 12], Rule[FontColor, RGBColor[0, Rational[2, 3], 0]], Rule[FontFamily, \\\\\\\"Helvetica\\\\\\\"], Rule[FontWeight, \\\\\\\"Bold\\\\\\\"], Rule[FontSlant, \\\\\\\"Plain\\\\\\\"], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\\\\) ]\\\"\", Rule[FontSize, 12], Rule[Background, None], Rule[FontFamily, \"Helvetica\"], Rule[FontWeight, \"Bold\"], Rule[FontColor, GrayLevel[0]], Rule[FontSlant, \"Plain\"], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\)"])}, {(Labeled[({{, , , , }}), "\!\(\*FormBox[StyleBox[\"\\\"\\\\!\\\\(\\\\*FormBox[StyleBox[\\\\\\\"\\\\\\\\\\\\\\\"bone•Volume•Wk20\\\\\\\\\\\\\\\"\\\\\\\", Rule[FontSize, 12], Rule[FontColor, RGBColor[0, 0, Rational[2, 3]]], Rule[FontFamily, \\\\\\\"Helvetica\\\\\\\"], Rule[FontWeight, \\\\\\\"Bold\\\\\\\"], Rule[FontSlant, \\\\\\\"Plain\\\\\\\"], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\\\\)  ⟹  3-var Ensemble (VC#1) [ \\\\!\\\\(\\\\*FormBox[StyleBox[\\\\\\\"\\\\\\\\\\\\\\\"ref = \\\\\\\\\\\\\\\\!\\\\\\\\\\\\\\\\(\\\\\\\\\\\\\\\\*FormBox[TagBox[InterpretationBox[StyleBox[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"101.26\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", Rule[ShowStringCharacters, False]], 101.26414318814311`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[8, 2], Rule[ExponentFunction, Function[If[Less[-2, Slot[1], 8], Null, Slot[1]]]], Rule[NumberPadding, List[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"0\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"]]]]], TraditionalForm]\\\\\\\\\\\\\\\\)\\\\\\\\\\\\\\\"\\\\\\\", Rule[FontSize, 12], Rule[FontColor, RGBColor[0, Rational[2, 3], 0]], Rule[FontFamily, \\\\\\\"Helvetica\\\\\\\"], Rule[FontWeight, \\\\\\\"Bold\\\\\\\"], Rule[FontSlant, \\\\\\\"Plain\\\\\\\"], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\\\\) ]\\\"\", Rule[FontSize, 12], Rule[Background, None], Rule[FontFamily, \"Helvetica\"], Rule[FontWeight, \"Bold\"], Rule[FontColor, GrayLevel[0]], Rule[FontSlant, \"Plain\"], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\)"])}, {(Labeled[({{, , , , }}), "\!\(\*FormBox[StyleBox[\"\\\"\\\\!\\\\(\\\\*FormBox[StyleBox[\\\\\\\"\\\\\\\\\\\\\\\"bone•Volume•Wk20\\\\\\\\\\\\\\\"\\\\\\\", Rule[FontSize, 12], Rule[FontColor, RGBColor[0, 0, Rational[2, 3]]], Rule[FontFamily, \\\\\\\"Helvetica\\\\\\\"], Rule[FontWeight, \\\\\\\"Bold\\\\\\\"], Rule[FontSlant, \\\\\\\"Plain\\\\\\\"], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\\\\)  ⟹  3-var Ensemble (VC#2) [ \\\\!\\\\(\\\\*FormBox[StyleBox[\\\\\\\"\\\\\\\\\\\\\\\"ref = \\\\\\\\\\\\\\\\!\\\\\\\\\\\\\\\\(\\\\\\\\\\\\\\\\*FormBox[TagBox[InterpretationBox[StyleBox[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"109.48\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", Rule[ShowStringCharacters, False]], 109.47558283814709`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[8, 2], Rule[ExponentFunction, Function[If[Less[-2, Slot[1], 8], Null, Slot[1]]]], Rule[NumberPadding, List[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"0\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"]]]]], TraditionalForm]\\\\\\\\\\\\\\\\)\\\\\\\\\\\\\\\"\\\\\\\", Rule[FontSize, 12], Rule[FontColor, RGBColor[0, Rational[2, 3], 0]], Rule[FontFamily, \\\\\\\"Helvetica\\\\\\\"], Rule[FontWeight, \\\\\\\"Bold\\\\\\\"], Rule[FontSlant, \\\\\\\"Plain\\\\\\\"], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\\\\) ]\\\"\", Rule[FontSize, 12], Rule[Background, None], Rule[FontFamily, \"Helvetica\"], Rule[FontWeight, \"Bold\"], Rule[FontColor, GrayLevel[0]], Rule[FontSlant, \"Plain\"], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\)"])}, {(Labeled[({{, , , , }}), "\!\(\*FormBox[StyleBox[\"\\\"\\\\!\\\\(\\\\*FormBox[StyleBox[\\\\\\\"\\\\\\\\\\\\\\\"bone•Volume•Wk20\\\\\\\\\\\\\\\"\\\\\\\", Rule[FontSize, 12], Rule[FontColor, RGBColor[0, 0, Rational[2, 3]]], Rule[FontFamily, \\\\\\\"Helvetica\\\\\\\"], Rule[FontWeight, \\\\\\\"Bold\\\\\\\"], Rule[FontSlant, \\\\\\\"Plain\\\\\\\"], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\\\\)  ⟹  4-var Ensemble [ \\\\!\\\\(\\\\*FormBox[StyleBox[\\\\\\\"\\\\\\\\\\\\\\\"ref = \\\\\\\\\\\\\\\\!\\\\\\\\\\\\\\\\(\\\\\\\\\\\\\\\\*FormBox[TagBox[InterpretationBox[StyleBox[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"105.46\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", Rule[ShowStringCharacters, False]], 105.45891454416144`, Rule[AutoDelete, True]], Function[NumberForm[Slot[1], List[8, 2], Rule[ExponentFunction, Function[If[Less[-2, Slot[1], 8], Null, Slot[1]]]], Rule[NumberPadding, List[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"0\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"]]]]], TraditionalForm]\\\\\\\\\\\\\\\\)\\\\\\\\\\\\\\\"\\\\\\\", Rule[FontSize, 12], Rule[FontColor, RGBColor[0, Rational[2, 3], 0]], Rule[FontFamily, \\\\\\\"Helvetica\\\\\\\"], Rule[FontWeight, \\\\\\\"Bold\\\\\\\"], Rule[FontSlant, \\\\\\\"Plain\\\\\\\"], Rule[Background, None], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\\\\) ]\\\"\", Rule[FontSize, 12], Rule[Background, None], Rule[FontFamily, \"Helvetica\"], Rule[FontWeight, \"Bold\"], Rule[FontColor, GrayLevel[0]], Rule[FontSlant, \"Plain\"], Rule[ShowStringCharacters, False], Rule[StripOnInput, False]], TraditionalForm]\)"])}}), "Response Comparison Plot"])

}}, Label -> None, FontSize -> 36]

Data Cubes & Summary Statistics

GridTable[{{
({{, (Labeled[({{}}), "(b) Summary 2-Var Ensemble"])}, {, }}),
GridTable[{
"RNA Single-Cell data ⟹ very noisy",
"Only 23 patients",
"Convert from 23 x 8,352 ⟹ 23 x 1,500 summary statistics",
"Summary statistics are more insightful",
"Issue:replicate data points!",
"Issue:Interstitial gaps!"
}, Label-> None, FontSize -> 20, Dividers -> None, Alignment -> Left]
}}, Label -> None, FontSize -> 36]

Trade-Off Analysis

GridTable[{{
,
GridTable[{
"Need to balance CD4 & CD8",
"The most predictive variables are not common",
"Need to choose a common set of inputs"
}, Label-> None, FontSize -> 20, Dividers -> None, Alignment -> Left]
}}, Label -> None, FontSize -> 36]

Conclusions

White-box models

Robust predictions

Trustable & Interpretable

Human-driven Variable & Model Selection from options

No need for hyper parameter tuning

Unique capability for super-wide data set analysis

Initialization

Needs["DataModeler`"]