Scales & Guides - 
How to Visualize Anything

Min-Hsuan Peng
Wolfram

Categorical data visualization

ListPlot[,
ScalingFunctions->{
NominalScale[Automatic,RotateLabel->True],
OrdinalScale[]
},
ImageSize->1100,AspectRatio->(1/2),LabelStyle->24,GridLines->All,ColorFunction->ColorData[{"Rainbow","Reverse"}]]

Overview

Categorical data types

Visualization of anything

Guide/example to use relational database

Introduction

Data are facts and statistics collected together; most often are categorical.

Explanatory variable: “Height”, “Blood type”, “Vegetarian”...

Response variable : 5’10”, “A”, True

Categorical Data Analysis (CDA)

...is to analyze the response variables. We might learn some of them when we study statistics including various measurements, statistic tests, model fitting, regression, and more.

What this talk is about and what this isn’t.

This talk is not focused on CDA but the visualization of data itself. Visualization of data helps understand data by using visual elements for finding trends, clusters, quantity, proportions, outliers, etc...

With that being said...

There is a intermediate area between visualizing raw data and CDA. A lot of data scientists would have to do this before they can do statistics analysis, and that is data combing.

Categorical Data

Types of data

Nominal data: {cat, dog, bunny}, {apple, orange, banana}, {horror, romance, suspect, action, fantasy}, affiliation, genre, ...

Ordinal data: {excellent, good, normal, below average}, {Gold medal, silver, bronze}, {Supreme court, Circuit, District}, size, quality, wellness, ...

Interval data: dates, pH, time, bond rating, temperature(C, F)

Ratio data: weight, height, width, length, age, temperature(K)

What’s the difference

TableForm[{{True,True,True,True},{"",True,True,True},{"","",True,True},{"","","",True}},TableHeadings->{{"Categories","Order","Meaningful distance","True 0"},{"Nominal","Ordinal","Interval","Ratio"}}]

Arithmetic

TableForm[{{True,True,True,True},{"",True,True,True},{"","",True,True},{"","","",True}},TableHeadings->{{"Equality (==, =!=)","Inequality (<, >)","Add","Multiply"},{"Nominal","Ordinal","Interval","Ratio"}}]

Numerical measures

TableForm[{{True,True,True,True},{"",True,True,True},{"","",True,True},{"","","",True}},TableHeadings->{{"Mode","Median","Arithmetic mean","Geometric mean"},{"Nominal","Ordinal","Interval","Ratio"}}]

Interval/Ratio scale

Any numerical responsive data can be in interval or ratio scale.

Ex: physical measurements, marker values, age, monetary, ...

Meaningful distance

relative/absolute 0

Example: Iris data

irisdata=ExampleData[{"Statistics","FisherIris"}];

Length[irisdata]

axeslabels=ExampleData[{"Statistics","FisherIris"},"ColumnHeadings"]

Take[irisdata,5]

WebImageSearch["setosa iris",3]

WebImageSearch["versicolor iris",3]

WebImageSearch["virginica iris",3]

Group the data by the species and extract axes labels:

data=GroupBy[irisdata,Last->Most];

data//Short

data//Keys

axeslabels

Plot with (RefLink[ParallelAxisPlot, paclet:ref/ParallelAxisPlot]):

ParallelAxisPlot[data,AxesLabel->axeslabels,]

Find the median, 25% and 75% quantiles of the data and stylize them in bold and dashed lines:

newdata=Join[
(*medium line*)
{Style[Median@#,Directive[Opacity[1],Thickness[0.01]]]},
(*quartile lines*)
Style[#,Directive[Opacity[1],Dashed,Thickness[0.005]]]&/@Transpose@Quantile[#,{0.25,0.75}]
]&/@data;

ParallelAxisPlot[newdata,AxesLabel->axeslabels,]

Bonus: changing plot range to be the same and see the difference

ParallelAxisPlot[newdata,PlotRange->8,AxesLabel->axeslabels,]

Ordinal scale data

Categories

fruitsIeat={"Banana","Orange","Banana","Banana","Banana","Apple","Banana"};

ListPlot[fruitsIeat,
ScalingFunctions->NominalScale[Automatic],
PlotStyle->PointSize[0.02`],LabelStyle->20]

Syntax

(RefLink[NominalScale, paclet:ref/NominalScale])[{(Subscript[cat, 1]),(Subscript[cat, 2]),…,(Subscript[cat, n])}] 
(RefLink[NominalScale, paclet:ref/NominalScale])[\[LeftAssociation](Subscript[cat, 1])->(Subscript[lab, 1]),…,(Subscript[cat, n])->(Subscript[lab, n])|>]

ListPlot[fruitsIeat,
ScalingFunctions->NominalScale[<|"Apple"->🍎,"Orange"->🍊,"Banana"->🍌|>],
]

Work with all the viz functions that support ScalingFunctions including charts.

Example - Visualize coded data

Dataset@

Use (RefLink[NominalScale, paclet:ref/NominalScale]) in a (RefLink[BubbleChart, paclet:ref/BubbleChart]):

state=NominalScale[<|1->"Illinois",2->"Indiana",3->"Wisconsin"|>];

cars=NominalScale[<|1->"Ford",2->"Chevrolet",3->"Honda",4->"Toyota",5->"Fiat"|>];

Tally[]

BubbleChart[Flatten/@Tally[],ScalingFunctions->{state,cars,None},
ColorFunction->"Rainbow",LabelStyle->20]

Bonus (Use it outside of viz)

Use it in (RefLink[GroupBy, paclet:ref/GroupBy]):

state["Label"][1]

KeySort/@GroupBy[,{state["Label",#[[1]]]&,cars["Label",#[[2]]]&},Length]

Dataset[%]

Bonus example

Visualize the natural state of the elements:

GraphicsColumn[
Table[
ListPlot[
Table[{e,ElementData[e,"Phase"]},{e,ElementData[]}],
ScalingFunctions->{
NominalScale[Association[Table[e->e["Abbreviation"],{e,elems}]]],NominalScale[<|Missing["NotAvailable"]->"n/a",(Entity["gas", Entity["MatterPhase","Gas"], "Entity[\"MatterPhase\", \"Gas\"]", "matter phase"])->"gas",(Entity["liquid", Entity["MatterPhase","Liquid"], "Entity[\"MatterPhase\", \"Liquid\"]", "matter phase"])->"liquid",(Entity["solid", Entity["MatterPhase","Solid"], "Entity[\"MatterPhase\", \"Solid\"]", "matter phase"])->"solid"|>]
},
],
{elems,Partition[ElementData[],UpTo[25]]}
],
ImageSize->800]

Ordinal scale data

Categories

Meaningful distance

ListPlot[,
ScalingFunctions->
OrdinalScale[
{"Totally Dissatisfied","Dissatisfied","Neutral","Satisfied","Totally Satisfied"},
{-1,0,1,3,6}
]
]

Syntax

(RefLink[OrdinalScale, paclet:ref/OrdinalScale])[{(Subscript[cat, 1]),(Subscript[cat, 2]),…,(Subscript[cat, n])}] 
(RefLink[OrdinalScale, paclet:ref/OrdinalScale])[{(Subscript[cat, 1]),…,(Subscript[cat, n])},{(Subscript[val, 1]),…,(Subscript[val, n])}]
(RefLink[OrdinalScale, paclet:ref/OrdinalScale])[\[LeftAssociation](Subscript[cat, 1])->(Subscript[val, 1]),…,(Subscript[cat, n])->(Subscript[val, n])|>]
(RefLink[OrdinalScale, paclet:ref/OrdinalScale])[{(Subscript[cat, 1]),…,(Subscript[cat, n])},vals,{(Subscript[lab, 1]),…,(Subscript[lab, n])}]

ListPlot[,
ScalingFunctions->OrdinalScale[
{"Totally Dissatisfied","Dissatisfied","Neutral","Satisfied","Totally Satisfied"},
Automatic,
{😡,😤,😐,😃,🤩}
],
LabelStyle->30]

Examples:

Dataset[]

BubbleChart[,
ScalingFunctions->{
NominalScale[Automatic],
OrdinalScale[{"Peewee","Small","Medium","Large","Extra Large","Jumbo","King"}],
None
}
]

States bond ratings

ListPlot[,
ScalingFunctions->{
NominalScale[Automatic,RotateLabel->True],
OrdinalScale[]
},
]

Wait, there is more...

OrdinalScale/NominalScale are designed to be operators

Properties of OrdinalScale/NominalScale

Possible properties include:
({{      , "CategoryAssociation", association of the form <|…,(Subscript[val, i])->(Subscript[cat, i]),…|>}, {      , "CategoryList", list of categories {(Subscript[cat, 1]),…,(Subscript[cat, n])}}, {      , "LabelAssociation", association of the form <|…,(Subscript[cat, i])->(Subscript[lab, i]),…|>}, {      , "LabelList", list of labels {(Subscript[lab, 1]),…,(Subscript[lab, n])}}, {      , "ValueAssociation", association of the form <|…,(Subscript[cat, i])->(Subscript[val, i]),…|>}, {      , "ValueList", list of values {(Subscript[val, 1]),…,(Subscript[val, n])}}, {      , "Properties", list of supported properties}})

OrdinalScale[{"hot","warm","lukewarm","cool","cold"}]["Properties"]

Example

Use (RefLink[OrdinalScale, paclet:ref/OrdinalScale]) value associations for classification:

woodhardnessscale=OrdinalScale[
\[LeftAssociation]
"Douglas Fir"->660,
"Yellow Pine"->870,
"Black Walnut"->1010,
"Red Oak"->1290,
"Hard Maple"->1450,
"Hickory"->1820
\[RightAssociation]];

woodhardnessscale["ValueAssociation"]

woodhardnessscale["CategoryAssociation"]

woodhardnessscale["Values"][{"Yellow Pine", "Yellow Pine", "Hard Maple"}]

Use Nearest to make a simple classification function:

woodhardness=Nearest[woodhardnessscale["ValueList"]]

similarwood[hardness_]:=woodhardnessscale["CategoryAssociation"][First@woodhardness[hardness]]

Given a hardness value, find a similar hardwood:

similarwood[785]

similarwood[1787]

Working with relational database

Carlo Barbieri’s workshop

Leonid Shifrin’s talk : Using the Dataset Query Language to Query Relational Database

tutorial/RelationalDatabasesQuickStart

guide/WorkingWithRelationalDatabases

guide/ComputationWithStructuredDatasets

Example

FindFile["ExampleData/ecommerce-database.sqlite"]

conn = DatabaseReference[FindFile["ExampleData/ecommerce-database.sqlite"]]

entity-relational diagram

rdb = RelationalDatabase[conn]

es = EntityStore[rdb]

EntityRegister[es]

We can query the database through the Entity framework:

EntityValue["offices","Properties"]

EntityValue[
"offices", 
{"officeCode", "addressLine1", "city", "state", "country"}, 
"Dataset"
]

EntityValue["employees","Properties"]

EntityValue["customers","Properties"]

EntityValue["employees",{"officeCode"},"Association"]

EntityValue["offices",{"officeCode","city"},"Association"]

employeecity=JoinAcross[Values@EntityValue["employees",{"officeCode"},"Association"],Values@EntityValue["offices",{"officeCode","city"},"Association"],Key["officeCode"]]

BarChart[Counts[#["city"]&/@employeecity],
ImageSize->600,BarSpacing->1,LabelStyle->16,
PlotLabel->"employees in cities",
ChartLabels->Placed[Automatic,Above]
]

Sales in $ per employee:

totalByEmployees=Total/@GroupBy[
JoinAcross[
JoinAcross[
JoinAcross[
Values@EntityValue["orders", {"orderNumber", "customerNumber"}, "Association"],
Values@EntityValue["customers",{"customerNumber", "salesRepEmployeeNumber"}, "Association"],
Key["customerNumber"]
],
Values[EntityValue["orderdetails",{"orderNumber","quantityOrdered","priceEach"},"Association"]],
Key["orderNumber"]
],
Values[EntityValue["employees",{"employeeNumber","lastName"},"Association"]],
Key["salesRepEmployeeNumber"]->Key["employeeNumber"]
],
Key["lastName"]->(#["quantityOrdered"]*#["priceEach"]&)
]

BarChart[Sort@totalByEmployees,
ImageSize->800,
BarSpacing->1,
LabelStyle->16,
ColorFunction->"Rainbow",
ChartLabels->Placed[Automatic,Above],
PlotLabel->"Sales in $ per employee"]

Thank you

Email: minhsuanp@wolfram.com

$PlotTheme={"Presentation","Bold","Large","LargeLabels"}