State of Typesetting in the FrontEnd

WolframTechnology Conference 2021

Jason Harris

jasonh@wolfram.com
Wolfram Research

In this presentation I will talk about the representation of typeset and evaluatable structures in the FrontEnd. (The box representation in the FrontEnd.) This will quickly lead into the discussion of TemplateBoxes and the now standard representation of many typeset and evaluatable expressions in the FrontEnd. You have likely used these structures all the time in daily Mathematica usage without knowing exactly what is going on under the hood. Many everyday notations and structures now utilise TemplateBoxes in the FrontEnd so it is an important building block for Mathematica developers. In this talk I will present one of the latest extensions in typesetting: Dynamic Template Boxes. Dynamic Template Boxes “unify” the Dynamic paradigm into TemplateBoxes and encapsulate much more than we could previously wrap into the TemplateBox framework. Dynamic Template Boxes will be a standard part of many future developments in the FrontEnd.

To access the TemplateBox material I will cover some background in Boxes, StyleSheets and MakeBoxes and MakeExpression rules.

In detail we will cover:

Boxes and the representation of typeset structures

MakeBoxes & MakeExpression and how the typesetting system interacts between the Kernel and the FrontEnd

TemplateBoxes, and how to use these typeset structures

StyleSheets and how to package up the typeset structures onto the style sheet

Association TemplateBoxes

Dynamic TemplateBoxes

In depth substitution and scoping in TemplateBox DisplayFunctions

Our Goal in Typesetting

Our goal is to introduce new notations which can do computations.

For instance something like (Subscript[a, ∂]) and make this parse and format as the operator ADiff, etc. Or make say our own commutator as we would use in Physics. Or say:

Everything is Boxes!

(x^3)+4

(∫_0^∞)(E^(-(x^z)))\[DifferentialD]x

Visual form vs. Internal form

theboxes = "";InputField[Dynamic@theboxes,Boxes,ContinuousAction->True]

Dynamic@Style[theboxes,ShowStringCharacters->True]

Generators and Graphics Boxes

All displayed graphics are actually boxes.

Graphics[Table[Line @ {{Random[],Random[]},{Random[],Random[]}},{7}], ImageSize->Small]

pts={{0,0},{1,1},{2,-1},{3,0}};
Graphics[{BezierCurve[pts],Green,Line[pts],Red,Point[pts]},ImageSize->Small]

Similarly all box control are also boxes:

3+Style[bob,FontSize->50]

Button["Hi There",Print @ "Hello"]

Slider[Dynamic[mn],{1,100,1}]

Dynamic[Factor[(y^mn)+1]]

Complex Structures:

• (RefLink[NetGraph, paclet:ref/NetGraph])

Its all about boxes! Everything is a Box.

MakeExpression & MakeBoxes

We need to be able to do computation with our boxes.

Generators are not sufficient.

Our goal is to introduce new notations which can do computations.

Evaluation Model

Parsing 	⟷ 	MakeExpression
Formatting 	⟷ 	MakeBoxes

(∂_y)((Sin[y])/2)

?MakeBoxes

?MakeExpression

Example PartialPlus

Consider a new notation for “Partial Plus” by sub-scripting a partial derivate by ‘+’ and having “FullForm” PDPlus.

Typically in creating a MakeExpression rule we transform to another MakeExpression of something Mathematica actually can make.

MakeExpression[RowBox[{
  SubscriptBox["∂", "+"], v_}],StandardForm]:=
MakeExpression[RowBox[{"PDPlus","[",v,"]"}],StandardForm]

In MakeBoxes we transform from expressions to boxes and we need to further call MakeBoxes on any subparts.

MakeBoxes[PDPlus[v_],StandardForm]:=
RowBox[{SubscriptBox["∂","+"],MakeBoxes[v,StandardForm]}]

Now we can

(∂_(+))(q_+t_):=(∂_(+))q+(∂_(+))t

(∂_(+))((x^2)+Sin[t])

Trick: Often it is best to create the notation you want using the 2D editing tools in Mathematica and then just copy the underlying boxes.

ConvertTo uses MakeBoxes and MakeExpression

Trick: You can use the menu Cell > Convert To > StandardForm (⌘⇧N) to format an expression in 2D. To enter tricky notations you can enter the full form of a notation and then convert the cell to StandardForm.

Conversions only work with syntactically valid structures.

Evaluation In MakeBoxes and MakeExpression

You have to be very careful that we don’t let evaluation occur as a side effect of our transformations. (Mostly in MakeBoxes)

Example Function Notation

(a|->(a^2))

Answer :

Example Up and Down

Answer :

(b^↑)+(t^↓)

FullForm @ %

TemplateBoxes: Introduction

Consider the display function for traditional form typesetting of Binomial. How is this done?

Binomial[n,k]//TraditionalForm

TemplateBox

RawBoxes @ TemplateBox[{"a","w"},"MyLegandreP",DisplayFunction:>(RowBox[{SubscriptBox["P",#1],"[",#2,"]"}]&),Tooltip->Automatic]

Putting it on the StyleSheet

So consider the following:

c+RawBoxes @ TemplateBox[{"a","b"},"MyCommutator",DisplayFunction:>(SubscriptBox[RowBox[{"[",RowBox[{#1,",",#2}],"]"}],"-"]&)]

c+RawBoxes @ TemplateBox[{"a","b"},"MyCommutator"]

<<InjectStyles`

Note: the option template box option InterpretationFunction->Automatic means Mathematica chooses the tag name by default as the function name.

Answer:

InjectStyles[InputNotebook[],"MyCommutator",TemplateBoxOptions->{DisplayFunction:>(SubscriptBox[RowBox[{"[",RowBox[{#1,",",#2}],"]"}],"-"]&)}]

BoxFunctions:

InjectStyles[InputNotebook[],"MyCommutator",TemplateBoxOptions->{
DisplayFunction:>(SubscriptBox[RowBox[{"[",RowBox[{#1,",",#2}],"]"}],"-"]&),
InterpretationFunction:>(RowBox[{"QuantumCommutator","[",RowBox[{#1,",",#2}],"]"}]&),
CopyFunction:>(RowBox[{"MathematicaCommutator","[",RowBox[{#1,",",#2}],"]"}]&)
}]

TemplateBoxes

Almost all composite typesetting in Mathematica is now done through TemplateBox. TemplateBox is:

TemplateBox[{(Subscript[box, 1]),(Subscript[box, 2]),...},tag]

is a low level box structure which displays as the DisplayFunction of the template box filled in with the (Subscript[box, i])

The DisplayFunction option is normally specified on the style sheet in order that when the style sheet changes all displayed instances of the template box change uniformly.

If the tag of the TemplateBox is a string then that style will be used for the TemplateBox.

TemplateBoxes can be nested to any level

TemplateBox[{TemplateBox[{"n","x"},"LegendreP"],"b"},"Binomial"]//RawBoxes//TraditionalForm

You can write MakeBoxes and MakeExpression rules which use TemplateBoxes in order to define new notations.

TemplateBoxes: Associations [12.1+]

A new feature of TemplateBoxes is you can specify the arguments as associations, and the DisplayFunction can then used named keys. Let’s look at an example of this:

(Subscript[ℒ, [X,Y]])T

This new notation generalizes to TemplateSlotSequence and other advanced concepts in TemplateBoxes.

LieDerivative Example

InjectStyles[InputNotebook[],"LieDerivative",TemplateBoxOptions->{DisplayFunction->(RowBox[{SubscriptBox["ℒ",RowBox[{"[",RowBox[{#x,",",#y}],"]"}]],#t}]&)}]

c+RawBoxes @ TemplateBox[<|"x"->"W","y"->"Y","t"->"T"|>,"LieDerivative"]

TemplateBoxes: Missing Arguments [12.1+]

If there are missing arguments

RawBoxes @ TemplateBox[{},"MyCommutator"]

RawBoxes @ TemplateBox[<||>,"LieDerivative"]

TemplateSlotSequences

TemplateSlotSequence allow us to specify a sequence of arguments riffled with a separator

TemplateBox[{"a","b","c","d","e"},
   "Foo", DisplayFunction->(RowBox@{TemplateSlotSequence[1]}&)]//RawBoxes

TemplateBox[<|"a"->"u","b"->"v","c"->"w", "d"->"x","e"->"y","f"->"z"|>,
   "Foo", DisplayFunction->(RowBox@{TemplateSlotSequence["b",";"]}&)]//RawBoxes

TemplateBoxes: Editing Highlighting [12.1+, 13+]

You can add an editing highlight to the arguments of a particular TemplateBox (or all TemplateBoxes) via options.

RawBoxes @ TemplateBox[<||>,"LieDerivative",
EditingHighlightStyle->{Background->RGBColor[1,1,0.8]},EditingHighlightArgumentStyle->{Background->RGBColor[0.8,1,1], FontSlant->Italic},
EditingHighlightBackgroundMargins->{{5,5},{10,10}},
EditingHighlightArgumentBackgroundMargins->{{2,4},{3,3}}
]

Dynamic Template Boxes [12.2+]

“DynamicTemplateBoxes” are TemplateBoxes whose DisplayFunction contains dynamics that feed back into the parameters of the TemplateBox

DynamicSlider

InjectStyles[InputNotebook[],"DynamicSlider",
TemplateBoxOptions->{DisplayFunction->(SliderBox[Dynamic[#a]]&)}]

RawBoxes@TemplateBox[<|"a"->0.3|>,"DynamicSlider"]

Further Examples

TemplateBox[<|"a"->0.3|>,"fly",DisplayFunction->(RowBox[{SliderBox[Dynamic[#a]],DynamicBox[ToBoxes[#a]]}]&)]//RawBoxes

TemplateBox[<|"a"->0.3|>,"fruit",DisplayFunction->(RowBox[{SliderBox[Dynamic[#a]],SliderBox[Dynamic[#a]]}]&)]//RawBoxes

TemplateBox[<|"a"->True|>,"fast",DisplayFunction->(RowBox[{CheckboxBox[Dynamic[#a]],DynamicBox[ToBoxes@#a]}]&)]//RawBoxes

Why Dynamic Template Boxes?

RGBColor[1,0,0]

RawBoxes @ TemplateBox[<|"color"->RGBColor[1,0,0]|>,"RGBColorSwatchTemplate"]

LinguisticAssistant

NamespaceBox["LinguisticAssistant",TemplateBox[<|{"query" -> "","boxes"->"None","allassumptions"->{},"assumptions"->{},"open" -> {1},"querystate"->{}}|>,"LinguisticAssistantTemplate"]]//RawBoxes

Dynamic Template Boxes: internals

DynamicTemplateBoxes work via an internal DynamicModuleBox.

DynamicModuleBox

A DynamicModuleBox scopes dynamic variables in a module like way.

Function Application

Function “Application”

(#1+((#2)^2)&)[a,b]

(#x+((#y)^2)&)[<|"x"->a,"y"->b|>]

DynamicModule[{x=a,y=b},x+(y^2)]

DynamicSlider revisited

RawBoxes@TemplateBox[<|"a"->0.3|>,"DynamicSlider2",
DisplayFunction->(RowBox[{SliderBox[Dynamic[#a]],DynamicBox[ToBoxes[#a]]}]&)]

Internals

The internal box generated by the c++ code TemplateBox::displayFunctionSubstitute for display will be:

(lldb) fp20 newChildAsExpr
Expr : 0x60000577e680 , RefCount : 1 : DynamicModuleBox[
 List[
  Set[
   FETemplate`$$a,
   1.000000
  ]
 ],
 SliderBox[
  Dynamic[
   FETemplate`$$a
  ]
 ]
]

The Notation package

<<Notation`

The notation package allows us to automate the process that has been described for creating notations. A notation statement is of the form:
  Notation[ExternalForm ⟺ InternalForm]

Example: ring addition, ring multiplication.

a(Subscript[*, ℛ])b(Subscript[+, ℛ])c

Notation Package Functioning

The notation package internally is a notation compiler. It takes a box structure representing the notation to be compiled and returns the equivalent MakeBoxes and MakeExpression rules.

Begin@"Notation`Private`";Notation[a_(Subscript[+, ℛ])b_ ⟺ RingPlus[a_,b_],Action->PrintNotationRules]
Notation[a_(Subscript[*, ℛ])b_ ⟺ RingTimes[a_,b_],Action->PrintNotationRules]
End[];

Example Up and Down Revisited

Quickly Entering Notations

Trick: You can quick create shortcuts for notations through an “add input alias” statement. (EscaddiaEsc)

AddInputAlias["opu"->(NotationTemplateTag[(\[SelectionPlaceholder]^↑)])]

Things I have not talked about

SyntaxForm

BaseStyle and DefaultBaseStyle

Context Dependent Parsing

TagBoxes and old style Parsing

Evaluation in MakeBoxes and MakeExpression

Many other things...

Tag Boxes

TagBoxes are invisible tags that have the form TagBox[boxes,tag]. tag can be any expression.TagBoxes are used to invisibly “tag” a box structure so we can identify it later.

( In Mathematica 8 onwards we transitioned to TemplateBoxes in the vast majority of cases.)

TagBoxes are still used to guide interpretation in some cases. Eg Grid

Grid[{{a,b},{c,d}}]

MakeTemplate creates generalized placeholders. These placeholders are based on TagBoxes.

The ⇥ key takes you between placeholders.

Another example of TagBox usage is in event handlers and other places like eg MouseAppearance

MouseAppearance[bob, "x"]

Typesetting for LegendreP[n,x]

To illustrate TagBoxes consider making a notation for LegendreP[n,x]⟷(Subscript[P, n])(x)

The Rules:

Here are the MakeExpression and MakeBoxes rules to do such a thing

MakeBoxes[LegendreP[n_,x_],StandardForm]:=RowBox[{SubscriptBox["P",MakeBoxes[n,StandardForm]],"[",MakeBoxes[x,StandardForm],"]"}]

MakeExpression[RowBox[{SubscriptBox["P",n_],"[",x_,"]"}],StandardForm]:=
MakeExpression[RowBox[{"LegendreP","[",RowBox[{n,",",x}],"]"}],StandardForm]

The Problem:

Subscript[P,n][x]

(Subscript[P, n])[x]+1//FullForm

Ouch!

Solution:

We use the following TagBox structure to distinguish the “P”

RawBoxes @ TagBox["P",SelectWithContents->True,Selectable->False]

LegendreP[n,x]+1

FullForm @ %

However whatever you come up with here, the P is likely to be detachable from the rest of the notation, as well as having other problems. This gives rise to TemplateBoxes.

BaseStyle

Consider the example of vector with a shaded background. Assume we had put the style box around the whole underscript to yield a structure like the following:

RawBoxes @ StyleBox[UnderscriptBox["v","~"],Background->GrayLevel[0.8]]

So instead we can add the style into the base style.

RawBoxes @ UnderscriptBox["v","~",BaseStyle->{Background->GrayLevel[0.8]}]

Sometimes there are lots of styles we would like to add to the base style so we can just encapsulate all these styles into a new style and add that to the style sheet.

BaseStyle and DefaultBaseStyle are options to almost every box.

Miscellaneous

I should talk here about precedence’s and SyntaxForm:

a+(∂_x)func+b

Also note that Graphics does context sensitive typesetting. Eg:

Point[{0,0}]

Graphics[ Point[{0,0}],ImageSize->50]

This is achieved by having the MakeExpression rules in Graphics be context sensitive.

Interesting Irrelevant note: The Mathematica box model to css box model, and BaseStyle to styles, ImageMargins, FrameMargins, to html margins and padding.

Conclusions

Steps to create a TemplateBox notation:

Enter a structure in 2D typesetting.

Copy the boxes for that structure into a display function replacing the arguments you want parameterized with or with #n or #var

Put the resulting style on the style sheet.

Create a MakeBoxes rule for the given notation.

Bonus points: create an input alias for the notation.

Put as much as you can onto the StyleSheet for forward compatibility, readability, and file size reduction.

Related FE Talks

John Fultz: State of the Notebook FrontEnd

Lou D’Andria: Extending the Notebook UI

James Lane: Graphics: New Features and Improvements

Yuzhu Lu: GLSL Support with Mathematica Graphics

Bignya Pathi & Jayanta Phadikar: Creating Custom Notations and Objects—A Refresher