Modern Diffing Tools

Lou D’Andria
Wolfram Research

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

In this talk, we will introduce recently released diffing tools for the Wolfram Language. We will explore how to compute and visualize changes between different types of objects, including cells and notebooks. Join us to gain a better understanding of how to use these tools effectively, and to learn about our future plans for their continued development.

SequenceAlignment: Lists and Strings

SequenceAlignment finds optimal alignments between lists or strings.

str1="Lorem ipsum dolor rutrum eu sit amet, consectetuer…";
str2="Lorem ipsum dolor sit amet, consectetuer adipiscing elit.";
SequenceAlignment[str1, str2]

list1={1,2,3,4,5};
list2={4,5,2,1,3};
SequenceAlignment[list1, list2]

This is great!

Battle tested since 2008 (7.0)

Optimal alignment is the key to diffing

Choice of methods and other options

Except when it’s not…

So. Many. Lists.

Output can be hard to visually parse, awkward to compute.

Only supports lists and strings.

Output is insufficient to tell whether inputs were SameQ.

SequenceAlignment[{{1},{}},{{1},{}}]===SequenceAlignment[{1},{}]

Wolfram/DiffTools

Brand new paclet on the Paclet Repository.

PacletInstall["Wolfram/DiffTools"]
Needs["Wolfram`DiffTools`"]

Main function: Diff

Diff[list1, list2]

Diff[str1, str2]

Returns a DiffObject with instant access to various views.

Diff supports lists, strings, files, cells, notebooks, and more to come.

By supporting cells, Diff effectively supports arbitrary expressions.

DiffObject

obj=Diff[Options[SequenceAlignment],Options[Diff]]

Ask for properties of a DiffObject.

obj["Properties"]

obj["SummaryString"]

DiffView displays a DiffObject in a particular format.

Available formats depend on the DiffObject.

DiffView[obj, "Formats"]

DiffView[obj, "SideBySide"]

DiffApply applies a DiffObject to an expression.

DiffApply[obj,Options[SequenceAlignment]]

% === Options[Diff]

We even support 3-way diffs -- two different edits of the same list.

This is very important in for source control systems.

Sometimes, those edits can be merged together.

Diff3[{1,3,5},{1,2,3,5},{1,3,4,5}]

Sometimes, they conflict.

Diff3[{1,3,5},{1,2,3,55},{1,22,3, 555}]

Diffing Cells

Some cells are string-like.

This example may be alarming, but is meant only to illustrate that Manipulate is a fully general function, not limited to exploring any fixed domain.

cellobj1=PreviousCell[];

This example may be somewhat alarming, but is meant only to illustrate that Manipulate is a fully general function, not limited to exploring any fixed domain of graphical or algebraic examples.

cellobj2=PreviousCell[];

Diff[cellobj1,cellobj2]

Some cells are not.

DiffInputFunction is a convenient way to turn expressions into cells.

Diff[∫(1/(1-(x^3)))\[DifferentialD]x,∫(2/(1-(x^3)))\[DifferentialD]x,DiffInputFunction->ExpressionCell]

No nesting support for in views yet.

Diff[∫(1/(1-(x^3)))\[DifferentialD]x,∫(1/(1+(x^3)))\[DifferentialD]x,DiffInputFunction->ExpressionCell]

Use the Compare Cells palette to diff cells in place.

Diffing Notebooks

{nbfile1, nbfile2} = {,}; 
;ArrangeNotebooks[nbfile1, nbfile2];

In some sense, a notebook is just a text file…

Diff[nbfile1,nbfile2, DiffGranularity->"Lines"]

But a notebook is also a WL expression containing a list of cells, grouping info, and options.

Diff[nbfile1,nbfile2]

Options like IgnoreElements let you customize the diff.

Still to come: DiffApply will be made to work for notebooks.

Use the Compare Notebooks palette to diff notebooks

Diffing Notebooks — Aligning Cells

Problem: Which cells were edited, and which were added?

Notebook[{
Cell["title", "Title"],
Cell["subtitle", "Abstract"]}]

Notebook[{
Cell["title", "Title"],
Cell["abstract", "Abstract"],
Cell["subtitle edited", "Subtitle"]}]

The solution has been around since v11.

ExpressionUUID

Notebook[{
Cell["title", "Title", ExpressionUUID -> "<1>"],
Cell["subtitle", "Abstract", ExpressionUUID -> "<2>"]}]

Notebook[{
Cell["title", "Title", ExpressionUUID -> "<1>"],
Cell["abstract", "Abstract", ExpressionUUID -> "<3>"],
Cell["subtitle edited", "Subtitle", ExpressionUUID -> "<2>"]}]

Diffing with a Common Ancestor

Say Alice and Bob each made edits to a given notebook. Can some process pick up both sets of edits?

{nb0,nb1,nb2}={,,};
ArrangeNotebooks[nb0, nb1, nb2];

Of course, a line-by-line merge of the notebook files is hopeless.

Diff3[nb0, nb1,nb2, DiffGranularity->"Lines"]

When taken cell-by-cell, we have more hope of a clean merge, and a better idea about what to do next.

Diff3[nb0, nb1,nb2]

Still to come: features for automating merging and conflict resolution.

We already have a good model for this in GitLink…

Interesting Examples

Examine all unsaved changes to a notebook.

Diff[File @ NotebookFileName[], EvaluationNotebook[],"Report",]

Visualize changes to a notebook file over time

;

NotebookHistory["StyleSheets/Default.nb"] // EchoTiming (* ~250 cells x 36 releases *)

NotebookHistory["StyleSheets/Core.nb"]//EchoTiming(* ~1500 cells x 36 releases *)

The DiffTools Paclet

See the DiffTools Paclet Repository page for complete docs.

https://resources.wolframcloud.com/PacletRepository/resources/Wolfram/DiffTools/

Or install it and view the docs locally, and try it out for yourself.

PacletInstall["Wolfram/DiffTools"]
Needs["Wolfram`DiffTools`"]

SystemOpen["paclet:Wolfram/DiffTools/guide/DiffTools"]

Directions

Packaging up diff and merge utilities as .wls files

Integration with common source control systems (like git)

Include native diffing and views for more objects

associations

bitmaps

directories / bundles

…

Better visualization of cell changes, especially diffs in invisible and barely-visible regions.

Interactive application of diffs, one “hunk” at a time.

Automating conflict resolution

Lots more…

Conclusion

The DiffTools prototype paclet is available today!

https://resources.wolframcloud.com/PacletRepository/resources/Wolfram/DiffTools/

Install and use it locally, or on Cloud.

PacletInstall["Wolfram/DiffTools"]
Needs["Wolfram`DiffTools`"]

We look forward to your feedback.