Using Paclets From the Wolfram Language Paclet Repository

Todd Gayley, Wolfram Research

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

Paclets are a way of packaging units of Wolfram system functionality so they can be easily found, distributed, updated, and loaded. Many paclets contain Wolfram Language packages, but a paclet can be as simple as a single documentation notebook, all the way up to a large application with multiple packages, documentation, front end palettes, libraries, etc. Paclets have been used internally in Wolfram products for years to componentize parts of the system, and make it easy to provide incremental updates to users. We are now enhancing the system and documenting it for users. We are also opening a public Paclet Repository that accepts submissions from users. This talk will introduce paclets and explain their capabilities.

Paclets

Paclets are heavily used within Wolfram Research (> 270 paclets in the Mathematica 13.0 layout).

We are now documenting and exposing the paclet system to users.

We are creating a Paclet Repository that will accept user submissions, hoping to create a vibrant ecosystem of easily-discoverable and accessible paclets.

If you are creating any kind of add-on content for Wolfram products that does not fit into one of our other Repositories (like a single function for the Function Repository), you should create it in the form of a paclet.

Introduction to Paclets

Paclets are units of Wolfram functionality, packaged up in a way that allows them to be discovered, installed, updated, and integrated seamlessly into the Wolfram environment.

Introduction to Paclets

Paclets are units of Wolfram functionality, packaged up in a way that allows them to be discovered, installed, updated, and to integrate seamlessly into the Wolfram environment.

The essential element that makes a paclet is the PacletInfo.wl file, a small, simple file of metadata that describes the paclet, its requirements, and the ways in which it extends the Wolfram environment.

Introduction to Paclets

Paclets are units of Wolfram functionality, packaged up in a way that allows them to be discovered, installed, and updated, and to integrate seamlessly into the Wolfram environment.

The essential element that makes a paclet is the PacletInfo.wl file, a small, simple file of metadata that describes the paclet, its requirements, and the ways in which it extends the Wolfram environment.

A component of the Wolfram Engine called the Paclet Manager is what manages paclets and provides functionality to find, install, update, remove, and manage paclets.

Introduction to Paclets

Paclets are versioned

Paclet Manager always uses the highest version number it finds

side-by-side installation of multiple versions, possibly with different requirements and compatibilities

Introduction to Paclets

Paclets are versioned

Paclet Manager always uses the highest version number it finds

side-by-side installation of multiple versions, possibly with different requirements and compatibilities

Paclets have requirements

declare that they are only compatible with certain versions of the Wolfram Engine (e.g., “12+“), certain $SystemID values (“MacOSX-x86-64”), and certain Product Names (“Mathematica”)

paclets that do not match the current system become invisible and will not be used

Introduction to Paclets

Paclets are versioned

Paclet Manager always uses the highest version number it finds

side-by-side installation of multiple versions, possibly with different requirements and compatibilities

Paclets have requirements

declare that they are only compatible with certain versions of the Wolfram Engine (e.g., “11+“), certain $SystemID values (“MacOSX-x86-64”), and certain Product Names (“Mathematica”)

paclets that do not match the current system become invisible and will not be used

Paclets have extensions

describe the way in which they extend the Wolfram system (“I have a palette”, “I have documentation”, “I have WL code files”, etc.)

Introduction to Paclets

Basic operation of the Paclet Manager (Fundamental Principle): When looking for a paclet by name, or a resource that paclets might provide, the Paclet Manager finds all the paclets that provide that resource, culls out any that are disabled or do not match the current system, then picks the highest version.

Introduction to Paclets

Basic operation of the Paclet Manager (Fundamental Principle): When looking for a paclet by name, or a resource that paclets might provide, the Paclet Manager finds all the paclets that provide that resource, culls out any that are disabled or do not match the current system, picks the highest version.

This is a smarter lookup than a simple search path like $Path. When looking for a file, the Paclet Manager is consulted first, and $Path is only used if a paclet-based file is not found.

The PacletInfo.wl File

It is the presence of a PacletInfo.wl file that makes a collection of files into a paclet

A simple file of metadata in Wolfram Language format

You can also name this file PacletInfo.m

The PacletInfo.wl File

PacletObject[<|
    "Name" -> "FileUtilities",  (* You can include comments *)
    "Version" -> "1.0",
    "Extensions" -> {
        {"Kernel", "Root" -> "Kernel", "Context" -> "FileUtilities`"}
    }
|>]

The PacletInfo.wl File

PacletObject[<|
    "Name" -> "FileUtilities",  (* You can include comments *)
    "Version" -> "1.0",
    "Extensions" -> {
        {"Kernel", "Root" -> "Kernel", "Context" -> "FileUtilities`"}
    }
|>]

Directory layout to match this PacletInfo.wl:

FileUtilities/
    PacletInfo.wl
    Kernel/
        FileUtilities.wl

The PacletInfo.wl File (pre-12.1 format, still supported)

Paclet[
    Name -> "FileUtilities",  (* You can include comments *)
    Version -> "1.0",
    Extensions -> {
        {"Kernel", Root -> "Kernel", Context -> "FileUtilities`"}
    }
]

The PacletInfo.wl File

Extensions

The Extensions section is where you declare the types of resources your paclet provides—the ways in which it “extends” the system.

Extension Types:

Kernel
        Wolfram Language code (.wl and .m files)

FrontEnd
        Front end resources like Palettes and Stylesheets

Documentation
        Documentation notebooks integrated with the Documentation viewer

Path
       Files that can be looked up as if the paclet’s location was on $Path

LibraryLink
       Libraries of native code

JLink
      Java classes and jar files that will automatically be found by J/Link

Asset
      Arbitrary files that can be looked up by a defined tag

Custom
      Any new extension you want to create, for example to implement a plug-in system

Example Paclet

Complex Example Paclet

PacletObject[<|
    "Name" -> "ExamplePaclet",
    "Version" -> "1.0.1234",
    "Description" -> "A really great paclet. A lot of people are 
                      saying it's the best ever.",
    "Extensions" -> {
        {"Kernel", "Root" -> "Kernel", "Context" -> {"MyPackage`", "MyOtherPackage`"}},
        {"Documentation"},
        {"FrontEnd"},
        {"Path"},
        {"Asset", "Assets" -> {{"sun", "Images/sun.jpg"}, {"moon", "Images/moon.jpg"}}}
    }
|>]

Example Paclet

Complex Example Paclet

This is the file layout of this paclet:

ExamplePaclet/
    PacletInfo.wl
    Kernel/
        MyPackage.wl
    FrontEnd/
        Palettes/
            SuperPalette.nb
    Documentation/
        English/
            Guides/
                SomeGuide.nb
    Assets/
        Images/
            sun.jpg
            moon.jpg

Example Paclet

CreatePacletArchive["/Users/tgayley/Documents/WTC2021/ExamplePaclet"]

PacletInstall[%]

Needs["MyPackage`"]

ExampleFunction[5]

ReadList["ExamplePaclet/hello.txt", String]

sunFile=PacletObject["ExamplePaclet"]["AssetLocation", "sun"]

Import[sunFile]

Documentation page link: ExamplePaclet/guide/SomeGuide

Solving the “Shadowing” Problem

We’ve all seen the “shadowing” warning message when we load two different packages that each define a symbol with the same name.

Solving the “Shadowing” Problem

We’ve all seen the “shadowing” warning message when we load two different packages that each define a symbol with the same name.

This is expected to become a much bigger problem as the paclet ecosystem grows

Context Aliasing

Context aliasing is a new feature in Mathematica 13 that lets you specify short contexts as placeholders for long ones:

Context1`Context2`Square[x_]:=x^2

$ContextAliases["cc`"]="Context1`Context2`";

cc`Square[5]

Context Aliasing

The real value of context aliasing comes with a new syntax for Needs:

Needs["Developer`" -> "dev`"]

This behaves like Needs always has, except for two important changes:

it establishes a context alias of “dev`” for “Developer`”

it doesn’t put “Developer`” on $ContextPath

Context Aliasing

The real value of context aliasing comes with a new syntax for Needs:

Needs["Developer`" -> "dev`"]

This behaves like Needs always does, except for two important changes:

it establishes a context alias of “dev`” for “Developer`”

it doesn’t put “Developer`” on $ContextPath

dev`PackedArrayQ[{1,2,3}]

Context Aliasing

Here is what that looks like for a paclet from the Paclet Repository:

PacletInstall[ResourceObject["Wolfram: CodeEquivalenceUtilities"]]

Paclets typically have a long context name, so it is convenient to declare an alias:

Needs["Wolfram`CodeEquivalenceUtilities`"->"ce`"]

Now you can use the short alias when calling functions from this paclet:

ce`CodeEquivalentQ[RandomInteger/@Range[5],Array[RandomInteger,5]]

The context from this paclet has not been placed on $ContextPath:

MemberQ[$ContextPath, "Wolfram`CodeEquivalenceUtilities`"]

Context Aliasing

This new syntax for Needs should be considered “best practice” when loading code from external packages. Get used to not having package symbols dumped into the global namespace, and instead having to invoke them with a short context alias prefix.

Needs["SomeSuperLongContext`" -> "slc`"]

PacletSymbol

PacletSymbol is a new function in version 13 that lets you invoke functions from paclets in a convenient way.

PacletSymbol["Wolfram: QuantumComputing","QuantumBasis"][<|Ket[0]->{1,0},Ket[1]->{1,0}|>,"Schrodinger"]

That single PacletSymbol call allows you to invoke a function from a paclet in the Paclet Repository without any preamble whatsoever.

PacletSymbol

PacletSymbol is a new function in version 13 that lets you invoke functions from paclets in a convenient way.

PacletSymbol["Wolfram: QuantumComputing","QuantumBasis"][<|Ket[0]->{1,0},Ket[1]->{1,0}|>,"Schrodinger"]

That single PacletSymbol call allows you to invoke a function from a paclet in the Paclet Repository without any preamble whatsoever.

PacletSymbol:

finds the named paclet or resource and installs it in a special way

loads the paclet’s main context but does not add the context to $ContextPath

evaluates to the named symbol in the paclet’s context

PacletSymbol

PacletSymbol installs the paclet in such a way that it is made visible in the current session, but will not be visible in subsequent sessions.

PacletSymbol

PacletSymbol installs the paclet in such a way that it is made visible in the current session, but will not be visible in subsequent sessions.

The paclet will not need to be downloaded again: If you invoke PacletSymbol in a future session, the paclet will be loaded and used immediately.

PacletSymbol

PacletSymbol installs the paclet in such a way that it is made visible in the current session, but will not be visible in subsequent sessions.

The paclet will not need to be downloaded again: If you invoke PacletSymbol in a future session, the paclet will be loaded and used immediately.

PacletSymbol allows you to quickly assess whether a paclet’s functionality meets your needs, without making any persistent changes to your environment. “Try before you buy”.

PacletSymbol

PacletSymbol installs the paclet in such a way that it is made visible in the current session, but will not be visible in subsequent sessions.

The paclet will not need to be downloaded again: If you invoke PacletSymbol in a future session, the paclet will be loaded and used immediately.

PacletSymbol allows you to quickly assess whether a paclet’s functionality meets your needs, without making any persistent changes to your environment. “Try before you buy”.

If you are satisfied with a paclet, you can use PacletInstall to install it in the usual way, making it available in future sessions.

Paclet System Documentation

The main Paclet guide page: guide/Paclets

All the Paclet Manager functions have their own reference pages

An in-depth tutorial on the paclet system will be present in version 13. Until that is released, you can find a useful reference at https://www.wolframcloud.com/obj/tgayley/Published/PacletDevelopment.nb

PacletTools  is an important collection of functionality for developing paclets, and authoring and building documentation

The Documentation Tools palette is what you will use to author documentation in standard Wolfram system format