Advanced Chat Notebook Use

Richard Hennigan, Wolfram Research

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

This presentation discusses advanced usage of Wolfram Language’s chat notebooks, including creating and programming chat notebooks, and developing new custom chat notebook behavior. It provides a hands-on approach with real code examples and tips for using chat notebook features effectively.

Topics

Creating chat notebooks

Chat notebook programming

Adding new behavior to chat notebooks

Modifying existing chat notebook behavior

Chat notebook LLM tools

Accessing and using default chat tools programmatically

Configuring default tools

Utilities for optimizing tool outputs for chat notebooks

Prerequisites

Install this prerelease version if you want to try the examples (close any currently open chat notebooks first):

PacletInstall["https://github.com/WolframResearch/Chatbook/releases/download/WTC2023-Prerelease/Wolfram__Chatbook-1.3.1.paclet"];
<<Wolfram`Chatbook`;

Get[ "Wolfram`Chatbook`" ];
CurrentChatSettings[ "Model" ]=.;
SetOptions[ #1, CellLabel -> Inherited ] & /@ Cells[ ];
$Line = 0;

Creating chat notebooks

Create a new chat notebook:

CreateChatNotebook[]

Specify initial content:

CreateChatNotebook[{Cell["Test notebook","Title"],Cell["Please ignore","Text"],Cell["Hello","ChatInput"]}]

Convert an existing notebook:

CreateChatNotebook[NotebookOpen[FindFile["ExampleData/document.nb"]]]

Available options:

Options[CreateChatNotebook]

Notebook-specific chat preferences:

CreateChatNotebook["Assistance"->True]

CreateChatNotebook[
"Model"->{"Anthropic","claude-2"},
"LLMEvaluator"->"RawModel"
]

Some settings that aren’t yet available through the UI:

CreateChatNotebook[
{},
(* Max number of cells to include in history: *)
"ChatHistoryLength"->3,
(* Specify generic chat notebook prompt components by name: *)
"BasePrompt"->{"Notebooks","Formatting"},
(* Additional system prompts: *)
"Prompts"->{"Use tools whenever possible."},
(* Do not merge adjacent cells into a single chat message: *)
"MergeMessages"->False,
(* Provide tools specific to the notebook: *)
"Tools"->{LLMTool[]},
(* Define a persona specific to the notebook: *)
"LLMEvaluator"-><|
"Name"->"PiratePainter",
"Prompts"->{LLMPrompt["PirateSpeak"][],"Keep responses short."},
"Icon"->Style["🏴‍☠️",FontSize->25]
|>,
(* Any other normal notebook options can be given as well: *)
WindowSize->{1920,1000},
Magnification->2,
WindowMargins->0
]

Chat notebook programming

ChatCellEvaluate—programmatically evaluate a cell as chat input

CurrentChatSettings—get or set chat-related settings for a particular scope

Chat notebook programming

ChatCellEvaluate

Programmatically evaluate a chat input:

nbo=CreateChatNotebook[{Cell["Hello","ChatInput"]},"Model"->"gpt-3.5-turbo"]

cell=First[Cells[nbo]]

ChatCellEvaluate[cell]

Chat notebook programming

ChatCellEvaluate

How is this different from NotebookEvaluate[...,InsertResults->True] or SelectionEvaluateCreateCell[...]?

Any cell can be sent as a chat input, even those that are normally not evaluatable.

nbo=CreateChatNotebook[,"Model"->"gpt-3.5-turbo"];

ChatCellEvaluate[Last[Cells[nbo]]]

Chat notebook programming

CurrentChatSettings

CurrentChatSettings[]

CurrentChatSettings["Model"]

Specify scope:

nbo=CreateChatNotebook["Model"->"gpt-3.5-turbo"]

CurrentChatSettings[nbo,"Model"]

Settings specified by delimiters are reflected in output:

CurrentChatSettings[#,"Model"]&/@Cells[nbo,CellStyle->"ChatInput"]

Default scope is $FrontEnd:

CurrentChatSettings[]===CurrentChatSettings[$FrontEnd]

Modify settings:

CurrentChatSettings[nbo,"Model"]={"Anthropic","claude-instant-1"}

CurrentChatSettings[#,"Model"]&/@Cells[nbo,CellStyle->"ChatInput"]

Update global preferences persistently:

CurrentChatSettings["Model"]="gpt-3.5-turbo"

CurrentChatSettings["Assistance"]=True

Restore default values:

CurrentChatSettings["Model"]=.

CurrentChatSettings["Assistance"]=.

Adding new behavior to chat notebooks

Chat notebooks accept HandlerFunctions, which can be specified to run arbitrary code at certain events during chat evaluations.

ChatPre—runs at the start of a chat evaluation

ChatPost—runs at the end of a chat evaluation

Other handlers supported by URLSubmit

More coming…

Adding new behavior to chat notebooks

See the arguments passed to the ChatPre and ChatPost handlers:

CreateChatNotebook[{Cell["Hello","ChatInput"]},HandlerFunctions-><|
"ChatPre"->EchoLabel["ChatPre"]@*Map[Iconize],
"ChatPost"->EchoLabel["ChatPost"]@*Map[Iconize]
|>]

Real examples

Automatically generate a window title for chat notebooks:

NotebookOpen["https://github.com/WolframResearch/Chatbook/raw/main/Notebooks/WTC2023/AutoNotebookTitle.nb"];

Attach a “suggestions” bar to each chat output:

NotebookOpen["https://github.com/WolframResearch/Chatbook/raw/main/Notebooks/WTC2023/SuggestionsBar.nb"];

Modifying existing chat notebook behavior

Modify chat notebook behavior by replacing functions used during chat evaluations with your own using the “ProcessingFunctions” option.

CellToChatMessage—converts Cell expressions into chat messages.

FormatChatOutput—converts the string response from the LLM into formatted notebook content.

WriteChatOutputCell—performs the final NotebookWrite when streaming output has completed.

More coming…

Modifying existing chat notebook behavior

CellToChatMessage

The function used to generate chat messages from notebook content

User messages:

CellToChatMessage[Cell["Hello","ChatInput"]]

Assistant messages:

CellToChatMessage[Cell["Hello","ChatOutput"]]

Automatic string conversion:

CellToChatMessage[Cell["My Section","Section"]]

CellToChatMessage[Cell[BoxData[MakeBoxes[1+1]],"Input",CellLabel->"In[1]:="]]

Modifying existing chat notebook behavior

CellToChatMessage

Customize how chat messages are created:

nbo=CreateChatNotebook[
{Cell["Testing","Section"],Cell["Hello","ChatInput"]},
"ProcessingFunctions"-><|
"CellToChatMessage"->(ToString[#,InputForm]&)
|>,
"Model"->"gpt-3.5-turbo"
]

NotebookEvaluate[nbo,InsertResults->True]

NotebookClose[nbo]

Modifying existing chat notebook behavior

CellToChatMessage

Ignore cells that don’t match certain styles:

onlyChatCells[cell:Cell[_,"ChatInput"|"ChatOutput",___],info_]:=CellToChatMessage[cell,info];
onlyChatCells[___]:=Nothing;

nbo=CreateChatNotebook[
{Cell["Testing","Section"],Cell["Hello","ChatInput"]},
"ProcessingFunctions"-><|"CellToChatMessage"->onlyChatCells|>,
"Model"->"gpt-3.5-turbo"
]

The section cell is ignored:

NotebookEvaluate[nbo,InsertResults->True]

NotebookClose[nbo]

Modifying existing chat notebook behavior

Replace images with AI-generated captions:

NotebookOpen["https://github.com/WolframResearch/Chatbook/raw/main/Notebooks/WTC2023/ImageCaptions.nb"];

Modifying existing chat notebook behavior

FormatChatOutput

The function used to format chat output cells using the string returned by the LLM:

FormatChatOutput["Show markdown styles such as _italic_ and **bold**"]

FormatChatOutput["Format inline code `f[x] := x + 1`"]

FormatChatOutput["Automatic documentation links: `NotebookObject`"]

FormatChatOutput["Create a list of numbers with `Table`:
```Wolfram
Table[i^2, {i, 10}]
```"]

Style[FormatChatOutput["Create a list of numbers with `Table`:
```wolfram
Table[i^2, {i, 10}]
```
# Formatting
Here is a [link](https://www.wolfram.com).

## Items
* First thing
* Second thing

## Text Styles
This is some _cool_ formatted **stuff**.
Math formatting: $\\forall _{x,a\\in \\mathbb{R}}a x+x^2+1>0$
This is an example of ``Wolfram`Chatbook`FormatChatOutput``

## Images
![cat](https://wolfr.am/1hAbN3sam)
Expression URI: "<>MakeExpressionURI[Plot[Sin[x],{x,-10,10}]]],"ChatOutput"]

Customization

nbo=CreateChatNotebook[
{Cell["Tell me a short story","ChatInput"]},
"ProcessingFunctions"-><|"FormatChatOutput"->(If[StringQ[#],StringReverse[#],#]&)|>,
"Model"->"gpt-3.5-turbo"
];

NotebookEvaluate[nbo,InsertResults->True];

Modifying existing chat notebook behavior

An example that uses FormatChatOutput and WriteChatOutputCell:

NotebookOpen["https://github.com/WolframResearch/Chatbook/raw/main/Notebooks/WTC2023/WLEvaluatorExample.nb"];

LLM tools

Accessing the default chat notebook LLMTools

Using them outside of chat notebooks (in LLMSynthesize etc.)

Configuring the default chat notebook tools

Creating new tools that integrate nicely with chat notebooks

LLM tools

Examples of the default tools:

NotebookOpen["https://github.com/WolframResearch/Chatbook/raw/main/Notebooks/WTC2023/DefaultToolExamples.nb"];

Accessing the default tools

Access any of the built-in chat notebook tools programmatically:

$DefaultTools["DocumentationLookup"]

$DefaultTools//Keys

These will also be available and documented in the LLM Tool Repository soon.

LLM tools

Using the default tools outside of chat notebooks

Use a default chat notebook tool in LLMSynthesize:

LLMSynthesize["Use the Wolfram Language evaluator to find the 123456789th prime",LLMEvaluator-><|"Tools"->$DefaultTools["WolframLanguageEvaluator"]|>]

Prime[123456789]

LLM tools

Configuring the default tools

Some of the tools have options that can be configured:

$DefaultToolOptions

nbo=CreateChatNotebook[{Cell["Can you try importing data.csv using the Wolfram Language evaluator tool? I want to make sure the sandbox is working correctly. It should fail with a message.","ChatInput"],Cell["Oops, I forgot to disable it. Try it again.","ChatInput"]},"Model"->"gpt-4"]

Disable read access to files:

SetToolOptions[nbo,"WolframLanguageEvaluator","AllowedReadPaths"->None]

LinkClose[$SandboxKernel]

LLM tools

Interpreting LLMTool outputs for chat notebooks

Many chat tool outputs will contain URIs of the form ![label](attachment://result-id) if they do not convert to plain text nicely.

Get the string output from the Wolfram Language evaluator tool:

string=LLMSynthesize["Plot sin(x) from -10 to 10",LLMEvaluator-><|"Tools"->$DefaultTools["WolframLanguageEvaluator"]|>]

Use GetExpressionURIs to expand URIs to their original expression:

GetExpressionURIs[string]

These will only be available in the kernel session that generated them:

GetExpressionURIs["Out[1]= ![image](attachment://result-23438b894cc78b15)"]

LLM tools

Generating LLMTool outputs for chat notebooks

Use MakeExpressionURI to generate these types of URIs:

uri=MakeExpressionURI[ResourceFunction["BirdSay"]["neat"]]

string="This is pretty neat: "<>uri

GetExpressionURIs[string]//Row

FormatToolResponse

For simple inputs, it behaves like TextString:

FormatToolResponse["just a string"]

FormatToolResponse[Now]

Graphics are automatically converted to URIs using MakeExpressionURI:

FormatToolResponse[Plot[Sin[x],{x,-5,5}]]

GetExpressionURIs[%]

Large outputs are formatted as a shortened version along with a URI for the full expression:

FormatToolResponse[Iconize[Range[1000]]]

GetExpressionURIs[%]

Here are some tools that do not work well in pure text:

tools={
LLMTool[
"BirdSay",
"text"->"String",
ResourceFunction["BirdSay"][#text]&
],
LLMTool[
"WolfieSay",
"text"->"String",
ResourceFunction["WolfieSay"][#text]&
]
};

data=LLMSynthesize["Use the BirdSay and WolfieSay tools to create a short exchange between Birdnardo and Wolfie where Birdnardo tells Wolfie a knock-knock joke.",All,LLMEvaluator-><|"Tools"->tools|>];

The LLM struggles to understand the return value:

data["CompletionText"]

data["CompletionToolsText"]

Use FormatToolResponse on the tool output and add some additional prompting to instruct the LLM how to make use of the result:

tools={
LLMTool[
{"BirdSay","You must use the resulting markdown link in your response in order for the user to see the formatted result."},
"text"->"String",
FormatToolResponse[ResourceFunction["BirdSay"][#text]]&
],
LLMTool[
{"WolfieSay","You must use the resulting markdown link in your response in order for the user to see the formatted result."},
"text"->"String",
FormatToolResponse[ResourceFunction["WolfieSay"][#text]]&
]
};

data=LLMSynthesize["Use the BirdSay and WolfieSay tools to create a short exchange between Birdnardo and Wolfie where Birdnardo tells Wolfie a knock-knock joke.",All,LLMEvaluator-><|"Tools"->tools|>];

The plain text result will have expression URIs:

data["CompletionText"]

Create nice formatted output:

GetExpressionURIs[%]//Column

Looking for feedback

What types of features would you like added?

Do you have an idea for a specific application that you want to use chat notebooks for?

https://github.com/WolframResearch/Chatbook

Feel free to create issues

Suggestions

Feature requests

Bug reports

The End

Questions?

CurrentChatSettings[EvaluationNotebook[],"ChatHistoryLength"]=1000;

StringLength["This presentation discusses advanced usage of Wolfram Language's chat notebooks, including creating and programming chat notebooks, and developing new custom chat notebook behavior. It provides a hands-on approach with real code examples and tips for using chat notebook features effectively."]

SetOptions[#,CellLabel->Inherited]&/@Cells[];