Workshop: LLM Tech

Giulio Alessandrini, Timothee Verdier, Bob Sandheinrich,
Christopher Wolfram, Avery Davis, Connor Gray
Wolfram Research

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

Setup Your LLM Access

To use LLM via external APIs you need a authorization key.

A key will be provided for the duration of this workshop

There  are several ways to store a key persistently

Create a new service connection and select

ServiceConnect["OpenAI","New"]

Store the key in your system keychain

SystemCredential["OPENAI_API_KEY"]="MY API KEY"

Let us know if you have any issue

Setup a Chat Notebook

Try to open a chat notebook

This can be done using the menu items

File ▶ New ▶ Chat–Enabled Notebook

File ▶ New ▶ Chat–Driven Notebook

Or directly from the keyboard

⌘ + ⌥ + n on macOS

Ctrl + ⌥ + n on Win/Linux

Start a new Chat

"'" insert a new chat input cell

"~" insert a new chat input cell

The Concept of Persona

LLM can give different answer when prompted with different initial input.
These are typically called personas.

Personas can be selected from the chat dropdown menu

The Concept of Persona

New personas can be installed from the Wolfram Prompt Repository.

You can add personas dynamically from the chat input

Press "@" and start typing to get suggestions

Personas can be enabled or disabled from the preference panel

Select Add & Manage Personas...

The Concept of Tools

Tools are an extremely powerful way to extend the LLM capabilities.

Tools work only with GPT4 out of the box

Tools involve multiple LLM calls

Application: Unit Tests Generation

Writing tests is boring but we have to do it

Use a tool to generate tests

Programmatic Use

In version 13.3+ of the Wolfram Language there are several functions that can help you interact with LLMs, depending on what you need to do

text completion: LLMSynthesize

parameter based completion: LLMFunction

example based completion: LLMExampleFunction

chat session: ChatObject/ChatEvaluate

specific application

ImageSynthetize

TextSummarize

Programmatic Use

Text Completion

Basic text completion. The LLM will try to continue the prompt or perform the task asked

LLMSynthesize["The best movie ever is "]

LLMSynthesize["list three strange colors"]

Programmatic Use

Parameter Based Completion

Useful to build up prompt programmatically
The result can be used as a normal function in your workflows

opposite=LLMFunction["The opposite of `` is "]

opposite["dark"]

opposite/@{"up","left"}

Programmatic Use

Example Based Completion

Build the prompt from input/output pairs

sentiment=LLMExampleFunction[{"I absolutely love the new update! Everything runs so smoothly now."->"Positive","The service was okay, but it took a while to get my order."->"Neutral","I'm disappointed with the product quality for the price I paid. Not happy!"->"Negative"}]

sentiment["The app's performance is alright, but it could use some more features to be really useful"]

This  can  be  useful  if  you  have  a  specific output in mind

show to the LLM that you want comma separated lists

list=LLMExampleFunction[{
"List two animals"->"cat, dog",
"List four cities"->"Paris, Cairo, Bogotá, Tokio"}]

list["list three strange colors"]

this is useful to conform the result

list=LLMExampleFunction[{
"List two animals"->"cat, dog",
"List four cities"->"Paris, Cairo, Bogotá, Tokio"},
DelimitedSequence["String"]]

list["list three strange colors"]

Programmatic Use

Chat Sessions

chat=ChatObject[]

chat//=ChatEvaluate["what it the heighest mountain?"]

chat//=ChatEvaluate["where is it?"]

LLMEvaluator

The Wolfram Language introduces the concept of LLMEvaluator
This is inspired by the frontend using specific evaluators to run code in buttons, cells and dynamic expressions

LLMEvaluator is an option of many LLM based function and allows precise fine tuning via a LLMConfiguration object

Specify the the function should use a model from Anthropic

LLMSynthesize["what it the heighest mountain?",LLMEvaluator->LLMConfiguration[<|"Model"->{"Anthropic","claude-instant-1"}|>]]

Anthropic is quite verbose. Let's try to do something about it

Specify a prompt modifier

LLMSynthesize["what it the heighest mountain?",LLMEvaluator->LLMConfiguration[<|"Model"->{"Anthropic","claude-instant-1"},"Prompts"->{"answer in a brief and concise way"}|>]]

The effect can be quite dramatic

Let's use a prompt from the repository via LLMPrompt

LLMSynthesize["what it the heighest mountain? ",LLMEvaluator->LLMConfiguration[<|"Model"->{"Anthropic","claude-instant-1"},"Prompts"->{LLMPrompt["NumericOnly"]
}|>]]

LLMEvaluator

If you use some settings over and over they can be baked into the default evaluator

$LLMEvaluator=LLMConfiguration[<|"Model"->{"Anthropic","claude-instant-1"},"Prompts"->{"answer in a brief and concise way"}|>]

LLMSynthesize["what it the heighest mountain?"]

LLMEvaluator

Some setting can affect the LLM sampling process

Make the output repeatable

LLMSynthesize["what it the heighest mountain?",LLMEvaluator->LLMConfiguration[<|"Temperature"->0|>]]

LLMSynthesize["what it the heighest mountain?",LLMEvaluator->LLMConfiguration[<|"Temperature"->0|>]]

Specify a maximum amount of token to sample

LLMSynthesize["what it the heighest mountain?",LLMEvaluator->LLMConfiguration[<|"Temperature"->0,"MaxTokens"->5|>]]

Specify one or more tokens that should act as stop signal

LLMSynthesize["what it the heighest mountain?",LLMEvaluator->LLMConfiguration[<|"Temperature"->0,"StopTokens"->{",","."}|>]]

LLMEvaluator

Use tools programmatically

define a tool that expands an expression

expand=LLMTool["expression expand","input"->"Expression",Expand[#input]&]

LLMSynthesize["what is the expansion of (x+4)^4. use the tool",LLMEvaluator->LLMConfiguration[<|"Tools"->{expand,series}|>]]

define a multi parameter tool that computes Taylor series

series=LLMTool[{"series expand","compute taylor series"},{"input"->"Expression","order"->"Integer","variable"->"Expression"},ToString[Normal@Series[#input,{#variable,0,#order}],InputForm]&]

requests and responses are automatically parsed

LLMSynthesize["compute the taylor approximation of w*Sin[w] to the tenth order",{"CompletionText","ToolRequests","ToolResponses"},LLMEvaluator->LLMConfiguration[<|"Tools"->{expand,series}|>]]

Specific Applications

ImageSynthesize["a workshop about LLM tech"]

TextSummarize[ExampleData[{"Text","AliceInWonderland"}]]

THE end