From Rock-Paper-Scissors to BabyAGI

Jofre Espigule-Pons, Machine Learning Team

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

In this presentation, we first explore large language models (LLMs) tackling the classic game of rock-paper-scissors, revealing nuanced decision-making and pattern recognition. Transitioning from game strategies, we then delve into the details of implementing an autonomous agent. Namely, a version of BabyAGI within the Wolfram Language ecosystem.

LLMs playing Rock-Paper-Scissors Game

*Image generated using DALL·E 3

LLMs are deterministic at temperature zero

Temperature in the context of LLMs

The output of an LLM is a probability distribution over every word (token) in the vocabulary, telling it the likelihood that any of these words might follow the input text.

This probability distribution is computed by using the softmax function:

The “temperature” is a hyperparameter used in the softmax function, which transforms model output logits into probabilities for the next token in the sequence. A higher temperature makes the distribution over the next token more uniform (increases randomness), while a lower temperature makes the distribution more focused on the most likely token (makes outputs more deterministic).

The connection to the physical temperature comes from the Boltzmann distribution, which looks like this:

(*Define the Boltzmann distribution function with temperature*)boltzmannDist[energy_List,temp_]:=Module[{Z},Z=Total[Exp[-energy/temp]];
Exp[-energy/temp]/Z];

(*Define some discrete energy states*)
energyStates=Range[0,9];

(*Plot Boltzmann probabilities for different temperatures*)
tempValues={0.1,1.0,10.0};
labels=StringJoin["T = ",ToString[#]]&/@tempValues;

ListLinePlot[Map[boltzmannDist[energyStates,#]&,tempValues],PlotRange->All,PlotLegends->labels,AxesLabel->{"Energy","Probability"},PlotLabel->"Effect of Temperature on Boltzmann Distribution"]

Here’s an example of how you might implement and visualize the effect of temperature scaling on a simple softmax function in Wolfram Language:

(*Define the softmax function with temperature*)softmaxWithTemp[logits_List,temp_]:=Module[{max=Max[logits]},Exp[(logits-max)/temp]/Total[Exp[(logits-max)/temp]]];

(*Randomly generate some logits*)
logits=RandomReal[{-1,1},10];

(*Plot softmax probabilities for different temperatures*)
tempValues={0.01,0.1,0.5,1.0,2.0};
labels=StringJoin["T = ",ToString[#]]&/@tempValues;

ListLinePlot[Map[softmaxWithTemp[logits,#]&,tempValues],PlotLegends->labels,PlotRange->All,AxesLabel->{"Index","Probability"},PlotLabel->"Effect of Temperature on Softmax Probabilities"]

Temperature 0

Repeating the same prompt 10 times, we obtain the exact same results each time:

Table[LLMSynthesize["Randomly select 1 or 2. Just write the number. DO NOT USE WORDS!",LLMEvaluator-><|"Model"->"gpt-4","Temperature"->0|>],10]

Temperature 1

Table[LLMSynthesize["Randomly select 1 or 2. Just write the number. DO NOT USE WORDS!",LLMEvaluator-><|"Model"->"gpt-4","Temperature"->1|>],10]

Rock-Paper-Scissors Game

Using a single prompt for both LLM players

Game rules are specified within the prompt.

*Image from Wikipedia: https://en.wikipedia.org/wiki/Rock_paper_scissors

Association to track played games

playedGames := <|
  "Your Plays" -> {},
  "Opponent Plays" -> {}
|>;

Failure Handler Function for OpenAI API

If the API fails it repeats the process, if it fails 5 times it returns $Failed:

apiFailureHandler[f_]:=Catch[Do[With[{res=f},If[StringQ[res],Throw[res]]],5];$Failed]

Agent Player definition

agent[playedGames_Association, model_String]:=apiFailureHandler@LLMSynthesize["You are an AI agent playing Rock-Paper-Scissors games against another player. Rock beats Scissors, Paper beats Rock and Scissors beats Paper. There are going to be 20 games in a match and you want to win the match. Here are the results of your previous games (from first played game to the last game played so far): "<>ToString[playedGames]<>"\nSelect your next play. DO NOT USE EXTRA WORDS! (Options: ROCK, PAPER, SCISSORS):",LLMEvaluator-><|"Model"->model,"Temperature"->0.001|>]

agent[playedGames,"gpt-4"]

agent[playedGames,"gpt-3.5-turbo"]

Helper functions to add plays

addPlay[playedGames_Association, selectedPlay_String] := "Your Plays" -> Append[playedGames["Your Plays"], selectedPlay];

player1Plays=addPlay[playedGames, agent[playedGames,"gpt-4"]]

addOpponentPlay[playedGames_Association, selectedPlay_String] := "Opponent Plays" -> Append[playedGames["Your Plays"], selectedPlay];

player2Plays=addOpponentPlay[playedGames, agent[playedGames,"gpt-3.5-turbo"]]

Association@{player1Plays,player2Plays}

Helper function to swap players in the playedGames association

swapPlayers[a_Association]:=AssociationThread[Keys[a]->Reverse[Values[a]]]

playedGames := <|
  "Your Plays" -> {"ROCK"},
  "Opponent Plays" -> {"PAPER"}
|>;

swapPlayers[playedGames]

Play one game

Alice is using GPT-4.

Bob is using GPT-3.5.

Clear[playAGame];

playAGame[previousGames_Association] := Block[{playedGames = previousGames, playedGamesForBob, alicePlays, bobPlays},
   playedGamesForBob = swapPlayers[playedGames];
   alicePlays = addPlay[playedGames, agent[playedGames, "gpt-4"]];
   bobPlays = addOpponentPlay[playedGamesForBob, agent[playedGamesForBob, "gpt-3.5-turbo"]];
   Association @{alicePlays, bobPlays}
   ]

playAGame[playedGames]

Play 20 games with Nest

playedGames := <|
  "Your Plays" -> {},
  "Opponent Plays" -> {}
|>;

results=Nest[playAGame, playedGames, 20]

gameResult[game_List]:= Which[
	{"ROCK", "SCISSORS"} == game, "Alice",
	{"ROCK", "PAPER"} == game, "Bob",
	{"ROCK", "ROCK"} == game, "Draw",
	{"PAPER", "ROCK"} == game, "Alice",
	{"PAPER", "SCISSORS"} == game, "Bob",
	{"PAPER", "PAPER"} == game, "Draw",
	{"SCISSORS", "SCISSORS"} == game, "Draw",
	{"SCISSORS", "PAPER"} == game, "Alice",
	{"SCISSORS", "ROCK"} == game, "Bob"]

Function to create a custom PieChart:

makePieChart[games_Association]:= With[{results = Counts@Map[gameResult, Thread@Values@games]},
	PieChart[results,
		ChartLabels -> Automatic, 
		ChartLegends -> Automatic,
		ChartStyle -> Keys@results/.{"Alice"->LightPink,"Bob"->LightBlue,"Draw"->LightYellow},
		ImageSize -> 400
		]
	]

makePieChart[results]

score[{myPlay_, opPlay_}] := Which[
   myPlay === opPlay, 0,
   {myPlay, opPlay} === {"ROCK", "SCISSORS"}, 1,
   {myPlay, opPlay} === {"SCISSORS", "PAPER"}, 1,
   {myPlay, opPlay} === {"PAPER", "ROCK"}, 1,
   True, 0  (* Loss also gives 0 *)
]

Function to create a ListLinePlot visualizing number of wins:

makeListLinePlot[results_Association]:= Block[{
	aliceScores = Accumulate[Map[score, Transpose[Values[results]]]], 
	bobScores = Accumulate[Map[score, Transpose[Reverse[Values[results]]]]] (* Inverting player roles for Bob's score *)
	},
	ListLinePlot[{aliceScores, bobScores}, 
		PlotLabels -> {"Alice", "Bob"},  
		AxesLabel -> {"Game \nNumber", "Wins"},
		GridLines -> Automatic,
		LabelStyle -> Directive[Black, 16],
		ImageSize -> Large
		]
	]

makeListLinePlot[results]

Results using all 9 possible initial game states configurations

Rock - Scissors

playedGames := <|
  "Your Plays" -> {"ROCK"},
  "Opponent Plays" -> {"SCISSORS"}
|>;

resultsRS=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsRS]]

moves = {"PAPER"->,"ROCK"-> ,"SCISSORS"->};

Thread@Values@resultsRS/.moves

Rock - Paper

playedGames := <|
  "Your Plays" -> {"ROCK"},
  "Opponent Plays" -> {"PAPER"}
|>;

resultsRP=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsRP]]

Thread@Values@resultsRP/.moves

Rock - Rock

playedGames := <|
  "Your Plays" -> {"ROCK"},
  "Opponent Plays" -> {"ROCK"}
|>;

resultsRR=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsRR]]

Paper - Rock

playedGames := <|
  "Your Plays" -> {"PAPER"},
  "Opponent Plays" -> {"ROCK"}
|>;

resultsPR=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsPR]]

Paper - Scissors

playedGames := <|
  "Your Plays" -> {"PAPER"},
  "Opponent Plays" -> {"SCISSORS"}
|>;

resultsPS=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsPS]]

Thread@Values@resultsPS/.moves

Paper - Paper

playedGames := <|
  "Your Plays" -> {"PAPER"},
  "Opponent Plays" -> {"PAPER"}
|>;

resultsPP=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsPP]]

Scissors - Paper

playedGames := <|
  "Your Plays" -> {"SCISSORS"},
  "Opponent Plays" -> {"PAPER"}
|>;

resultsSP=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsSP]]

Scissors - Rock

playedGames := <|
  "Your Plays" -> {"SCISSORS"},
  "Opponent Plays" -> {"ROCK"}
|>;

resultsSR=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsSR]]

Scissors - Scissors

playedGames := <|
  "Your Plays" -> {"SCISSORS"},
  "Opponent Plays" -> {"SCISSORS"}
|>;

resultsSS=Nest[playAGame, playedGames, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsSS]]

Overall Results

With[{results=Counts@Map[gameResult,Thread@Map[Flatten,Thread@{Values@resultsRS,Values@resultsRP,Values@resultsRR,Values@resultsSR,Values@resultsSP,Values@resultsSS,Values@resultsPS,Values@resultsPR,Values@resultsPP}]]},PieChart[results,ChartLabels->Automatic,ChartLegends->Automatic,ChartStyle->Keys@results/.{"Alice"->LightPink,"Bob"->LightBlue,"Draw"->LightYellow}]]

Using a different prompt

This time we will display the games as a list of associations with players plays alongside the game result of each round:

playedGames := {<|"Game Number"->1,"Your Play" -> "SCISSORS","Opponent Play" -> "SCISSORS","Game Result"->"Draw"|>};

Agent Player definition

agent[playedGames_List, model_String]:=apiFailureHandler@LLMFunction["You are an AI agent playing Rock-Paper-Scissors games against another player. Rock beats Scissors, Paper beats Rock and Scissors beats Paper. There are going to be 20 games in a match and you want to win the match. For the current match, here are the results of your previous games (from first played game to the last game played so far): `PlayedGames`\nSelect your next play. DO NOT USE EXTRA WORDS! (Options: ROCK, PAPER, SCISSORS):",LLMEvaluator-><|"Model"->model,"Temperature"->0.001|>][<|"PlayedGames"->playedGames|>]

agent[playedGames,"gpt-4"]

agent[playedGames,"gpt-3.5-turbo"]

Play one game

Alice is using GPT - 4.

Bob is using GPT - 3.5.

gameResult[game_List]:= Which[
	{"ROCK", "SCISSORS"} == game, "You win",
	{"ROCK", "PAPER"} == game, "You lose",
	{"ROCK", "ROCK"} == game, "Draw",
	{"PAPER", "ROCK"} == game, "You win",
	{"PAPER", "SCISSORS"} == game, "You lose",
	{"PAPER", "PAPER"} == game, "Draw",
	{"SCISSORS", "SCISSORS"} == game, "Draw",
	{"SCISSORS", "PAPER"} == game, "You win",
	{"SCISSORS", "ROCK"} == game, "You lose"]

Clear[playAGame];

playAGame[{playedGamesForAlice_List, playedGamesForBob_List}] := Block[{alicePlay, bobPlay},
   alicePlay = agent[playedGamesForAlice, "gpt-4"];
   bobPlay = agent[playedGamesForBob, "gpt-3.5-turbo"];
   {Append[playedGamesForAlice, <|"Game Number"-> Last[playedGamesForAlice]["Game Number"]+1, "Your Play" -> alicePlay, "Opponent Play" -> bobPlay, "Game Result" -> gameResult[{alicePlay, bobPlay}]|>],
   Append[playedGamesForBob, <|"Game Number"-> Last[playedGamesForBob]["Game Number"]+1, "Your Play" -> bobPlay, "Opponent Play" -> alicePlay, "Game Result" -> gameResult[{bobPlay, alicePlay}]|>]}
   ]

Results using all 9 possible initial game states configurations

Rock - Scissors

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "ROCK","Opponent Play" -> "SCISSORS","Game Result"->"You win"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "SCISSORS","Opponent Play" -> "ROCK","Game Result"->"You lose"|>};

resultsRS=Nest[playAGame, {playedGamesForAlice,playedGamesForBob}, 19]

Function to create a custom PieChart:

makePieChart[games_List]:= With[{results = Counts[First[games][[All,"Game Result"]]/.{"You win"->"Alice (GPT-4)","You lose"->"Bob (GPT-3.5)"}]},
	PieChart[results,
		ChartLabels -> Automatic, 
		ChartLegends -> Automatic,
		ChartStyle -> Keys@results/.{"Alice (GPT-4)"-> LightPink, "Bob (GPT-3.5)" -> LightBlue, "Draw" -> LightYellow},
		ImageSize -> 400
		]
	]

Function to create a ListLinePlot visualizing number of wins:

score[{myPlay_, opPlay_}] := Which[
   myPlay === opPlay, 0,
   {myPlay, opPlay} === {"ROCK", "SCISSORS"}, 1,
   {myPlay, opPlay} === {"SCISSORS", "PAPER"}, 1,
   {myPlay, opPlay} === {"PAPER", "ROCK"}, 1,
   True, 0  (* Loss also gives 0 *)
]

makeListLinePlot[results_List]:= Block[{
	aliceScores = Accumulate[Map[score, Values@First[results][[All,{"Your Play","Opponent Play"}]]]], 
	bobScores = Accumulate[Map[score, Values@Last[results][[All,{"Your Play","Opponent Play"}]]]]
	},
	ListLinePlot[{aliceScores, bobScores}, 
		PlotLabels -> {"Alice", "Bob"},  
		AxesLabel -> {"Game \nNumber", "Wins"},
		GridLines -> Automatic,
		LabelStyle -> Directive[Black, 16],
		ImageSize -> Large
		]
	]

Column@Through[{makePieChart,makeListLinePlot}[resultsRS]]

Values@First[resultsRS][[All,{"Your Play","Opponent Play"}]]/.moves

Rock - Paper

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "ROCK","Opponent Play" -> "PAPER","Game Result"->"You lose"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "PAPER","Opponent Play" -> "ROCK","Game Result"->"You win"|>};

resultsRP=Nest[playAGame,{playedGamesForAlice,playedGamesForBob}, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsRP]]

Values@First[resultsRP][[All,{"Your Play","Opponent Play"}]]/.moves

Rock - Rock

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "ROCK","Opponent Play" -> "ROCK","Game Result"->"Draw"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "ROCK","Opponent Play" -> "ROCK","Game Result"->"Draw"|>};

resultsRR=Nest[playAGame,{playedGamesForAlice,playedGamesForBob}, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsRR]]

Paper - Rock

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "PAPER","Opponent Play" -> "ROCK","Game Result"->"You win"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "ROCK","Opponent Play" -> "PAPER","Game Result"->"You lose"|>};

resultsPR=Nest[playAGame,{playedGamesForAlice,playedGamesForBob}, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsPR]]

Paper - Scissors

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "PAPER","Opponent Play" -> "SCISSORS","Game Result"->"You lose"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "SCISSORS","Opponent Play" -> "PAPER","Game Result"->"You win"|>};

resultsPS=Nest[playAGame,{playedGamesForAlice,playedGamesForBob}, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsPS]]

Paper - Paper

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "PAPER","Opponent Play" -> "PAPER","Game Result"->"Draw"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "PAPER","Opponent Play" -> "PAPER","Game Result"->"Draw"|>};

resultsPP=Nest[playAGame,{playedGamesForAlice,playedGamesForBob}, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsPP]]

Scissors - Paper

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "SCISSORS","Opponent Play" -> "PAPER","Game Result"->"You win"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "PAPER","Opponent Play" -> "SCISSORS","Game Result"->"You lose"|>};

resultsSP=Nest[playAGame,{playedGamesForAlice,playedGamesForBob}, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsSP]]

Scissors - Rock

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "SCISSORS","Opponent Play" -> "ROCK","Game Result"->"You lose"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "ROCK","Opponent Play" -> "SCISSORS","Game Result"->"You win"|>};

resultsSR=Nest[playAGame,{playedGamesForAlice,playedGamesForBob}, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsSR]]

Scissors - Scissors

playedGamesForAlice := {<|"Game Number"->1,"Your Play" -> "SCISSORS","Opponent Play" -> "SCISSORS","Game Result"->"Draw"|>};
playedGamesForBob := {<|"Game Number"->1,"Your Play" -> "SCISSORS","Opponent Play" -> "SCISSORS","Game Result"->"Draw"|>};

resultsSS=Nest[playAGame,{playedGamesForAlice,playedGamesForBob}, 19]

Column@Through[{makePieChart,makeListLinePlot}[resultsSS]]

Values@First[resultsSS][[All,{"Your Play","Opponent Play"}]]/.moves

Overall Results

With[{results=Counts@Flatten@Map[First[#][[All,"Game Result"]]/.{"You win"->"Alice (GPT-4)","You lose"->"Bob (GPT-3.5)"}&,{resultsRS,resultsRP,resultsRR,resultsSR,resultsSP,resultsSS,resultsPS,resultsPR,resultsPP}]},PieChart[results,ChartLabels->Automatic,ChartLegends->Automatic,ChartStyle->Keys@results/.{"Alice (GPT-4)"->LightPink,"Bob (GPT-3.5)"->LightBlue,"Draw"->LightYellow}]]

Counts@Flatten@Map[First[#][[All,"Game Result"]]/.{"You win"->"Alice (GPT-4)","You lose"->"Bob (GPT-3.5)"}&,{resultsRS,resultsRP,resultsRR,resultsSR,resultsSP,resultsSS,resultsPS,resultsPR,resultsPP}]

Alice (GPT- 4) win rates

Using First Prompt

winRateAliceExperiment1=PercentForm[69/(69+75+36)//N]

Using Second Prompt

winRateAliceExperiment2=PercentForm[72/(72+68+40)//N]

Bob (GPT-3.5) win rates

Using First Prompt

winRateBobExperiment1=PercentForm[36/(69+75+36)//N]

Using Second Prompt

winRateBobExperiment2=PercentForm[40/(72+68+40)//N]

Counts of Moves

We can observe that Alice (GPT-4) took advantage of Bob (GPT-3.5) by playing more times PAPER, since Bob was bias to play ROCK and he only played SCISSORS four times. Indicating that GPT-4 is learning from previous games of a match.

With[{results=Counts@Flatten@Map[First[#][[All,"Your Play"]]&,{resultsRS,resultsRP,resultsRR,resultsSR,resultsSP,resultsSS,resultsPS,resultsPR,resultsPP}]},PieChart[results,ChartLegends->Automatic,PlotLabel->Style["Alice (GPT-4)",Large],ChartStyle->Keys@results/.{"ROCK"->LightRed,"PAPER"->LightBlue,"SCISSORS"->LightGreen}]]

With[{results=Counts@Flatten@Map[First[#][[All,"Opponent Play"]]&,{resultsRS,resultsRP,resultsRR,resultsSR,resultsSP,resultsSS,resultsPS,resultsPR,resultsPP}]},PieChart[results,ChartLegends->Automatic,PlotLabel->Style["Bob (GPT-3.5)",Large],ChartStyle->Keys@results/.{"ROCK"->LightRed,"PAPER"->LightBlue,"SCISSORS"->LightGreen}]]

Pair of Plays Counts

PieChart[Counts@Values@Flatten@Map[First[#][[All,{"Your Play","Opponent Play"}]]&,{resultsRS,resultsRP,resultsRR,resultsSR,resultsSP,resultsSS,resultsPS,resultsPR,resultsPP}],ChartLegends->Automatic]

LLMs Playing Against Humans

gameResult[game_List]:= Which[
	{"ROCK", "SCISSORS"} == game, "You win",
	{"ROCK", "PAPER"} == game, "You lose",
	{"ROCK", "ROCK"} == game, "Draw",
	{"PAPER", "ROCK"} == game, "You win",
	{"PAPER", "SCISSORS"} == game, "You lose",
	{"PAPER", "PAPER"} == game, "Draw",
	{"SCISSORS", "SCISSORS"} == game, "Draw",
	{"SCISSORS", "PAPER"} == game, "You win",
	{"SCISSORS", "ROCK"} == game, "You lose"]

Clear[playAGame];

playAGame[{playedGamesForAlice_List, playedGamesForBob_List}, humanPlay_String] := Block[{alicePlay, bobPlay = humanPlay},
   alicePlay = agent[playedGamesForAlice, "gpt-4"];
   bobPlay = humanPlay;
   {Append[playedGamesForAlice, <|"Game Number"-> Last[playedGamesForAlice]["Game Number"]+1, "Your Play" -> alicePlay, "Opponent Play" -> bobPlay, "Game Result" -> gameResult[{alicePlay, bobPlay}]|>],
   Append[playedGamesForBob, <|"Game Number"-> Last[playedGamesForBob]["Game Number"]+1, "Your Play" -> bobPlay, "Opponent Play" -> alicePlay, "Game Result" -> gameResult[{bobPlay, alicePlay}]|>]}
   ]

Scissors - Scissors initial game states

Using Manipulate:

(* Initial game states *)
playedGamesForAlice = {<|"Game Number" -> 1, 
    "Your Play" -> "SCISSORS", "Opponent Play" -> "SCISSORS", 
    "Game Result" -> "Draw"|>};
playedGamesForBob = {<|"Game Number" -> 1, "Your Play" -> "SCISSORS",
     "Opponent Play" -> "SCISSORS", "Game Result" -> "Draw"|>};

Manipulate[
 If[Length[playedGamesForAlice] < 21,
  Grid[playedGamesForBob /. a_Association :> Normal[a]],
  "Game Over"
 ],
 {{bobPlay, "ROCK", "Bob's Play"}, {"ROCK", "PAPER", "SCISSORS"}},
 Button["Play Next Game",
  {playedGamesForAlice, playedGamesForBob} = playAGame[{playedGamesForAlice, playedGamesForBob}, bobPlay];,
  Method -> "Queued"
 ]
]

Counts[playedGamesForAlice[[All,"Game Result"]]/.{"You win"->"Alice (GPT-4)","You lose"->"Bob (Human)"}]

PieChart[Counts[playedGamesForAlice[[All,"Game Result"]]/.{"You win"->"Alice (GPT-4)","You lose"->"Bob (Human)"}],ChartLabels->Automatic,ChartLegends->Automatic]

Scissors - Paper initial game states

(* Initial game states *)
playedGamesForAlice = {<|"Game Number" -> 1, 
    "Your Play" -> "SCISSORS", "Opponent Play" -> "PAPER", 
    "Game Result" -> "You win"|>};
playedGamesForBob = {<|"Game Number" -> 1, "Your Play" -> "PAPER",
     "Opponent Play" -> "SCISSORS", "Game Result" -> "You lose"|>};

Manipulate[
 If[Length[playedGamesForAlice] < 21,
  Grid[playedGamesForBob /. a_Association :> Normal[a]],
  "Game Over"
 ],
 {{bobPlay, "ROCK", "Bob's Play"}, {"ROCK", "PAPER", "SCISSORS"}},
 Button["Play Next Game",
  {playedGamesForAlice, playedGamesForBob} = playAGame[{playedGamesForAlice, playedGamesForBob}, bobPlay];,
  Method -> "Queued"
 ]
]

Counts[playedGamesForAlice[[All,"Game Result"]]/.{"You win"->"Alice (GPT-4)","You lose"->"Bob (Human)"}]

PieChart[Counts[playedGamesForAlice[[All,"Game Result"]]/.{"You win"->"Alice (GPT-4)","You lose"->"Bob (Human)"}],ChartLabels->Automatic,ChartLegends->Automatic]

Scissors - Rock initial game states

(* Initial game states *)
playedGamesForAlice = {<|"Game Number" -> 1, 
    "Your Play" -> "SCISSORS", "Opponent Play" -> "ROCK", 
    "Game Result" -> "You lose"|>};
playedGamesForBob = {<|"Game Number" -> 1, "Your Play" -> "ROCK",
     "Opponent Play" -> "SCISSORS", "Game Result" -> "You win"|>};

Manipulate[
 If[Length[playedGamesForAlice] < 21,
  Grid[playedGamesForBob /. a_Association :> Normal[a]],
  "Game Over"
 ],
 {{bobPlay, "ROCK", "Bob's Play"}, {"ROCK", "PAPER", "SCISSORS"}},
 Button["Play Next Game",
  {playedGamesForAlice, playedGamesForBob} = playAGame[{playedGamesForAlice, playedGamesForBob}, bobPlay];,
  Method -> "Queued"
 ]
]

Reference for human experiments playing RPS game

Social cycling and conditional responses in the Rock-Paper-Scissors game

Next Steps

Playing against SequencePredict

Play Rock–Paper–Scissors Using SequencePredict: https://www.wolfram.com/language/12/high-level-machine-learning/play-rock-paper-scissors-using-sequencepredict.html

Using other models from HuggingFace in Mathematica 14.0

Most open models are not good enough yet:

SystemCredential["HUGGINGFACE_API_KEY"]="Add your api key here..."

so=ServiceConnect["HuggingFace",Authentication->{"apikey"->SystemCredential["HUGGINGFACE_API_KEY"]}]

ServiceExecute[so,"RawTextGeneration",{"model"->"meta-llama/Llama-2-7b-chat-hf","temperature"->0,
"inputs"->"You are an AI agent playing Rock-Paper-Scissors games against another player. Rock beats Scissors, Paper beats Rock and Scissors beats Paper. There are going to be 20 games in a match and you want to win the match. Here are the results of your previous games (from first played game to the last game played so far): <|Your Plays -> {PAPER, ROCK, ROCK, ROCK, ROCK}, Opponent Plays -> {SCISSORS, PAPER, PAPER, PAPER, PAPER}|>\nSelect your next play. DO NOT USE EXTRA WORDS!"
}]

Other related work

SmartPlay : A Benchmark for LLMs as Intelligent Agents

Github repo: https://github.com/microsoft/SmartPlay

It’s a benchmark and methodology for evaluating the abilities of large language models (LLMs) as agents by playing games including Rock-Paper-Scissors.

Generative Agents: Interactive Simulacra of Human Behavior: https://arxiv.org/pdf/2304.03442.pdf

Understanding Social Reasoning in Language Models with Language Models: https://arxiv.org/abs/2306.15448

Lang chain Agent simulations https://python.langchain.com/docs/use_cases/agent_simulations/

ProAgent: Building Proactive Cooperative AI with Large Language Models https://arxiv.org/abs/2308.11339v1

A Survey on Large Language Model based Autonomous Agents https://arxiv.org/abs/2308.11432v1

Playing with an autonomous agent: BabyAGI

*Image generated using DALL·E 3

BabyAGI Main Loop

BabyAGI uses three main sub-agents (Execution, Task Creation, and Prioritization Agents) to perform tasks based on a given objective. This basic example showcases how an autonomous agent can be utilized for generating lectures using Wolfram Language.

*Image from GitHub repository for BabyAGI:   https://github.com/yoheinakajima/babyagi/

Adding Wolfram Language documentation examples to the memory of the Auto Agent

Initialization

ClearAll[autoAgent, setObjective, addTask, addCompletedTask,executionAgent,taskCreationAgent,prioritizationAgent,iterateAutoAgent, iterateAutoAgentStep,nfunction];

Define its main components

Adding Wolfram Documentation examples to the memory of BabyAGI

Paclet to get OpenAI embeddings

Paclet needed to obtain OpenAIEmbedding:

PacletInstall["ChristopherWolfram/OpenAILink"];
Needs["ChristopherWolfram`OpenAILink`"]

OpenAIEmbedding["Create a 3D pie chart"]//Short

Note: Official way to get embeddings using OpenAI Service Connection

https://reference.wolfram.com/language/ref/service/OpenAI.html

sentences=StringSplit[ExampleData[{"Text","TheRaven"}],"."];
sentences//Length

sentenceEmbeddings=ServiceExecute["OpenAI","Embedding",{"Input"->sentences}];
sentenceEmbeddings//Short

Retrieve related WL documentation examples based on a certain task

The file for the nearest function based on documentation examples can be found on amoeba: https://amoeba.wolfram.com/index.php/s/Kb7tfE4axK7rRLi

nfunction:= Import["/Users/jofre/WRI related/ML/WL related/Agents/examplesNearestFunction.wl"];

nfunction

Testing the nearest function:

task="Create a 3D pie chart";

wlExamplesContext = nfunction[OpenAIEmbedding[task],2]

Execution Agent

Execution Agent takes an objective, a task, and a list of embeddings as inputs. It uses the GPT-4 model to execute a single task based on the given objective. For context, it considers three of the most similar completed tasks to the current task.

executionAgent[objective_String, task_String, embeddings_List] := StringTrim@LLMFunction[ "You are an AI who performs one task based on the following objective: `objective`. Take into account these previously completed tasks: `context`. Check the following related Wolfram Language examples for insipiration: `contextExamples`. Your task: `task`\nResponse:",LLMEvaluator->\[LeftAssociation]"Model"->"gpt-4","Temperature"->0\[RightAssociation]][<|"objective"->objective,"context"->Nearest[embeddings, OpenAIEmbedding[task],3, DistanceFunction -> CosineDistance],"contextExamples"->nfunction[OpenAIEmbedding[task],3],"task"->task|>];

Task Creation Agent

Task Creation Agent is designed to generate new tasks based on the result from the Execution Agent. It considers the objective, the last task result, its description, and a list of incomplete tasks. It ensures that the new tasks created do not overlap with the incomplete tasks.

taskCreationAgent[objective_String, result_String, taskDescription_String, taskList_List] :=LLMFunction[ "You are a task creation AI that uses the result of an execution agent to create new tasks with the following objective: `objective`. The last completed task has the result: `result`. This result was based on this task description: `taskDescription. These are incomplete tasks: "<>StringRiffle[taskList, ", "]<>". Based on the result, create new tasks to be completed by the AI system that do not overlap with incomplete tasks.",LLMEvaluator->\[LeftAssociation]"Model"->"gpt-4","Temperature"->0\[RightAssociation]][<|"objective"->objective,"result"->result,"taskDescription"->taskDescription|>];

Prioritization Agent

Prioritization Agent takes in a list of tasks and an objective. It cleans up the formatting of the task list and reprioritizes the tasks, keeping in mind the main objective. The agent does not remove any tasks. The tasks are returned in a prioritized order.

prioritizationAgent[taskList_List, objective_String] := Module[{ response, newTasks},
response =LLMFunction[ "You are a task prioritization AI tasked with cleaning the formatting of and reprioritizing the following tasks: `taskList`. Consider the ultimate objective of your team: `objective`. Do not remove any tasks. Return the result as a numbered list, like:\n#. First task\n#. Second task\n ...",LLMEvaluator->\[LeftAssociation]"Model"->"gpt-4","Temperature"->0\[RightAssociation]][<|"taskList"->taskList,"objective"->objective|>];
  newTasks = StringSplit[response, "\n"];
  newTasks = DeleteCases[newTasks, s_String /; StringTrim[s] == ""];
newTasks = StringTrim[StringReplace[newTasks, (StartOfString ~~ DigitCharacter .. ~~ ". ") -> ""]];
  newTasks
];

Iterate Auto Agent n times

This is where the main logic of the BabyAGI lies:

setObjective sets the main objective of the autoAgent.

addTask adds a task to the agent’s task list.

addCompletedTask adds a task to the list of completed tasks.

iterateAutoAgentStep carries out one iteration of the agent’s operation. It fetches and executes a task, adds it to the completed tasks, updates the task list, creates new tasks, and reprioritizes the task list.

iterateAutoAgent runs the single-step iteration n times.

setObjective[autoAgent_Association, objective_String] := Association[autoAgent, "objective" -> objective];
addTask[autoAgent_Association, task_String] := Association[autoAgent, "taskList" -> Append[autoAgent["taskList"], task]];
addCompletedTask[autoAgent_Association, completedTask_]:= Association[autoAgent, "completedTaskList" -> Append[autoAgent["completedTaskList"], completedTask]];

(* Iterate AutoAgent n times *)
iterateAutoAgentStep[autoA_Association] := Module[{autoAgent= autoA, task, result, newTasks, newTaskList},
   (* Step 1: Pull the first task *)
   task = First[autoAgent["taskList"]];
   
   (* Step 2: Execute the task *)
   result = executionAgent[autoAgent["objective"], task, autoAgent["completedTaskList"]];
   
   (* Step 3: Add completed tasks to memory *)
   autoAgent = addCompletedTask[autoAgent, With[{taskPlusResult = StringJoin["Task: ", task, "\n Result:\n", result]}, OpenAIEmbedding[taskPlusResult] -> taskPlusResult]];

   (* Update the task list *)
   autoAgent = Association[autoAgent, "taskList" -> Rest[autoAgent["taskList"]]];
   
   (* Step 4: Create new tasks *)
   newTasks = taskCreationAgent[
     autoAgent["objective"],
     result,
     task,
     autoAgent["taskList"]
   ];
   
   (* Parse the generatedTasks string into a list *)
   newTaskList = StringSplit[newTasks, "\n"];
   newTaskList = DeleteCases[newTaskList, s_String /; StringTrim[s] == ""];
   newTaskList = StringTrim[StringReplace[newTaskList, (StartOfString ~~ DigitCharacter .. ~~ ". ") -> ""]];
   
   (* Add the new tasks to autoAgent *)
   autoAgent = Fold[addTask, autoAgent, newTaskList];
   
   (* Step 5: Reprioritize tasks *)
   autoAgent["taskList"] = prioritizationAgent[
     autoAgent["taskList"],
     autoAgent["objective"]
   ];
   autoAgent
];

iterateAutoAgent[autoAgent_Association, n_Integer] := Nest[iterateAutoAgentStep, autoAgent, n];

Example 1: Generating CA examples

(* Initialize Auto Agent *)
autoAgent := <|
  "taskList" -> {},
  "objective" -> "",
  "completedTaskList" -> {}
|>;
autoAgent = setObjective[autoAgent, "Create Wolfram Language examples about Cellular Automata"];
autoAgent = addTask[autoAgent, "Make the list of tasks"];
autoAgent = addCompletedTask[autoAgent, OpenAIEmbedding["The AutoAgent has been initialized with an objective and initial task."] -> "The AutoAgent has been initialized with an objective and initial task."];

(* Number of iterations *)
n = 3;

autoAgent=iterateAutoAgent[autoAgent,n];

TabView[Drop[Values@autoAgent["completedTaskList"],1]]

Manipulate[
 ArrayPlot[CellularAutomaton[rule, {{1}, 0}, steps]],
 {rule, 0, 255, 1}, {steps, 1, 100, 1}]

ArrayPlot[CellularAutomaton[90, {{1}, 0}, 100]]

ArrayPlot[CellularAutomaton[110, {{1}, 0}, 20]]

Example 2: Generate questions and answers for a lecture on the WL symbol UnitConvert

(* Initialize Auto Agent *)
autoAgent := <|
  "taskList" -> {},
  "objective" -> "",
  "completedTaskList" -> {}
|>;
autoAgent = setObjective[autoAgent, "Generate questions and answers for a lecture on the Wolfram language symbol UnitConvert"];
autoAgent = addTask[autoAgent, "Make the list of tasks"];
autoAgent = addCompletedTask[autoAgent, OpenAIEmbedding["The AutoAgent has been initialized with an objective and initial task."] -> "The AutoAgent has been initialized with an objective and initial task."];

(* Number of iterations *)
n = 3;

autoAgent=iterateAutoAgent[autoAgent,n];

TabView[Drop[Values@autoAgent["completedTaskList"],1]]

UnitConvert[Quantity[8.2, "Miles"], "Kilometers"]

UnitConvert[Quantity[1, "Kilograms"], "AtomicMassUnit"]

Example 3: Create a lecture using Wolfram language code examples on Geometry

(* Initialize Auto Agent *)
autoAgent := <|
  "taskList" -> {},
  "objective" -> "",
  "completedTaskList" -> {}
|>;
autoAgent = setObjective[autoAgent, "Create a lecture using Wolfram language code examples on Geometry."];
autoAgent = addTask[autoAgent, "Make the list of tasks"];
autoAgent = addCompletedTask[autoAgent, OpenAIEmbedding["The AutoAgent has been initialized with an objective and initial task."] -> "The AutoAgent has been initialized with an objective and initial task."];

(* Number of iterations *)
n = 4;

autoAgent=iterateAutoAgent[autoAgent,n];

TabView[Drop[Values@autoAgent["completedTaskList"],1]]

Some examples from this auto generated lecture on Geometry

ParametricPlot[{t*Cos[t], t*Sin[t]}, {t, 0, 10*Pi}]

Graphics[SierpinskiMesh[4]]

MandelbrotSetPlot[]

Graphics3D[Torus[]]

The list of tasks can get quite long after several iterations:

autoAgent["taskList"]

Acknowledgments

Colleagues at Wolfram

WSS23 students and mentors

BabyAGI creator Yohei Nakajima. Twitter: @yoheinakajima