New in 11.2: 
ExternalEvaluate

Wolfram Technology Conference 2016
Ian Johnson, Wolfram Research

Overview

Runs code in other languages through REPL

Return result back as WL expr

Currently Python + NodeJS, more coming soon

Basics

Quit

ExternalEvaluate;ExternalEvaluate`Private`resetCache[];

Evaluations

Simple string specifier

ExternalEvaluate["Python","2+2"]

ExternalEvaluate["NodeJS","2+2"]

New session each evaluation (no persistent state)

Table[ExternalEvaluate["Python","import os; os.getpid()"],4]

$ProcessID

By default, attempts to turn the result into an expression:

ExternalEvaluate["Python","2"]//IntegerQ

ExternalEvaluate["Python","import os; os"]

Can use “ReturnType”->”String” for these cases

ExternalEvaluate["Python"->"String","import os; os"]

ExternalEvaluate["Python"->"String","2"]//IntegerQ

Sessions

StartExternalSession gives you a persistent session

session=StartExternalSession["Python"];

ExternalEvaluate[session,"x=4"]

ExternalEvaluate[session,"x"]

With sessions can also use “_” for previous result

ExternalEvaluate[session,"_"]

Get all running sessions with ExternalSessions

Shallow[ExternalSessions[],3]

Stop sessions with DeleteObject

DeleteObject[ExternalSessions[]]

ExternalSessions[]

Objects

Can evaluate File, URL, CloudObject, LocalObject

Useful for supporting packages, etc.

script=FileNameJoin[{$TemporaryDirectory,"functions.py"}];
Export[script,"def stringReverse(s):
	return s[::-1]

def stringAllCaps(s):
	return s.upper()
","Text"];
session=StartExternalSession["Python"];
ExternalEvaluate[session,File[script]];
ExternalEvaluate[session,"stringReverse('hello world')"]
ExternalEvaluate[session,"stringAllCaps('hello world')"]

Supported Systems

Evaluator system classes

Python

NodeJS

User extensible - look at ExternalEvaluate`AddHeuristics and ExternalEvaluate`Private`$LanguageHeuristics- (but note that this will definitely change between 11.2 and 11.3)

?ExternalEvaluate`AddHeuristics

Evaluator system instances - i.e. python2 vs python3

Add with RegisterExternalEvaluator, remove with UnregisterExternalEvaluator

Find automatically / list evaluators with FindExternalEvaluators

FindExternalEvaluators[]

Registered evaluators are persistent between kernel sessions

$UserBaseDirectory

PersistentValue["ExternalEvaluate`EvaluatorCache","Local"]//Short

Under the hood

Basic overview

Confirm that evaluator is usable

Start evaluator running custom “repl” script

Script can be found at :

PacletResource["ExternalEvaluate",#]&/@{"PythonREPL","NodeJSREPL"}

SystemOpen/@%

Evaluator prints off a host + port the script is listening on commands for with a ZMQ Pair socket to stdout

Wolfram Language reads that in and connects to the host + port with a ZMQ Pair socket using SocketConnect

Wolfram Language sends input to the socket using JSON encoding of <|”input”->”input string”|>

Evaluator then evaluates the command and sends back the result - catching exceptions, errors, etc. and sending that back as “error” keys in JSON

Wolfram Language interprets the result and returns that to user