Calling Wolfram Language from Java, JavaScript, Python, Bash, and back forth

Daniel Carvalho
dcarvalho@wolfram.com
International Business Development Executive
Wolfram Research

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

A key feature for current technologies that are making the 4th industrial revolution happen, is the capability to talk and communicate dynamically around different programming languages. Let’s see examples of how to call and be called from Wolfram Language with other popular programming languages, and easily integrate and have the advantage of Wolfram technology with its symbolic, algorithms, math, curated data, graphics, IA, and ML capabilities to any existing legacy tech, to create innovative resources and automation.

({{, }})

Popular Programming Languages

Tiobe index keeps this programming languages list based in a interesting data science workflow combining information from different sources

SQL - Will be ever popular, because is the language to talk to relational databases (back-end) such as MySQL, SQLite, Oracle, MS SQL Server, etc

JavaScript - Will be ever popular, because it is the only Browser language, all the WEB front-end runs just JavaScript

C/C++ - Will be ever popular, because it is used in embedded systems, operational systems, system kernels, games, and so on

Java and Python - They are largely used in enterprise application, large scale and production environment. And both used in Data Science and many big data applications

Visual Basic - As it is available as VBA in MS Office, they are very common tools in business automation

Python, JavaScript (node) and Java processing float point values:

$ python
Python 3.8.5 (default, Sep  4 2020, 07:30:14) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type “help”, “copyright”, “credits” or “license” for more information.
>>> .1 + .2
0.30000000000000004
>>> exit()
$ node
Welcome to Node.js v12.18.2.
Type “.help” for more information.
> .1 + .2
0.30000000000000004
> 
(To exit, press ^C again or ^D or type .exit)
> 
$ jshell
|  Welcome to JShell -- Version 14
|  For an introduction type: /help intro

jshell> .1 + .2
$1 ==> 0.30000000000000004

jshell> /exit
|  Goodbye
$

.1 + .2

https://www.tiobe.com/tiobe-index/
https://floating-point-gui.de/

Wolfram Language

Wolfram Language a higher level language, made for humans and productivity, for business and science.

Wolfram Language is very friendly to talk to other technologies, and run in different environments such as:

Linux (Mathematica, WolframScript, Wolfram Engine for Developers)

Mac (Mathematica, WolframScript, Wolfram Engine for Developers)

Windows (Mathematica, WolframScript, Wolfram Engine for Developers)

RaspberryPi (IoT and electronics Projects) - Linux

Android (Apps: Wolfram Player)

iOS (Apps: Wolfram Cloud, Wolfram Player)

WEB (Wolfram Cloud, WEB APIs, Wolfram|Alpha)

Wolfram tech compute on the desktops, mobile, servers or cloud

https://www.wolfram.com/language/

Languages supported by Mathematica Notebook Interface (UI) in cells computation

Wolfram Language calling X language from Mathematica notebook UI

Wolfram Language examples calling:

Bash (OS command line)

Python

Java

JavaScript (NodeJS)

SQL

Lua

Wolfram Language (Mathematica) calling Bash (Shell)

Bash came from UNIX Bourne shell and now are available at Linux, Mac and Windows too

At Wolfram notebook, start a new cell with “>” character and select the external language. The external language should be appropriately available in your OS, for Bash use Shell

date

uname -a

cal

pwd

awk "BEGIN {print (20+5)/2}"

Directory[]

?*Dir*

NotebookDirectory[]

Some very common utilities in Data Science and WEB development are wget and curl:

sudo apt install wget
sudo apt install curl

Working with external tools it is important to set the current directory and create a folder for each project, tools like Python, NodeJS, SQLite, Git are tied to hidden folders and additional configuration files...

SetDirectory[NotebookDirectory[]]

wget to download files at command line...

wget https://datahub.io/core/finance-vix/r/vix-daily.csv

vix=SemanticImport["vix-daily.csv"]

wget --version

ls -l *.csv

curl to call WEB APIs at command line...

curl https://api.coinbase.com/v2/prices/spot?currency=USD

Head[%]

FullForm[%5]

%5["StandardOutput"]

Wolfram Language (Mathematica) calling Python

It is possible to run code in Python inside a Mathematica cells
Of course Python should be appropriately installed and available in your environment (OS)
It is useful to call your own libraries and code already build for your work or research, instead of recreate in Wolfram Language, you can just call it.

[x*2 for x in range(-10,10)]

BarChart[%,ColorFunction->Hue]

[[x*y for x in range(-10,10)] for y in range(-10,10)]

ArrayPlot[%,ColorFunction->Hue]

import requests

response = requests.get("https://api.github.com/users/danielscarvalho")
response.json()

Dataset[%]

The same job with Wolfram Language

Dataset[Association[Import["https://api.github.com/users/danielscarvalho"]]]

FindExternalEvaluators["Python"]

34/2

Wolfram Language (Mathematica) calling Java

new java.util.Date().toString()

System.getProperty("java.version");

Runtime.version().toString();

System.getenv("JAVA_HOME")

System.getenv("PATH")

Simple JavaBean source code, with two values id (int) and val (float):

class Info {

	private int id = 0;
	private float val = 0.0f;
	
	public Info() {
	}
	
	public Info(int id, float val) {
		this.setId(id);
		this.setVal(val);
	}
	
	public void setVal(float v) {
		this.val = v;
	}
	
	public float getVal() {
		return this.val;
	}
	
	public void setId(int id) {
		this.id = id;
	}
	
	public int getId() {
		return this.id;
	}
	
	public String toString() {
		return "[id = " + this.id +
		       ", val = " + this.val + "]\n";
	}

} //Info

Test the JavaBean:

Info i = new Info();

i.setId(10);
i.setVal(120.33f);
i.getVal();

i.toString()

Lets create a collection with the JavaBean:

import java.util.HashMap;

HashMap hm = new HashMap<Integer, Info>();

import java.util.Random;

Random r = new Random();

for(int i = 0; i < 10; i++)
   hm.put(i, new Info(i, (float)1000*r.nextFloat()));

hm.get(2).toString()

Java calling Wolfram Language: J/Link

All secrets of the mysterious J/Link can be clearly deciphered right next to you, in your Mathematica installation folder, such as:

/opt/Wolfram/Mathematica/12.3/SystemFiles/Links/JLink

You can find there documentation and sample code, hints about how to compile and run.
There are many different ways to call Wolfram Language from Java with J/Link. The one I most like is this bellow, because it is analogous to SQL, we can just embed Wolfram Language code as a text source code inside a Java program as string, and get a string result back:

import com.wolfram.jlink.*;

public class CallWl {

	public static void main(String[] argv) {

		KernelLink ml = null;

		try {

			ml = MathLinkFactory.createKernelLink(argv);

		} catch (MathLinkException e) {
			System.out.println(“Fatal error opening link: “ + e.getMessage());
			return;
		}


		try {
		
			// Get clear
     		ml.discardAnswer();

			// If you want the result back as a string, use evaluateToInputForm or
			// evaluateToOutputForm. The second arg for either is the requested page
			// width for formatting the string. Pass 0 for PageWidth->Infinity.
			// These methods get the result in one step--no need to call waitForAnswer.
			
			String wlcode = “NIntegrate[x^2+Sinc[x],{x,-10,10}]”; //The magic goes here!
			String strResult = ml.evaluateToOutputForm(wlcode, 0);

			System.out.println(strResult);

		} catch (MathLinkException e) {
			System.out.println(“MathLinkException occurred: “ + e.getMessage());
		} finally {
			ml.close();
		}
	}
}

The result is back in a string: “669.983”

To compile at Linux command line:

javac -classpath JLink.jar CallWl.java

To run at Linux command line:

java -classpath JLink.jar CallWl -linkmode launch -linkname ‘math -mathlink’

Of course we need JLink.jar lib at CLASSPATH, as usual in Java.

It is possible to call a remote kernel in the network too, just change parameters appropriately. Check firewall permissions.

J/Link is there for a while, to learn more options about calling Wolfram Language from Java with J/Link, please take a look at the documentation:
https://reference.wolfram.com/language/JLink/tutorial/Overview.html

Wolfram Language calling JavaScript (NodeJS)

It is appropriate to set the current directory as the notebook directory, those external languages are tied to director structures.

SetDirectory[NotebookDirectory[]]

RegisterExternalEvaluator["NodeJS","/opt/node-v12.18.2-linux-x64/bin/node"]

FindExternalEvaluators["NodeJS"]

ExternalEvaluate["NodeJS","345/2"]

234/444-4**10

Math.PI

randomArray = (length, max) => [...new Array(length)]
    .map(() => Math.round(Math.random() * max));
    
randomArray(100,100)

BarChart[%,ColorFunction->Hue]

jsresult = new Array(100)
jsresult = jsresult.fill(0).map(() => Math.random());

ListPlot[%,ColorFunction->Hue,Filling-> Axis]

Lets try some JavaScript code:

jsresult.length

function add(accumulator, a) {
    return accumulator + a;
}

jsresult.reduce(add)

jsresult.reduce(function(a, b) { return a + b; }, 0);

jsresult.filter(value => value > .9);

jsresult.map(value => value*100).filter(value => value > 90)

JavaScript (NodeJS) calling Wolfram Language

WolframScript makes really easy any programming language call Wolfram Language code.
This is an example with JavaScript:

const { exec } = require(“child_process”);

let wlcode = “wolframscript -code NIntegrate[x^2*Sinc[x],{x,-100,100}]”; //Magic!!!

exec(wlcode, (error, stdout, stderr) => {
    if (error) {
        console.log(`error: ${error.message}`);
        return;
    }
    if (stderr) {
        console.log(`stderr: ${stderr}`);
        return;
    }
    console.log(`stdout: ${stdout}`);
});

Output: stdout: -173.4765057397561

SQL - Enterprise data stand at the relational database

In times of big and small data it is very important to call clearly information from the massive enterprise databases with SQL, we can do it from Mathematica cells easily.
Companies have terabytes of data in relational databases for CRM, ERP, Data Warehouse and Data Marts, and other enterprise systems...
Even non structured text data files in Apache Hadoop can be seen as SQL with Hive database

Here we gonna see:

SQLite

MySQL

SQL - SQLite

SQLite is a very popular single user database, used in mobile apps and for local data science projects too.

List files in the current folder:

FileNames[]

Connect to SQLite database file:

db = DatabaseReference[FindFile["chinook.db"]]

Check the database connection:

ExternalEvaluate[db,
"SELECT 1+2 CALC, date('now') NOW, sqlite_version() VERSION"]

The result from SQL queries are datasets (data frame):

Head[%]

List all tables:

ExternalEvaluate[db,"SELECT name FROM sqlite_master WHERE name NOT LIKE 'sqlite_%'"]

ExternalEvaluate[db,
"select * from playlists"]

View the first 10 lines of tracks table:

ExternalEvaluate[db,
"select * from tracks limit 10"]

SQL - MySQL

MySQL is the most popular free relational database, very popular in WEB applications with the LAMP stack. MariaDB is a fork from MySQL they are compatible.

Define connection parameters, use strong password, do not save the pwd in your notebook! ;-)

mysql = DatabaseReference[
<|"Backend"->"mysql",
"Name"->"datascience",
"Port"->Automatic,(* 3306 *)
"Host"->"localhost",
"Username"->"datascience",
"Password"->"Y5jZuxQUS8FQY5nc"|>]; 
(* OK! I will change the pwd!! :-) *)

Connect to the database:

dbmysql = DatabaseReference[mysql]

Query that didn’t need a table, just to test the connection:

ExternalEvaluate[dbmysql,
"SELECT now() NOW, version() VERSION, 12+100 CALC"]

Usually to inspect a table, try using limit to get just some lines:

ExternalEvaluate[dbmysql,
"SELECT * from contacts limit 10"]

Check the total number of lines in the table:

ExternalEvaluate[dbmysql,
"SELECT count(1) count from contacts"]

Get user info from NY, notes the backslash at the query to escape the double quote:

ExternalEvaluate[dbmysql,
"select first_name, last_name, email from contacts where state=\"NY\""]

List NY addresses:

ExternalEvaluate[dbmysql,
"select address, zip from contacts where state=\"NY\""]

Check the number of addresses by state:

ExternalEvaluate[dbmysql,
"select count(1) qtt, city, state from contacts group by city, state"]

%[All, Interpreter["City"][#city]-> #qtt &]

DeleteCases[Normal[%79],Rule[Failure[__], _]]

GeoRegionValuePlot[%,ImageSize->Large]

We can keep the Big Data at the server side with a relational database, companies have terabytes of data in their internal systems. We can combine this data with Wolfram curated data, and bring to Mathematica at the client side just the important data, exploring the small data, where the value really is.

Additional references:
https://www.sqlite.org/
https://www.sqlitetutorial.net/sqlite-sample-database/
https://www.mysql.com/

Gartner Says 70% of Organizations Will Shift Their Focus From Big to Small and Wide Data By 2025
https://www.gartner.com/en/newsroom/press-releases/2021-05-19-gartner-says-70-percent-of-organizations-will-shift-their-focus-from-big-to-small-and-wide-data-by-2025

Python at Jupyter Notebook calling Wolfram Language (Option 1)

Example:

Another example:

https://github.com/WolframResearch/WolframLanguageForJupyter

https://reference.wolfram.com/language/WolframClientForPython/

https://twitter.com/danielscarvalho/status/1222581978953912321

WolframLanguage from Jupyter Notebook (Option 2)

Wolfram Notebook (UI) is an original environment for science and exploratory computation
Now inspired by Wolfram Notebook, recently Jupyter Notebook are getting very popular for Data Science projects

You can call Wolfram Language from Jupyter Notebook cells too, without using Python or other programming language:

...then select Wolfram Language from the Jupyter Notebook drop down menu:

Outputs are either strings, for simple textual results, or images, for graphics and typeset results:

For more details:
https://github.com/WolframResearch/WolframLanguageForJupyter

Mathematica, WolframScript and Wolfram Engine for Developers at Bash

You can call wolframscript from command line from Linux, Mac or Linux

$ wolframscript -code WolframAlpha[\”What\ is\ the\ meaning\ of\ life\”,\”SpokenResult\”]
42

Notes that it is required to escape double quotes and space with back slash

Another option is to embrace the Wolfram Language code with single quote as fallows

$ wolframscript - code' WolframAlpha["What is the meaning of life", "SpokenResult"]'
42

WolframScript make it easy to call Wolfram Language code as a batch text file.

At this example, we cam make Wolfram|Alpha queries at the command line:

#!/usr/bin/env wolframscript
Print[“Query: “ <> $ScriptCommandLine[[2]]]
Print[“Response: “ <> WolframAlpha[$ScriptCommandLine[[2]],”SpokenResult”]]

Usage:

$ waquery.wls “What is Sonia Braga age?”
Query: What is Sonia Braga age?
Response: Sonia Braga was born 71 years ago, on Thursday, June 8, 1950

$ waquery . wls "Distance from NYC to Paris"
Query : Distance from NYC to Paris
Response : The distance from the center of New York City, United States to the center of Paris is about 5851 kilometers

https://www.wolfram.com/wolframscript/

Calling Wolfram Language from Lua language

Lua is a lightweight script language, powerful and popular in AI and games. Made by PUC-RJ in Brazil to attend Petrobras (Oil company) projects, it gets a global usage. Lua can be executed interactively in the command line or as all script languages in a batch text file.

As Lua language is able to call OS commands by os.execute command, and then, you can call wolframscript easily, even when there is not a official package or integration library. It is required to have Mathematica, wolframscript or Wolfram Engine for Developers installed in your system alongside Lua.

$ lua
Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio
> os.execute “wolframscript -code NIntegrate[x^2+3x+2*Sinc[x],{x,0,100}]”
348336.07709019806
true	exit	0
> os.execute “wolframscript -code N[Sum[x^2+3x+2*Sinc[x],{x,0,100}]]”
353504.1208578768
true	exit	0
> os.execute ‘wolframscript -code WolframAlpha[\\”What\\ is\\ Sonia\\ Braga\\ age?\\”,\\”SpokenResult\\”]’
Sonia Braga was born 71 years ago, on Thursday, June 8, 1950
true	exit	0
os.execute ‘wolframscript -code WolframAlpha[\\”distance\\ from\\ NYC\\ to\\ Paris\\”,\\”SpokenResult\\”]’
The distance from the center of New York City, United States to the center of Paris is about 5851 kilometers
true	exit	0
> os.execute [==[wolframscript -code ‘WolframAlpha[“What is the boiling point of water at mount Everest?”,”SpokenResult”]’]==]
The answer is about 70.4 degrees Celsius
true	exit	0
>

https://www.lua.org/
https://simion.com/info/calling_external_programs.html
https://www.wolfram.com/wolframscript/

Other languages to integrate

Wolfram technology can integrate well with other platforms

R

Ruby

Julia

Octave

C/C++

Databases (SQL and NoSQL too)

WEB API (Wolfram Cloud)

Bonus: Wolfram on Spreadsheets

Spreadsheets are lingua franca in business!
And we can communicate very well with Wolfram Language!

Call Wolfram Data from Excel (Option 1)

You can call Wolfram data from Excel
Cloud connector for Excel

https://www.wolfram.com/microsoft-integration/excel/

https://www.wolfram.com/cloud-connector-for-excel/ways-to-use/index.php.pt-br?source=footer

Calling Wolfram Language functions from Excel

You can call Wolfram Language from an Excel spreadsheets too
Cloud connector for Excel

({{, }})

Wolfram and Microsoft Excel Data Integration (Option 2)

Wolfram Data Intelligence: New in Microsoft

Microsoft and Wolfram partnership to intelligently add meaning to data. Identify and auto-fill thousands of data points from hundreds of data types directly in Microsoft Excel. Expertly curated data provides instant interactivity and immediate answers.

https://www.wolfram.com/cloud-connector-for-excel/
https://www.wolfram.com/microsoft-integration/excel/
https://templates.office.com/en-us/collection-data-types-in-excel-templates

Wolfram Language talks very well with “X” programming language

To learn more: https://www.wolfram.com/wolfram-u/
How do you integrate your projects and research??
Wolfram experienced consulting team can help to integrate systems! 

How can we help??

Thanks for attending this talk!!