Abstract: R is a open source programming language for statistical computing and graphics. This language is used for Data Science and Machine learning.The Wolfram Language is a general multi-paradigm programming language used for symbolic computation, functional programming, and rule-based programming. It will always be useful to get a communication between these languages. R language evaluation results can be called from Wolfram language [
1
, 2
]. The goal of the project is to get Wolfram language evaluation result in an R session to make the both way communication possible. Wolfram Language code is sent from R to Wolfram language using ZeroMQ socket. Then Wolfram language evaluation result is sent to R using the same socket communication.The result is then used in R. At present, we can convert only few data types between Wolfram language(WL) and R, but there is scope of extending the idea discussed here.Communication protocol
Communication protocol
Sending and receiving messages between R and WL
Sending and receiving messages between R and WL
We created a WL server by opening a socket in WL using SocketOpen [
3
] functionality. A client from R side is connected to the server. Then WL code is sent from R to WL over ZeroMQ socket. We are using Socket functionality in R with the help of “rzmq” package [4
, 5
] to send the messages. WL starts listening on the opened socket and whenever data is received on the socket it evaluates the code and sends the evaluation result back to the R session. This is implemented using SocketListen functionality [6
]. Exporting evaluated result in WL
Exporting evaluated result in WL
We are sending WL code as strings from R. WL receives the data as ByteArray. We then convert the ByteArray to String and evaluate the code-string. Once the evaluation is over we export the result as a JSON string and send it to R client. Using WL function Developer`WriteRawJSONString we convert the evaluated result to a JSON String. This full process is implemented by a call-back function “processdata” in our code.
Example: Converting WL expression to JSON string:
In[]:=
Developer`WriteRawJSONString[<|"Output"->1|>,"ConversionRules"->{_Missing|None->Null},"Compact"->True]//FullForm
Out[]//FullForm=
"{\"Output\":1}"
We could use ExportString function too:
In[]:=
ExportString[<|"Output"->1|>,"RawJSON","Compact"->True]//FullForm
Out[]//FullForm=
"{\"Output\":1}"
Processing the evaluation result in R
Processing the evaluation result in R
Once the evaluated result is received in R, we can extract the result by de-serializing the content into R objects using the fromJSON() function of “rjson” package [7, 8] .
Here is an example of converting JSON-string into R objects. We used to denote R side code.
> library(rjson)
> fromJSON("{\"Output\":1}")$Output
[1] 1
> fromJSON("{\"Output\":1}")$Output
[1] 1
We can then use the result in R session
> res = fromJSON("{\"Output\":1}")$Output
> res + 2
[1] 3
> res + 2
[1] 3
Implementation details
Implementation details
WL side Code:
WL side Code:
The server is created by opening a Socket in WL
s=SocketOpen[9003,"ZMQ_PAIR"]
Out[]=
SocketObject
Processing and sending the data to R:
◼
As described earlier, the evaluation is done using “processData” call back function.
◼
processData function takes 2 arguments: dataByteArray and sourceSocket.
◼
dataByteArray is the message sent from the R session in the form of ByteArray.
◼
sourceSocket is the SocketObject where the R client is connected.
◼
It creates two variables dataString to store the string data sent from R session and jsonWrite to store the response in JSON string.
◼
Upon receiving a valid message from R, header part of the byte array is removed and stored the remaining part of the message as a String.
◼
Using WL function ToExpression on dataString evaluated result eval is obtained.
◼
Using Developer`WriteRawJSONString evaluated result is converted to JSON string.
◼
Using StringToByteArray function the JSON string is converted to ByteArray and the ByteArray is sent to R using BinaryWrite function.
processData[dataByteArray_,sourceSocket_]:=Module[{dataString,jsonWrite},If[!ByteArrayQ[dataByteArray],Print["ByteArray Error"];];Print[dataByteArray];dataString=ByteArrayToString@dataByteArray[[40;;]];Print[dataString];eval=ToExpression[dataString];Print[eval];jsonWrite=Quiet[Developer`WriteRawJSONString[<|"Output"->eval|>,"ConversionRules"->{_Missing|None->Null},"Compact"->True]];If[FailureQ[eval],jsonWrite=Developer`WriteRawJSONString[<|"Output""Evaluation Failed"|>,"ConversionRules"->{_Missing|None->Null},"Compact"->True]];If[FailureQ[jsonWrite],jsonWrite=Developer`WriteRawJSONString[<|"Output""Conversion Failed"|>,"ConversionRules"->{_Missing|None->Null},"Compact"->True]];BinaryWrite[sourceSocket,StringToByteArray[jsonWrite]]]
WL starts listening on the opened socket and whenever data is received, the data is processed:
SocketListen[s,processData[#DataByteArray,#SourceSocket]&,HandlerFunctionsKeys->{"DataByteArray","SourceSocket"}]
Out[]=
SocketListener
Note:- The WL server needs to be run first and the client side R code should be run after that to make the communication working .
R Side Code:
R Side Code:
◼
An R-script is made to implement the client side code.
◼
This script uses 2 libraries.
◼
rzmq to communicate with WL using ZeroMQ sockets.
◼
rjson to convert the JSON string to R object.
◼
The client from R is connected to the Socket opened in WL.
◼
The function wolfram_evaluate takes an input of string and sends it to WL via socket. Then WL evaluates it and sends back the output as JSON string. The wolfram_evaluate function returns the output evaluated in WL as R objects.
library(rzmq)
library(rjson)
context = init.context()
socket = init.socket(context,"ZMQ_PAIR")
connect.socket(socket,"tcp://localhost:9001")
wolfram_evaluate <- function(expression){
send.socket(socket,expression)
msg = rawToChar(receive.socket(socket, unserialize=FALSE, dont.wait=FALSE))
wlinput<- fromJSON(msg)$Output
return(wlinput)
}
library(rjson)
context = init.context()
socket = init.socket(context,"ZMQ_PAIR")
connect.socket(socket,"tcp://localhost:9001")
wolfram_evaluate <- function(expression){
send.socket(socket,expression)
msg = rawToChar(receive.socket(socket, unserialize=FALSE, dont.wait=FALSE))
wlinput<- fromJSON(msg)$Output
return(wlinput)
}
Examples
Examples
In this section, multiple examples are shown.
WL Integer to R integer:
> wolfram_evalute(“2+2”)
[1] 4
[1] 4
Screenshot from R session
WL True|False to R logical:
> wolfram_evalute(“PrimeQ[2]”)
[1] TRUE
[1] TRUE
Screenshot from R session
WL Real to R double:
> wolfram_evalute(“123.33 * 3.6”)
[1] 443.988
[1] 443.988
Screenshot from R session
WL String to R character:
> wolfram_evaluate("StringJoin[\"Hello,\",\"World!\"]")
[1] "Hello,World!"
[1] "Hello,World!"
Screenshot from R session
WL List to R vector:
> wolfram_evaluate("Range[1,10,0.25]")
[1] 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 3.25 3.50
[12] 3.75 4.00 4.25 4.50 4.75 5.00 5.25 5.50 5.75 6.00 6.25
[23] 6.50 6.75 7.00 7.25 7.50 7.75 8.00 8.25 8.50 8.75 9.00
[34] 9.25 9.50 9.75 10.00
[1] 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 3.25 3.50
[12] 3.75 4.00 4.25 4.50 4.75 5.00 5.25 5.50 5.75 6.00 6.25
[23] 6.50 6.75 7.00 7.25 7.50 7.75 8.00 8.25 8.50 8.75 9.00
[34] 9.25 9.50 9.75 10.00
Screenshot from R session
WL list to R matrix:
> wolfram_evaluate("arr={};For[i=1,i<=3,i++,For[j=1,j<=2,j++,AppendTo[arr,{i,j}]]];arr")
[[1]]
[1] 1 1
[[2]]
[1] 1 2
[[3]]
[1] 2 1
[[4]]
[1] 2 2
[[5]]
[1] 3 1
[[6]]
[1] 3 2
[[1]]
[1] 1 1
[[2]]
[1] 1 2
[[3]]
[1] 2 1
[[4]]
[1] 2 2
[[5]]
[1] 3 1
[[6]]
[1] 3 2
Screenshot from R session
WL Associations to R list
> wolfram_evaluate("<|\"a\"->2,\"b\"->3|>")
$a
[1] 2
$b
[1] 3
$a
[1] 2
$b
[1] 3
Screenshot from R session
Solve differential equations in WL and get the result in R:
> wolfram_evaluate(expression = "s1 = NDSolve[{y'[x] == y[x] Cos[x + y[x]], y[0] == 1}, y, {x, 0, 30}];Table[Evaluate[y[x] /. s1], {x, 0, 30}]")
[1] 1.00000000 0.99138734 0.52835782 0.20660353 0.08807179 0.07731209
[7] 0.16258009 0.37356039 0.35841070 0.16652814 0.06434901 0.04249864
[13] 0.07040728 0.18168240 0.26281366 0.15467941 0.05918596 0.03101026
[19] 0.03943762 0.09847241 0.19102585 0.14884701 0.06092202 0.02682244
[25] 0.02587096 0.05711399 0.13399362 0.14110671 0.06625568 0.02620211
[31] 0.01930176
> list = wolfram_evaluate(expression = "s1 = NDSolve[{y'[x] == y[x] Cos[x + y[x]], y[0] == 1}, y, {x, 0, 30}];Table[Evaluate[y[x] /. s1], {x, 0, 30}]")
[1] 1.00000000 0.99138734 0.52835782 0.20660353 0.08807179 0.07731209
[7] 0.16258009 0.37356039 0.35841070 0.16652814 0.06434901 0.04249864
[13] 0.07040728 0.18168240 0.26281366 0.15467941 0.05918596 0.03101026
[19] 0.03943762 0.09847241 0.19102585 0.14884701 0.06092202 0.02682244
[25] 0.02587096 0.05711399 0.13399362 0.14110671 0.06625568 0.02620211
[31] 0.01930176
> list = wolfram_evaluate(expression = "s1 = NDSolve[{y'[x] == y[x] Cos[x + y[x]], y[0] == 1}, y, {x, 0, 30}];Table[Evaluate[y[x] /. s1], {x, 0, 30}]")
Screenshot from R session
We can use the WL evaluated data and plot that:
> plot(list, type="l")
Screenshot from R session
Fit using FindFit in WL and get the result in R
> wolfram_evaluate(expression="data = {{1.0, 12.}, {1.9, 10.}, {2.6, 8.2}, {3.4, 6.9}, {5.0, 5.9}};model = a Exp[-k t];fit = FindFit[data, model, {a, k}, t];modelf = Function[{t}, Evaluate[model /. fit]];modelf[3]")
[1] 7.939335
[1] 7.939335
Screenshot from R session
Data Types Conversion
Data Types Conversion
We showed in the above section that these data conversions are possible.
Further Scope
Further Scope
There are many scope of improvement. Here are some of the things that can be improved.
◼
Exporting Complex numbers as JSON string is not possible now.
◼
Bigger data size is breaking the communication protocol but that can be improved using multiple Socket messages.
◼
WL Plot, Image conversion is not possible now.
◼
It should be possible to open a WL server from R.
Concluding remarks
Concluding remarks
This project enables R programmers to directly evaluate WL code and get the output in their R session. I have checked data conversion is possible for Integer, Real, String, List, Association. We also checked functions like NDSolve and FindFit. Currently, conversion of complex numbers, Images and Plots from WL to R is not possible. I want to add these data type support in the future. Currently we need to run the client side script in a R session but we want to make this as an R-Package so that it can be imported easily.
Keywords
Keywords
◼
R calling Mathematica
◼
R Calling Wolfram language
◼
ExternalEvaluate R
◼
Multi-language communication
Acknowledgment
Acknowledgment
I would like to express my special thanks of gratitude to my mentor Suman Banerjee, who gave me this wonderful project and also provided support in completing my project. I would like to thank M. Bahrami, Tuseeta Banerjee for their support all through the school and all the associates who helped me complete this School. Suman wants to thank Todd Gayley at Wolfram Research for sharing his knowledge about communication protocols. Many of the ideas discussed with Todd is used in this project. I would like to express my special thank to Stephen Wolfram for the idea of the Wolfram school. I want to thank my friend Yeswanth Sai Yadla for his help in graphics.
References
References