This project explores the integration of Large Language Models (LLMs) with the Wolfram Language stack to build semi-autonomous agents capable of performing diverse data science tasks. Leveraging Wolfram Language’s well-curated and computable knowledge base and functional programming approach, this project demonstrates the use of LLMFunction and LLMTool as potential building blocks of agent like architectures. This work presents practical examples and future explorations towards developing autonomous workflow agents that generate Wolfram Language code for AI and data science applications. This could be a great tool to increase access to AI in the scientific community by simplifying data preparation and computability.
"Composable Intelligence for Science"-GPT4o
AI for Science - Data challenges and LLM Agents
AI for Science - Data challenges and LLM Agents
The widespread adoption of AI in scientific research is significantly hampered by data challenges, including the preparation, transformation and integration of diverse datasets to computation . Large Language Models (LLMs) offer a promising solution by enabling the creation of agents that can autonomously handle these tasks, making data more accessible and usable for AI applications. Wolfram Language stands out as a great tool for this purpose due to its extensive, codified knowledge-base that spans multiple scientific disciplines, and its functional approach to programming. The LLMFunction object in Wolfram Language allows for the composition of sophisticated, autonomous workflows that leverage LLM capabilities to make data more computable.
In this notebook, we will explore the potential of LLMFunctions as symbolic objects that can be used to build agentic workflows, similar to what is seen in frameworks like Langchain,OpenAI functions etc. We will demonstrate how these functions can be composed and utilized to perform a variety of data science tasks, as illustrated in the agent architecture in Section 2.Through a series of examples, we will highlight the versatility and power of Wolfram Language in integrating LLM technology to streamline and enhance scientific workflows, ultimately aiming to broaden access across the sciences by making data more readily computable and actionable.
In this notebook, we will explore the potential of LLMFunctions as symbolic objects that can be used to build agentic workflows, similar to what is seen in frameworks like Langchain,OpenAI functions etc. We will demonstrate how these functions can be composed and utilized to perform a variety of data science tasks, as illustrated in the agent architecture in Section 2.Through a series of examples, we will highlight the versatility and power of Wolfram Language in integrating LLM technology to streamline and enhance scientific workflows, ultimately aiming to broaden access across the sciences by making data more readily computable and actionable.
Wolfram Language’s data ecosystem
Wolfram Language’s data ecosystem
Wolfram Language is equipped with a comprehensive and computable data ecosystem , making it an effective compositional environment for creating LLM-based agents capable of performing date transformation tasks across domains. The examples below illustrate how Wolfram Language can handle data in diverse fields, leveraging its extensive functions repository,data repository and entity stores construct.
Example 1: Biology - Analyzing Genomic Data
Wolfram Language provides powerful tools for genomic analysis, enabling users to access and analyze genetic data with ease. For instance, we can retrieve and examine the full sequence of the SCNN1A gene, which plays a critical role in sodium transport in epithelial cells.
Example 1: Biology - Analyzing Genomic Data
Wolfram Language provides powerful tools for genomic analysis, enabling users to access and analyze genetic data with ease. For instance, we can retrieve and examine the full sequence of the SCNN1A gene, which plays a critical role in sodium transport in epithelial cells.
GenomeData["SCNN1A","FullSequence"]//Short
Out[]//Short=
CCGGCCAGCGGGCGGGCTCCCCAGCCAGGCCGCTGCACCTGTCAGGT…CTGCTTGGAGCAAGTAGGCAGGAGCTCAATAAATGTTTGTTGCATGAA
StringLength[%]
Out[]=
28703
Example 2: Chemistry - Molecular Structure and Properties
In the field of chemistry, Wolfram Language offers detailed information about molecular structures and properties. By using the Entity function, we can explore the molecular structure of caffeine and visualize it.
In the field of chemistry, Wolfram Language offers detailed information about molecular structures and properties. By using the Entity function, we can explore the molecular structure of caffeine and visualize it.
caffeine=Entity["Chemical","Caffeine"];caffeineStructure=MoleculePlot[caffeine]
Out[]=
Example 3: Econometrics - Country Data Analysis
Wolfram Language’s entity framework allows for the analysis of comprehensive country-specific data. By querying the Entity function for France, we can obtain its population, meat production, and natural resources data.
Wolfram Language’s entity framework allows for the analysis of comprehensive country-specific data. By querying the Entity function for France, we can obtain its population, meat production, and natural resources data.
Out[]=
Out[]=
Out[]=
,,,,,,,,,,,,,,,,,
These are some very basic examples of data objects and computational approaches available in the Wolfram language. For a more holistic overview of the
Example 4: Training neural networks
In addition to the above modalities for data storage and handling, Wolfram language also provides very easy and intuitive ways of building your own AI models or retrieving models from elsewhere, including the wolfram neural net repository.
In addition to the above modalities for data storage and handling, Wolfram language also provides very easy and intuitive ways of building your own AI models or retrieving models from elsewhere, including the wolfram neural net repository.
net=NetChain[{LinearLayer[],LogisticSigmoid}];trained=NetTrain[net,{1->False,2->False,3->True,4->True}]
Out[]=
net=NetModel["LeNet Trained on MNIST Data"]
Out[]=
These are some very basic examples of data objects and computational approaches available in the Wolfram language. For a more holistic overview of the data and computational capabilities of the wolfram stack, I would highly recommend reading through Stephen Wolfram’s announcement note on the Wolfram Data Repository, which provides a very systematic way of understanding how WL structures its data and computational layers.
Wolfram Language functions for building data-science agents
Wolfram Language functions for building data-science agents
Functions as building blocks
Functions as building blocks
The functional programming paradigm offers a powerful and elegant approach to building LLM agents. Its emphasis on composability, immutability, and expressive abstractions aligns well with the needs of scientific AI systems.
Agent building involves combining LLMs with various tools, memory systems, and human input in a repeatable and recursive manner. Wolfram Language offers a modular and composable approach to achieving this through its LLMFunction and other related constructs. Using these functions, LLMs can be packaged with any kind of data, context, and tools to perform different data science tasks, effectively becoming symbolic objects that encapsulate LLM reasoning. This is a templatized view of these objects :
Agent building involves combining LLMs with various tools, memory systems, and human input in a repeatable and recursive manner. Wolfram Language offers a modular and composable approach to achieving this through its LLMFunction and other related constructs. Using these functions, LLMs can be packaged with any kind of data, context, and tools to perform different data science tasks, effectively becoming symbolic objects that encapsulate LLM reasoning. This is a templatized view of these objects :
◼
Code
Combining functions to build agents
Combining functions to build agents
Combining these symbolic objects, a representational state and a UI for human input can lead to the creation of semi-autonomous task based and domain specialized agents.In the below figure one such architecture is depicted, borrowing from some best practices from SWE Agent design patterns. This is a very active field of research currently, and new architectures are being explored at a rapid pace.
Canonical Functions
Canonical Functions
Functions provide a modular way of thinking about and creating an ‘Action Space’ for these kinds of semi-autonomous agent. That way a more granular level of control, guidance and explainability can be implemented over LLM driven actions. Below are some canonical examples explored in the short duration of this project
The agent leverages the LLMFunction construct to handle specific tasks, integrating data, human input, and tools in a streamlined workflow. For example, a planning prompt function can be represented as:
This function generates the next task based on the current state, data, and human input. Similarly, a code generation prompt can be formulated as:
This function generates Wolfram Language code for the specified task. And finally a function with the wolfram function as a tool can be used to update both the data and the internal state of the agent :
The combination of these functions consolidates planning, code generation, and tool usage into a cohesive agent capable of performing complex data science tasks with Human in the loop controls and guidance.
The importance of “Expert Knowledge”
The importance of “Expert Knowledge”
While LLM constructs are powerful tools for building automated workflows, their true effectiveness is significantly enhanced when combined with expert knowledge. This expertise can be incorporated through carefully crafted prompts or integrated knowledge bases, via Retrieval-Augmented Generation (RAG). However, a considerable amount of work remains in distilling and integrating domain-specific knowledge from various scientific fields into these agents. A good model of codifying this kind of knowledge is seen in the LLM prompt repository in Wolfram Language.
Demonstrations : Wolfram language functions in action
Demonstrations : Wolfram language functions in action
In this section, we will demonstrate the Wolfram Language functions in action using simple toy problems. The problems chosen are straightforward and serve to illustrate the functionality of the LLMFunctions and other related constructs. Each demonstration is a representation of one of the canonical functions presented in the agent architecture above. There are many more similar functions that can be constructed in a modular and repeatable way. For those interested in seeing a fully integrated demo, I recommend checking out this Multi-agent implementation for education by Jofre Espigule-Pons.
The “State” of the agent
The “State” of the agent
Wolfram language has some powerful frameworks for creating data stores that can be leveraged for maintaining and updating important context related information. These stores can serve as internal space representations for the Agent, proving level 2 cognitive guidance for discrete tasks. For eg. here is an example of space representation as a simple table:
And here’s what it might look like after a few iterations of the agent loop :
Function 1 : Planning with LLMs
Function 1 : Planning with LLMs
One of the key things that is of interest to distil from LLM’s is their planning and reasoning ability. LLMFunctions provide an easy wrapper for passing different kinds of planning tasks to LLMs and getting the output back in customizable and computable ways :
Ok, that was a very simple example. But the real power of these functions becomes evident as you start playing around with the different parameters :
That’s just a minor formatting change, but it shows the different parameters that are available and which can be used to control the behaviour of the LLMs in very precise ways.
Function 2 : Coding with LLMs
Function 2 : Coding with LLMs
The other functionality of LLMs which could be very useful in data science contexts is the ability to generate code. As mentioned before, Wolfram language itself has some really powerful data science functions and capabilities and most LLM’s are already pretrained on Wolfram documentation and can generate working code for prevalent data-science tasks :
The great thing about doing all of this natively in a wolfram language notebook is that we can quickly convert that code into an actual working function that can be applied locally to our dataset (avoiding the need to send large context bundles to the API) :
Now, let’s create a simple function that converts the LLM-returned code into an expression that can be evaluated by the Wolfram Kernel :
And finally we apply the created function to our dataset.
Function 3 : Providing tools to LLMs
Function 3 : Providing tools to LLMs
So far, we have mainly been using the built-in capabilities of LLMs to generate the responses that are of interest to us. But many times we come across situations, where we are interested in making sure that the LLM uses a specific tool to generate a result. Wolfram Language provides a native way of defining these tools and passing them to the LLM :
In the above example, we asked the LLM to use the Tool X, and that means running the anonymous function (represented by #&). But, wolfram language has some in-built LLM Tools that can also be used in these LLM calls.
For eg. suppose you want to ask the LLM to look up some wolfram language documentation for a specific task, wolfram has a pre-built tool for that called “DocumentationLookup”
For eg. suppose you want to ask the LLM to look up some wolfram language documentation for a specific task, wolfram has a pre-built tool for that called “DocumentationLookup”
These are representative parts of a workflow that can be stitched together using the functional compositional approach of wolfram language, which lends itself well to building discrete yet computationally connected autonomous units.
Further reading and resources
Further reading and resources
To further explore the capabilities and applications of Wolfram Language in building data science agents, several resources can help you get started and deepen your understanding.
First, delve into the LLMFunction documentation, which provides comprehensive details on how to create and utilize LLMFunctions within the Wolfram Language. This function allows for the integration of LLM technology into various workflows, making it a good tool for constructing agent architectures.
For those interested in leveraging tools alongside LLMs, the LLMTool documentation is also a good resource. It explains how to use LLMTool objects to enhance the functionality of your agents, enabling them to perform a wide range of tasks with more precision and predictability.
To understand the broader context and potential of these technologies, Stephen Wolfram’s writings are invaluable. His article, "The New World of LLM Functions: Integrating LLM Technology into the Wolfram Language", offers insights into the integration of LLM capabilities within the Wolfram ecosystem and highlights various practical applications .
Additionally, the Wolfram Prompt Repository provides a collection of pre-written prompts that can be used to guide LLM-based workflows. These prompts can be customized and integrated into your projects, streamlining the process of building complex agents.
First, delve into the LLMFunction documentation, which provides comprehensive details on how to create and utilize LLMFunctions within the Wolfram Language. This function allows for the integration of LLM technology into various workflows, making it a good tool for constructing agent architectures.
For those interested in leveraging tools alongside LLMs, the LLMTool documentation is also a good resource. It explains how to use LLMTool objects to enhance the functionality of your agents, enabling them to perform a wide range of tasks with more precision and predictability.
To understand the broader context and potential of these technologies, Stephen Wolfram’s writings are invaluable. His article, "The New World of LLM Functions: Integrating LLM Technology into the Wolfram Language", offers insights into the integration of LLM capabilities within the Wolfram ecosystem and highlights various practical applications .
Additionally, the Wolfram Prompt Repository provides a collection of pre-written prompts that can be used to guide LLM-based workflows. These prompts can be customized and integrated into your projects, streamlining the process of building complex agents.
Concluding Remarks
Concluding Remarks
What I have presented in this notebook is just a rough exploration of what’s possible within the confines of the notebook and using high-level functions. The language also has good connectors to other services as well as ways to design integrations in more sophisticated ways. These connected capabilities require a bit of engineering to operationalize . LLMs present a novel modality for engaging with this information ecosystem, and constructs like LLMFunctions are making it more accessible to newcomers like me. These tools bridge the gap between the symbolic world of Wolfram Language and the statistical AI capabilities of LLMs, creating a uniquely powerful combination. My plan is to continue exploring these tools to tackle more complex data science challenges and build sophisticated agents capable of solving real-world scientific problems. Stay tuned for more developments, as I share my progress and learnings in future posts.
Acknowledgments
Acknowledgments
This project and the entailed explorations would not have been possible without the support of the Wolfram summer school, and my amazing mentors - Christopher Wolfram and John McNally. I would also like to express my gratitude to Jofre Espigule-Pons, Christian Pasquel and Phileas Dazeley-Gaist for their invaluable help and support throughout this project. Thank you for getting me started on this journey.
References
References
1
.BabyAGI à la Wolfram: a task-driven autonomous agent using Wolfram Language
by Jofre Espigule-Pons
Wolfram Community, STAFF PICKS, March 5, 2024
https://community.wolfram.com/groups/-/m/t/3135678
by Jofre Espigule-Pons
Wolfram Community, STAFF PICKS, March 5, 2024
https://community.wolfram.com/groups/-/m/t/3135678
2
.Stephen Wolfram (2023), “The New World of LLM Functions: Integrating LLM Technology into the Wolfram Language,” Stephen Wolfram Writings. writings.stephenwolfram.com/2023/05/the-new-world-of-llm-functions-integrating-llm-technology-into-the-wolfram-language.
Cite This Notebook
Cite This Notebook