With the rapid expansion of scientific literature, it is becoming increasingly difficult for researchers to identify the right computational tools for their work. The AI Aided Wolfram Tech Recommender for Scientific Articles is a system that intelligently connects research papers with relevant Wolfram Language functions and technologies. Using large language models and carefully designed prompts, the system analyzes article content and generates a tailored list of Wolfram functions that match the research methods and analyses described. It then creates an interactive Wolfram Notebook report that not only recommends functions but also explains how each one can be applied to specific parts of the research. This context-aware approach enhances understanding, speeds up experimentation, and promotes reproducibility by linking modern scientific writing with symbolic computation tools in a meaningful way.
Introduction:
Introduction:
Scientific research is advancing faster than ever, and so is the need for tools that can help researchers navigate and apply computational methods effectively. With thousands of powerful functions available in the Wolfram Language, selecting the right ones to complement a specific research paper can be overwhelming, especially for those new to symbolic computation.
The AI-Aided Wolfram Tech Recommender for Scientific Articles was created to help bridge this gap. This project explores how AI can assist researchers by analyzing the content of scientific articles and suggesting relevant Wolfram functions tailored to their methodology, goals, and analysis needs. At its core, the system uses large language models to generate initial recommendations, which are then filtered and validated to reduce hallucinations, one of the biggest challenges when working with generative AI.
The development pipeline begins with a set of definitions, including a master list of valid Wolfram Language functions. This list serves as the benchmark for identifying whether a recommended function is truly part of the language. Every recommendation is checked against this list, and false suggestions are flagged automatically, helping to maintain accuracy and trust in the tool.
In parallel with the main system, a form based interface is also being developed to offer an even more streamlined user experience. Although the form is still a work in progress and not yet fully integrated into the pipeline, the core system successfully generates a detailed and personalized Wolfram Notebook. This notebook includes curated function recommendations and clear explanations of how each function relates to specific parts of the research article, making it easier for users to explore, adapt, or build upon the original work.
To better understand the reliability of the model, I also ran a small study using 50 scientific articles across different fields. The results showed that hallucinated functions were minimal, indicating that the validation system works as intended and that this pipeline can be scaled for broader scientific use.
The AI-Aided Wolfram Tech Recommender for Scientific Articles was created to help bridge this gap. This project explores how AI can assist researchers by analyzing the content of scientific articles and suggesting relevant Wolfram functions tailored to their methodology, goals, and analysis needs. At its core, the system uses large language models to generate initial recommendations, which are then filtered and validated to reduce hallucinations, one of the biggest challenges when working with generative AI.
The development pipeline begins with a set of definitions, including a master list of valid Wolfram Language functions. This list serves as the benchmark for identifying whether a recommended function is truly part of the language. Every recommendation is checked against this list, and false suggestions are flagged automatically, helping to maintain accuracy and trust in the tool.
In parallel with the main system, a form based interface is also being developed to offer an even more streamlined user experience. Although the form is still a work in progress and not yet fully integrated into the pipeline, the core system successfully generates a detailed and personalized Wolfram Notebook. This notebook includes curated function recommendations and clear explanations of how each function relates to specific parts of the research article, making it easier for users to explore, adapt, or build upon the original work.
To better understand the reliability of the model, I also ran a small study using 50 scientific articles across different fields. The results showed that hallucinated functions were minimal, indicating that the validation system works as intended and that this pipeline can be scaled for broader scientific use.
Definitions:
Definitions:
The codes below give us the full master list of valid Wolfram Language Functions and and the boolean function to validate them.
In[]:=
validFunctionsListFull=Names["System`*"];validFunctionsList=Drop[Drop[validFunctionsListFull,168],-307];isValidWolframFunction[func_String]:=MemberQ[validFunctionsList,func];
This code wraps the AI-LLM pipeline in the reusable Wolfram Language Recommender Function.
In[]:=
recommendWolframFunctions[articleText_String]:=Module[{prompt,rawOutput,lines,cleanedSplitOutput},prompt="Read the following scientific article and return a list of Wolfram Language functions that are relevant to the methods or analysis described.\n\nOutput the list in the exact format:\nFunctionName - Short explanation\n\nOnly include one function per line. Do not include numbering, bullet points, or any text before or after the list.\n\nExample:\nImport - Import data from various sources\nListPointPlot3D - Visualize 3D data\n\nArticle:\n"<>articleText<>"\n\nThis is a list of the available Wolfram Language functions:\n\n"<>StringRiffle[Drop[Drop[validFunctionsList,168],-307],"\n"]<>"\n\nDO NOT SUGGEST FUNCTIONS THAT ARE NOT IN THIS LIST";rawOutput=LLMFunction[prompt];(*thisreturnsawrappedobject*)lines=rawOutput["String"];(*unwraptheplainstring*)cleanedSplitOutput=Select[StringSplit[#," - "]&/@StringSplit[lines,"\n"],Length[#]==2&];Dataset[AssociationThread[{"Function","Explanation"},#]&/@cleanedSplitOutput]]
Part I - Building the Pipeline and Generating the Wolfram Function Report:
Part I - Building the Pipeline and Generating the Wolfram Function Report:
Importing the article and Images:
Importing the article and Images:
In[]:=
articleText=Import["https://arxiv.org/pdf/2506.08229",{"PDF","Plaintext"}];paperImages=Import["https://arxiv.org/pdf/2506.08229",{"PDF","Images"}];
Wolfram Functions Recommendation Prompt:
Wolfram Functions Recommendation Prompt:
To generate function recommendations, I created a prompt that asks the language model to read the content of a scientific article and return a list of relevant Wolfram Language functions. The prompt is passed into an LLMFunction, which allows natural language tasks to be processed directly within the Wolfram environment.
In the prompt, I clearly define the expected output format: each line should include a function name, followed by a short explanation, with no bullet points or extra formatting. For example, a correct response might look like:
Import - Import data from various sources
Once the prompt is sent and the model responds, the output is stored in rawOutput. I then split the output into separate lines using StringSplit, and further break each line into two parts based on the hyphen: one for the function name and one for its explanation. I filter the results using Select to make sure only pairs of length two are kept, this removes any incomplete or malformed lines.
Finally, I convert the cleaned list into a Dataset, where each row shows a function and its corresponding explanation. This structured format makes it easy to display and later use the recommendations throughout the rest of the pipeline.
In the prompt, I clearly define the expected output format: each line should include a function name, followed by a short explanation, with no bullet points or extra formatting. For example, a correct response might look like:
Import - Import data from various sources
Once the prompt is sent and the model responds, the output is stored in rawOutput. I then split the output into separate lines using StringSplit, and further break each line into two parts based on the hyphen: one for the function name and one for its explanation. I filter the results using Select to make sure only pairs of length two are kept, this removes any incomplete or malformed lines.
Finally, I convert the cleaned list into a Dataset, where each row shows a function and its corresponding explanation. This structured format makes it easy to display and later use the recommendations throughout the rest of the pipeline.
In[]:=
recommendationPrompt=LLMFunction["Read the following scientific article and return a list of Wolfram Language functions that are relevant to the methods or analysis described.\n\nOutput the list in the exact format:\nFunctionName - Short explanation\n\nOnly include one function per line. Do not include numbering, bullet points, or any text before or after the list.\n\nExample:\nImport - Import data from various sources\nListPointPlot3D - Visualize 3D data\n\nArticle:\n"<>articleText];rawOutput=recommendationPrompt[];cleanedSplitOutput=Select[StringSplit[#," - "]&/@StringSplit[rawOutput,"\n"],Length[#]==2&];Dataset[AssociationThread[{"Function","Explanation"},#]&/@cleanedSplitOutput]
Out[]=
After Importing the article as a pdf, the above code generates the list of Wolfram functions that the LLM thinks will be useful for research purposes for this specific article but the problem is that all of these functions are not actual Wolfram Language functions and the LLM hallucinated some of these functions so our next step is to validate whether all these functions belong to the master list of Wolfram Language functions that we have in our “Definitions” section of this notebook. Before I validate these functions, I wrapped this code in a reusable function.
Wrapping the recommendation in a Function:
Wrapping the recommendation in a Function:
To simplify reuse, I wrapped the recommendation logic into a function called recommendWolframFunctions. It takes the article text as input, builds the prompt, and sends it to LLMFunction. The model’s response is unwrapped, split into lines, and cleaned to extract only valid Function - Explanation pairs. The final output is returned as a structured Dataset, ready for validation and later use in the notebook generation step.
In[]:=
recommendWolframFunctions[articleText_String]:=Module[{prompt,rawOutput,lines,cleanedSplitOutput},prompt="Read the following scientific article and return a list of Wolfram Language functions that are relevant to the methods or analysis described.\n\nOutput the list in the exact format:\nFunctionName - Short explanation\n\nOnly include one function per line. Do not include numbering, bullet points, or any text before or after the list.\n\nExample:\nImport - Import data from various sources\nListPointPlot3D - Visualize 3D data\n\nArticle:\n"<>articleText;rawOutput=LLMFunction[prompt];(*thisreturnsawrappedobject*)lines=rawOutput["String"];(*unwraptheplainstring*)cleanedSplitOutput=Select[StringSplit[#," - "]&/@StringSplit[lines,"\n"],Length[#]==2&];Dataset[AssociationThread[{"Function","Explanation"},#]&/@cleanedSplitOutput]]
Usage / Test:
Usage / Test:
In[]:=
articleText=Import["https://arxiv.org/pdf/2506.08229",{"PDF","Plaintext"}];recommendWolframFunctions[articleText];
Wolfram Functions Validation:
Wolfram Functions Validation:
After generating the list of function recommendations, I needed to verify whether each function was actually part of the Wolfram Language. This validation step ensures the accuracy of the output and helps identify hallucinated suggestions made by the language model.
First, I extract the list of function names and their explanations from the dataset using splitResults. Then, I apply a function called isValidWolframFunction to each function name, which checks whether it exists in the master list of valid Wolfram Language symbols.
Using MapThread, I combine the function names, their explanations, and the validation results into one unified structure. The final output is stored in a variable called validatedData, which is then displayed as a Dataset. This table clearly shows which recommended functions are real (True) and which are hallucinated (False), helping users trust and act on the valid results.
First, I extract the list of function names and their explanations from the dataset using splitResults. Then, I apply a function called isValidWolframFunction to each function name, which checks whether it exists in the master list of valid Wolfram Language symbols.
Using MapThread, I combine the function names, their explanations, and the validation results into one unified structure. The final output is stored in a variable called validatedData, which is then displayed as a Dataset. This table clearly shows which recommended functions are real (True) and which are hallucinated (False), helping users trust and act on the valid results.
In[]:=
results=recommendWolframFunctions[articleText];
splitResults=Normal[results][[All,{"Function","Explanation"}]];
functionNames=splitResults[[All,1]];validityFlags=isValidWolframFunction/@functionNames;
validatedData=MapThread[Association["Function"->#1,"Explanation"->#2,"IsValid"->#3]&,{splitResults[[All,1]],splitResults[[All,2]],validityFlags}];
In[]:=
Dataset[validatedData]
Out[]=
Form Page:
Form Page:
The Form Page shown below serves as a simple web interface where users can paste a URL to a scientific article. It then generates a list of recommended Wolfram Language functions and checks whether each one is valid. While this does achieve part of the intended goal, it currently operates separately from the main pipeline and produces a new, different set of function recommendations. At this stage, it does not yet generate a personalized Wolfram Notebook. I’m including this feature in the notebook because it’s a useful addition, and in future work, I hope to expand it so that it not only validates the functions but also generates a complete Wolfram Function Recommender Notebook based on the article provided.
FormPage[{"Article"->"String"},Module[{articleText=Import[#Article,{"PDF","Plaintext"}],results,splitResults,functionNames,validityFlags,validatedData},results=recommendWolframFunctions[articleText];splitResults=Normal[results][[All,{"Function","Explanation"}]];functionNames=splitResults[[All,1]];validityFlags=isValidWolframFunction/@functionNames;validatedData=MapThread[Association["Function"->#1,"Explanation"->#2,"IsValid"->#3]&,{splitResults[[All,1]],splitResults[[All,2]],validityFlags}];Grid[Prepend[Values[validatedData],{"Function Name","Description","Valid WL?"}],Frame->All,FrameStyle->LightGray,Spacings->{1,1},Background{None,{{LightGray,None}}}]]&,AppearanceRules-><|"Title"->"Wolfram Function Recommender Tool","Description"->"Enter a URL to a scientific article","SubmitLabel"->"Analyze Article"|>]
Form Page (Image):
Form Page (Image):
Below is the image of the Form Page working to generate a list of Wolfram Functions and then it validates whether the recommended function is a valid Wolfram Function or not in the “ValidWL?” column.
Creating the Wolfram Functions Report:
Creating the Wolfram Functions Report:
Before generating the final Wolfram Notebook, I needed to clean the list of function recommendations so that only the valid ones were used. I did this by filtering out all entries marked as False in the validation step. The remaining, confirmed Wolfram Language functions were joined into a space-separated string called realWlFunctions.
This list was then passed into LLMSynthesize as part of a detailed prompt. The goal of this step was to automatically generate a rich, markdown-formatted report that explains how the valid Wolfram functions relate to the article’s content. The prompt asks the LLM to include key sections like a title, abstract, methodology, tech integration, example demos, and references, using only the validated functions in the explanation. The result is stored in reportResult, which can then be displayed as a complete, personalized Wolfram Tech Recommendation Report.
This list was then passed into LLMSynthesize as part of a detailed prompt. The goal of this step was to automatically generate a rich, markdown-formatted report that explains how the valid Wolfram functions relate to the article’s content. The prompt asks the LLM to include key sections like a title, abstract, methodology, tech integration, example demos, and references, using only the validated functions in the explanation. The result is stored in reportResult, which can then be displayed as a complete, personalized Wolfram Tech Recommendation Report.
Now, we generate the report:
Part II - LLM Hallucination Research Experiment
Part II - LLM Hallucination Research Experiment
An AI hallucination happens when a language model like ChatGPT or Wolfram’s LLMFunction makes something up that sounds real but isn’t. It could be a fake fact, a wrong explanation, or in my case, a Wolfram function that doesn’t actually exist. These models don’t know what’s true, they just predict what looks right based on patterns in the data they were trained on. So when I asked the model to suggest Wolfram Language functions, it sometimes gave confident but incorrect answers like “BayesianOptimization” or “NeuralNetwork,” which aren’t official functions.
When using language models in real-world applications, hallucinations can quietly create serious risks. If an AI tool recommends something that looks correct but isn’t, users may not always catch the mistake, especially in fields like scientific research, education, or even marketing where precision matters. For example, a hallucinated Wolfram function might seem useful but could break a notebook or lead to wrong results. In research, this can mean wasted time, invalid experiments, or misleading conclusions.
For newer applications like AI-driven recommendation systems, the danger is that users might trust the output without realizing it’s incorrect. This is especially risky when the AI is being used by students, researchers, or non-experts who rely on it to guide their work. In fast-moving fields, even a small error can lead to bigger consequences downstream. That’s why it’s not enough to just generate answers, we need to validate them and build systems that are aware of these risks and designed to prevent them.
For newer applications like AI-driven recommendation systems, the danger is that users might trust the output without realizing it’s incorrect. This is especially risky when the AI is being used by students, researchers, or non-experts who rely on it to guide their work. In fast-moving fields, even a small error can lead to bigger consequences downstream. That’s why it’s not enough to just generate answers, we need to validate them and build systems that are aware of these risks and designed to prevent them.
For this project, I built a validation step that checks every recommended function against a real list of Wolfram Language functions. If a function doesn’t match, the system flags it. This helped me make the results more accurate and more useful, and it’s something I plan to keep improving as I test the system in different fields like book recommendations or marketing.
In this next part of my Notebook, I do a research experiment with 50 different scientific articles from arXiv to observe how much the LLM hallucinates.
In this next part of my Notebook, I do a research experiment with 50 different scientific articles from arXiv to observe how much the LLM hallucinates.
Importing 50 PDF articles for our LLM Hallucination Research Experiment:
Importing 50 PDF articles for our LLM Hallucination Research Experiment:
In this part of the experiment, I collected 50 articles from arXiv to test how often the language model hallucinates functions across a broader dataset. After importing all article links, I used a filter to keep only the URLs that point to PDF files. This was done using Select and StringContainsQ, which checks if each URL contains the word “pdf.”
Once filtered, the list of PDF links was stored in papersList, and I used Import to extract the plaintext content from each article. This gave me clean text data from all 50 articles, which I then used to run the function recommendation and validation process on a larger scale.
Once filtered, the list of PDF links was stored in papersList, and I used Import to extract the plaintext content from each article. This gave me clean text data from all 50 articles, which I then used to run the function recommendation and validation process on a larger scale.
Uploading the Articles List to the Cloud:
Uploading the Articles List to the Cloud:
To keep things organized and avoid losing data during the experiment, I saved the full list of imported articles to the Wolfram Cloud using CloudPut. This stored the list under the name “articlesTextForAnalysis”, which created a cloud object that I could access later from any notebook or environment.
Later, I retrieved the saved data using CloudGet, which pulled the articles back into the variable papersList. This step ensured the list of 50 article texts was safely backed up and easy to reuse without having to re-import everything from arXiv.
Later, I retrieved the saved data using CloudGet, which pulled the articles back into the variable papersList. This step ensured the list of 50 article texts was safely backed up and easy to reuse without having to re-import everything from arXiv.
Visual Graphics for the Articles List:
Visual Graphics for the Articles List:
Next, we check the article details visually through a BarChat, Histogram, and Smooth Histogram graphics. On x-axis, we have the “Articles”, and on y-axis, we have the “Character Count”.
Getting Wolfram Function Recommendations for these articles:
Getting Wolfram Function Recommendations for these articles:
This function, getFunctionsRecommendations, is where the AI does its core job. It takes the full text of a scientific article (paperText) and uses LLMSynthesize to ask the large language model for 20 Wolfram Language functions that might be useful based on the content of the paper. The prompt is carefully written to ensure the output is simple and clean, just a list of function names in single quotes, one per line, with no extra text, Markdown, or formatting. This makes it much easier to process the output later on in the pipeline.
Variable for Accumulation:
Variable for Accumulation:
Processing Articles with AI LLM (One by One):
Processing Articles with AI LLM (One by One):
In this step, I processed each of the 50 scientific articles one by one using the language model. I used a Table loop to go through the list of articles stored in papersList, and for each one, I called getFunctionsRecommendations to generate a list of suggested Wolfram Language functions.
The results were stored in the variable functionRecommendations, which kept getting updated using Append after each article was processed. I also added a short pause between each iteration using Pause[1] to give the model time to respond and avoid overloading it. The styled PrintTemporary was just a visual marker to help track progress while running the loop. This loop gave me a complete set of raw function recommendations across all 50 articles.
The results were stored in the variable functionRecommendations, which kept getting updated using Append after each article was processed. I also added a short pause between each iteration using Pause[1] to give the model time to respond and avoid overloading it. The styled PrintTemporary was just a visual marker to help track progress while running the loop. This loop gave me a complete set of raw function recommendations across all 50 articles.
List Dimensions:
List Dimensions:
Head of the first element in the list:
Head of the first element in the list:
Wolfram Language (WL) Functions Recommendations for the First Article:
Wolfram Language (WL) Functions Recommendations for the First Article:
WL Functions Recommendations for the Last Article:
WL Functions Recommendations for the Last Article:
Cleaning the Function Recommendations:
Cleaning the Function Recommendations:
After running the AI model on 50 articles, I noticed that most outputs were valid strings of Wolfram function names, but a few failed and returned a Failure object instead. To ensure the data is clean and consistent, I first checked the “head” of each element using Head /@ functionRecommendations, which revealed the presence of one Failure among all the String entries. To fix this, I used DeleteFailures to remove any invalid results. This gave me a reliable list of string outputs only. Finally, I performed some light post-processing: joining the valid strings into one large string and stripping out unwanted spaces and quotation marks using StringJoin and StringReplace. This left me with a clean, uniform string containing only validated function names, ready for analysis or further transformation.
Word Cloud Visualization of AI-Recommended Functions:
Word Cloud Visualization of AI-Recommended Functions:
The graphic above shows a Word Cloud generated from all the function names suggested by the AI LLM across 50 scientific articles. At this point in the process, we have not yet filtered out the incorrect or hallucinated Wolfram Language (WL) functions, so the cloud contains a mix of valid and invalid recommendations. This visualization helps highlight which functions the model frequently suggests. For example, functions like “ArrayPlot”, “RandomSample”, and “Manipulate” appear prominently, indicating their high frequency and relevance according to the LLM’s understanding. Although this Word Cloud gives us a good sense of common recommendations, it doesn’t yet confirm whether those functions are valid WL functions. The next step is to differentiate real WL functions from hallucinated ones, allowing us to better evaluate the AI’s accuracy and usefulness in assisting scientific research.
Validating Wolfram Language Functions:
Validating Wolfram Language Functions:
The function shown above, checkFunction, is used to verify whether each AI-recommended function is a valid Wolfram Language (WL) function or simply a hallucination. It works by comparing each item in a given list to the full list of recognized functions in the Wolfram System context (`System``).
Internally, it first retrieves all valid WL function names using Names[“System*”], then checks each item in the input list (wlf) to see if it exists in that official list. The result is returned as pairs of the original function name along with a Boolean value: True, if it’s a real WL function, and False, if it’s not. This step is crucial in filtering out invalid or made-up outputs, which are common in large language model responses.
Internally, it first retrieves all valid WL function names using Names[“System*”], then checks each item in the input list (wlf) to see if it exists in that official list. The result is returned as pairs of the original function name along with a Boolean value: True, if it’s a real WL function, and False, if it’s not. This step is crucial in filtering out invalid or made-up outputs, which are common in large language model responses.
Validating the Recommended Functions:
Validating the Recommended Functions:
After collecting all the function names recommended by the AI LLM across 50 scientific articles, the next step was to verify which of those suggestions were actually valid Wolfram Language (WL) functions and which ones were hallucinated (i.e., not part of the official Wolfram Language system). To do this, I used a helper function called checkFunction, which compares each recommended name against Wolfram’s internal list of system-defined functions. This was done using the command:
This returned a list of pairs like {FunctionName, True} for valid functions or {FunctionName, False} for hallucinated ones. I then visualized the results using Dataset[%] and used the Tally[Flatten[Last /@ %%]] command to count how many of the total functions were real versus hallucinated.
This step was important because it allowed me to assess the accuracy of the LLM’s recommendations. For example, suggestions like “ArrayPlot” or “Import” were confirmed as valid WL functions, while others like “TranslateImport” or “GaussianProcessRegression” were not recognized by the system and were therefore classified as hallucinations. This helped me better understand both the potential and the limitations of relying on LLMs for tool recommendations in scientific workflows. In the code above, we generated a total of 961 recommended functions across all 50 articles. Out of these, 727 were validated as real Wolfram Language functions, while 234 were identified as false or hallucinated. This indicates a strong proportion of accurate recommendations made by the AI LLM, highlighting its potential usefulness in scientific tool discovery.
This step was important because it allowed me to assess the accuracy of the LLM’s recommendations. For example, suggestions like “ArrayPlot” or “Import” were confirmed as valid WL functions, while others like “TranslateImport” or “GaussianProcessRegression” were not recognized by the system and were therefore classified as hallucinations. This helped me better understand both the potential and the limitations of relying on LLMs for tool recommendations in scientific workflows. In the code above, we generated a total of 961 recommended functions across all 50 articles. Out of these, 727 were validated as real Wolfram Language functions, while 234 were identified as false or hallucinated. This indicates a strong proportion of accurate recommendations made by the AI LLM, highlighting its potential usefulness in scientific tool discovery.
Visualizing the Results: Pie Chart
Visualizing the Results: Pie Chart
To better understand the accuracy of the AI LLM’s function recommendations, the code above generates a pie chart. Specifically, it uses the tally of the validated functions; whether each function is a real Wolfram Language function (True) or a hallucinated one (False), and creates a visual breakdown.
- Last /@ %: Extracts the validation results (True/False) from the list of {functionName, True/False} pairs.
- First /@ %: Extracts the corresponding function names.
- ChartLabels → First /@ %: Assigns the function names as labels on the chart.
- PlotLabel → “AI-LLM Function Recommendations”: Adds a title to the chart for clarity.
This chart helps visualize how many of the AI-generated function recommendations were valid versus invalid, offering a quick snapshot of the AI’s performance in recognizing real WL functions.
- Last /@ %: Extracts the validation results (True/False) from the list of {functionName, True/False} pairs.
- First /@ %: Extracts the corresponding function names.
- ChartLabels → First /@ %: Assigns the function names as labels on the chart.
- PlotLabel → “AI-LLM Function Recommendations”: Adds a title to the chart for clarity.
This chart helps visualize how many of the AI-generated function recommendations were valid versus invalid, offering a quick snapshot of the AI’s performance in recognizing real WL functions.
View all the WL Functions suggested for the 50 Articles by the AI LLM:
View all the WL Functions suggested for the 50 Articles by the AI LLM:
In this section, we store and view the complete list of all Wolfram Language (WL) functions recommended by the AI LLM across the 50 scientific articles. Each function is paired with a boolean value: True if it’s a valid WL function and False if it’s not. This paired structure allows for easy identification and separation of genuine versus hallucinated function suggestions.
Separating the AI LLM Hallucinations:
Separating the AI LLM Hallucinations:
To evaluate the accuracy of the AI LLM’s suggestions, we filter out all the functions that were incorrectly identified as WL functions (i.e., those marked False). By extracting these hallucinated recommendations, we can analyze which non-existent or incorrect function names appear most frequently. This step is crucial for understanding common patterns of AI-generated hallucinations and improving prompt strategies in future iterations.
Word Cloud (Graphics): Most Common Hallucinated Functions
Word Cloud (Graphics): Most Common Hallucinated Functions
The word cloud above visualizes the most frequently recommended functions by the AI LLM that turned out to be hallucinations; not actual Wolfram Language (WL) functions. Each term represents a function name the model suggested, but that does not exist in the WL system. The size of each word corresponds to how often that particular hallucinated function appeared across the 50 articles.
One striking observation is the prominence of the term “Statistics”, which appears much larger than the rest, indicating that it is the most commonly hallucinated function. Other frequent hallucinations include “RegressionModel”, “GroupByImport”, and “MonteCarlo”. Interestingly, many of these terms are conceptually valid or closely resemble real WL functions, which may explain why the AI confidently (but mistakenly) recommended them.
This reveals a valuable insight: the hallucinations are not entirely random. Instead, they show that there might be a gap in Wolfram Language’s current set of functions, or at least, these are areas where AI models expect functions to exist but can’t find them easily. In fact, one could argue that these hallucinated terms present an opportunity for future function development in Wolfram Language. Functions like “Statistics” or “RegressionModel” may represent high-demand computational concepts that could benefit from clearer, more centralized implementation. Overall, this step helps us better understand not only the LLM’s limitations but also user expectations and potential areas of enhancement in the WL ecosystem.
One striking observation is the prominence of the term “Statistics”, which appears much larger than the rest, indicating that it is the most commonly hallucinated function. Other frequent hallucinations include “RegressionModel”, “GroupByImport”, and “MonteCarlo”. Interestingly, many of these terms are conceptually valid or closely resemble real WL functions, which may explain why the AI confidently (but mistakenly) recommended them.
This reveals a valuable insight: the hallucinations are not entirely random. Instead, they show that there might be a gap in Wolfram Language’s current set of functions, or at least, these are areas where AI models expect functions to exist but can’t find them easily. In fact, one could argue that these hallucinated terms present an opportunity for future function development in Wolfram Language. Functions like “Statistics” or “RegressionModel” may represent high-demand computational concepts that could benefit from clearer, more centralized implementation. Overall, this step helps us better understand not only the LLM’s limitations but also user expectations and potential areas of enhancement in the WL ecosystem.
Concluding Remarks
Concluding Remarks
Building this project has been a journey of asking a simple question: how can we make Wolfram technologies more accessible to researchers who might not know where to start? The AI Aided Wolfram Tech Recommender for Scientific Articles was my attempt to answer that. From reading a scientific article to generating a personalized Wolfram Notebook with curated functions and explanations, this pipeline was designed to act like a bridge between language models and symbolic computation.
One of the biggest technical challenges we faced was hallucination; when the AI recommends functions that sound convincing but do not actually exist. In order to bypass the hallucination issue typical of AI LLM algorithms, we created a validation system that checks each recommended function against the official Wolfram Language function list. If a function doesn’t match, it’s flagged. This step was essential not just for accuracy but for building trust in the system, especially for people who are using Wolfram tools in research for the first time.
The system works in steps. First, the LLM reads and analyzes the article, then suggests a list of functions that it believes match the methods or analyses. Those functions go through a filtering process where only the valid ones move forward. The final product is a customized Wolfram Notebook that highlights those functions and explains how they relate to different parts of the paper. It’s not just a list, it’s a tool that can actually help someone think through and expand their work.
To test the reliability of the system, we also ran a hallucination study using 50 different scientific articles from arXiv. The model generated 966 function suggestions in total. Out of those, 726 were valid Wolfram functions and 240 were hallucinated. That means the system was accurate about 75 percent of the time, not perfect, but a strong starting point, especially when paired with a built-in validation layer.
Although the form-based web interface is still in progress, it’s another piece of this vision. The idea is to make the tool more interactive, more accessible, and easier to use without needing to open a notebook or write code. There’s also room to take this core idea and experiment with it in other domains, like recommending books, mapping concepts in marketing, or building research support tools in other industries.
In conclusion, this project is about making powerful technology feel approachable. It’s about helping researchers spend less time guessing and more time building. And it’s about understanding where Artificial Intelligence excels, where it slips up, and how we can work with it more thoughtfully to create tools that are not just smart, but reliable and helpful too.
One of the biggest technical challenges we faced was hallucination; when the AI recommends functions that sound convincing but do not actually exist. In order to bypass the hallucination issue typical of AI LLM algorithms, we created a validation system that checks each recommended function against the official Wolfram Language function list. If a function doesn’t match, it’s flagged. This step was essential not just for accuracy but for building trust in the system, especially for people who are using Wolfram tools in research for the first time.
The system works in steps. First, the LLM reads and analyzes the article, then suggests a list of functions that it believes match the methods or analyses. Those functions go through a filtering process where only the valid ones move forward. The final product is a customized Wolfram Notebook that highlights those functions and explains how they relate to different parts of the paper. It’s not just a list, it’s a tool that can actually help someone think through and expand their work.
To test the reliability of the system, we also ran a hallucination study using 50 different scientific articles from arXiv. The model generated 966 function suggestions in total. Out of those, 726 were valid Wolfram functions and 240 were hallucinated. That means the system was accurate about 75 percent of the time, not perfect, but a strong starting point, especially when paired with a built-in validation layer.
Although the form-based web interface is still in progress, it’s another piece of this vision. The idea is to make the tool more interactive, more accessible, and easier to use without needing to open a notebook or write code. There’s also room to take this core idea and experiment with it in other domains, like recommending books, mapping concepts in marketing, or building research support tools in other industries.
In conclusion, this project is about making powerful technology feel approachable. It’s about helping researchers spend less time guessing and more time building. And it’s about understanding where Artificial Intelligence excels, where it slips up, and how we can work with it more thoughtfully to create tools that are not just smart, but reliable and helpful too.
Acknowledgments
Acknowledgments
I’m grateful to my mentor, Daniel Carvalho, whose expertise in Wolfram Language and thoughtful feedback helped me improve both the structure and clarity of this project. A huge thank you as well to Christian Pasquel for his technical guidance and practical suggestions, especially when I got stuck. As part of the project’s development, we held a scope meeting with Stephen Wolfram, many thanks to him for his valuable insights. And a big thank you to the Wolfram Summer School 2025 faculty and Program Director, Stephanie Bowyer, for creating such a supportive space to learn and build.
References
References
1
.Wolfram Research, Inc., 2024, Mathematica, Version 14.2, Champaign, IL.
2
.Wolfram Research, Inc., 2025, Wolfram Language & System Documentation Center, Champaign, IL, https://reference.wolfram.com/language/.
3
.Wolfram, S., 2023, What Is ChatGPT Doing ... and Why Does It Work?. Wolfram Media Inc.
4
.5
.Bernard, Etienne, 2021, Introduction to Machine Learning. Wolfram Media Inc,
https://www.wolfram.com/language/introduction-machine-learning/.
https://www.wolfram.com/language/introduction-machine-learning/.
6
.Wolfram, S., 2015, An Elementary Introduction to the Wolfram Language. 3Ed. Wolfram Media Inc,
https://www.wolfram.com/language/elementary-introduction/3rd-ed/.
https://www.wolfram.com/language/elementary-introduction/3rd-ed/.
7
.Gemini, Google, 2024, Prompting Guide 101: A quick-start handbook for effective prompts https://services.google.com/fh/files/misc/gemini-for-google-workspace-prompting-guide-101.pdf
8
.Boonstra, Lee, 2024, Prompt Engineering, OpenAI, https://www.gptaiflow.tech/assets/files/2025-01-18-pdf-1-TechAI-Goolge-whitepaper_Prompt%20Engineering_v4-af36dcc7a49bb7269a58b1c9b89a8ae1.pdf
9
.10
.Cloud, Google, 2025, Prompt engineering: overview and guide, https://cloud.google.com/discover/what-is-prompt-engineering.
11
.Anthropic, 2025, 6 Techniques for Effective Prompt Engineering, https://www-cdn.anthropic.com/62df988c101af71291b06843b63d39bbd600bed8.pdf
Cite This Notebook
Cite This Notebook
AI-Aided Wolfram Tech Recommender System for Scientific Articles (TechMatch)
by Muskaan Shahzad
Wolfram Community, STAFF PICKS, July 9, 2025
https://community.wolfram.com/groups/-/m/t/3495300
by Muskaan Shahzad
Wolfram Community, STAFF PICKS, July 9, 2025
https://community.wolfram.com/groups/-/m/t/3495300