Anton Antonov​
​MathematicaForPrediction at WordPress​
​MathematicaForPrediction at GitHub

Introduction

In this notebook we use the Large Language Model (LLM) prompts”ThemeTableJSON” and "ExtractArticleWisdom" to facilitate the reading and comprehension of Stephen Wolfram’s article "Can AI Solve Science?", [SW1].
Remark: We use “simple” text processing, but since the article has lots of images multi-modal models would be more appropriate.
Here is an image of article's start:
Out[]=
The computations are done with Wolfram Language (WL) chatbook. The LLM functions used in the workflows are explained and demonstrated in [SW2, AAn1÷ AAn4]. The workflows are done with OpenAI's models. Currently the models of Google's (PaLM, Gemini) and MistralAI cannot be used with the workflows below because their input token limits are too low.
The "rendering" of the LLM results that are in Markdown format can be done:
◼
  • Manually -- that is quick and easy enough
    • ◼
  • Using ExternalEvaluate and the Raku package "Markdown::Grammar", [AAp1], discussed in [AAn5]
    • ◼
  • Using LLM functions as discussed in [AAn2, AAn6].

    • Structure

    The structure of the notebook is as follows:
    1
    .
    Getting the article's text and setup
    Standard ingestion and setup.
    2
    .
    Article's structure
    TL;DR via a table of themes.
    3
    .
    Flowcharts
    Get flowcharts relating article's concepts.
    4
    .
    Extract article wisdom
    Get a summary and extract ideas, quotes, references, etc.

    Ingest text

    Here we get the plain text of the article:
    In[]:=
    txtArticleOrig=Import["https://writings.stephenwolfram.com/2024/03/can-ai-solve-science/","Plaintext"];​​StringLength[txtArticleOrig]
    Out[]=
    103350
    Here we clean article’s text :
    In[]:=
    txtArticle=StringTake[txtArticleOrig,{1,StringPosition[txtArticleOrig,"Posted in:"]〚1,1〛}];​​StringLength[txtArticle]
    Out[]=
    98644

    Themes

    Here we extract the themes found in the article and tabulate them (using the prompt "ThemeTableJSON"):
    In[]:=
    tblThemes=LLMResourceFunction["ThemeTableJSON"][txtArticle];​​ResourceFunction["GridTableForm"]@Dataset[tblThemes//FromJSON]
    Out[]=
    #
    theme
    content
    1
    Introduction to AI in Science
    Stephen Wolfram discusses the potential and limitations of AI in science, emphasizing its role in leveraging computational reducibility and its limitations due to computational irreducibility. He explores AI's practical uses and its comparison to traditional scientific methods, highlighting the combination of AI and computational paradigms as a powerful approach for advancing science.
    2
    AI's Role and Limitations
    AI, particularly through trained neural nets, offers a new way to utilize computational reducibility, capturing regularities in a manner aligned with human cognition. However, its effectiveness is limited in the presence of computational irreducibility, where deep, irreducible computations are necessary to uncover new discoveries.
    3
    Practical Uses of AI in Science
    AI provides a new tool for science, useful for making approximate predictions and identifying candidate cases for exploration. It excels in workflows where an '80%' solution is sufficient, but struggles with tasks requiring precise and perfect outcomes due to its inherent limitations in dealing with computational irreducibility.
    4
    Combining AI with Computational Paradigms
    The greatest potential for advancing science lies in combining AI's strengths with the formal computational paradigm. This approach leverages AI's ability to handle computational reducibility and the computational paradigm's capacity for conducting irreducible computations, enabling the exploration of new scientific frontiers.
    5
    AI Measurements and Formalized Science
    AI enables new kinds of measurements, extracting meaningful features from unstructured data. These 'AI measurements' can potentially lead to formal relationships and predictions, extending the domain of 'exact sciences' to areas previously dominated by less formal approaches.
    6
    Exploring New Directions with AI
    AI's potential to suggest new scientific directions is limited by its training on existing human material, making it better suited for extending established paradigms rather than discovering fundamentally new science. True exploration requires venturing into the 'wilds of the ruliad' beyond AI's current capabilities.
    7
    Conclusion and Acknowledgments
    Wolfram concludes by acknowledging the assistance and discussions that have shaped his exploration of AI in science. He notes the beginning nature of his work and the potential for future engineering innovations to impact his conclusions about AI's role in scientific discovery.
    Remark: A fair amount of LLMs give their code results within Markdown code block delimiters (like "```".) Hence, (1) the prompt "ThemeTableJSON" does not use Interpreter["JSON"], but Interpreter["String"], and (2) we use above the function FromJSON in order to convert the LLM input into a WL data structure. (See the definition of FromJSON see the "Setup" section below.)

    Related definitions

    After reading the breakdown in the table above we might want to get a few clarifications and definitions of certain concepts. For those, we might (i) just use general knowledge of the LLM or (ii) articles's text. Here are a few examples:
    In[]:=
    LLMSynthesize["What are computational irreducibility and computational reducibility according to Stephen Wolfram?"]
    Out[]=
    According to Stephen Wolfram, computational irreducibility refers to systems or processes that cannot be simplified or predicted efficiently. These are systems whose behavior cannot be determined without explicit simulation or computation. This means that there is no shortcut or formula that can predict the outcome, and one has to go through the entire process to understand it fully.​Conversely, computational reducibility refers to systems or processes that can be simplified or predicted efficiently. In these cases, it is possible to find a formula, algorithm, or shortcut that can determine the outcome without performing all the individual steps. The behavior of these systems is said to be reducible to a manageable calculation or procedure.​Wolfram's ideas on computational irreducibility and reducibility stem from his exploration of complexity in various systems, particularly through his work on cellular automata and the field of computational thinking. He argues that many naturally occurring phenomena and processes are computationally irreducible, meaning that their full behavior cannot be easily deduced from simple rules or formulas.
    In[]:=
    LLMSynthesize[{"What are computational irreducibility and computational reducibility according to the following text?","Give concise definitions.\n\n",txtArticle}]
    Out[]=
    Computational irreducibility, according to Stephen Wolfram's analysis, implies that for certain processes and systems, there isn't a way to shortcut through the computational work required to reach a result. This principle suggests that certain complex processes need to be computed step-by-step to understand their final state, because there's no simpler way to predict the outcome. It highlights fundamental limits in our ability to predict or simplify the behavior of systems, as everything unfolds at a level of complexity that is essentially equivalent across different types of computations. This is pivotal in understanding the limits of what we can expect to achieve with both traditional scientific methods and AI in predicting or understanding complex systems.​In contrast, computational reducibility refers to situations or aspects of systems where it is possible to find shortcuts or simpler methods to predict their behavior without having to simulate every step of the process. This concept is crucial in areas of science and mathematics where laws or formulas can be derived to predict the outcomes of systems efficiently rather than simulating them—highlighting pockets within the vast sea of computational irreducibility where prediction and simplification are possible. These pockets allow for the development of theories, models, and predictions about certain aspects of the world in a computationally efficient manner, enabling the progress of science within the realms of computational irreducibility.

    Flowcharts

    In this section we LLMs to get Mermaid-JS flowcharts that correspond to the content of [SW1].
    Remark: Below we use the paclet "AntonAntonov/MermaidJS", [AAp2], but the resource function "MermaidInk" can be used instead.

    Big picture concepts

    Here we generate Mermaid-JS flowchart for the “big picture” concepts:
    In[]:=
    mmdBigPicture=​​LLMSynthesize[{​​"Create a mermaid-js graph for the connections between the concepts in the article:\n\n",​​txtArticle,​​LLMPrompt["NothingElse"]["mermaid-js"]​​}]
    Here we define “big picture” styling theme:
    In[]:=
    mmdBigPictureTheme="%%{ init: { 'theme': 'forest', 'themeVariables': { 'backgroundColor': '#FFF' } }}%%\n";
    Here we create the flowchart from LLM’s specification:
    We made several “big picture” flowchart generations. Here is the result of another attempt:

    Fine grained

    Here we derive a flowchart that refers to more detailed, finer grained concepts:
    Here we define “fine grained” styling theme:
    Here we create the flowchart from LLM’s specification:
    We made several “fine grained” flowchart generations. Here is the result of another attempt:

    Summary and ideas

    Here we get a summary and extract ideas, quotes, and references from the article:
    The result is rendered below.

    SUMMARY

    Stephen Wolfram explores the potential and limitations of AI in science, discussing its ability to leverage computational reducibility and its implications for scientific discovery and understanding.

    IDEAS:

    ◼
  • AI's recent successes suggest it might eventually "do everything," but there are limits to its capabilities in science.
    • ◼
  • The transformation of science by computational representation is a major shift, akin to the introduction of mathematics.
    • ◼
  • AI, particularly machine learning, offers a new way to access and utilize computational capabilities.
    • ◼
  • Computational irreducibility sets a hard limit on what can be predicted or solved, even with AI.
    • ◼
  • Pockets of computational reducibility within the vastness of computational irreducibility allow for scientific progress.
    • ◼
  • AI can streamline the discovery of computationally reducible phenomena but cannot overcome computational irreducibility.
    • ◼
  • The historical quest to "mechanically solve" mathematics mirrors the current exploration of AI's potential in science.
    • ◼
  • Systematic enumeration and AI can lead to unexpected discoveries in science, showcasing the power of computation.
    • ◼
  • AI's role in predicting outcomes is limited by the inherent computational irreducibility of many processes.
    • ◼
  • The potential of AI to identify computational reducibility and assist in scientific workflows is significant, yet bounded by computational irreducibility.
    • ◼
  • AI's ability to emulate human intelligence and potentially understand natural processes directly is intriguing but limited.
    • ◼
  • The use of AI in solving equations and predicting complex system behaviors showcases both its potential and limitations.
    • ◼
  • AI's role in multicomputational processes and exploring spaces of systems opens new avenues for scientific exploration.
    • ◼
  • The concept of science as narrative and the challenge of finding what's interesting in science are influenced by computational irreducibility.
    • ◼
  • The extension of "exact sciences" to traditionally less formal areas through AI measurements and computational language represents a new frontier in science.

    • QUOTES:

    ◼
  • "AI will be able to 'do everything', or at least everything we currently do."
    • ◼
  • "The answer is inevitably and firmly no."
    • ◼
  • "AI can't importantly help the progress of science."
    • ◼
  • "A fundamentally computational representation of the world."
    • ◼
  • "The hard limit of computational irreducibility."
    • ◼
  • "There are always certain aspects of a system about which things can be said using limited computational effort."
    • ◼
  • "AI has the potential to give us streamlined ways to find certain kinds of pockets of computational reducibility."
    • ◼
  • "Computational irreducibility is what will prevent us from ever being able to completely 'solve science'."
    • ◼
  • "AI provides a new way to make use of computational reducibility."
    • ◼
  • "The challenge of finding what's interesting in science are influenced by computational irreducibility."

    • FACTS:

    ◼
  • AI's recent successes have raised expectations about its potential to solve scientific questions.
    • ◼
  • Science has been transformed by computational representation, following the earlier transformation by mathematics.
    • ◼
  • Computational irreducibility limits the ability to predict or solve many scientific questions.
    • ◼
  • Despite computational irreducibility, science is possible due to pockets of computational reducibility.
    • ◼
  • AI can help identify computationally reducible phenomena but cannot overcome computational irreducibility.
    • ◼
  • Historical attempts to "mechanically solve" mathematics parallel current explorations of AI in science.
    • ◼
  • AI and systematic enumeration can lead to discoveries that were not anticipated by human intuition.
    • ◼
  • AI's predictive capabilities are constrained by computational irreducibility.
    • ◼
  • AI can assist in scientific workflows by identifying computationally reducible phenomena.
    • ◼
  • The extension of "exact sciences" through AI measurements and computational language represents a significant development in science.

    • REFERENCES:

    ◼
  • Wolfram Language
    • ◼
  • Neural networks
    • ◼
  • Machine learning
    • ◼
  • Computational irreducibility
    • ◼
  • Cellular automata
    • ◼
  • Multicomputational processes
    • ◼
  • Generative AI
    • ◼
  • Automated theorem proving
    • ◼
  • Computational language
    • ◼
  • Wolfram|Alpha

    • RECOMMENDATIONS:

    ◼
  • Explore the potential of AI in identifying computationally reducible phenomena within scientific research.
    • ◼
  • Utilize AI to streamline scientific workflows, recognizing its limitations due to computational irreducibility.
    • ◼
  • Investigate the use of AI in solving equations and predicting behaviors in complex systems.
    • ◼
  • Consider the role of AI in multicomputational processes and its ability to explore new scientific spaces.
    • ◼
  • Embrace the concept of science as narrative and the challenge of finding what's interesting through computational irreducibility.
    • ◼
  • Extend the domain of "exact sciences" to traditionally less formal areas using AI measurements and computational language.
    • ◼
  • Leverage the power of computation and AI to make unexpected discoveries in science.
    • ◼
  • Recognize the limitations of AI in predicting outcomes due to computational irreducibility.
    • ◼
  • Explore the use of AI in generating scientific narratives and identifying interesting scientific questions.
    • ◼
  • Investigate the potential of AI to assist in the formalization of science, particularly in areas previously considered less formal.

    • References

    Articles

    [SW1] Stephen Wolfram, "Can AI Solve Science?", (2024), Stephen Wolfram's writings.
    [SW2] Stephen Wolfram, "The New World of LLM Functions: Integrating LLM Technology into the Wolfram Language", (2023), Stephen Wolfram's writings.

    Notebooks

    [AAn1] Anton Antonov, "Workflows with LLM functions (in WL)", (2023), Wolfram Community.
    [AAn2] Anton Antonov, "LLM aids for processing of the first Carlson-Putin interview", (2024), Wolfram Community.
    [AAn3] Anton Antonov, "LLM aids for processing Putin's State-Of-The-Nation speech given on 2/29/2024", (2024), Wolfram Community.
    [AAn4] Anton Antonov, "LLM over Trump vs. Anderson: analysis of the slip opinion of the Supreme Court of the United States", (2024), Wolfram Community.
    [AAn5] Anton Antonov, "Markdown to Mathematica converter", (2023), Wolfram Community.
    [AAn6] Anton Antonov, "Monte-Carlo demo notebook conversion via LLMs", (2024), Wolfram Community.

    Packages, paclets

    [AAp1] Anton Antonov, "Markdown::Grammar" Raku package, (2022-2024), GitHub/antononcube.
    [AAp2] Anton Antonov, "MermaidJS" paclet, (2023), Wolfram Language Paclet Repository.

    CITE THIS NOTEBOOK

    Comprehension AI aids for Stephen Wolfram's article "Can AI Solve Science?"​
    by Anton Antonov​
    Wolfram Community, STAFF PICKS, March 8, 2024
    ​https://community.wolfram.com/groups/-/m/t/3137323