In his article ‘Generative AI Space and the Mental Imagery of Alien Minds’ , Stephen writes that concepts can be thought of as islands in a large space “and between these islands there lie huge expanses of what we might call ‘inter-concept space’”. My project focuses on exploring these spaces in a linguistic context, essentially investigating that if we consider two pieces of text as points in space, what lies between them- and does it hold any meaning? The inspiration for the project is the process of ‘text diffusion’, wherein a clean text is corrupted and a model is trained to return the original text when provided a corrupted one. Through my investigation, I found that while it’s quite hard to obtain the text that lies in the exact space between two other texts, it is possible to manually combine the texts in different combinations- and in that way understand what lies between them.

Introduction

The general aim of my research was to explore the space that is created from blending two different pieces of text. The investigation began with Semantic Search Indices, which I used to assign vectors to all pieces of writing I worked with. I began with the simplest way of creating inter-concept space, by blending the vectors embeddings of two paragraphs from an article using weighted sum to create a new one. An existing paragraph that corresponded most closely to the new paragraph found using the Nearest Neighbours function became the foundation of generating new text, which was done in 3 ways. I applied Markov Chain on the existing paragraph- which uses n-grams to generate text based on the word pool it’s given. LLM calls were applied on both the existing paragraph and the Markov generated text to rephrase them, thereby producing more ‘original’ texts.’ After initially using paragraphs as my concepts, I also repeated the investigation with sentences to expand the universe and allow for better and closer nearest neighbours. Additionally, I tried feature extraction, wherein I extracted sentiment, topics and structure from the matched items and only provided those to the LLM for more ‘original’ final text. In the second phase of the project, I used the neural network BERT and relied on ability to interpret. Instead of creating a search index, I created vector embeddings within BERT, and allowed it to find nearest neighbours based on its own understandings. This was attempted with both single words and paragraphs as the concepts.

Setup

Here you can get the Paclet ‘LinguisticInterconcepts’ I created with all the functions developed in the project:
In[]:=
CloudGet@CloudObject[
https://www.wolframcloud.com/obj/dagarwal14/WSRP25/LinguisticInterconcepts.wl
]
These are the functions it consists of:
In[]:=
ResourceFunction["GridTableForm"][{#,Information[#,"Usage"]}&/@Names["LinguisticInterconcepts`*"],TableHeadings->{"Function name","Description"}]
Out[]=
#
Function name
Description
1
BlendVector
BlendVector[index, y] blends semantic vectors in the index.
2
GenerateMarkovText
GenerateMarkovText[text, maxSentences] generates new Markov-style text.
3
LLMRephraseByFeatures
LLM rephrase by sentence features.
4
MakeIndex
MakeIndex[text_String, opts] creates index from text.
5
MakeNGramTrie
MakeNGramTrie[text_String, ngram_Integer] generates new Markov-style text.
6
RephraseAndShow
RephraseAndShow[text] rephrases text using LLM and prints both versions.
7
SentencePOS
SentencePOS[s] gives a list of POS rules for the given sentence.

What is a concept?

The objective of exploring inter-concept space raises the question of what a concept even constitutes. The Oxford Languages dictionary describes the noun “concept” as ‘An abstract idea; a general notion.’ Interestingly enough, in the philosophical sense the dictionary describes it as ‘an idea or mental picture of a group or class of objects formed by combining all their aspects’. By this definition, a paragraph or a sentence is actually a better representation of a concept than a word. When discussing inter-concept space in the context of visuals, a concept is considered to be an image- hence image diffusion is considered an accurate exploration of the same. My argument is that an image in its simple sense is a collection of pixels- the same way a sentence or paragraph is a collection of words. A word only holds any meaning when defined through a sentence or paragraph- which means it can be said that all these forms of text are simply different ways of representing the same concept. Subsequently, I believe that is it acceptable that the linguistic analog of inter-concept space or ‘text diffusion’ can in fact be explored through paragraphs/sentences. Some might argue that pixels do not inherently hold any meaning of their own- while words do. This is true. The ultimate idea is that words too are concepts, but so are a collection of words.

Semantic Search Index

It is essential (and convenient and fast) to begin with creating an index, as it allows us to store the data in the desired form, in this case paragraphs and create an easily searchable dataset. More importantly, creating an index assigns vector embeddings to each data item, basically represents them as numbers.
Get the text:
In[]:=
text=Import["https://writings.stephenwolfram.com/2023/07/generative-ai-space-and-the-mental-imagery-of-alien-minds/","Plaintext"];
Split into paragraphs:
In[]:=
paragraphs=Select[StringTrim/@StringSplit[text,RegularExpression["\\n\\h*\\n"]],StringLength[#]>0&];​​Length[paragraphs]
Out[]=
103
Create a semantic search index with the paragraphs:
In[]:=
index=CreateSemanticSearchIndex[paragraphs];
Assign a variable to hold the vector embeddings:
In[]:=
embeddings=Normal[index["Embeddings"]];​​embeddings//Length
Out[]=
124

Vector blending

The merging of any two paragraphs can easily be done using weighted sum, because all the paragraphs have been represented as a set on numbers. If our dataset included all sentences ever written, nearest neighbour would provide us with an exact match to the new vector. Of course since that isn’t possible, we have reduced our ‘universe’ to Stephen’s article in this investigation.
Map the vector embeddings to the indices using a nearest neighbours function :
In[]:=
nfVecs=Nearest[embeddings->"Index"];
Randomly pick two vectors from the embeddings:
In[]:=
V3=RandomChoice[embeddings];​​V4=RandomChoice[embeddings];
Blend and create a new vector using weighted sum (an interpolation weight value of 0.5 evenly combines both vectors):
In[]:=
y=0.5;​​Vnew=V3+y*V4;​​Vnew//Length
Out[]=
384
Extract the index positions of both original vectors:
In[]:=
posV3=nfVecs[V3]//First​​posV4=nfVecs[V4]//First
Out[]=
59
Out[]=
62
We now use nearest neighbours function to find existing paragraphs in the text that correspond most closely to the new bended vector, excluding the original vectors themselves:
In[]:=
nns=Complement[nfVecs[Vnew,5],{posV3,posV4}]
Out[]=
{55,64,66}
In[]:=
index["Items"][[nns//First]]
Out[]=
Minds in Rulial Space We can think of what we’ve done so far as exploring some of the “natural history” of what’s out there in generative AI space—or as providing a small taste of at least one approximation to the kind of mental imagery one might encounter in alien minds. But how does this fit into a more general picture of alien minds and what they might be like? With the concept of the ruliad we finally have a principled way to talk about alien minds —at least at a theoretical level. And the key point is that any alien mind—or, for that matter, any mind—can be thought of as “observing” or sampling the ruliad from its own particular point of view, or in effect, its own position in rulial space.
View the original paragraphs themselves for comparison:
In[]:=
index["Items"][[posV3]]
Out[]=
It’s a nontrivial fact of physics that “pure motion” in physical space is possible ; in other words, that an “object” can be moved “without change” from one place in physical space to another. And now, in a sense, we’re asking about pure motion in rulial space : can we move something “without change” from one mind at one place in rulial space to another mind at another place? In physical space, things like particles—as well as things like black holes—are the fundamental elements that are imagined to move without change. So what’s now the analog in rulial space? It seems to be concepts—as often, for example, represented by words.
In[]:=
index["Items"][[posV4]]
Out[]=
In a sense the whole arc of the intellectual development of our civilization can be thought of as corresponding to an expansion in rulial space: with us progressively being able to think in new ways, and about new things. And as we expand in rulial space, we are in effect encompassing more of what we previously would have had to consider the domain of an alien mind. When we look at images produced by generative AI away from the specifics of human experience—say in interconcept space, or with modified rules of generation—we may at first be able to make little from them. Like inkblots or arrangements of stars we’ll often find ourselves wanting to say that what we see looks like this or that thing we know.
The BlendVector function in the package can simply be provided with the index variable and an interpolation weight value to return the original and new paragraphs:
BlendVector[index_,y_:0.5]

Markov Chain Text Generation

Prefix trees and Markov Chain work together to analyze a text, understand how frequently the words appear in relation to each other and then generate new text based on these findings. Since the nearest neighbour paragraphs aren’t exact matches to the blended vectors, Markov Chain will be used to generate more ‘original’ text. An existing paclet ‘TriesWithFrequencuies’ was edited to create functions for the LinguisticInterconcepts paclet.
​
Given the general larger text as a and a n-gram integer, the function MakeNGramTrie generates a prefix tree with predictions about the text flow:
In[]:=
tr=MakeNGramTrie[text,3];​​LeafCount[tr]
Out[]=
34311
The GenerateMarkovText function is then given the prefix tree, desired number of sentences and starting seed words to generate new text:
In[]:=
finaltext=GenerateMarkovText[tr,2,{"mental","imagery"}]
Out[]=
Mental imagery of alien” neural net like a human brain suffers a stroke in a very recognizable“ real-world-inspired” images— perhaps like a way to 3. Here are not anatomically correct) line from an array of possible images, essentially mathematical pattern.

Applying LLM Calls

Since neither the nearest neighbour paragraph or Markov Chain generated text are entirely original, LLMs can be used for rephrasing . This is particularly essential for the Markov Chain text as it is syntactically correct but essentially meaningless.
We create a function that uses the LLMFunction and add the specific command we want to implement:
In[]:=
rephraseLLM=LLMFunction["Rephrase the following paragraph to make it more clear and concise:\n\n```text\n`1`\n```"];
Then we apply the function on any text we have generated.
Nearest existing paragraph to blended vector:
In[]:=
new=index["Items"][[nns//First]];​​rephrased=rephraseLLM[new]
Out[]=
**Minds in Rulial Space** So far, we've explored the "natural history" of generative AI, offering a glimpse into the mental imagery that might exist in alien minds. But how does this relate to our understanding of these minds? The concept of the ruliad provides a theoretical framework for discussing alien minds. Essentially, any mind—alien or otherwise—can be viewed as "observing" or sampling the ruliad from its unique perspective within rulial space.
Markov chain text:
In[]:=
rephrasedmarkov=rephraseLLM[finaltext];
Both original and rephrased texts are then outputted with corresponding titles:
In[]:=
Grid[{​​{"Original Paragraph:",new},​​{"Rephrased Paragraph:",rephrased},​​{"Original Markov chain text:",finaltext},​​{"Rephrased Markov chain text:",rephrasedmarkov}},Dividers->All,Alignment->Left]

Larger Item Pool

Repeating the investigation with the same article but distributing it into sentences essentially allows us to expand our universe. The availability of more items in the index and fewer ideas in one item (a sentence) can significantly improve the accuracy of the nearest neighbours matched with the blended vector. There is a much narrower scope to misinterpret the main context and miss significant words in a sentence when compared to a paragraph.
The ‘LinguisticInterconcepts’ paclet can be used to repeat the experiment with a larger item pool- ie. sentences.
OR

Feature Extraction

The ultimate goal of text diffusion is to produce an original text that is a blend of the two input texts. While vector embeddings made it easy to blend the texts, the difficult part has been producing the new text- since our ‘universe’ is too small to find an entirely accurate nearest neighbour. One method to produce text that is both more original and closely aligned to the blended vector is to provide a LLM with the text features, instead of the text itself. This allows for the idea and meaning of the text to be captured without simply rephrasing it.
We extract the topic and sentiment of the Markov chain generated text:
To extract the parts of speech in the text we use the SentencePOS function from the paclet:
We provide the LLMRephraseByFeatures function in the paclet with the extracted features (in the following order) to generate the final text:
Remark: Feature extraction can also be performed on the matched item itself, instead of generating Markov chain text first.

Using BERT as an LLM

An important realization at this point in the project was that LLMs such as BERT which we were using for the semantic search index already have knowledge and understanding of their own. Hence it’s unnecessary to create a search index and limit the LLM to the text it can search and use. Instead, we can simply provide the neural network with a narrower universe and rely on its ability to interpret it.

Sentence embedding in BERT

Obtain a custom sentence embedding network that converts text strings into sentence vectors by appending a pooling layer to the pretrained BERT model:
Each sentence vector has 768 dimensions :

Single words as concepts

Here, we are using animal names as the dataset, as they are easy for the network to understand interpret.
Obtain the data of animal names:
Create vector embeddings for the data:
Map the vector embeddings to the indices using a nearest neighbours function :
Test the function by finding the nearest neighbours for a single animal name:

Merging words

Use weighted sum to create a blended vector of two animal names:
Remark: We do not filter out the blended concept-words in order to be able to visually verify the correctness of the nearest neighbor finding -- the blended concept-words should be in the result by Nearest. (See the commented out code for how such filtering can be done.)

Including the definitions

In the beginning I discussed how paragraphs and words are essentially just different ways of representing the same concept. Adhering to that idea, the inter-concept space of animals can also be explored using definitions of the animals.
Import the animal dictionary data:
This is what the data looks like:
The data is now searchable by animal name :
We can not use the Wolfram Function Repository BERT function “SentenceBERTEmbedding” for sentence embedding as it uses HTTPS call to HaggingFace which is a slow process and gives HTTP Error 429:
Create vector embeddings and map them to the indices using nearest neighbours function :
Blend the vectors using weighted sum :
Obtain the nearest neighbours:

Conclusion

This project set out to approximate the “space between” two texts by blending their vector embeddings and retrieving meaningful intermediates using nearest neighbour searches. By applying Markov Chains and LLM-based rephrasing, I generated variations that reflected shared semantic features of the original inputs. Moreover, the use of different scales (paragraphs versus sentences) and different embedding approaches (semantic indices versus BERT-based representations) underscored the importance of granularity and model interpretability in such tasks. This suggests that exploring inter-concept space is not only about finding mathematical averages of meaning, but also about understanding the linguistic structures and contexts that shape semantic transitions. Although I did not implement true text diffusion—which would involve training a model to recover original text from noisy variants—this work demonstrates that combining embeddings and guided generation can produce coherent, interpretable transitions between concepts.

Limitations and Future Plans

The major limitations in the project were that I was not able to program the actual process of text diffusion, and hence unable to find text that corresponds most closely to the blended vectors.
One of the most common models for text diffusion is corrupting and adding noise to text and then training a model to output the original text when given a noisy version.
I want to attempt this using prefix trees and Markov Chain and creating a database with innumerous noisy variations of all sentences in a large text. Then I will train a neural network with the data to be able to return the original sentence when given any variation. Due to the high levels of computing power and the extended amount of time that this extension of the project would take, it’s something I want to attempt in the future.
Here is a description of an alternative much simpler algorithm for iterative refinement of word sequences that is a type of text diffusion.
2
.
Randomly select words from the descriptions of concepts-to-be-merged
6
.
2
.
Go to step 3
7
.
1
.
Otherwise go to step 3
Other text diffusion procedures using specially trained neural network for the refinement step. (Step 3 above.)

Acknowledgements

I would like to sincerely thank my mentor Anton Antonov for not only guiding me with every possible aspect and step of this project, but being just as deeply invested as me throughout. I also want to thank the program director, Megan for greatly assisting me in both personal and project-related contexts. Lastly, I’d like to thank WSRP for providing me this opportunity to explore my interest in computational thinking and greatly deepen my understanding.

References

Articles

[SW1] Stephen Wolfram, “Generative AI Space and the Mental Imagery of Alien Minds”, (2023), Stephen Wolfram Writings.
[M1] Charan H U, “Diffusion Language Models (DLMs): A New Frontier in Text Generation”, (2024), Medium.

Notebooks

[AAn1] Anton Antonov, “Using Prefix trees for Markov chain text generation”, (2023), Wolfram Community.
[AAn2] Anton Antonov, “Rorschach mask animations projected over 3D surfaces”, (2022), Wolfram Community.

Functions, paclets, other tools

[AAp1] Anton Antonov, "TriesWithFrequencies", (2023), Wolfram Language Paclet Repository.
[WNNR1] “Bert”, Wolfram Neural Net Repository.

Data

[AZA1] "All Animals A-Z List"
[B4A] “List of Animal Names”

CITE THIS NOTEBOOK

What is the linguistic analog of Inter-concept space?​
by Delisha Agarwal​
Wolfram Community, STAFF PICKS, July 10, 2025
​https://community.wolfram.com/groups/-/m/t/3499159