The Wolfram Emerging Leaders Program is a 4 month long project based program designed for gifted high school students to take a deep dive into a topic of their choice. Students work remotely in small groups to take their project from ideation to completion, with the guidance of Wolfram experts. Wolfram Emerging Leaders Program participants are selected from the Wolfram High School Summer Camp, which is open to talented, STEM oriented students age 17 and under.
This essay was written by:
This essay was written by:
Abstract
Abstract
Many figures of speech are common expressions in natural language, but not all are. Humans can create new figures of speech on the fly that others can understand. We wanted to see whether we could computationally generate figures of speech that would make sense to humans. We focused on similes, figures of speech that compare unlike things, typically using the words “like” or “as,” because they have predictable structures. The end result is a cloud application where users can input a body of text containing similes and get the text back with new similes with similar meanings to the original ones.
Introduction
Introduction
Have you ever looked at your English paper and thought that it need a little spice? Are you a teacher and want to show your students some creative similes they haven’t seen before? Or are you just a writer looking for some inspiration? Well, we have the solution for you!
We created a function where one can input a text and output replaced similes with the same general meaning. The general process of the function is extracting the similes from the text, replacing the similes, and then putting the replaced similes back into the text. The text can be in the form of a poem, a paragraph, or an excerpt from your favorite book.
We embarked on this perilous journey for a singular purpose: to prove that it could be done. We successfully created a website using a cloud form to make it easy for anybody to use!
While one may use this just for fun or to spice up a bland poem, this could also have broader applications for those learning the English language. By diving deeper into the computational aspects of a more traditionally creative process, we could expand this in the future to other figures of speech and creative writing techniques.
The following paragraph is used several times in this post to demonstrate intermediate steps:
In[]:=
text = "You are as ugly as a fly. As long as I can remember, you have been an atrocious orange.I must say that I am as stuffed as a turkey.As you sleep, think of this. We are as pretty as flowers. I am like the lustrous lavender.Yes, I am as beautiful as the wild hibiscus. While you are as funny and ugly as a cow, I am like George Washington, a wise general.";
We use this text because it contains multiple structures for sentences and similes. It also contains some phrases that are not similes but contain some of the same elements. This is to test whether our function can distinguish between similes and other phrases. Lastly, this sample text is humorous, especially if you, like us, have spent hours coding without a break and have gained an unconventional sense of humor throughout the course of your project.
Before blast-off: making text lowercase before replacement
Before blast-off: making text lowercase before replacement
In order to be able to extract similes and their adjectives and nouns, the text must be formatted uniformly. When adjectives and nouns are taken from the similes to be replaced, the proper nouns must not be replaced since they may hold specific meanings that cannot be replicated with a replacement. The rest of the text is placed in lower case in order for the pattern matching to work.
The formatting function properNounSaver decapitalizes the entire text except for proper nouns.
In[]:=
properNounSaver[text_]:=StringReplace[ToLowerCase[text], Thread[ToLowerCase[TextCases[text, "ProperNoun"]] TextCases[text, "ProperNoun"]]]
formattedText=properNounSaver[text]
Out[]=
you are as ugly as a fly. as long as i can remember, you have been an atrocious orange.i must say that i am as stuffed as a turkey.as you sleep, think of this. we are as pretty as flowers. i am like the lustrous lavender.yes, i am as beautiful as the wild hibiscus. while you are as funny and ugly as a cow, i am like George Washington, a wise general.
Let's find similes! Identifying and extracting similes
Let's find similes! Identifying and extracting similes
To replace similes in a text with new ones, we first must find the original similes. The two types of similes, "like" and "as" similes, have very different structural patterns, thus there are different functions for each pattern.
That's what I "Like"
That's what I "Like"
So, what are "like" similes? "Like" similes compare one thing to another using "like." One example is "You are like a horse." We need to take out "like a horse" from this sentence so that we can replace "horse" with a synonym. Coincidentally, "like a horse" is a prepositional phrase according to the function TextCases. A prepositional phrase is a phrase including an object ("you"), a preposition, ("like") and the thing you are comparing it to, ("a horse"). This allows the function takeLikeSimiles to be a nice and short function that is pretty flexible. Here are some more "like" similes:
Showing the structure of a prepositional phrase:
TextStructure["You are like a horse. I look like a deranged zebra."]
Out[]=
| |
| ||||||||||||||||||||||
Sentence |
It’s important to remember that not every use of like is a simile. For example, "I really like horses" is not a simile.
"I really like horses" does not contain a prepositional phrase because "like" is being used as a verb, not a preposition.
TextStructure["I really like horses."]
Out[]=
| ||||||||||||||||||
Sentence |
takeLikeSimiles outputs a list of "like" similes from the word "like" to the end of the phrase:
In[]:=
takeLikeSimiles[text_]:=Module[{prep, wordsinprep, nodoubleprep, reSentSimile, takelike }, prep = TextCases[text, "PrepositionalPhrase"];wordsinprep = TextCases[prep, "Word"];nodoubleprep = DeleteDuplicates[wordsinprep, SubsetQ];reSentSimile = StringRiffle/@nodoubleprep;takelike = reSentSimile[[First/@Position[nodoubleprep, "like"]]];takelike];
takeLikeSimiles[formattedText]
Out[]=
{like the lustrous lavender,like George Washington a wise general}
"As" you wish
"As" you wish
"As" similes are a bit different from "like" similes. The structures of "as" similes are not as grammatically uniform as the structures of "like" similes. "As" similes can contain prepositional phrases, verb phrases, noun phrases, etc. Thus, we can not analyze their grammatical structure using TextCases.
Here are two examples of "as" similes. Note the different parts of speech that appear in each:
TextStructure["Horses are as cool as cake. Joseph is as slimy and red as a vivacious speaker."]
Out[]=
| |
| ||||||||||||||||||||||||||||||||||
Sentence |
But, all hope is not lost. Unlike "like" similes, "as" similes are extremely structured (for better or for worse). "As" similes have the general structure as follows: [subject] is/are/ext. "as" __ "as" __.
Sadly, this means that every possible scenario of "as" similes needs to be written, but with the help of sequences, we make it work. Many of these structures rely on determiners. Determiners are words like "a," "the," and "an."
Sadly, this means that every possible scenario of "as" similes needs to be written, but with the help of sequences, we make it work. Many of these structures rely on determiners. Determiners are words like "a," "the," and "an."
Taking "as" similes that have determiners:
In[]:=
findASimiles[text_]:=Flatten/@ SequenceCasesPartition[StringSplit[text],1], {{"as"},_,{"as"},{"a"},_} {{"as"},_,{"as"},{"an"},_} {{"as"},_,{"as"},{"the"},_} {{"as"},_,{"and"},_,{"as"},{"a"},_}{{"as"},_,{"and"},_,{"as"},{"an"},_} {{"as"},_,{"and"},_,{"as"},{"the"},_}{{"as"},_,{"and"},{"as"},_,{"as"},{"a"},_} {{"as"},_,{"and"},{"as"},_,{"as"},{"an"},_}{{"as"},_,{"and"},{"as"},_,{"as"},{"the"},_};
findASimiles[formattedText]
Out[]=
{{as,ugly,as,a,fly.},{as,stuffed,as,a,turkey.},{as,beautiful,as,the,wild},{as,funny,and,ugly,as,a,cow,}}
Sometimes a determiner isn't present before the noun, usually because the noun is plural. In such cases, a sequence with a more general form that ends with any word following the second "as" is used to extract the similes. However, since this sequence will also take parts of the sequences found by findASimiles, the findPluralSimiles function deletes all cases that were already found by findASimiles.
Taking "as" similes that don't have determiners :
In[]:=
findPluralSimiles[text_]:=Flatten/@ DeleteCasesSequenceCases Partition[StringSplit[text],1], {{"as"},_,{"as"},_}{{"as"},_,{"and"},_,{"as"},_}{{"as"},_,{"and"},{"as"},_,{"as"},_}, (* Cases to delete *) {{"as"},_,{"and"},_,{"as"},{"the"}} {{"as"},_,{"and"},_,{"as"},{"an"}} {{"as"},_,{"and"},_,{"as"},{"a"}} {{"as"},_,{"and"},{"as"},_,{"as"},{"a"}} {{"as"},_,{"and"},{"as"},_,{"as"},{"an"}} {{"as"},_,{"and"},{"as"},_,{"as"},{"the"}} {{"as"},_,{"as"},{"a"}} {{"as"},_,{"as"},{"the"}} {{"as"},_,{"as"},{"an"}};
At this point, a few of the similes taken may end in an adjective instead of a noun because the sequence cuts off one space too early. To correct this, findPluralAdjSim and findSingularAdjSim do essentially the same thing as the findPluralSimiles and findASimiles functions, except their sequences take an extra word after the original sequences, just in case an adjective directly precedes the noun in the simile.
findPluralAdjSim and findSingularAdjSim take the text and output the similes plus one word:
The function findAdjectiveSimiles joins the findPluralAdjSim and findSingularAdjSim lists and also appends {"quickly.", "quickly"} to the list. This prevents it from being the empty set and provides it with enough elements to allow later functions to run without throwing errors. The word "quickly" was used because, as an adverb, it is deleted from the later output lists and so doesn't interfere with the desired results.
Joining lists of similes and adding a filler list:
All of the lists in the last output are only supposed to contain words that are inside a simile. Unfortunately, there are extra words at the end of many of them. The function getAdjSimile eliminates lists where this is the case. It does this by deleting cases where the second to last word has punctuation after it, indicating that the simile likely ends before the last word that was included.
Removing the lists that have extraneous words:
Some similes that don't have a adjective directly before the noun still slip through the cracks, so simList ensures that the last word of the taken simile can be classified as a noun.
getOnlyAdjList removes similes that end in words that can be classified as pronouns or conjunctions.
To get the final complete list of "as" similes from the given text, the check function cuts off the last words in the similes that have an adjective directly before the noun. Then findsSimiles replaces those cut - short similes that are in the merged list of all the similes with the newly found similes that include both the noun and the adjective that comes directly before it. The output is a list of all the "as" similes in the text.
The function check cuts off the last words in the similes that have an adjective directly before the noun.
findsSimiles creates a final list of all the "as" similes in the text.
As you may have noticed, the phrase "as long as I can" makes it through this whole process despite the fact that it is not a simile (sigh). This is because it matches one of the possible sequence structures. It also goes through all the functions that are meant to find similes that have a adjective right before the noun because "can" may technically be classified as a noun.
Thankfully though, our other functions that take the adjectives and nouns from similes to replace them recognize that “can” is a verb and so the phrase itself is not treated like the other similes and remains unchanged.
Tying it all together
Tying it all together
The function findAllSimiles takes all of the previous functions that take the similes from the text and compile them into one large list so that it takes an input of the formatted text from properNounSaver and outputs a list of all the similes in the text.
Creating a final list of all similes:
Taking the nouns and adjectives out of the similes
Taking the nouns and adjectives out of the similes
Our replacement functions take a list of zero to two adjectives and a noun as input. To account for this adjustment in formatting, pairNounAdj groups each of the consecutive adjectives and nouns in the similes.
pairNounAdj outputs the pairs with many levels of lists and must be simplified. The following line of code is for quick formatting purposes.
What is the right word? Replacing nouns with and without context
What is the right word? Replacing nouns with and without context
After the nouns and adjectives in the similes are identified for replacement, substitutes must be found for those words. Sometimes, only a noun must be replaced, such as in the sentence "She is like a superhero."
Other times, the simile has a noun and one or more adjectives, such as in the sentence “The package was as light as a feather.” In the second case, the adjectives in the sentence provide context as to why something is being compared to the noun. This context is useful for finding a substitute noun. To handle the two cases, we wrote two functions: replaceNounWithContext and replaceNounWithoutContext.
As speedy as a . . . turtle? Replacing nouns with context
As speedy as a . . . turtle? Replacing nouns with context
When given a simile with the structure "As (adjective) as a (noun)", finding a substitute word for the noun won't be accurate if we simply replaced it with a synonym for the noun. This is because in order for the replacement to make sense, the replacement noun needs to match the meaning given by the adjective. For example, for the simile "as strong as a horse", a replacement noun such as "gorilla" would be sufficient, but when the adjective is changed to "speedy", nouns like "cheetah" or "car" would be more appropriate.
In order to generate possible replacement nouns that account for this given context, the Datamuse API was used. The Datamuse API was useful because of its large and diverse database of words and, more importantly, its word-finding query engine that can find synonyms of a word and rank the results in order of relatedness to a second word. The Datamuse API was used with the Import function to generate a list of possible replacement nouns for each adjective-noun pair in a simile, as seen below:
In order to generate possible replacement nouns that account for this given context, the Datamuse API was used. The Datamuse API was useful because of its large and diverse database of words and, more importantly, its word-finding query engine that can find synonyms of a word and rank the results in order of relatedness to a second word. The Datamuse API was used with the Import function to generate a list of possible replacement nouns for each adjective-noun pair in a simile, as seen below:
The find function takes in an adjective and a noun extracted from a simile, and then imports the DatamuseAPI to generate a list of nouns that are similar to the given noun and ranked in order to their relatedness to the given adjective:
This outputs a list of nouns that have some relation to the word "cookie", ranked by how relevant they are to the adjective "sweet." An example simile that this function would be used to replace the noun of is "You are as sweet as a cookie."
The way that Datamuse API orders these lists of nouns is by assigning each word a score of relatedness (which will be referred to as the API score). Originally, solely using the API score to generate the replacement noun produced a decent model, but there were many unrelated outliers that received a high API score and skewed the model's performance. So, the average concept weight of the word was also incorporated into the model to avoid these outliers, because we observed that the outliers tended to be uncommon and long words.
The avg function find the average concept weight of the word to give a rough measure of how frequently the word is used in a grammatical context.
This displays the average concept weight of the word "fish" and "eviscerated" in general.
The replace function uses this avg function along with the Datamuse API to create a list of possible replacement nouns given an adjective-noun pair ranked based on API score and average concept weight (gives importance to API score through a hyperparameter) and picks a random word from there.
The first part of the replace function displays the list of replacement nouns generated by the Datamuse API, with each word's relative positions in the list when ordered according to API score and concept weight. For example, as seen in the second element of the output, the word caramel appears second when the list is ranked according to API score, and thirteenth when ranked according to concept weight.
The second part of the replacement function as indicated by the comment below, combines the API score and concept weight into a new score that the final list of replacement nouns can be ordered on. This new score is a weighted average of the API score and the concept weight, with the popScore variable controlling the weighting of the concept weight score. Basically, the popScore determines the level of importance that the concept weight of a word in the list has in ranking them. A hyperparameter like popScore is necessary because the API score is much more important than the concept weight in ranking the best replacement words than the concept weight, which is accounted for solely to minimize outliers.
After some experimentation further described below, we determined that a popScore of 0.1295 displayed the best results for noun replacement. Check it out!
Optimization Nation
Optimization Nation
How did we determine the popScore to be 0.1295? To determine which value of popScore was the best, we developed a Manipulate function, nounGenerateTest, that graphed how well the list was ordered for each value of popScore (between 0 and 1).
nounGenerateTest takes in an adjective and its corresponding noun as well as a list of nouns (which will be referred to as the scoring list). The scoring list is a manually selected list of nouns that we decided would be the best replacement nouns for the given adjective noun pair. Using this list, we can create a new score as an indicator of how well the list is ordered. This new score would be the average position of all of the words in the scoring list, and because we wanted all of the best words to be ranked as highly as possible, a high average position indicates good ordering. Check out the Manipulate function below!
nounGenerateTest takes in an adjective and its corresponding noun as well as a list of nouns (which will be referred to as the scoring list). The scoring list is a manually selected list of nouns that we decided would be the best replacement nouns for the given adjective noun pair. Using this list, we can create a new score as an indicator of how well the list is ordered. This new score would be the average position of all of the words in the scoring list, and because we wanted all of the best words to be ranked as highly as possible, a high average position indicates good ordering. Check out the Manipulate function below!
This Manipulate function shows a slider that is used to alter popScore, the hyperparameter. As popScore is changed, you can see how the ordering of the list changes, as well as the average position seen below that. There is also an interactive graph that plots the average position of the best words vs. the popScore value. By sliding the popScore slider until it aligns with the lowest point in the graph, we can determine the best value of popScore for a given adjective-noun pair.
After many tests, the best value of popScore was determined to be 0.1295. Below are examples of some tests that we ran with specific adjective-noun pairs and scoring lists with their corresponding popScore value that yielded the best results.
Once the list of replacement nouns was optimized, the final function would take a random word from the list and output it as the replacement noun to be used. Experimentation with this random selection indicated that taking a random sample from the first 20 words in the optimized list when possible yields the best replacement noun.
replaceNounWithContext’s definition when given a list of strings:
replaceNounWithContext’s definition when given a list of three strings:
Dude, you gave me no context!
Dude, you gave me no context!
We found that we could use WordData and the ConceptWeight property of words to find reasonable substitutes for nouns that did not have adjectives associated with them in their similes. This is what we did in replaceNounWithoutContext. The function replaceNounWithoutContext uses two helper functions: findNounSpec and findSimilarNouns. The first, findNounSpec finds a relevant WordData word specification for a string.
The function findNounSpec takes in a noun and returns a relevant WordData word specification for it. This is necessary because words can have multiple senses, some of which are not even nouns. One example of this is the many senses of the word "light."
The word "light" can be a noun, an adjective, a verb, or an adverb, and it has a variety of meanings:
findNounSpec outputs a word specification that is a noun. It also tries to pick a relevant sense by using the WordData property ConceptWeight.
Using findNounSpec to find a word specification for "light" :
Here is findSimilarNouns. This function creates a list of word specifications sorted by ConceptWeight from which replaceNounWithoutContext can draw noun replacements for a particular noun. It is a helper function to replaceNounWithoutContext. It draws from Synonyms and NarrowerTerms. Synonyms gives a list of words with the same meaning as the original word. NarrowerTerms gives a list of words that describe a subset of the original word.
Narrower terms related to the word "bird" include several types of birds:
The function findSimilarNouns calls findNounSpec to use the word specification it returns to find similar nouns. If findNounSpec does not return a word specification in the first place, findSimilarNouns returns the original word.
Using findSimilarNouns to find nouns similar to "light" :
After finding the list of possible replacements with findSimilarNouns, replaceNounWithoutContext randomly chooses from a few of the substitutions with the highest ConceptWeights. It outputs a single word. It returns the original noun if findSimilarNouns does not find a replacement.
This often does not perform as well as replaceNounWithContext. However, it is capable of generating good replacements, or at least funny replacements.
Another example of replaceNounWithoutContext:
I am as happy, felicitous, glad, well-chosen
I am as happy, felicitous, glad, well-chosen
replaceAdjective uses WordData to find synonyms for an adjective and then picks one from a few with the highest concept weights. If it does not find any synonyms, it returns the original word.
Finding a synonym for "small":
With context or without context? That is the question
With context or without context? That is the question
The function replaceAdjectiveNoun takes a list that contains either only a noun or one/two adjectives and a noun as input. It returns a list of similar words to be used in the replacement simile. It does this by calling replaceAdjective and replaceNounWithContext or by calling replaceNounWithoutContext.
"Put that thing back where it came from, or so help me!" Putting the replaced adjectives and nouns back into the original text
"Put that thing back where it came from, or so help me!" Putting the replaced adjectives and nouns back into the original text
Now that there is a list of adjective and noun groups, the words need to be put back into the similes that they were taken from, and then those similes need to be put into the text. The adjective and noun groups can't be replaced directly into the text because then words outside the similes may be replaced.
replacedNAToSimile2 takes the replaced list of adjectives and nouns and threads replacements with it so that they can be placed back into the similes the adjectives and nouns came from.
Replacing nouns and adjectives into the similes:
When replacing the nouns and adjectives, a common issue is small grammar mistakes such as "an dog" or "a rats." The function grammarFix looks at the noun in the noun phrase to determine whether to use "a", "an", or to pluralize the noun. A future improvement with this function is to look at the word after the determiner ("a" or "an" or "the") instead of the noun.
Fixing grammatical errors within the new similes:
After the replaced nouns and adjectives have been put back into the similes, the replaced similes must go back into the original text.
putBackToText takes the replaced similes and puts it into the original text:
The final function
The final function
replaceSimiles
replaceSimiles
The final function, replaceSimiles, makes use of all of the previous functions and Module to wrap it all into one. It functions in the order that was explained above.
Partly cloudy: the cloud app
Partly cloudy: the cloud app
We deployed a simple form to the cloud where a user can type an input and get back a text with replaced similes. Here is an example of use:
This takes Cloud Credits, so we did not include a public link for you to use. If you would like to, feel free to try the web interface though with the code below.
Evaluate the rest of the code in this notebook and try this out:
Conclusion
Conclusion
From insult generator to simile simulator
From insult generator to simile simulator
In our initial code, we created a funny insult generator. It's pretty simple, but you get a real kick out of it. We didn't end up using it because it couldn't detect similes and the replacements are buggy, but it was the start of a great journey. Try out this code yourself and see what you come up with!
Looking back at our first lines of code, we see that we have come a long way. Below is the execution of the final functions in order as the appear in replaceSimiles so you can see exactly what's going on.
The first step was to format the text with properNounSaver. We had to include this because our code started to replace "George Washington" with "Curious George Harrison" since it all became lower case and thus was recognized as regular nouns.
Next, we had to find all the similes and replace the nouns and adjectives.
originalNA has what seems like is an overly complicated way to remove brackets. A problem we ran into was that Flatten flattened everything and got rid of all groupings, it also kept the empty spaces. We ended up having to use DeleteCases to get rid of the space, leveling nounAdj to get rid of the first brackets and then flattening once more to format correctly.
We then had to replace the adjectives using the simplified list and then put it back into the similes.
Last (but definitely not least) we had to correct grammar (or at least most of it), put the similes back into the paragraph, and double-check to make sure the last paragraph had correct capitalization.
Voila! We have our replacing function!
In our paragraph, you may notice that it takes out "as long as I can" even though it isn't a simile. This is because it follows the pattern of the "as" similes. This also happens sometimes with "like" similes and taking out phrases that aren't similes, but it doesn't ever end up replacing it because they don't have nouns or adjectives unless it is a simile. In the end, nothing gets replaced and it gets put back in the original text like nothing happened.
Look above in the walk-through of all the steps. originalSimiles includes "as long as I can", but in the final text it's normal!
Look above in the walk-through of all the steps. originalSimiles includes "as long as I can", but in the final text it's normal!
Reflection
Reflection
Wow. That was a lot. We feel like our brains are like french fries, or should we say that they are like french croutons? But it feels good. We learned a lot, including about parts of speech we have never heard of before (like a noun phrase) and patterns we never knew would pop up. Did you know that "like" similes were basically buff prepositional phrases? We didn't. Finding patterns with "as" similes was also very rewarding. Below is some of our original code for finding "as" similes. We had a function for every. Single. One. Luckily, we found out how to use the conditional Or symbol which greatly sped up our code.
We also learned the ins and outs of adjective and noun replacement. It was fascinating to see how many different definitions of the words there were. For example, the noun "light" in the sentence "You are as light as a sheet of paper." would replace to "You are as lightbulb as a sheet of paper." These would make sense in a different context, but it didn't quite work the way we intended.
Something that we tried but abandoned was using metaphors and other figures of speech. This was not only because of the time crunch, but also because lots of comparisons and figures of speech can stretch throughout whole chapters, texts, or even books! Even shorter excerpts of figures of speech are also much harder to computationally identify due to their abstractness, and it is very difficult to decide what you want to replace, even as a human. After contemplating what to use, we set forth on conquering the similes, and let's just say that we rocked it.
Our favorite part of our function is the magnificent insults that you can create. Seriously, just plug a few in! It's HILARIOUS!
Although our function is great at generating insults, it could also be a great tool for ESL teachers or teachers in general. ESL teachers can use this to create fun similes with new words for the students to learn. By expanding vocabulary through familiar and unfamiliar diction, students can expand their vocabulary with new words and uses for words that they haven't thought of before.
It could also be a great inspiration tool for writers. Maybe they can’t quite find what they are looking for in a comparison, but have a general idea of what they want. BOOM! Use this! Maybe they are stuck in a slump and need something to shake things up. BOOM! Use this! Maybe you are a casual writer aiming to make things a bit funny. BOOM! Use this!
There are so many uses for our function that we couldn’t possibly name them all, so go forth and have fun!