Wolfram Cloud Document

Creating word clouds of character names from works of fiction
by Christine Tsu

Introduction

Characters provide the driving force of works of fiction, adding a sense of relatability, personality, and emotion as well as forming a medium through which readers can experience the story. Using a program to identify character names in works of fiction can help reveal trends in the ways characters are written, and visualizations can enhance a reader’s understanding of and connection with the text. My project uses proper nouns and key words, such as "said," "asked," and "believed," to identify character names in works of fiction. It displays those names in a word cloud based on the frequency of each name and also creates a graph of the character relationships, with thicker edges indicating a closer relationship.

Identifying Proper Nouns

This specific code will work for any book entities that have the "Plaintext" property, but most of the code could easily be adapted to work for any plaintext that the user inputs. To begin finding the character names in a work of fiction, I first identified the proper nouns in the work.

Getting the plaintext of a book:

In[]:=

getBook[title_]:=EntityValue[Interpreter["Book"][title],"Plaintext"]

Identifying the proper nouns in a text:

In[]:=

getProperNouns[book_]:=TextCases[book,"ProperNoun"]//DeleteDuplicates

Testing it on Alice’s Adventures in Wonderland by Lewis Carroll:

getProperNouns[getBook["Alice's Adventures in Wonderland"]]

Out[]=

{Alice,White,ORANGE MARMALADE,Dinah,DRINK ME,II,English,Poor Alice,Duchess,Dear,O Mouse,William,Conqueror,Dodo,Lory,Eaglet,III,A CAUCUS-RACE,Ahem,Edwin,Morcar,Earls,Mercia,Northumbria,Stigand,Canterbury,Edgar Atheling,Prizes,Mine,Canary,White Rabbit,Mary Ann,Run,W.,Pat,Bill,Nay,Brandy,ADVICE FROM A CATERPILLAR,Caterpillar,Pigeon,Ugh,Serpent,VI,Fish-Footman,croquet,Frog-Footman,Queen,Footman,Cheshire-Cat,Cheshire-Cats,Cheshire-Puss,Cat,March Hare,Hatter,Dormouse,CROQUET,Miss,First,Kings,Queens,Knave,Hearts,King,Majesty,Herald,March,Pepper,ALICE}

Using Dialogue Tags

For the most part, only characters are able to speak, so dialogue tags (such as “he said” or “she exclaimed”) will help identify character names and distinguish them from other proper nouns (names of places, months, holidays, etc.).

I didn’t want the article “the” to stay in the text because it is not actually part of someone’s name. For example, I wanted to pull out “Hatter” instead of “the Hatter.” I split the text at whitespace and punctuation characters, which left me with a list of words and spaces that I could then use to identify the locations of dialogue words, and subsequently, the locations of character names.

Removing "the" and splitting the plaintext at whitespace and punctuation characters:

In[]:=

getText[title_]:=StringSplit[StringReplace[getBook[title]," the "->" "],{Whitespace,PunctuationCharacter}];

I wanted to catch two-word names, such as "Mary Jones" and "Ms. Jones," as well as one-word names. However, names with abbreviations, like Ms. Jones, will now appear with a blank space between the abbreviation and the last name. To format all two-word names in the same way, I removed the space between “Mrs” and “Jones” in the list.

Example of splitting a string with "Mary Jones" and "Ms. Jones":

splitString=StringSplit["Mary Jones and Ms. Jones",{Whitespace,PunctuationCharacter}]

Out[]=

{Mary,Jones,and,Ms,,Jones}

Deleting the space after an abbreviation:

In[]:=

Delete[splitString,Position[splitString,"Ms"]+1]

Out[]=

{Mary,Jones,and,Ms,Jones}

I applied this logic to create a function that will do the same for the entire text.

A function that deletes the space after abbreviations:

In[]:=

shortenAbbreviations[text_]:=Module[{abbreviationIndices=Position[text,"Mrs"|"Mr"|"Ms"|"Dr"|"Mx"]},Delete[text,abbreviationIndices+1]];

Next, I needed to find the indices of dialogue words (like "said," "demanded," "exclaimed," etc.). I caught two-word names as well as one-word names by checking if there was an uppercase word located at an index either two above or two below the index of the dialogue word. If not, I just took the single words on either side of the speech word.

A function that finds the indices of speech words and puts the names around them into a list:

In[]:=

findSpeechWords[text_]:=ModulespeechIndices=FlattenPositiontext,

Speech Words

,speechWords,speechWords=Flatten[Table[Which[StringStartsQ[text[[speechIndices[[i]]+2]],ToUpperCase[Alphabet[]]]&&StringStartsQ[text[[speechIndices[[i]]+1]],ToUpperCase[Alphabet[]]],StringJoin[text[[speechIndices[[i]]+1]]," ",text[[speechIndices[[i]]+2]]],StringStartsQ[text[[speechIndices[[i]]-2]],ToUpperCase[Alphabet[]]]&&StringStartsQ[text[[speechIndices[[i]]-1]],ToUpperCase[Alphabet[]]],StringJoin[text[[speechIndices[[i]]-2]]," ",text[[speechIndices[[i]]-1]]],True,{text[[speechIndices[[i]]-1]],text[[speechIndices[[i]]+1]]}],{i,Length[speechIndices]}]]//DeleteDuplicates

Since I removed the periods after abbreviations like "Ms.," I put them back in by adding a line of code to the end of the function above.

A function that finds the indices of speech words, puts the words around them into a list, and adds the periods back to abbreviations:

In[]:=

findSpeechWords[text_]:=ModulespeechIndices=FlattenPositiontext,

Speech Words

Testing the function on Alice’s Adventures in Wonderland:

In[]:=

findSpeechWords[shortenAbbreviations[getText["Alice's Adventures in Wonderland"]]]

Out[]=

{,Alice,she,and,anxiously,this,these,it,nothing,Mouse,Lory,Duck,rather,Dodo,Eaglet,voices,in,Fury,to,cunning,voice,hear,now,master,you,Caterpillar,very,hastily,had,Pigeon,Footman,two,Duchess,She,last,Cat,they,out,he,was,aloud,March Hare,Dormouse,Hatter,no,Seven,Queen,severely,a,Rabbit,Somebody,White Rabbit,King,cook,who,Knave,her}

Using Other Key Verbs

Aside from dialogue tags, I also used other verbs that tend to be exclusively used for characters (like "thought" or "believed"). The same method can be used to create a list of words that appear before these verbs.

A function that creates a list of words that appear before specific verbs:

In[]:=

findOtherVerbs[text_]:=Module[{verbIndices=Flatten[Position[text,"bought"|"buys"|"heard"|"hears"|"thought"|"thinks"|"sat"|"sits"|"believed"|"believes"]],verbWords},verbWords=DeleteCases[Flatten[Table[Which[StringStartsQ[text[[verbIndices[[i]]-2]],ToUpperCase[Alphabet[]]]&&StringStartsQ[text[[verbIndices[[i]]-1]],ToUpperCase[Alphabet[]]],StringJoin[text[[verbIndices[[i]]-2]]," ",text[[verbIndices[[i]]-1]]],StringStartsQ[text[[verbIndices[[i]]-1]],ToUpperCase[Alphabet[]]],text[[verbIndices[[i]]-1]]],{i,Length[verbIndices]}]],Null]//DeleteDuplicates;verbWords=If[StringContainsQ[#,"Mrs"|"Mr"|"Ms"|"Dr"|"Mx"],StringReplace[#," "->". "],#]&/@verbWords];

Testing the function on Pride and Prejudice:

In[]:=

findOtherVerbs[shortenAbbreviations[getText["Pride and Prejudice"]]]

Out[]=

{Mr. Bingley,I,Elizabeth,Mrs. Hurst,He,Maria,Mrs. Collins,You,Lydia,Mrs. Gardiner,Mr. Bennet,Colonel Forster,Jane,She,Mrs. Bennet}

Removing Duplicates

From the work above, I gathered a list of proper nouns, words from dialogue tags, and words from key verbs. I first combined the words from dialogue tags and words from key verbs to catch as many character names as possible. Then, I found the intersection of this new list with the list of proper nouns.

A function that uses all three lists to create a new list:

In[]:=

getIntersection[wordList1_,wordList2_,properNounList_]:=Flatten[Intersection[properNounList,Union[wordList1,wordList2]]];

Testing it on The Secret Garden:

In[]:=

shortenedText=shortenAbbreviations[getText["The Secret Garden"]];getIntersection[findSpeechWords[shortenedText],findOtherVerbs[shortenedText],getProperNouns[getBook["The Secret Garden"]]]

Out[]=

{Archibald Craven,Basil,Ben,Ben Weatherstaff,Colin,Colonel McGrew,Dickon,Dr. Craven,Martha,Mary,Master Craven,Mem Sahib,Mester Colin,Miss Mary,Mistress Mary,Mother,Mr. Craven,Mr. Roach,Mrs. Medlock,Mrs. Sowerby,Nut,Rajah,Roach,robin,Susan Sowerby,Weatherstaff}

There is some overlap here. For instance, "Mary," "Mary Lennox," "Miss Mary," and "Mistress Mary" all refer to the same person. To remove these duplicates, I first separated the names into a list of one-word names and a list of two-word names.

Separating a list of character names into one-word and two-word names:

I went through each item in the list of twoWordNames and checked if part of it was already included in oneWordNames (for example, "Miss Mary" includes "Mary"). Then, I collected each group of duplicates in a list.

Collecting the groups of duplicate characters in a list:

I started to create a list of all of the characters by first adding the characters that were not included in the duplicate list.

Collecting the character names that are not duplicates:

I chose the shortest item from each list of duplicate characters (for example, “Mary” instead of “Miss Mary”) and added it to the list of all of the characters.

Add the shorter name from each of the duplicate character lists to the list of all characters:

Organize the list and delete duplicates:

Putting It All Together

I wrapped all of the steps above in a function that removes duplicates in a list of characters. One issue is that the function isn't able to tell if names like "Mrs. Sowerby" and "Susan Sowerby" refer to the same person or not. I decided to leave the duplicates in rather than only including one name that has "Sowerby" in it because it is possible that two people have the same last name (for instance, if they are related to each other).

A function that removes duplicates in a list of characters:

Creating the Final List

In my final character list, I decided not to include words that were only prepositions, determiners, interjections, and pronouns, all of which would most likely not be character names. In case some character names had different capitalization styles, I also removed duplicates after changing the entire list to lower case. I reformatted the list after that to make sure that the capitalization style was correct again.

A function that cleans the final character list:

Testing the function on The Secret Garden:

Creating a Word Cloud of the Character Names

I used a word cloud to display the character names. Since character names that appear a greater number of times in the text are larger in the word cloud, it provides a visual of the importance of different characters.

First-person narratives present a challenge because the narrator's name may not appear. For instance, if Mary was the narrator of The Secret Garden, it is unlikely that "said Mary" or "Mary thought" would appear in the text. As a temporary work-around for this, I decided to create a function that tests if the work was written in the first person. If it is, a caption under the word cloud will let the user know that the narrator's name may not appear in the cloud.

Testing If the Work Was Written in the First Person

My function first deletes the dialogue in the work of fiction. This way, the program will not have to sift through dialogue that may contain the word "I." Next, the function counts the number of times that the word "I" appears in the work (not including the dialogue) and divides this number by the total word count to come up with a percentage. Through testing, I found that if the percentage is over 1.5%, the work of fiction tends to be in the first person.

A function that tests if the work of fiction as written in the first person:

Testing it on a book written in the first person (The Time Machine) and a book not written in the first person (The Secret Garden):

Displaying the Word Cloud

I wrote a function that uses the final list of characters to create a word cloud based on how often each character name appeared in the text. The word cloud also includes a caption below it, which will display the first-person message above if the work was written in the first person.

A function that creates a word cloud of the character names:

Testing it on The Secret Garden:

Creating a Graph of the Character Relationships

My graph provides a visualization of how strong/weak character relationships are by correlating edge thickness with how often two character names appear together. Counting the number of times two character names appear in the same chapter would be a good way to measure this. However, different texts have different formatting for chapters, so it would be more complex to separate different books by chapter. I instead decided to count how many times two character names appear in the same 30-paragraph section.

A function that separates the plaintext into 30-paragraph sections:

Counting How Many Times Two Character Names Appear in the Same Section

Creating a list of character names for each section:

Placing the lists of character names that include more than one name in a new list:

Count the number of times two specific character names appear in a sublist:

I created a function to do the three steps above.

A function that counts how many times two characters appear in the same section:

Testing it on The Secret Garden:

Converting the Counts to Edge Weights for the Graph

I wanted to scale the edge weights of my graph so that character relationships with greater counts would have larger weights. I decided to standardize the scale by setting the maximum value to 7 and adjusting the other values accordingly.

A function that converts the counts to a format that will set the edge weights of the graph:

Testing it on The Secret Garden:

Creating the Graph

I created a graph using the paired character names. Then, I set the weight of the edges to the association created above and linked those edge weights to the thickness of the edges. Like the word cloud, the graph will also include a caption below it that displays the first-person message if the work was written in the first person.

A function to create a graph with edge thicknesses corresponding to closeness of character relationships:

A function that puts all of the steps above into a single function:

Testing it on The Secret Garden:

Putting Everything Together

I wanted to make the program easy to use by placing everything into a function that takes the title of the work of fiction and either "word cloud" or "graph" and returns the word cloud or graph for that work of fiction.

The final function that uses all of the steps above:

Testing it on different works of fiction:

Conclusion

I found that my program identified character names very effectively for some works of fiction, such as Alice’s Adventures in Wonderland, which had distinct character names and was written in the third person. However, there were a few types of cases that introduced error. General names, like “Wicked Witch” from The Wonderful Wizard of Oz, were not picked up by the list of proper nouns. Characters that were not explicitly named, such as the maid from The Strange Case of Dr. Jekyll and Mr. Hyde, were also not picked up. Additionally, when two characters had similar names, the program sometimes only identified one. For instance, the Red Queen and the White Queen from Through the Looking-Glass were both referred to as “the Queen.” This caused my program to only keep “Queen” and delete “Red Queen” and “White Queen,” which were considered duplicates.

In the future, I would like to improve the accuracy of the program so that it doesn’t miss too many character names and doesn’t add erroneous ones. Adding more verbs to my list of key verbs (without introducing incorrect names) will enhance my program’s ability to pick up minor characters’ names and allow it to adapt more easily to a wide range of writing styles.

In the case of first-person narratives, the program had trouble identifying the narrator’s name. Their name would sometimes be mentioned in dialogue, but because characters speak in different ways, it was difficult to find general patterns that could be used to identify the narrator’s name. I would like to spend more time studying this so that the program will be able to consistently pick up narrators’ names in the future.

Finally, the graph of character relationships worked best when there were just a few central characters and a small number of minor characters. However, in some works of fiction, like Pride and Prejudice, there were a lot of character names, and many of them appeared together very frequently. This caused their graphs to seem very disorganized. There was also only a little contrast in the edge thicknesses of the graph, which meant it was more difficult to compare the strengths of character relationships. I would like to work on refining these graphs to make them more accurate and clear, hopefully with strong contrasts between close relationships and distant ones.