Textual Parsing deals with the task of splitting a sentence into smaller parts based on grammatical rules . The purpose of this project is to explore Textual Parsing on various types of text using Wolfram's TextStructure function to investigate the underlying statistics behind it and find some trends.
Introduction
Introduction
It is quite an understatement to say that there is text everywhere, because everything there is are described using text. Describing such text requires complex rules and structure which are dealt in the field of Linguistics. Natural Language Processing on the other hand, deals the same but in regards to how a piece of text is interpreted and worked upon by a Computer. Textual Parsing plays a vital role in such process as it helps to break texts into sub-components based upon grammatical rules which provide the structure of a sentence.
Textual Parsing, again is broken into two parts :
Textual Parsing, again is broken into two parts :
◼
Dependency Parsing, which deals with how a word in sentence is dependent to another word.
◼
Constituent Parsing, which shows the structure of a sentence in regards to their Grammatical Units.
A typical picture showing dependency and constituent parse trees.
My aim in this project is to investigate the statistics of Textual Parse Trees using various types of texts and find trends among the results.
Collecting and Processing Data
Collecting and Processing Data
For this project, I have analyzed Textual Parsing on three different types of text; Literature Data, Narrative Data, and Informational Data. Additionally, I also analyzed Textual Parsing on various long sentences which contained over 100 words in each sentence.
◼
Literature Data consisted of various books from William Shakespeare, Jane Austen, and Arthur Conan Doyle.
◼
Narrative Data consisted of various movie scripts from directors like Quentin Tarantino, Christopher Nolan, and Martin Scorsese.
◼
Informational Data consisted of various articles from Wikipedia and New York Times.
◼
For Long Sentences, I have created a dataset of around thirty long sentences.
{EmmaStructured,northhangerAbbeyStructured,PersuassionStructured,prideandprejusticeStructured,senseandsensibilityStructured,ConanDoyleStructured,houndofthebaskervillesStructured,returnofsherlockStructured,signofthefourStructured,thelostworldstructured,alliswellstructured,merchantofveniceStructured,midsummernightsdreamStructured,phoenixandturtleStructured,romeoandhulietStructured,twogentlemenofveronaStructured,BatmanBeginsStructured,DarkknightStructured,InterstellarStructured,CasinoStructured,MeanStreetsStructured,DjangoStructured,IngloriousStructured,PulpfictionStructured,uefaStructured,sherlockHolmesStructured,usaStructured,bitcoinStructured,ethereumStructured,cryptocurrencyStructured,wikipediaStructured,ElizabethStructured,HollywoodStructured,LinguisticsStructured,WolframStructured,MathematicaStructured,NetflixStructured,AmazonStructured,NYTdatastructured,ShakespearePOSData,JanePOSData,ArthurPOSData,QuentinPOSData,NolanPOSData,MartinPOSData,WikipediaPOSData,NYTposData,Emma,northhangerAbbey,Persuassion,prideandprejustice,senseandsensibility,ConanDoyle,houndofthebaskervilles,returnofsherlock,signofthefour,thelostworld,alliswell,merchantofvenice,midsummernightsdream,phoenixandturtle,romeoandjuliet,twogentlemenofverona,longestSentences,longSentencesStructured,Depgraphs,Congraphs,BatmanBegins,Darkknight,Interstellar,MeanStreets,Casino,Django,Pulpfiction,Inglorious,LiteratureData,NarrativeData,InformationalData,Congraphs,Depgraphs,ShakespeareDependencyGraphs,ShakespeareConstituentGraphs,JaneDependencyGraphs,JaneConstituentGraphs,ArthurDependencyGraphs,ArthurConsituentGraphs,QuentinDependencyGraphs,QuentinConstituentGraphs,NolanDependencyGraphs,NolanConstituentGraphs,MatindependencyGraphs,MartinConstituentGraphs,WikipediaDependencyGraphs,WikipediaConstituentGraphs,NYTdataDependencyGraphs,NYTdataConstituentGraphs}=CloudImport["https://www.wolframcloud.com/env/praneeth041/finaldataset"];
{LongestSentencesDependencyComplexity,LongestSentencesGrammaticalComplexity,LiteratureDependencyComplexity,LiteratureGrammaticalCompmlexity,NarrativeDependencyComplexity,NarrativeGrammaticalCompmlexity,InformationalDependencyComplexity,InformationalGrammaticalComplexity,LongestGrammaticalStability,LongestDependencyStability,LiteratureGrammaticalStability,LiteratureDependencyStability,NarrativeGrammaticalStability,NarrativeDependencyStability,InformationalGrammaticalStability,InformationalDependencyStability,uefa,sherlockHolmes,usa,bitcoin,ethereum,cryptocurrency,wikipedia,Elizabeth,Hollywood,Linguistics,Wolfram,Mathematica,Netflix,Amazon,ShakespeareData,JaneData,ArthurData,MartinData,QuentinData,NolanData}=CloudImport["https://www.wolframcloud.com/obj/praneeth041/finaldataset4"];
WikipediaArticles = List[ConstituentAssociation/@{uefaStructured,sherlockHolmesStructured,usaStructured,bitcoinStructured,ethereumStructured,cryptocurrencyStructured,wikipediaStructured,ElizabethStructured,HollywoodStructured,LinguisticsStructured,WolframStructured,MathematicaStructured,NetflixStructured,AmazonStructured}]//Flatten;
Functions Used to Pre-Process Data
Build an association which maps the Frequency of Grammatical Units in a piece of text :
structuredData[data_String] := TextStructure[TextSentences[data],"ConstituentStrings"]//Flatten;ConstituentAssociation[structuredData_] := Association#Count[StringCases[structuredData,Shortest["("~~a:#~~", "~~b:__~~")"]{a,b}]//Flatten,#]&/@;
Build an association of average Frequency of Grammatical Units given a list of ConstituentAssociation outputs :
AVGConstituentAssociation[ConstituentAssociation_]:=Association#Round[Total[Table[ConstituentAssociation[[n]][#],{n,Length[ConstituentAssociation]}]]/Length[ConstituentAssociation]//N]&/@;
Make an association of the frequency of Parts of Speech in a piece of text :
POSAssociation[data_String]:=ModulePOS = TextCasesdata,,Association#Length[POS[#]]&/@;
MakeanassociationoftheaveragefrequencyofPartsofSpeechgivenalistofPartofSpeechAssociations:
AVGposAssociation[POSAssociation_]:=Association#(Total[Table[POSAssociation[[n]][[#]],{n,Length[POSAssociation]}]]/Length[POSAssociation]//N)&/@;
In[]:=
Functions used to analyze Constituent/Dependency Graphs
Functions used to analyze Constituent/Dependency Graphs
Compute the depth of a Dependency Parse Tre :
DependencyDepthFunction[Graph_]:=Module[vertexout=DeleteCases[Table[If[Length[n]==1,n],{n,VertexOutComponent[Graph,#]&/@VertexList[Graph]}],Null]//Flatten, vertexin=DeleteCases[Table[If[Length[n]==1,n],{n,VertexInComponent[Graph,#]&/@VertexList[Graph]}],Null]//Flatten, Max[Table[Length[n],{n,FindShortestPath[Graph,#,vertexout[[1]]]&/@vertexin}]]] DependencyDepthFunction[TextStructure["A cat sat on a mat","DependencyGraphs"][[1]]]
Compute the depth of a Constituent Parse Tree :
ConstituentDepthFunction[Graph_]:=Module[vertexout=DeleteCases[Table[If[Length[n]==1,n],{n,VertexInComponent[Graph,#]&/@VertexList[Graph]}],Null]//Flatten, vertexin=DeleteCases[Table[If[Length[n]==1,n],{n,VertexOutComponent[Graph,#]&/@VertexList[Graph]}],Null]//Flatten, Max[Table[Length[n],{n,FindShortestPath[Graph,vertexout[[1]],#]&/@vertexin}]]]
It is necessary to convert the edges of a graph in the case of Constituent Graph since the vertices of constituent graphs are undirected and it is impossible to find it's depth based on the vertices if they are undirectional.
ConvertEdges[Graph_]:=Table[DirectedEdge[x[[1]],x[[2]]],{x,EdgeList[Graph]}]
ConstituentDepthFunction[Graph[ConvertEdges[TextStructure["A cat sat on a mat.","ConstituentGraphs"][[1]]]]]
Highlight the Depth of a Dependency Parse Tree :
Highlight the depth of a Constituent Parse Tree :
Compute the number of Leafs at Maximum Depth level of a Constituent Parse Tree :
Compute the Number of Leafs at Maximum Depth level of a Dependency Parse Tree :
Compute the total leafs of a Dependency Parse Tree :
Convert Undirected Edges into Directed Edges. Necessary in order to graph Constituent Parse Trees :
Compute the total leafs of a Constituent Parse Tree :
Functions used to to compute Lexical Richness
Compute the Type Token Ratio of a piece of text :
Compute Duagst's Uber Index for a piece of text :
Analysis and Visualization
Analysis and Visualization
I have done 5 studies based on different metrics and analyzed them accordingly.
◼
Study 1 : Comparing the Frequency of Grammatical Units among various types of texts.
◼
Study 2 : Comparing average Parts of Speech used in a sentence among various types of texts
◼
Study 3 : Computing Tree Stability, Complexity, and Depth of Constituent and Dependency Parse Trees
◼
Study 4 : Analyzing Statistics on Tree Stability and Complexity of various types of texts
◼
Study 5 : Analyzing Word Clouds.
This Computational Essay presents the summary and results of each study in a crisp and compact way.
Please refer to the Final Project Notebook to gain more information on each study.
In[]:=
Stability and Complexity of a Sentence using their Parse Trees
Stability and Complexity of a Sentence using their Parse Trees
In the below blocks of code, I’ve come up with formulas which compute the Complexity and Stability of a Parse Tree.
◼
The ComplexityTreeDepth function takes the depth, number of leafs at maximum depth, and the graph of a parse tree in order to compute the Complexity of a particular parse tree. After working out various formulas, I found out that the average outgoing degree of non-leaf nodes is a good metric for complexity along with the depth and the number of leafs at maximum depth. During further exploration, adding and rescaling them seemed to provide viable results. Bases on this formula, a sentence is more complex when it reaches a deeper level (in either tree) and also when the vertices spawn more vertices. Ultimately, the function gives a value in between 0 - 70 as far as I've observed in this project. (It might go even higher if run on more complex sentences which contain 1000's of words in a sentence.)
◼
The TreeStability function on the other hand; takes the complexity of a sentence, its tree depth, number of leafs at maximum depth, and the number of leaves as it's input. Since a sentence tends to become less stable as its complexity increases, I divided the number of leaves with complexity. Then, I multiplied the result with the addition of Tree Depth and number of leaves at maximum depth. After working on various formulas, this seemed to provide feasible results when subtracted from 100.
The following visualizations are cut down to 5 sentences from each dataset for the essay. Please refer to the Final Project Notebook to see all the visualizations which contains 31 sentences from longest sentences dataset and 150 sentences from each of the other datasets.
Comparing Complexity and Stability with the Constituent Graphs of Five Sentences from Each dataset:
Figure 1 : Comparing Stability, Depth. and Complexity with their Constituent Graphs
Comparing Complexity and Stability with the Dependency Graphs of Five different sentences from each dataset
Figure 2 : Comparing Stability, Depth, and Complexity with their Dependency Graphs
In[]:=
Are Movie Scripts as Complex and Stable as Shakespeare's Plays?
Are Movie Scripts as Complex and Stable as Shakespeare's Plays?
Well, the answer to the question comes in two ways. Let's dig into the statistics of Complexity and Stability among various datasets and find the answer.
The following statistics are based on the stability and complexity of 31 sentences from the dataset of long sentences and 150 random sentences each from various types of texts.
Figure 3
Figure 4
You could now say from the graphs that sentences in Literature Data are more complex than sentences in Movies Scripts. In fact, the average complexity of a sentence in Movie Scripts are relatively lower as compared to other types of texts. However, this is not the case while comparing the Stability of a Sentence; it get's interesting from here on.
Figure 5
Figure 6
When it comes to stability, Sentences of Movie Scripts seem to be most stable than other types of texts. Although one could argue that the stability function was formulated in such a way that less complexity equals more stability, I investigated deeper into Narrative data and found some interesting trends.
Figure 7
From this plot, we can infer that Narrative Data has a higher rate of Lexical Richness and Lexical Readability (since lower score on Flesch Kincaid Scale means higher grade level), but when it comes to Sentence Complexity, they are relatively low. Since dialogues in movies are usually not so complex, it could be a reason as to why this happens. However, movie dialogues tend to use distinct types of words in each sentence which may also explain the peaks in Lexical Richness and Lexical Readability.
When it comes to the initial question on whether Movie Scripts are as complex and stable as Shakespeare’s Plays, the answer seems to be yes. The average complexity of a sentence in Shakespeare’s plays is indeed higher than that of Movie Scripts, but the lexical complexity and lexical richness are similar in both. This is something I wish to call as “Disguised Complexity” where the sentences are complex but does not seem so in comparison to their parse trees. On the stability part, I think it is pretty clear that Movie Scripts are much stable than Shakespeare’s plays. One reason for this might also be because of the change in the language due to time but something I’ve found out using the graphs is that the “Disguised Complexity” phenomenon in Movie Scripts seems to make them more stable since lesser complexity tends to equal higher stability.
When it comes to the initial question on whether Movie Scripts are as complex and stable as Shakespeare’s Plays, the answer seems to be yes. The average complexity of a sentence in Shakespeare’s plays is indeed higher than that of Movie Scripts, but the lexical complexity and lexical richness are similar in both. This is something I wish to call as “Disguised Complexity” where the sentences are complex but does not seem so in comparison to their parse trees. On the stability part, I think it is pretty clear that Movie Scripts are much stable than Shakespeare’s plays. One reason for this might also be because of the change in the language due to time but something I’ve found out using the graphs is that the “Disguised Complexity” phenomenon in Movie Scripts seems to make them more stable since lesser complexity tends to equal higher stability.
In[]:=
Other Trends found while Comparing Complexity and Stability over various types of texts
Other Trends found while Comparing Complexity and Stability over various types of texts
◼
We can also infer from the graph of Average Stability and Average Complexity that the Long Sentences dataset has the highest average complexity of a sentence, but also the lowest sentence stability in comparison other types of texts. This again strengthens the point that it is difficult to maintain stability as complexity increases in a sentence. However, there are a few peaks in the graph of the Long Sentences Stability which contradicts the previous statement and proves that not every sentence that is complex necessarily needs to be less stable.
Figure 8
◼
Among all the types of data, Informational data seems to be the most consistent in both Average Complexity of a Sentence and Average Stability of a Sentence (Refer to Figure 3 and 5). Most Informational data is precise and compact; which maybe the reason why a sentence is arguably stable and complex at the same time.
In[]:=
Comparing the Frequency of Grammatical Units
Comparing the Frequency of Grammatical Units
Using the functions ConstituentAssociation and AVGConstituentAssociation, I was able to produce a list of associations which map the frequency of Grammatical Units in a piece of text. I was able to plot Box Whisker Charts and List Line Plots using these results and compared them.
Figure 9
Figure 10
Figure 11
We can honestly find many observation from the above graphs but these are what I found.
Trends
Trends
◼
The Box Whisker Charts for the frequency of Informational and Narrative Data seemed to be inconsistent as compared to the graph of Literature Data for the same. Literature Data seems to exploit the most number of grammatical units.
◼
Regardless of other Grammatical Units, the frequency of Adjectives in each types of data seems to be higher.
◼
The number of AdverbPhrases, NounPhrases, or VerbPhrases are higher in the case of narrative data as compared to Literature and Informational Data. One possible reason for this might be because Movie Scripts in Narrative Data seem to use sentences with less words but different types of words.
In[]:=
Parts of Speech : Do they make a sentence complex?
Parts of Speech : Do they make a sentence complex?
After investigating the statistics on the frequency of Grammatical Units, I furthered into the statistics of average parts of speech in a sentence; which gave me an answer to whether Parts of Speech make a sentence complex.
Figure 12
Figure 13
Figure 14
As you can see from the graph above, Informational data had the greatest number of Parts of Speech in a sentence as compared to other types of data. Although one of the reasons for this might be because we had a lot of Literature data as compared to Informational data; I’d like to keep that aside for now.
From the graphs of Parse Tree Stability and Complexity, Informational Data seemed to be mildly complex and stable. But when we take the Lexical Readability and Complexity in regards, we observe that Informational data are complex lexically, but they seem to have lesser lexical readability as compared to that of narrative data. This gave me an idea on whether more usage of parts of speech in a sentence, might reduce the lexical readability of a sentence.
After further investigation, I found out that this argument seems to be true even in the case of narrative data. If you remember, sentences in Narrative data had high values Lexical Complexity and Readability; but when you look at the above graph on Average Parts of Speech in sentence, Narrative Data seemed to have the least number of average parts of speech in a sentence. This again strengthens the argument that more parts of speech might reduce the Lexical Readability of sentence. However, I believe further research can be carried out on this topic.
To answer the question as a whole, it seems like more number of parts of speech in a sentence do increase Lexical Complexity but maybe at the loss of Lexical Readability.
From the graphs of Parse Tree Stability and Complexity, Informational Data seemed to be mildly complex and stable. But when we take the Lexical Readability and Complexity in regards, we observe that Informational data are complex lexically, but they seem to have lesser lexical readability as compared to that of narrative data. This gave me an idea on whether more usage of parts of speech in a sentence, might reduce the lexical readability of a sentence.
After further investigation, I found out that this argument seems to be true even in the case of narrative data. If you remember, sentences in Narrative data had high values Lexical Complexity and Readability; but when you look at the above graph on Average Parts of Speech in sentence, Narrative Data seemed to have the least number of average parts of speech in a sentence. This again strengthens the argument that more parts of speech might reduce the Lexical Readability of sentence. However, I believe further research can be carried out on this topic.
To answer the question as a whole, it seems like more number of parts of speech in a sentence do increase Lexical Complexity but maybe at the loss of Lexical Readability.
In[]:=
Other trends found while comparing Average Parts of Speech among various types of text.
Other trends found while comparing Average Parts of Speech among various types of text.
Figure 15
◼
While comparing the average parts of speech in a sentence used by each author, I found out that Jane Austen and Arthur Conan Doyle seemed to have similar number of average parts of speech in a sentence. I think that further research involving only the books of Jane Austen and Arthur Conan Doyle might show more similarities.
In[]:=
Word Clouds
Word Clouds
Using text cases and Word Cloud function, I have been able to create a function which takes a piece of text (a book presumably) as one input, and a Grammatical Unit as another input. The function returns a Word Cloud based on the words which match the Grammatical Unit.
After playing with this function, I was able to find some interesting trends; and cool visualizations.
A demonstration of the WordCloudGrammaticalUnit function:
As we can observe from above figures, Word Clouds of Proper Nouns seems to return the names of various characters when presented with narrative and literature data (which seems reasonable), but when it comes to Informational data like a Wikipedia article on UEFA, it provides a diminished sense of what the article is all about; and presents the article as sort of a Word Cloud.
◼
Further testing on other types of Grammatical Units seemed to provide other important data such as the most frequent determiners or Pronouns occurred in a piece of text. I believe that further research could be conducted using this data in the Field of Natural Language Generation.
Final Results and Discussions
Final Results and Discussions
While working on the project, I obviously got to learn a lot about the Wolfram Language as a whole and explored NLP in Wolfram Language for the first time. I definitely found some interesting trends while analyzing the statistics on various metrics. One such result which astonished me was regarding the Narrative Data when I found out that they are comparatively as stable and complex as Shakespeare’s plays. I also enjoyed playing with the WordCloudGrammaticalUnit function which gave some cool visualisations. One more vital component which I really liked and am proud of about my project are the functions I created to get the Depth, Complexity, and Stability of Text Parse Trees. I agree, these functions would’ve been much better but considering the fact that I could come up with a formula to compute something that significant makes me feel good.
Future Work
Future Work
◼
Investigate the Frequency of Grammatical Units in each level of a Constituent Parse Tree.
◼
Continue Research on the Word Clouds function and find out other trends using it.
◼
Improve the current formula for Stability and Complexity of a Parse Tree.
◼
Improvise the project by working on more amounts of data.
Acknowledgements
Acknowledgements
I would wholeheartedly thank my mentor, Aaron Enright, who tremendously helped me on the project and guided me towards the right way. I would also like to thank mentor Mark Greenberg and TA’s - Anika Patel and Sheldon Rego who helped me on one of the core ideas of the project. The mentor team, TA team, and administration team did an amazing job in solving everyone’s problems on Slack and maintaining the decorum of the camp.