Introduction
Introduction
Motivation
Motivation
Linguistic history parallels cultural history
Every language is like a one-of-a-kind species. It captures unique conceptualizations of the world and has its own ways of constructing words, phrases and sentences for communicating ideas. As we compare the words and structures of various languages, we come to a greater understanding of our world and how it came to be.
Every language is like a one-of-a-kind species. It captures unique conceptualizations of the world and has its own ways of constructing words, phrases and sentences for communicating ideas. As we compare the words and structures of various languages, we come to a greater understanding of our world and how it came to be.
Abstract
Abstract
A language family is a group of languages related through descent from a common ancestral language. The term "family" reflects the tree model of origination similar to that of a biological family tree. In linguistics, lexical similarity is a measure of the degree to which the word sets of two given languages are similar. A lexical similarity of 1 (or 100 %) would mean a total overlap between vocabularies, whereas 0 would imply that there are no common words. Lexical similarity can be used to evaluate the genetic relationship between two languages. My goal with this project was to graph the relations between language families according to their lexical similarities.
Using Lexical Similarity Data
Using Lexical Similarity Data
Generating the Data
Generating the Data
We shall start by getting the lexical similarity data for a particular language, this should generate a list of languages that have a similar vocabulary.
Getting the Lexical Similarities of different Languages
In[]:=
GetSimilarityData[mylang_String]:=LanguageData[Interpreter["Language"][mylang],"LexicalSimilarity"]
In[]:=
GetSimilarityData["German"]
Out[]=
,,,,
Now, to organise the data for each language - with the edge weight representing the Lexical Similarity
In[]:=
OrganiseSimilarityData[mylang_String]:=Column[With[{l=Interpreter["Language"][mylang]},weights = Property[l #[[1]],EdgeWeight N[Quantity[#[[2]],"Percent"]]]&/@l["LexicalSimilarity"]]]
In[]:=
OrganiseSimilarityData["French"]
Out[]=
Property |
Property |
Property |
Property |
Property |
Property |
Property |
Property |
Property |
Graphing Lexical Similarity
Graphing Lexical Similarity
This allows us to visualise the similarities between languages
To graph the lexical similarity for a particular language
In[]:=
Graph[weights,VertexLabelsPlaced["Name",Below],EdgeLabels"EdgeWeight",ImageSize {700,200},GraphLayout{"PackingLayout""LayeredLeft"}]
Out[]=
Creating Language Families
Creating Language Families
Based on Specific Languages
Based on Specific Languages
Once we are able to generate the lexical similarity data for individual languages, we can move on to creating language families based on their similarities.
The function "similarityGraph" generates the languages similar to the specified language.
As the function generates language pairs, we need to delete all the duplicate cases - as well as any languages that do not have lexical data.
The function "similarityGraph" generates the languages similar to the specified language.
As the function generates language pairs, we need to delete all the duplicate cases - as well as any languages that do not have lexical data.
Generating the Lexical Values for various languages
In[]:=
similarityGraph[ent_Entity]:=similarityGraph[{ent}];similarityGraph[ents:{__Entity}]:=Module[{simrules,getedges},simrules=DeleteCases[Thread[entsEntityValue[ents,"LexicalSimilarity"]],__Missing];(* "getedges" represents the weight of the edge (lexical similarity) between two languages *)getedges=Flatten[Function[{r,d},Map[Property[r#[[1]],{EdgeWeightN[Quantity[#[[2]],"Percent"]]}]&,d]]@@@simrules];getedges=DeleteDuplicatesBy[getedges,Sort[#[[1]]]&];getedges=getedges/.Interval[{min_,max_}]min;Graph[getedges,VertexLabels"Name",GraphLayout{"LayeredDigraphEmbedding","RootVertex"ents[[1]]},FormatTypeStandardForm]]//If[EmptyGraphQ[#],Missing["NotAvailable"],#]&;
Deleting Duplicates and Missing Cases
In[]:=
similarityGraph[l_,n:_Integer:0]/;(n≥0):=similarityGraph[NestWhile[DeleteDuplicates[Flatten[Join[{#},DeleteMissing[Flatten[EntityValue[#,"LexicalSimilarity"]]][[All,1]]]]]&,{l},UnsameQ,All,n]];
Graphing the Language Families
In[]:=
LanguageFamilyGraph[lang_String]:=CommunityGraphPlot[similarityGraph[Interpreter["Language"][lang],1],ImageSize{800,450},PlotLabel lang,LabelStyle"Section"]
This allows us to get a detailed idea about the language family a particular language belongs to.
The circles around languages represent the boundaries of a language family.
The distance between two languages indicates the similarity between them. Being dialects of the same language, notice how Northern, Eastern and Western Frisian are placed extremely close to each other.
The circles around languages represent the boundaries of a language family.
The distance between two languages indicates the similarity between them. Being dialects of the same language, notice how Northern, Eastern and Western Frisian are placed extremely close to each other.
In[]:=
LanguageFamilyGraph["English"]
Out[]=
In[]:=
LanguageFamilyGraph["French"]
Out[]=
Based on Geographical Regions
Based on Geographical Regions
In addition to creating language families for individual languages, we can also create families for all the languages spoken in a geographical region.
This is done by extracting all the languages spoken in a region and calculating their relative lexical similarity to each other.
This is done by extracting all the languages spoken in a region and calculating their relative lexical similarity to each other.
Language Families based on Geographical Regions
As we can see, there exist discrete language families in large geographical locations - not all languages can be connected to each other.
Evolution of Language Families
Evolution of Language Families
Cardinal Numbers as a Linguistic Database
Cardinal Numbers as a Linguistic Database
In my search for a database for classifying languages, I stumbled upon cardinal numbers.
Though often overlooked, cardinal numbers can give us a lot of information about the similarities between two languages. This is because cardinal numbers tend to form the foundation of a language - and are also prone to be "borrowed" by other languages. To get a better sense of their true potential, let us look at cardinal numbers in some European languages.
Though often overlooked, cardinal numbers can give us a lot of information about the similarities between two languages. This is because cardinal numbers tend to form the foundation of a language - and are also prone to be "borrowed" by other languages. To get a better sense of their true potential, let us look at cardinal numbers in some European languages.
Notice how the languages : French, Italian, Portuguese and Spanish have cardinal representations that are extremely similar.It may come as no surprise that they are related, each belonging to the family of "Romance Languages".
Evolution of Language Families with the Database
Evolution of Language Families with the Database
We can visualise the growth of language families as a function of the size of the database. A larger dataset allows for more accurate results.
Graphing Language Families as a function of the database size
We can now group these languages together in the form of language families
Grouping the languages together
We can now create a tool that allows us to visualise the evolution of language families parallel to that of our dataset.
As the size of the dataset increases, the accuracy of the formation of language families also increases.
As the size of the dataset increases, the accuracy of the formation of language families also increases.
Creating an interactive visualisation of language families
Conclusion
Conclusion
Future Work
Future Work
I wish to work on this project in the near future and use letter-to-letter comparisons between languages to classify them. This would create a more robust framework for mapping language families and could also help us visualise the exact lexical differences between different languages.
Acknowledgements
Acknowledgements
I'd like to thank my mentor - Mae Jennings, the TAs and the camp directors at the Wolfram High School Summer Camp for a wonderful experience!