Abstract
Abstract
In recent years, machine learning has helmed technological innovation with its powerful potential for applications in a wide range of fields. From image classification to natural language processing, machine learning has fundamentally changed our daily lives. Yet, the applications of machine learning in the study of human names (anthroponomastics) have been limited. This could be accredited to the complexity of human names given vast cultural differences and personal preferences, but is still interesting because the existence of personal names, as abstract linguistic symbols for individual people, dates back to 3000 BC and is pervasive as an indispensable part of human societies. In this project, I explore how names can be represented and categorized computationally to create an algorithm to separate human names into first and last by factoring in the cultural origin of the name, and consider its merits and demerits in the process.
Introduction
Introduction
The idea of individual identity with a name distinguishes humans from the rest of the animal kingdom, such that it is fundamental to human existence. However, names vary drastically across cultures and an individual’s name is the first denotation of an individual's cultural background, marital status, and in some places, societal standing. Naming customs are so personal and distinct that they vary within countries themselves - in India, a North Indian is likely unaware of naming customs in South India and vice versa. Furthermore, with increasing westernization and adoption of English as the Lingua Franca globally, many people, especially in Asia, have adopted anglicized names in addition to their traditional given names.
These complexities have lent themselves to the limited computational exploration of names in the past, so the goal of this project is to consider the cultural variations in names and then computationally explore them to create an algorithm to split names.
These complexities have lent themselves to the limited computational exploration of names in the past, so the goal of this project is to consider the cultural variations in names and then computationally explore them to create an algorithm to split names.
Dataset Creation
Dataset Creation
Initial Data Collection
Initial Data Collection
Given the lack of a comprehensive dataset on individual names and their corresponding places of birth / origins, I had to curate my own dataset for the project. The data was collected from WikiData, the common source of open data for Wikimedia projects like Wikipedia.
Due to the large size of the data that was needed for the project, it was difficult to obtain the same with Wolfram's in-built WikidataData function. Therefore, with the assistance of my mentor, SPARQL queries on Wolfram's WikiData mirror were ran to obtain a dataset consisting of full names and birthplaces, consisting of information about over 3.2 million individuals.
Due to the large size of the data that was needed for the project, it was difficult to obtain the same with Wolfram's in-built WikidataData function. Therefore, with the assistance of my mentor, SPARQL queries on Wolfram's WikiData mirror were ran to obtain a dataset consisting of full names and birthplaces, consisting of information about over 3.2 million individuals.
The dataset was stored in the variable "namesandplaces" as a list of associations.
In[]:=
<<GraphStore`
In[]:=
namesandplaces=CloudImport[CloudObject["https://www.wolframcloud.com/obj/unnathiutpal6/namesandplaces"]];
In[]:=
RandomChoice[namesandplaces]
Out[]=
othernamesRDFString[Francisca D. Weinberg,en],nameRDFString[Francesca Zijlstra,en],birthplaceURL
Cleaning Data
Cleaning Data
The dataset, at this point, contains the name and birthplace of individual people, but the birthplace is in the form of the city name. However, it would be difficult and inefficient to create an accurate classifier with thousands of city names (and thus classes), so the city names had to be converted to their respective country names.
First, all the individual birthplaces needed to be linked to their associated countries in a rule.
This was done by finding their associated country with the WikidataData function, in batches of 1000 at a time.
birthplacesToBirthCountries=Thread[#->WikidataData[#,"P17"]]&/@Partition[Cases[Union[Lookup[namesandplaces,"birthplace"]],_URL],UpTo[1000]];
However, not all 585 countries obtained from Wikidata were still valid. Some of the countries from Wikidata included historical states like "Principality of Wallachia," which would not be valid training data in context of a name-country classifier for the 21st century.
Therefore, to find valid countries, all the countries found from Wikidata were mapped to Wolfram Language country entities (as those would only consist of valid countries).
validCountries=#->Interpreter["Country"][Replace[WikidataData[#,"Label"],{s_String}->s]]&/@Union[Flatten[Values[Flatten[birthplacesToBirthCountries]]]];
While this converted valid countries into country entities, it did not remove countries that were not valid from the list, so invalid countries had to be removed from the list next.
Invalid countries mapped to “Failures” in place of entities, so this characteristic was used to remove them.
validCountries=DeleteCases[validCountries,_->_Failure];
Some birthplaces mapped to a list instead of a single country, so were not converted to country entities. They needed to be removed as well.
validCountries=DeleteCases[validCountries,_->_List];
At this point, we have a rule that transforms Wikidata country entities into valid Wolfram country entities (denoted by validCountries) and another rule that transforms Wikidata URLs for birthplaces into Wikidata country entities (denoted by birthplacesToBirthCountries). We now need a consolidated rule to transform Wikidata URLs for birthplaces into valid Wolfram country entities.
A replacement rule was applied to transform Wikidata URLs for birthplaces to valid Wolfram country entities.
URLtoCountry=birthplacesToBirthCountries/.validCountries;
URLtoCountry consisted of URLs to a list with single country entities, but the list had to be removed for it to be uniform.
URLtoCountry=URLtoCountry/.{e_Entity}:>e
Since the data obtained from Wikidata wraps the names in an RDF string wrapper, the RDF string wrapper needs to be removed.
namesAndBirthplaces=Replace[#["name"],RDFString[name_,_]:>name]->#["birthplace"]&/@namesandplaces;
Next, URLtoCountry had to be applied to namesAndBirthplaces to convert URLs of birthplaces to country entities.
Since the replacement rules in URLtoCountry were extensive and long, a Dispatch was used instead.
URLtoCountryDispatch=Dispatch[Flatten[URLtoCountry]]
namesToCountries=namesAndBirthplaces/.URLtoCountryDispatch;
In[]:=
RandomChoice[namesToCountries]
Out[]=
María San Gil
The data then had to be grouped such that all names corresponding to a particular country entity were in a list linked to that particular country entity to be used in the training process.
nameToCountryClean=Cases[nameToCountries,HoldPattern[_String->_Entity]];
grouped=First/@#&/@GroupBy[nameToCountryClean,Last];
Splitting Data for Training and Validation
Splitting Data for Training and Validation
We now have a dataset with 900 000+ values, but it must be split into training and validation so that the accuracy of the classifier can be calculated later. The data was split in an 80:20 ratio for training and testing.
classificationData=RandomSample[#,UpTo[Ceiling[Length[#].8]]]&/@grouped;
Since the same data could not be used for testing, the complement of classification data was taken as test data.
testData=#->Complement[grouped[#],classificationData[#]]&/@Keys[classificationData]
The testing data, however, could not be in the same shape as the classification data, i.e., the training data had to be in the form _Entity -> {_String, _String} but the testing data had to be in the form {_String -> _Entity, _String -> _Entity, ...}.
The testing data was reshaped to the required format.
testData2=Flatten[With[{entity=First@#,strings=Last@#},#->entity&/@strings]&/@testdata];
This classification and validation data can now be used to train and test the classifier respectively.
Name-to-Country Classifier
Name-to-Country Classifier
Creating the Classifier
Creating the Classifier
The Classify function (from the Wolfram function repository) creates a machine learning classifier based on the data fed into it.
The classification data curated in the previous steps was used to train the classifier.
nameToCountryClassifier=Classify[classificationData];
In[]:=
nameToCountryClassifier["Unnathi Kumar"]
Out[]=
Testing the Classifier
Testing the Classifier
The classifier was tested and achieved an accuracy of ~57%.
Visualizations
Visualizations
In the exploration of the name-to-country classifier, two functions were written to visualize the outputs in the form of geographic heat maps.
This function plots a map with varying colours depending on the probabilities of a name belonging from a particular geographic region.
In[]:=
plotProbabilityDensityHeatMap[s_String]:=GeoDensityPlot[nameCountryClassifiernoUSA[s,"Probabilities"]]
In[]:=
plotProbabilityDensityHeatMap["Unnathi Kumar"]
Out[]=
This function plots a shaded map depending on the probabilities of a name belonging from a particular geographic region.
In[]:=
plotProbabilityHeatMap[s_String]:=GeoRegionValuePlot[nameCountryClassifiernoUSA[s,"Probabilities"]]
In[]:=
plotProbabilityHeatMap["Unnathi Kumar"]
Out[]=
Name Splitter
Name Splitter
Grouping Countries by Language/Cultures
Grouping Countries by Language/Cultures
Some countries with similar languages share cultures and thus naming customs as well. A prime example of this is Spanish-speaking countries like Costa Rica, Mexico, and Spain. Therefore, it would be easier to write the name splitter algorithm if Spanish-speaking countries were grouped together.
All countries are imported and their first languages stored in a variable.
All countries with common first languages are grouped together.
Spanish-speaking countries form the 9th part of the language groups list.
Besides grouping countries by language, it would also be advantageous in the interest of cleaner, more readable code to group culturally-similar countries by the regional organizations they are part of.
For example, countries in the Arab League are fairly culturally similar, with similar naming customs, so they must be grouped together.
Creating a Name Splitter
Creating a Name Splitter
This W3 article on "Personal names around the world" served as a framework to understand how names differ across cultures. Rules and guidelines from this webpage, along with individual Wikipedia pages on naming across the world (such as this one), were adapted into this name splitter algorithm.
Notable rules include:
◼
Patronymic names are used in Iceland (which differs according to gender as well), Malaysia, Russia, and Indonesia.
◼
Japan, Korea, Hungary, Vietnam, and China order names as family name followed by given name(s).
◼
The Chinese also occasionally use generational names along with given/family names.
◼
Spanish-speaking people commonly have two family names, ordered as paternal family name followed by maternal family name.
◼
Arabic countries have a very different naming system, consisting of multiple parts. You can find out more here.
In the consolidated algorithm, the name was tokenized and its individual parts were used to make splitting it up easier, while considering the origin of the names.
A consolidated name splitter algorithm was created.
While this algorithm could not be tested extensively due to the lack of a sufficient dataset, informal testing proved it to be working in most cases.
Discussion
Discussion
Limitations
Limitations
◼
Certain assumptions were made in the creation of the name splitter algorithm.
◼
Firstly, cultural differences in naming customs within countries were not considered due to the wide variety across the board. For example, in India itself, naming customs vary drastically from North India to South India, but since the classifier was not built in such a way that it could classify the specific regional origin of the name, the most general form of names in India had to be considered. While I was able to recognize this as a native of India, there must be multiple countries for which common naming customs could not be considered due to cultural differences.
◼
Secondly, since the names were tokenized before splitting, it was considered that parts of names are always separated by whitespace in the regions considered.
◼
Limited training data was available for certain regions. For example, less than 10 data points were available on names from the Gaza strip, Western Sahara, Nauru, Tuvalu, Palau, Niue, Marshall Islands, and Kiribati. Additionally, around 140 000 data points corresponded to the United States, while the next country closest (United Kingdom) to that number stood at around 70 000. Therefore, limited data was available on more obscure regions, which made it difficult to accurately predict origin. Furthermore, considering that the United States is home to millions of immigrants, the classifier could be biased with that in mind as well.
◼
The accuracy of the classifier was less than expected. This could either be due to 3 reasons, as outlined below.
◼
Lack of sufficient training data: as stated previously, sufficient training data was unavailable for certain regions and exponentially more data was available for others.
◼
Similarities in naming customs across cultures: some neighbouring countries would likely have similar naming countries, but do not share languages or regional organizations for us to group them together easily. Therefore, while the classifier might classify a Chinese name to Japanese incorrectly, this would be incorrect for the purpose of the classifier but would be useful for the splitter, as eastern countries, regardless, share the custom of family name before given name.
◼
Irrelevant historical data: even though I tried to remove most historical data by removing historical countries, I was not able to remove historical data that still mapped to country entities. For example, historical duchesses and dukes still mapped to the existing country entity of UK, but were not relevant to the classifier.
◼
Limited information was available from secondary sources on how names are split in certain regions, so the name splitter is currently only able to accurately split names from certain regions on which sufficient information was available.
Future Additions
Future Additions
◼
I could make a classifier that separates names regionally rather than nationally (this would, however, take substantially more training time and require more training data) and increasing the accuracy of the current classifier.
◼
An approach that was initially taken was to feed the classifier with data about what is considered a first name and what is considered a last name, but this approach was highly inaccurate. Yet, this approach could be investigated further as it would help the name splitter be more accurate than by only considering the origin of names. An approach that we did not get to explore within this was with creating feature vectors with first names and last names, so this would be something I would want to look into in the future.
◼
I could collect more data on how names are split across cultures from primary sources to increase the applicability of the algorithm.
Conclusions
Conclusions
Some conclusions that can be drawn from the work done in this project include:
◼
There are some merits associated with using the origin of name to split human names.
◼
There is great scope for exploration of human names with reference to machine learning and technology. The intersection of anthroponomastics, technology, and linguistics could be explored in greater depth.
In general, the work done seems to have great potential applications in the real world. For example, it could realistically be used in multiple administrative and management software. Yet, its merits as a computational exploration of human names are more valuable as it provides an understanding into how complex human names are and how technology must be moulded to account for that.
Keywords
Keywords
◼
Machine Learning
◼
Linguistics
◼
Names
Acknowledgements
Acknowledgements
I would like to thank my mentor, Aaron Enright, for the help and guidance throughout. I would also like to thank the TAs and everyone else associated with the Summer Camp as this would not have been possible without their support.