Machine learning has recently gained popularity for analysing language, syntax and semantics from large databases . This is not just limited to humans, but other species as well; machine learning can be used to analyse acoustic communication in non human species . Cetaceans like dolphins and whales have the second biggest brain to body ratio (after humans) and are both capable of vocal production learning, which means that they can mimic sounds and change them accordingly. They also have specialized brain cells called spindle neurons, which is linked to advanced capabilities like memory, problem-solving, communicating and reasoning. Sperm whales, in particular are considered to be highly intelligent; they usually communicate in coda click sounds; since the clicks are discrete, machine learning tools can be applied to discover and analyse existing patterns. The goal of the project was to use machine learning techniques to analyse sperm whale bioacoustics. A convolutional neural network was trained to create an echolocation click detector to analyse spectrograms from acoustic data. Then, the different types of codas were classified using a Recurrent Neural Network and Classify; different sperm whale clans were distinguished using the same strategy, and the codas were converted into musical notes.
Create a training set using spectrograms from several audio files
Create a training set using spectrograms from several audio files
◼
The audio files were imported from Watkins Marine Mammal Sound Database
In[]:=
a1=Import[File["/Users/apple/Desktop/1_93570006.wav"]]
Out[]=
Partition the audio files into small windows containing no more than one "click"
Partition the audio files into small windows containing no more than one "click"
◼
Spectrograms were obtained for each audio file and were then partitioned, so that every cell contains just one click; partitioning and resizing also helped maintain homogeneity, which further contributed to a better trained CNN.
In[]:=
Spectrogram[AudioTrim[a1,2],AspectRatioAutomatic,FrameFalse]
Out[]=
In[]:=
ImagePartition
,{50,444}
Out[]=
,
,
,
,
,
,
,
,
,
,
,
,
,
In[]:=
MapImageResize[#,{64,64}]&,
,
,
,
,
,
,
,
,
,
,
,
,
,
Out[]=
,
,
,
,
,
,
,
,
,
,
,
,
,
Build a training set and a test set of examples containing "clicks" and "no clicks"
Build a training set and a test set of examples containing "clicks" and "no clicks"
◼
Cells containing clicks and no clicks were randomly selected to create a training set; the remaining cells were used as a test set
In[]:=
clicks=RandomSample
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
;
In[]:=
noClicks=RandomSample
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
;
Create a simple Classify
Create a simple Classify
In[]:=
whaleClickClassify=Classify[trainingSet,PerformanceGoal"Quality"]
Out[]=
ClassifierFunction
In[]:=
ClassifierMeasurements[whaleClickClassify,testSet]
Out[]=
Classify type of Codas (type of messages) using a Classify
Classify type of Codas (type of messages) using a Classify
◼
Sperm whales are part of highly matrilineal families in which the females and the offspring make group decisions to travel. Family members are known to defend and raise calves too. Sperm whales usually communicate in short, 2 second duration clicks that form different patterns and can be categorized into recognizable types called codas. Each clan usually contains over 20 codas- these are thought to contain information about the whale’s identity, however what it communicates is still a mystery. Here a simple classifier method is used to distinguish between different types of codas. It can also be seen that the class 1+1+3 has been observed quite a few times, suggesting that the particular coda is an integral part of whale communication (might be whale for “hello”).
In[]:=
dataset2=SemanticImport["/Users/apple/Downloads/Classifying types of codas.xlsx"]
Out[]=
Distinguish different Clans using Classify (different communities of sperm whales)
Distinguish different Clans using Classify (different communities of sperm whales)
Converting codas into sound notes
Converting codas into sound notes
◼
Create an ordered scale of codas by using a nearest function
Creating sound notes for each coda and generating music based on their frequency
Creating sound notes for each coda and generating music based on their frequency
Concluding Remarks
Concluding Remarks
While in this project, sperm whale communication patterns have been analysed, the same approach can be applied to other whale species as well, such as humpbacks, which are said to communicate in songs instead of clicks. A larger data set can be used to train the CNN better, depending on availability of audios. Another possible extension would be to compare the musical notes generated to existing codas and comment on the similarity between them both. Advances in Machine Learning have now made it possible to obtain extraordinary insight into non-human communication; understanding cetacean better also holds the potential to appreciate and discover the complexity and diversity of inter-species communication as well as advancing other technologies like gentle robots and powerful underwater acoustic sensors.
Key Words
Key Words
◼
Wolfram Language
◼
Biological Sciences
◼
Machine Learning
◼
Audio Visualizations
◼
Data Science
◼
Natural Language processing
Acknowledgement
Acknowledgement
◼
I would like to thank Dr. Stephen Wolfram for the idea behind the project and the approaches to take with the same. This project would not have been possible without the support and help from my incredible mentor Jofre Espigule . I would also like to thank Dr Tuseeta Banerjee for providing helpful insight into deep learning approaches that could aid this project.
References
References