Temporal evolution of cond-mat topics on Arxiv​
​by Jiahao Fan

Introduction

Wouldn’t it be interesting to know how your scientific field evolves over time? I am pursuing a PhD in condensed matter theory, so I like to see how this field has progressed over the last three decades. To achieve that, we investigated the article data on ArXiv. We extracted the essential information via neural network transformers from all the titles and abstracts in condensed matter physics. To created feature vectors that encode the content of an article, we trained a long-short-term-memory network to reproduce the topic keys provided by ArXiv. Dimension reduction then helped to project the high-dimensional feature vectors into 2- and 3-dimensional feature plots. The resulting dynamics of the point clouds reveals the evolution of the field.

Web crawling the ArXiv articles related to cond-mat from 1992-2020.

First, we use Python from within Mathematica to obtain the data via an ArXiv API. We downloaded the information of the articles related to condensed matter physics from 1992-2020 in an XML format.

Generate article information list from the XML file.

Then we import the XML file into Mathematica and extract the useful information like article IDs, titles and abstracts into an Association. For further processing we store the information in wxf-files group by the publication year and month.

Using SciBERT to embed the title and abstract information in arrays.

Then we use the neural network SciBert in the Wolfram Network Repository to convert the title and abstract texts into arrays. This is called “embedding” in the NLP community. The reason we do this embedding before training is that we do not need to add embedding layers to our training-used neural network so as to facilitate the training process.

Train a network to learn the connection between the title-abstract and the topic of articles.

To obtain just one feature vector per article, we had to train our own LSTM neural network. We applied supervised learning based on the relation between title & abstract and the topic keys that were provided with each article. The idea behind this training is the hope to generate some high-dimensional feature vector inside the network which then represents the information encoded in the titles and abstracts.

Dimension reduction of the feature vectors and see the evolution through time

In the final step, we apply dimension reduction to the feature vectors using tSNE. Thereby, every article is represented by a point in a feature space. Using ListAnimate youu can see the evolution of the reduced feature vectors through time. Here I only show the evolution from 1992-1998 due to the limited time.
Out[]=
cond-mat
cond-mat.stat-mech
cond-mat.str-el
cond-mat.mes-hall
cond-mat.supr-con
cond-mat.mtrl-sci
cond-mat.dis-nn
cond-mat.soft
cond-mat.dis-nn cond-mat.stat-mech
cond-mat.str-el cond-mat.supr-con

Conclusions

Although it is not a very long time. The evolution could tell us something already. First of all we could see that the number of the articles increases every year, which is obvious. Secondly we could see that some clusters come close to each other, like in 1996 the Mesoscale and Nanoscale Physics (mess-hall) categories cluster is very close to Strongly Correlated Electrons (str-el), indicating the connection between this two fields.In 1998 we could see that the superconductor (supr-con) and str-el clusters come very close, indicating an intimate connection.

Keywords

◼
  • Natural Language processing
    • ◼
  • Neural Network
    • ◼
  • Dimension Reduction
    • ◼
  • Condensed Matter Physics

    • Acknowledgment

    Mentor: Markus van Almsick
    I would like to thank Markus for his patient and kind help on my project, I learned a lot of useful skills and ideas about Wolfram Language progamming.