Apple mobility trends data visualization
Apple mobility trends data visualization
Version 0.9
Anton Antonov
MathematicaForPrediction at WordPress
SystemModeling at GitHub
April 2020
May 2020
December 2020
MathematicaForPrediction at WordPress
SystemModeling at GitHub
April 2020
May 2020
December 2020
Introduction
Introduction
I this notebook we ingest and visualize the mobility trends data provided by Apple, [APPL1].
We take the following steps:
1
.Download the data
2
.Import the data and summarise it
3
.Transform the data into long form
4
.Partition the data into subsets that correspond to combinations of geographical regions and transportation types
5
.Make contingency matrices and corresponding heat-map plots
6
.Make nearest neighbors graphs over the contingency matrices and plot communities
7
.Plot the corresponding time series
Data description
Data description
From Apple’s page https://www.apple.com/covid19/mobility
From Apple’s page https://www.apple.com/covid19/mobility
About This Data
The CSV file and charts on this site show a relative volume of directions requests per country/region or city compared to a baseline volume on January 13th, 2020.
We define our day as midnight-to-midnight, Pacific time. Cities represent usage in greater metropolitan areas and are stably defined during this period. In many countries/regions and cities, relative volume has increased since January 13th, consistent with normal, seasonal usage of Apple Maps. Day of week effects are important to normalize as you use this data.
Data that is sent from users’ devices to the Maps service is associated with random, rotating identifiers so Apple doesn’t have a profile of your movements and searches. Apple Maps has no demographic information about our users, so we can’t make any statements about the representativeness of our usage against the overall population.
The CSV file and charts on this site show a relative volume of directions requests per country/region or city compared to a baseline volume on January 13th, 2020.
We define our day as midnight-to-midnight, Pacific time. Cities represent usage in greater metropolitan areas and are stably defined during this period. In many countries/regions and cities, relative volume has increased since January 13th, consistent with normal, seasonal usage of Apple Maps. Day of week effects are important to normalize as you use this data.
Data that is sent from users’ devices to the Maps service is associated with random, rotating identifiers so Apple doesn’t have a profile of your movements and searches. Apple Maps has no demographic information about our users, so we can’t make any statements about the representativeness of our usage against the overall population.
Observations
Observations
The observations listed in this subsection are also placed under the relevant statistics in the following sections and indicated with “Observation”.
◼
The directions request volumes reference date for normalization is 2020-01-13 : all the values in that column are .
100
◼
From the community clusters of the nearest neighbor graphs (derived from the time series of the normalized driving directions requests volume) we see that countries and cities are clustered in expected ways. For example, in the community graph plot corresponding to “{city, driving}” the cities Oslo, Copenhagen, Helsinki, Stockholm, and Zurich are placed in the same cluster. In the graphs corresponding to “{city, transit}” and “{city, walking}” the Japanese cities Tokyo, Osaka, Nagoya, and Fukuoka are clustered together.
◼
In the time series plots the Sundays are indicated with orange dashed lines. We can see that from Monday to Thursday people are more familiar with their trips than say on Fridays and Saturdays. We can also see that on Sundays people (on average) are more familiar with their trips or simply travel less.
Load packages
Load packages
In[]:=
Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/DataReshape.m"]Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/Misc/HeatmapPlot.m"]
Data ingestion
Data ingestion
Apple mobile data was provided in this WWW page: https://www.apple.com/covid19/mobility , [APPL1]. (The data has to be download from that web page -- there is an “agreement to terms”, etc.)
In[]:=
dsAppleMobility=ResourceFunction["ImportCSVToDataset"]["~/Downloads/applemobilitytrends-2021-01-15.csv"]
Out[]=
Observation: The directions requests volumes reference date for normalization is 2020-01-13 : all the values in that column are .
100
Data dimensions:
In[]:=
Dimensions[dsAppleMobility]
Out[]=
{4691,375}
Data summary:
In[]:=
Magnify[ResourceFunction["RecordsSummary"][dsAppleMobility],0.6]
Out[]=
Number of unique “country/region” values:
In[]:=
Length[Union[Normal[dsAppleMobility[Select[#["geo_type"]=="country/region"&],"region"]]]]
Out[]=
63
Number of unique “city” values:
In[]:=
Length[Union[Normal[dsAppleMobility[Select[#["geo_type"]=="city"&],"region"]]]]
Out[]=
295
All unique geo types:
In[]:=
lsGeoTypes=Union[Normal[dsAppleMobility[All,"geo_type"]]]
Out[]=
{city,country/region,county,sub-region}
All unique transportation types:
In[]:=
lsTransportationTypes=Union[Normal[dsAppleMobility[All,"transportation_type"]]]
Out[]=
{driving,transit,walking}
Data transformation
Data transformation
It is better to have the data in long form (narrow form). For that I am using the package "DataReshape.m", [AAp1].
In[]:=
(*lsIDColumnNames={"geo_type","region","transportation_type"};*)(*FortheinitialdatasetofApple'smobilitydata.*)lsIDColumnNames={"geo_type","region","transportation_type","alternative_name","sub-region","country"};dsAppleMobilityLongForm=ToLongForm[dsAppleMobility,lsIDColumnNames,Complement[Keys[dsAppleMobility〚1〛],lsIDColumnNames]];Dimensions[dsAppleMobilityLongForm]
Out[]=
{1730979,8}
Remove the rows with “empty” values:
In[]:=
dsAppleMobilityLongForm=dsAppleMobilityLongForm[Select[#Value≠""&]];Dimensions[dsAppleMobilityLongForm]
Out[]=
{1709416,8}
Rename the column “Variable” to “Date” and add a related “DateObject” column:
In[]:=
AbsoluteTiming[dsAppleMobilityLongForm=dsAppleMobilityLongForm[All,Join[KeyDrop[#,"Variable"],<|"Date"#Variable,"DateObject"DateObject[#Variable]|>]&];]
Out[]=
{714.062,Null}
Add “day name” (“day of the week”) field:
In[]:=
AbsoluteTiming[dsAppleMobilityLongForm=dsAppleMobilityLongForm[All,Join[#,<|"DayName"->DateString[#DateObject,{"DayName"}]|>]&];]
Out[]=
{498.026,Null}
Here is sample of the transformed data:
In[]:=
SeedRandom[3232];RandomSample[dsAppleMobilityLongForm,12]
Out[]=
Here is summary:
Partition the data into geo types × transportation types:
Basic data analysis
Basic data analysis
We consider relative volume o directions requests for the last date only. (The queries can easily adjusted for other dates.)
Here we plot histograms and Pareto principle adherence:
Heat-map plots
Heat-map plots
Remark: Using the contingency matrices prepared for the heat-map plots we can do further analysis, like, finding correlations or nearest neighbors. (See below.)
Cross-tabulate dates with regions:
Make a heat-map plot by sorting the columns of the cross-tabulation matrix (that correspond to countries):
(We use Rasterize in order to reduce the size of the notebook.)
Here we take closer look to one of the plots:
Nearest neighbors graphs
Nearest neighbors graphs
Graphs overview
Graphs overview
Here we create nearest neighbor graphs of the contingency matrices computed above and plot cluster the nodes:
Closer look into the graphs
Closer look into the graphs
Here we endow each nearest neighbors graph with appropriate vertex labels:
Here we plot the graphs with clusters:
Observation: From the community clusters of the nearest neighbor graphs (derived from the time series of the normalized driving directions requests volume) we see that countries and cities are clustered in expected ways. For example in the community graph plot corresponding to “{city, driving}” the cities Oslo, Copenhagen, Helsinki, Stockholm, and Zurich are placed in the same cluster. In the graphs corresponding to “{city, transit}” and “{city, walking}” the Japanese cities Tokyo, Osaka, Nagoya, and Fukuoka are clustered together.
Time series analysis
Time series analysis
Time series
Time series
In this section for each date we sum all cases over the region-transportation pairs, make a time series, and plot them.
Remark: In the plots the Sundays are indicated with orange dashed lines.
Here we make the time series:
Here we plot them:
Observation: In the time series plots the Sundays are indicated with orange dashed lines. We can see that from Monday to Thursday people are more familiar with their trips than say on Fridays and Saturdays. We can also see that on Sundays people (on average) are more familiar with their trips or simply travel less.
“Forecast”
“Forecast”
He we do “forecast” for code-workflow demonstration purposes -- the forecasts should not be taken seriously.
Fit a time series model to the time series:
Plot data and forecast:
References
References
[AAp1] Anton Antonov, Data reshaping Mathematica package, (2018), MathematicaForPrediciton at GitHub.