This notebook scrapes state by state current coronavirus data from the New York Times web page shown in WebExecute[...]. This page is more frequently updated than the GitHub database.
Note: this notebook may have to be revised if web page is changed. Select the desired browser below.
Note: this notebook may have to be revised if web page is changed. Select the desired browser below.
In[]:=
session=StartWebSession[(*"Chrome"*)"Firefox"(*,Visible->False*)];
In[]:=
WebExecute["OpenPage""https://www.nytimes.com/interactive/2020/us/coronavirus-us-cases.html"];
If you don’t have a subscription you may have to scroll down the page to have the ‘Show more’ button appear or you can just click the button manually to get the whole table, rather than find the button and click it with the procedures below.
Chrome XPath
//*[@id=”coronavirus-us-cases”]/div/div/div[3]/div/button
//*[@id=”coronavirus-us-cases”]/div/div/div[3]/div/button
Firefox XPath
/html/body/div[1]/main/article/section/div/div/div[3]/div/button
/html/body/div[1]/main/article/section/div/div/div[3]/div/button
In[]:=
Clear[element];If[session["Browser"]"Chrome",element=WebExecute["LocateElements""XPath""//*[@id=\"coronavirus-us-cases\"]/div/div/div[3]/div/button"]];If[session["Browser"]"Firefox",element=WebExecute["LocateElements""XPath""/html/body/div[1]/main/article/section/div/div/div[3]/div/button"]];Pause[1];WebExecute["ClickElement"element[[1]]]
Out[]=
Success
After the full table is open this can be executed, otherwise only the abbreviated table will be loaded.
In[]:=
table=WebExecute["LocateElements""Tag""table"]
Out[]=
WebElementObject,WebElementObject
In[]:=
tableText=WebExecute["ElementText"#]&/@table;
In[]:=
tableText[[1]]
Out[]=
STATE CONFIRMED CASES DEATHSNew York 142 0Washington 141 20California 112 2Massachusetts 28 0Texas 24 0Nebraska 16 0Oregon 13 0Florida 13 2Georgia 11 0New Jersey 11 0Colorado 9 0Illinois 7 0Pennsylvania 7 0South Carolina 6 0Arizona 5 0Maryland 5 0New Hampshire 4 0Nevada 4 0Kentucky 4 0Rhode Island 3 0Tennessee 3 0Virginia 3 0Iowa 3 0Ohio 3 0Utah 2 0North Carolina 2 0Indiana 2 0Minnesota 2 0Hawaii 2 0Wisconsin 1 0Oklahoma 1 0Kansas 1 0District of Columbia 1 0Missouri 1 0Vermont 1 0Connecticut 1 0Louisiana 1 0
In[]:=
tableText[[2]]
Out[]=
HOW VIRUS WAS CONTRACTED CONFIRMED CASESUnknown 258Cluster connected to a community in New Rochelle, N.Y. 54Nursing facility in Kirkland, Wash. 51Diamond Princess cruise ship 43Personal contact in U.S. 39Travel overseas 32Travel in Italy 28Business conference in Boston 22Travel in Egypt 21Grand Princess cruise in February 18
In[]:=
DeleteObject[session]
In[]:=
dataStrs=StringSplit[tableText[[1]],"\n"];data0=StringCases[#,{LetterCharacter..,DigitCharacter..,DigitCharacter..}]&/@dataStrs;data=If[Length[#]>3,{StringJoin@Riffle[Take[#,Length[#]-2]," "],ToExpression@#[[-2]],ToExpression@#[[-1]]},{#[[1]],ToExpression@#[[-2]],ToExpression@#[[-1]]}]&/@data0;nytDataset=AssociationThread[{"State","Cases","Deaths"},#]&/@Rest[data]//Dataset
Out[]=
In[]:=
nytDataset[Total,{"Cases","Deaths"}]
Out[]=
In[]:=
chartelements=Map[PieChart[{#[[1]],#[[2]]}]&,Values@Normal@nytDataset[All,{"Cases","Deaths"}]];
GeoBubbleChart[Map[If[#[[3]]>0||#[[2]]>15,Callout[Interpreter["USState"][#[[1]]]#[[2]],Column[{Style[#[[1]],Bold],"Cases:",#[[2]],"Deaths:",#[[3]]}],Right],Interpreter["USState"][#[[1]]]#[[2]]]&,Values@Normal@nytDataset[All,{"State","Cases","Deaths"}]],ChartElementschartelements,ImageSize900,GeoRange{{24.,50.},{-130.,-60.}}]