eia.org : US Petroleum & Other Liquids
Dara O Shayda
dara@compclassnotes.com
dara@compclassnotes.com
In[]:=
DateObject[]
Out[]=
Abstract
Since 2007 there has been a significant increase in the export of US refined petroleum products to the point that many professional voters, taxpayers and business owners have become concerned. This live-code notebook contains the proper access to the cloud systems at eia.gov and accurate visualization and wavelet analysis of the export datasets. The findings are conclusive on the gigantic increase of the US petroleum based products.
© 2012-Present CCN StudiosCreative Commons Attribution-NonCommercial-ShareAlike 4.0
Data Source
● https://www.eia.gov/dnav/pet/pet_move_exp_dc_nus-z00_mbblpd_m.htm
● Click on Sort Results right column
https://www.eia.gov/opendata/documentation.php
● https://www.eia.gov/dnav/pet/pet_move_exp_dc_nus-z00_mbblpd_m.htm
● Click on Sort Results right column
https://www.eia.gov/opendata/documentation.php
Distillate Fuel Oil: export
A general classification for one of the petroleum fractions produced in conventional distillation operations. It includes diesel fuels and fuel oils. Products known as No. 1, No. 2, and No. 4 diesel fuel are used in on-highway diesel engines, such as those in trucks and automobiles, as well as off-highway engines, such as those in railroad locomotives and agricultural machinery. Products known as No. 1, No. 2, and No. 4 fuel oils are used primarily for space heating and electric power generation. [1]
(*consulttheeia.govsitetoobtainyourownapikey*)apikey=CloudSymbol["epi_api_key"];url="https://api.eia.gov/v2/petroleum/move/exp/data?api_key="<>apikey<>"&data[]=value&frequency=monthly&start=1950-01-31";data=Import[url];
Visualize data
In[]:=
i=1;Association[Association[data[[2]][[2]]]["data"][[i]]]
Out[]=
period1994-01,duoareaR20-Z00,area-namePADD 2,productEPDM10,product-nameDistillate Fuel Oil, Greater than 15 to 500 ppm Sulfur,processEEX,process-nameExports,seriesMD1EX_R20-Z00_1,series-descriptionMidwest (PADD 2) Exports of Distillate Fuel Oil, Greater than 15 to 500 ppm Sulfur (Thousand Barrels),value1,unitsMBBL
product-name
In[]:=
Highlighted@Association[Association[data[[2]][[2]]]["data"][[i]]]["product-name"]
Out[]=
process-name
In[]:=
Association[Association[data[[2]][[2]]]["data"][[i]]]["process-name"]
Out[]=
Exports
series-description
In[]:=
Highlighted@Association[Association[data[[2]][[2]]]["data"][[i]]]["series-description"]
Out[]=
units
In[]:=
Highlighted@Association[Association[data[[2]][[2]]]["data"][[i]]]["units"]
Out[]=
MBBL
In U.S. energy reporting, Mbbl (or MBBL) stands for one thousand barrels. It is derived from the standard abbreviation “bbl” for barrel (42 US gallons), with the ‘M’ representing the Roman numeral for thousand.
process
In[]:=
Association[Association[data[[2]][[2]]]["data"][[i]]]["process"]
Out[]=
EEX
EU ETS and EEX: Oil refineries are covered under the European Union Emissions Trading System (EU ETS), which utilizes the European Energy Exchange (EEX) for auctioning emission allowances.
Does EEX indicate the petroleum products export to EU?
● unit: use the unit value to combine or filter the MBBL/D vs. MBBL. In this case set to 0 since we only seek the Monthly.
In[]:=
nums=Map[If[("value"/.#)=!=Null&&("process-name"/.#)=!=Null,If[("units"/.#)==="MBBL/D",unit=0,unit=1];(*Print[("units"/.#)];*)tmp=({"period","value"}/.#);tmp[[1]]=DateObject[tmp[[1]]];tmp[[2]]=unit*ToExpression[tmp[[2]]];tmp,Nothing]&,Association[data[[2]][[2]]]["data"]];(*grouptocombinedtheperioddatee.g.month*)assoc=GroupBy[nums,First];Clear[f];(*SwitchbetweenTotal[]toMean[]oranyotherfunctionoftwoarguments*)f[a_,b_]:={a,Total[Flatten[b/.{a->Nothing}]]};series=KeyValueMap[f,assoc];
Visualize series
In[]:=
Short[series,6]
Out[]//Short=
,6244,,2576,,3182,,2436,,1498,,3153,,1658,,2818,,2242,,3509,,4650,364,,4748,,1502,,2492,,2038,,2651,,2150,,3078,,4316,,33989,,36147
In[]:=
ticks=Rotate[DateString[#,{"Year"}],Pi/2.5]&/@DateRange[Min@series[[All,1]],Max@series[[All,1]],{2,"Year"}];DateListPlot[series,PlotRange->All,ImageSize->500,PlotRangePaddingNone,Frame->False,Ticks{Thread[{DateRange[Min@series[[All,1]],Max@series[[All,1]],{2,"Year"}],ticks}],Automatic},PlotLabel->Highlighted@"Distillate Fuel Oil",AxesLabel->{Automatic,Highlighted@"Mbbl"}]
Out[]=
Key Takeaways
● April 2007: Export 3,558 Mbbl
● July 2017: Export 55,700 Mbbl
● 15x spike that has never recovered from
● Such large spike never experiences in the US Oil markets since 1920!
● July 2017: Export 55,700 Mbbl
● 15x spike that has never recovered from
● Such large spike never experiences in the US Oil markets since 1920!
Crude Oil: export
A mixture of hydrocarbons that exists in liquid phase in natural underground reservoirs and remains liquid at atmospheric pressure after passing through surface separating facilities. Depending upon the characteristics of the crude stream, it may also include:
Small amounts of hydrocarbons that exist in gaseous phase in natural underground reservoirs but are liquid at atmospheric pressure after being recovered from oil well (casinghead) gas in lease separators and are subsequently commingled with the crude stream without being separately measured. Lease condensate recovered as a liquid from natural gas wells in lease or field separation facilities and later mixed into the crude stream is also included;
Small amounts of nonhydrocarbons produced with the oil, such as sulfur and various metals;
Drip gases, and liquid hydrocarbons produced from tar sands, oil sands, gilsonite, and oil shale. [
Liquids produced at natural gas processing plants are excluded. Crude oil is refined to produce a wide array of petroleum products, including heating oils; gasoline, diesel and jet fuels; lubricants; asphalt; ethane, propane, and butane; and many other products used for their energy or chemical content. [1]
Small amounts of hydrocarbons that exist in gaseous phase in natural underground reservoirs but are liquid at atmospheric pressure after being recovered from oil well (casinghead) gas in lease separators and are subsequently commingled with the crude stream without being separately measured. Lease condensate recovered as a liquid from natural gas wells in lease or field separation facilities and later mixed into the crude stream is also included;
Small amounts of nonhydrocarbons produced with the oil, such as sulfur and various metals;
Drip gases, and liquid hydrocarbons produced from tar sands, oil sands, gilsonite, and oil shale. [
Liquids produced at natural gas processing plants are excluded. Crude oil is refined to produce a wide array of petroleum products, including heating oils; gasoline, diesel and jet fuels; lubricants; asphalt; ethane, propane, and butane; and many other products used for their energy or chemical content. [1]
In[]:=
apikey=CloudSymbol["epi_api_key"];url="https://api.eia.gov/v2/petroleum/move/wkly/data/?api_key="<>apikey<>"&frequency=weekly&start=1990-01-31&data[]=value&facets[product][]=EPC0";data=Import[url];
In[]:=
Association[data[[2]][[2]]]["data"][[1]]
Out[]=
{period1991-02-08,duoareaNUS-Z00,area-nameU.S.,productEPC0,product-nameCrude Oil,processEEX,process-nameExports,seriesWCREXUS2,series-descriptionU.S. Exports of Crude Oil (Thousand Barrels per Day),value138,unitsMBBL/D}
In[]:=
nums=Map[If[("value"/.#)=!=Null&&("process-name"/.#)=!=Null,If[("units"/.#)==="MBBL/D",unit=1,unit=0];(*Print[("units"/.#)];*)tmp=({"period","value"}/.#);tmp[[1]]=DateObject[tmp[[1]]];tmp[[2]]=unit*ToExpression[tmp[[2]]];tmp,Nothing]&,Association[data[[2]][[2]]]["data"]];assoc=GroupBy[nums,First];Clear[f];f[a_,b_]:={a,Total[Flatten[b/.{a->Nothing}]]};series=KeyValueMap[f,assoc];
All Exports, Liquids and Gas
Frequencies
Scalogram
Scalogram 3D
Decompositions
References