The aim of this project is the implementation of an optimal fractional differentiation procedure for time series that fit the ‘long-memory’ condition. Moreover, since there is a trade-off between the informative dependence of the time series and the stationarity needed to make statistical inferences, this project proposes a recursive binary search method to get the optimal (minimum) non-integer order of differentiation to make the time series (weakly) stationary, while preserving as much as memory as possible. This procedure will be tested on several Fractional Brownian motion simulations using different values for the Hurst exponent between 1/2 and 1. This will allow to get an empirical relationship between the latter as a measure of the degree of dependency in a time series, and the optimal non-integer order of differentiation. The hypothesis is that this relationship tends to be linear, being the classical d(1) order of differentiation only optimal mostly when H is closer to 1. Thus, the project starts by implementing the non-integer differentiation method for time series developed by Lopez de Prado (2018). Then, a new recursive binary search algorithm to find the optimal non-integer “d” is proposed, due to its efficient computational time, degree of precision and simplicity. This will be tested over the Fractional Brownian Motion simulations, proving the linear relationship between the Hurst exponent and the optimal “d”. Finally, some general considerations about the differentiation procedure implemented are outlined, drawing possible improvements for it, as well as for the developed search algorithm.
Why Fractional Differentiation for Time Series?
Why Fractional Differentiation for Time Series?
The first thing necessary to understand is why fractional differentiation might be a better method in the context of time series analysis. Essentially, some time series show a high level of statistical dependence between two points over time, well-known as “long-memory”. This informational dependency is a consequence of the fact that each new state depends upon a “long” history of previous states. However, statistical modelling faces some difficulties in dealing with this condition since most of its procedures work over the assumption of IID processes. Thus, practitioners are constantly seeking for stationarity by implementing integer differentiations as the standard transformation tool. Then, they try to recap the marginal signal in this new transformed time series, leading with either a restricted set of models or a complex combination of them. In the quantitative finance area, for instance, a time series of prices is transformed by applying a differentiation of order d(1) as a rule of thumb, which leads to work with a series of difference of prices. Although this transformation achieves stationarity, it comes at the expense of removing all memory from the original time series; that is, its trend, which is the key feature of any model’s predictive power. Note that any differentiation on a time series will remove some of its memory by default. However, in the majority of cases, it is not necessary to remove all of them just to achieve the stationarity condition. Additionally, since time series are mostly non-invariant processes, it is also difficult to achieve strong stationarity. This is why, commonly, it is more than enough to reach weak stationarity to afford any statistical inference. Therefore, instead of looking for an integer differentiation, such as d(1), a fractional differentiation procedure could be done instead, such as:
d(*),0<(*)<1
In that sense, the project could be divided as follows. In the first part (Part I), the Fractional Differentiation Method for Time Series will be formally described, including its implementation using Wolfram language. In the same section, the project is going to define a recursive binary search algorithm to find the optimal order of differentiation. Then, in the second part (Part II), this search algorithm to find will be tested over many Fractional Brownian Motion simulations for different values of the Hurst Exponent , that, in the context of long-memory processes, lies between 1/2 and 1. Finally, the project finishes with the “Concluding Remarks” section that summarizes the key ideas of all of these procedures.
d(*)
H
Part I: The Fractional Differentiation Method
Part I: The Fractional Differentiation Method
Formal Definition
Formal Definition
Say a time series can be differentiated by the Finite Difference Operator , such as =, ∀n∈ℕ, where = is the Backward Shift Operator. Thus, the goal is:
X={, , ,...}
x
t
x
t−1
x
t−2
n
∇
n
∇
x
t
n
(1-B)
x
t
n
B
x
t
x
t-n
𝑛∈ℕ→𝑑∈
So, considering the Binomial Series Expansion for non-integer exponents =
,d∈, it is possible to define a Finite Difference Operator for non-integer exponents as well, such as:
d
(1+x)
∞
∑
k=0
d |
k |
k
x
d
∇
d
(1−B)
∞
∑
k=0
k
(−B)
d |
k |
Now, considering , it is feasible to use , where . However, since the Gamma Function is a meromorphic function when, could be better to use the generalized definition of the binomial coefficients instead, such as:
=
n |
k |
n!
k!(n-k)!
=,∀𝑑∈
d |
k |
Γ(d+1)
k!Γ(d-k+1)
Γ(*):=
(*)!->
(*)<0
==(d−i)=
d |
k |
d(d−1)(d−2)…(d−k+1)
k!
k−1
∏
i=0
k!
k−1
∏
i=0
d−i
k−i
So, replacing this term in our difference operator, it is viable to get:
d
∇
d
(1−B)
∞
∑
k=0
k
k−1
∏
i=0
d−i
k−i
Simplifying, the Fractional Difference Operator will be:
d
∇
∞
∑
k=0
k
w
d,k
w
d,k
k
(−1)
k−1
∏
i=0
d−i
k−i
d
∇
d(d−1)
2!
2
B
d(d−1)(d−2)
3!
3
B
Hence, by applying the above definition in , such as = B, it is possible to define the Fractional Differentiation Method for Time Series:
X
d
∇
x
t
∞
∑
k=0
k
w
d,k
x
t
d
∇
x
t
∞
∑
k=0
w
d,k
x
t-k
w
d,k
d(d−1)
2!
d(d−1)(d−2)
3!
k−1
∏
i=0
k!
Finally, Lopez de Prado (2018) proposes an Iterative Fixed-Window method to update {} over the time series for a given , such as:
w
d,k
d
w
d,k
−w
d,k−1
(d−k+1)
k
Particularly, we are looking for all the cases:
||>τ,τ->0
w
d,k
Being τ := minimum weight (tolerance) to consider any data point as useful information to be preserved.
Finally, it is possible to use Wolfram language to implement the method defined above.
Finally, it is possible to use Wolfram language to implement the method defined above.
Wolfram Implementation
Wolfram Implementation
In[]:=
WeightRule[diff_,threshold_]:=Module[{d=diff,t=threshold,k=1,wgtlist={1.},wformulae=(-#1[[-1]]*((#2-#3+1)/#3))&},While[True,lastweight=wformulae[wgtlist,d,k];If[Abs@lastweight<t,Break[],k++];AppendTo[wgtlist,lastweight]];Return@Reverse@wgtlist]
In[]:=
FractionalDSeries[ts_TemporalData,d_,t_:0.0001]:=Module[{timeSeries=ts,diffOrder=d,tolerance=t},FractionalDSeries::tol="Argument `1` causes more weights than total observations. Please, increase it.";weights=WeightRule[diffOrder,tolerance];totalWeights=Length@weights;timeSeriesValues=QuantityMagnitude@timeSeries["Values"];If[Length@timeSeriesValues<totalWeights,Message[FractionalDSeries::tol,"'t'"]&&Abort[]];idxDiff=Range[totalWeights,Length@timeSeriesValues];fdSeries=(weights.timeSeriesValues[[#[[1]];;#[[2]]]])&/@Transpose[{Range[1,idxDiff//Length],idxDiff}];Return@TimeSeries[fdSeries,{timeSeries["Dates"][[totalWeights;;]]},TemporalRegularity->True];]
So, let’s start by testing this function using some financial data; particularly, the time series of logarithmic prices of NYSE: $GE (General Electric Inc.) since 2000 until today:
ts=Log@QuantityMagnitude@FinancialData["GE","Close","Jan. 1, 2000"];
Now, by defining a specific set of {} values between 0 and 1, it is possible to get a nice representation of the effects of the fractional differentiation procedure. Also, to see the consequences of the differentiation on the memory, an Auto-Correlation Function (ACF) is plotted with a maximum lag of 20 data-points.
d
In[]:=
baseDs=Range[0,1,0.005];setFSeries=Table[{d,FractionalDSeries[ts,d]},{d,baseDs}];
In[]:=
ListAnimate[Table[dTested=listfseries[[1]];tsData=listfseries[[2]];autoCorrData=CorrelationFunction[tsData,{1,20}];minfseries=Min@tsData;maxfseries=Max@tsData;minACF=Min@autoCorrData;maxACF=Max@autoCorrData;Row[{ListLinePlot[tsData,PlotLabel->Style["Fractional Diff. TSeries | d = "<>ToString@dTested,Lighter@Black,13,Bold],PlotStyle->{Lighter@Blue},PlotRange->{If[Negative@minfseries,minfseries*1.05,minfseries*.95],maxfseries*1.05},ImageSize320,AxesLabel->{"Time","(?)"}],ListPlot[autoCorrData,PlotLabel->Style["ACF | d = "<>ToString@dTested,Lighter@Black,13,Bold],PlotStyle->{Lighter@Blue},PlotRange->{If[Negative@minACF,minACF*1.2,minACF*0.8],maxACF*1.02},ImageSize320,Filling->Axis,AxesLabel->{"Lag","AC"}]}," "],{listfseries,setFSeries}],AnimationRunningFalse,SaveDefinitionsTrue]
Out[]=
It is quite obvious that, when , the correlogram shows less memory in the time series for different lags, in such a way that for all memory is erased. Thus, the closer , the better to the aim of memory preservation. Note also that the length of the differentiated time series is not the same for all the values of , because of the effect of the Iterative Fixed-Window method. Finally, the reader might notice also there is a “(?)” character that defines the y-axis in the differentiated time series plot. Just skip it by now, because it will be clarified at the end of this Part I.
d->1
d=1
d->0
d
Now, considering these observations, it is possible to get the optimal by the implementation of a binary search algorithm.
d(*)
The Recursive Binary Search Algorithm to find d(*)
The Recursive Binary Search Algorithm to find d(*)
A binary search is a search algorithm that allows to find the position of a specific value in a sorted set of elements. Essentially, given the array of values, such as , where <=<=<=...<=, the index position of a target φ value could be found by following the pseudocode defined below:
A
q
A={,,,...,}
A
1
A
2
A
3
A
q
A
1
A
2
A
3
A
q
function RecursiveBinarySearch [A, φ, min, max] is middlePoint := min + floor ((max - min) / 2) while min < max do if φ < A[middlePoint] then max := middlePoint + 1 run RecursiveBinarySearch[A, φ, min, max] else if φ > A[middlePoint] then min:= middlePoint - 1 run RecursiveBinarySearch[A, φ, min, max] else return middlePoint return Error
Part II: The Fractional Brownian Motion (fBm) Experiment
Part II: The Fractional Brownian Motion (fBm) Experiment
Please, visit the full study: https://www.wolframcloud.com/obj/fsyrosas/Published/fracdiff_fbm_study.nb
Concluding remarks
Concluding remarks
◼
A typical unit root test (i.e., Dickey Fuller F test) is a good starting point to assess stationarity, but it is not necessarily sufficient.
◼
Getting results for d < 0 (fractional integration) in time series is possible, but...
This is probably the most immediate and visible drawback of this method for real implementations. As already explained, a fractional differentiation does not allow to understand exactly the type of information that is currently available for the aim of modelling. In some particular contexts, this could be a problem. For instance, considering the point of view of Explainable Machine Learning, working with this type of time series as the base dataset would not contribute to being able to understand the results obtained by the modeling from a discretionary point of view, since, essentially, the new differentiated series is practically incomprehensible. This requires a delicate judgment of the practitioner to find the appropriate environment for the implementation of this method.
Keywords
Keywords
◼
Fractional Differentiation
◼
Fractional Brownian motion
◼
Stochastic analysis
◼
Time Series
Acknowledgment
Acknowledgment
The author wishes to express his immense gratitude to all the people who made the development of this project possible. First of all, to Gosia Konwerska, for her kindness, support and brilliant ideas on this subject that were useful to accomplish the goal of the program: definitely, this project would not have gotten to where it is without her. Likewise, the author would like to express his gratitude to Oleg Marichev, for his courtesy and brilliance, as well as for being an inspiration in the deeper branch of this subject that the author seeks to learn. Likewise, the author also wishes to express his gratitude to his mentor Jesse Galef, for providing a practical and adequate guidance for an appropriate development within the program. Last but not least, the author expresses his enormous gratitude to Stephen Wolfram, for having insisted (and how right he was!) on the development of this topic as a project. Of course, the author extend this thanks to the entire team behind the organization of the WSS22, especially to Mads Bahrami, for being the architect who allowed the author to have this amazing opportunity.
Full Notebook
Full Notebook
There are some conclusions that could be better understood by taking a look at the full notebook: https://www.wolframcloud.com/obj/fsyrosas/Published/fracdiff_fbm_study.nb
References
References
◼
Lopez de Prado, M. (2018). Advances in Financial Machine Learning. Wiley, New Jersey, (75-88).
◼
Samko, S., Kilbas, A. and Marichev, O. (1993). Fractional integrals and derivatives: theory an applications. Gordon and Breach Science Publishers, Philadelphia, (7-23).
◼
Walasek, R. and Gajda, j. (2021). “Fractional differentiation and its use in machine learning”. International Journal of Advances in Engineering Sciences and Applied Mathematics, Vol.13, No. 3, pp. 270-277. https://doi.org/10.1007/s12572-021-00299-5
◼
Dai, W. and Heyde, C.C. (1996). “Ito’s formula with respect to Fractional Brownian motion and its application”. Journal of Applied Mathematics and Stochastic Analysis, Vol. 9, No. 4, pp. 439-448. https://doi.org/10.1155/S104895339600038X