Nonparametric Additive Modeling by Smoothing Splines: Robust Unbiased-Risk-Estimate Selector and a Nonisotropic-Smoothing Improvement
Nonparametric Additive Modeling by Smoothing Splines: Robust Unbiased-Risk-Estimate Selector and a Nonisotropic-Smoothing Improvement
This Demonstration is dedicated to the memory of François de Crécy.
An earlier series of Demonstrations [1–3] concerned the well-known cross-validation approach to optimally estimate smooth univariate regression functions. Notably, [2] analyzes, on simple examples, how one can construct good confidence statements (asymptotically justified) about the -optimal amount of smoothing, and it is demonstrated by [3] that the selector (a variant of cross-validation) can be easily robustified via randomized choices. The present Demonstration discusses an extension of the latter robustification method to a simple bivariate context. Specifically, we model and compute using the well-known backfitting algorithm, by additive cubic smoothing splines. Furthermore, this Demonstration introduces a natural nonisotropy in the smoothing operators.
L
2
C
L
In detail, this Demonstration considers the estimation of a smooth additive function of two variables, here over . It is given that only triplets (,,) are known for , which satisfy the model =μ+++σ, where the are independent, standard normal random variables. However, for the sake of simplicity, we also assume that the variance is known.
f(,)=μ+()+()
X
1
X
2
f
1
X
1
f
2
X
2
[0,1]×[0,1]
n
X
1,i
X
2,i
Y
i
i1,2,…,n
Y
i
f
1
X
1,i
f
2
X
2,i
ϵ
i
ϵ
i
2
σ
In a first step, solve
(λ):+λ(x)dx+(x)dx
min
c,(·),(·)
g
1
g
2
n
∑
i=1
2
c++-
g
1
X
1,i
g
2
X
2,i
Y
i
2
2
d
d
2
x
g
1
2
2
d
d
2
x
g
2
for all the in a grid of equally spaced (in log scale) trial values. Recall that assuming a standard mean-zero restriction on each of the components and (see [4]) for any , generically gives a unique solution , denoted by , the being omitted when there is no ambiguity, where the two components and are cubic spline functions.
λ
g
1
g
2
λ>0
(λ)
c=,,
1
n
n
∑
i=1
Y
i
g
1
g
2
,,
μ
f
1,λ
f
2,λ
λ
f
1,λ
f
2,λ
Of course, giving the same weight (here ) to the two integrals of criterion (1) (the two so-called smoothness energies) is rather arbitrary, and there is room for improvement even without introducing differing weights, which would complicate their robust selection by percentiles of randomized unbiased-risk estimators.
λ
As a second step, introduce a dilation of one of the two explanatory variables, say , and denoting by the dilation factor, write =α,i=1,…,n. At the new observation sites, we consider the second problem:
X
2
α
X
2,i
X
2,i
(α,λ):+λ(x)dx+()d.
min
c,(·),(·)
g
1
g
2
n
∑
i=1
2
c++-
g
1
X
1,i
g
2
X
2,i
Y
i
2
2
d
d
2
x
g
1
2
2
d
d
2
x
g
2
x
x
We solve this problem again for a grid of values, after having tuned by the simple formula (see the Details section for its justification):
λ
α
α
1/3
(x)dx(x)dx
2
2
d
d
2
x
f
2,
λ
2
2
d
d
2
x
f
1,
λ
where and are the two components given by , with selected by the robust method corresponding to the percentile. (This is similar to the univariate context in [3], except that here, the randomized degrees of freedom are drawn from a list that has been previously computed.)
f
1,
λ
f
2,
λ
λ
λ
C
L
th
99
Let be the minimizer of , where is selected, among the grid of values mentioned above, by a fast robust method. "Fast" means here that instead of precisely computing a percentile, we chose only the fifth largest among six randomized choices. This Demonstration displays the two smooth components ↦(),↦, taking into account the tuned dilation that was used, denoted by .
(c,,)
g
1
g
2
,
α
λ
λ
λ
C
L
λ
X
1
g
1
X
1
X
2
g
2
α
X
2
,
f
1
f
2