QuadraticApproach::usage="Given a function f of two variables, its quadratic approximation at the point (a, b) it is given by:
Q
(a,b)
​(x, y) = f(a, b) +
f
x
​(a, b) (x - a) +
f
y
​(a, b) (y - b) +
1
2
​​
f
xx
​(a, b) x - a
2
)
+ 2
f
xy
​(a, b) (x - a) (y - b) +
f
yy
​(a, b) (y - b
2
)
​). It must be considered that this graph may be a paraboloid (elliptical, circular or hyperbolic)​, so this will be the best approach to the surface z = f(x, y) at the point (a, b, f(a, b)​). References: Hollis, Selwyn. CalcLabs with Mathematica, Multivariable ​​ Calculus. 4a. Ed, Cengage Learning, pp. 197-200, 2010; Personal.";
In[]:=
quadApprox[ξ_,χ_,a_]:=If[Length[χ]==Length[a],Module[{subs=Table[χ[[i]]a[[i]],{i,Length[χ]}]},(ξ/.subs)+(Evaluate[Grad[ξ,χ]]/.subs).(χ-a)+((χ-a)/2).(Evaluate[Grad[Grad[ξ,χ],χ]]/.subs).(χ-a)]]
In[]:=
f[x_,y_]:=Sin[
2
(x-y)
]+
2
Sin[(x+y-2)]
;​​χ:={1,1};​​q[x_,y_]=quadApprox[f[x,y],{x,y},χ]//Expand
Out[]=
4-4x+2
2
x
-4y+2
2
y
In[]:=
Show[Plot3D[f[x,y],{x,-1,2},{y,0,3}],Plot3D[q[x,y],{x,0,2},{y,0,2}],PlotRange{0,2},BoxRatiosAutomatic]
Out[]=
The graph above shows the function and its quadratic approximation in (1, 1).
Newton’s method to minimize:
In[]:=
f[{x_,y_,z_}]:=Exp[
2
(x-y)
]+
2
Sin[x+z-2]
+
2
(
2
z
-1)
;​​χ={.8,.9,1.5};​​quadf=quadApprox[f[{x,y,z}],{x,y,z},χ]//Expand
Out[]=
20.3423-3.22786x+1.85559
2
x
-0.0040402y-2.0605xy+1.03025
2
y
-30.2319z+1.65067xz+12.3253
2
z
In[]:=
Grad[quadf,{x,y,z}]//Simplify
Out[]=
{-3.22786+3.71117x-2.0605y+1.65067z,-0.0040402-2.0605x+2.0605y,-30.2319+1.65067x+24.6507z}
In[]:=
{x,y,z}/.Solve[Grad[quadf,{x,y,z}]==0,{x,y,z}][[1]]
Out[]=
{0.784019,0.785979,1.17391}
A simpler method:
In[]:=
newStep3D[χ_]:=Module[{quadf},quadf=quadApprox[f[{x,y,z}],{x,y,z},χ];​​{x,y,z}/.Solve[Grad[quadf,{x,y,z}]==0,{x,y,z}][[1]]];​​newStep3D[{0.8,0.9,1.5}]
Out[]=
{0.784019,0.785979,1.17391}
With NestList, iterations will be made more easily:
In[]:=
NestList[newStep3D,{0.7840185523508929,0.7859793366646184,1.173913043478261},5]
Out[]=
{{0.784019,0.785979,1.17391},{0.967792,0.967792,1.03231},{0.998544,0.998544,1.00146},{0.999997,0.999997,1.},{1.,1.,1.},{1.,1.,1.}}
Exercises:
1) The function f(x, y, z) =
2
(y-x+z)
+
2
3-x-
2
y

+6-x-
2
y
-
2
3
z
)
it has a minimum local very close to (2, 1, 1.5). Calculate 4 steps with Newton’s method and find the approximation that minimizes it. Calculate the values of f and ∇f in the resulting approximation. Compare the result by FindMinimum.
​
In[]:=
f[{x_,y_,z_}]:=
2
(y-x+z)
+
2
(3-x-
2
y
)
+
2
(6-x-
2
y
-
3
z
)
;​​χ={2,1,1.5};​​quadf=quadApprox[f[{x,y,z}],{x,y,z},χ]//Expand
Out[]=
213.406-35.5x+3
2
x
-72.5y+6xy+9.75
2
y
-195.75z+11.5xz+29.yz+49.9375
2
z
In[]:=
grad[quadf,{x,y,z}]//Simplify
Out[]=
grad[213.406+3
2
x
+9.75
2
y
-195.75z+49.9375
2
z
+6.x(-5.91667+y+1.91667z)+y(-72.5+29.z),{x,y,z}]
In[]:=
NestList[newStep3D,χ,4]
Out[]=
{{2,1,1.5},{2.27886,0.866269,1.44602},{2.28648,0.845633,1.44209},{2.28678,0.844528,1.44225},{2.28678,0.844526,1.44225}}
In[]:=
FindMinimum[f[{x,y,z}],{x,2},{y,1},{z,1.5}]
Out[]=
{2.75851×
-19
10
,{x2.28678,y0.844526,z1.44225}}
Much easier with FindMimimum.
2) It’s the same statement, but I’ll only FindMimimum use:
In[]:=
FindMinimum[
2
(
2
y
-z)
+
2
(
2
x
-y)
+ArcTan[
2
(1.3-x)
+
2
(.8-z)
],{x,1},{y,1},{z,1}]
Out[]=
{0.0896289,{x1.04228,y0.963337,z0.864168}}
3) It’s the same statement, but I’ll only FindMimimum use:
In[]:=
FindMinimumExp-
2
x
+3
2
y
Cos
2
x
+y+2
2
y
,{x,3},{y,2}
Out[]=
{-0.0707241,{x2.1206,y0.5}}
In[]:=
Plot3DExp-
2
x
+3
2
y
Cos
2
x
+y+2
2
y
,{x,1,3},{y,-1,2},AxesLabel->(Style[#,15,Blue]&/@{"X","Y","Z"}),AxesOrigin{0,0,0},AxesTrue,BoxedFalse,PlotRangeAll,PerformanceGoal"Quality"
Out[]=
We will also do it by the original method (modifying newStep3D in 2D):
In[]:=
f[x_,y_]:=Exp-
2
x
+3
2
y
Cos
2
x
+y+2
2
y
;​​χ={3,2};
In[]:=
quadf=quadApprox[f[x,y],{x,y},χ]
Out[]=
-
21

Cos[
19
]+(-2+y)-2
3
7
-
21

Cos[
19
]-
9
-
21

Sin[
19
]
2
19
+(-3+x)-
3
7
-
21

Cos[
19
]-
3
-
21

Sin[
19
]
19
+(-2+y)
1
2
(-2+y)
345
532
-
21

Cos[
19
]-
3
7
3
7
-
21

Cos[
19
]+18
3
133
-
21

Sin[
19
]-
71
-
21

Sin[
19
]
76
19
+
1
2
(-3+x)
39
266
-
21

Cos[
19
]+
2
7
3
7
-
21

Cos[
19
]+
3
2
21
19
-
21

Sin[
19
]+
27
-
21

Sin[
19
]
38
19
+(-3+x)
1
2
(-3+x)-
6
133
-
21

Cos[
19
]+
1
7
3
7
-
21

Cos[
19
]-
-
21

Cos[
19
]
21
+6
3
133
-
21

Sin[
19
]-
10
-
21

Sin[
19
]
19
19
+
1
2
(-2+y)
39
266
-
21

Cos[
19
]+
2
7
3
7
-
21

Cos[
19
]+
3
2
21
19
-
21

Sin[
19
]+
27
-
21

Sin[
19
]
38
19
In[]:=
Grad[quadf,{x,y}]//Simplify
Out[]=

1
106134
-
21

-798(21+19
21
)Cos[
19
]-76(63+38
21
)xCos[
19
]+57(273+76
21
)yCos[
19
]-126
19
(126+247
21
)Sin[
19
]+84
19
(-35+57
21
)xSin[
19
]+441
19
(9+19
21
)ySin[
19
],
1
70756
-
21

-532(231+38
21
)Cos[
19
]+38(273+76
21
)xCos[
19
]-57(-805+76
21
)yCos[
19
]-14
19
(1267+2565
21
)Sin[
19
]+294
19
(9+19
21
)xSin[
19
]+7
19
(-497+1368
21
)ySin[
19
]
This method didn’t work as we see. We will trace the graphs:
The quadratic approximation does work.