In[]:=
deploy
Mon 15 Aug 2022 12:43:16

Find learning rate which guarantees drop

In[]:=
Needs["Notation`"];​​distinguishMatrixPowerAndPower[A_,d_]:=If[SquareMatrixQ[A],MatrixPower[A,d],Power[A,d]];​​Notation
d_
A_
⟺
distinguishMatrixPowerAndPower[A_,d_]
;​​getLR2[theta_,noise_]:=​​H=DiagonalMatrix[{4,1}];​​Print["H=",H//MatrixForm];​​h=MatrixPower[H,1/2];​​rot=RotationMatrix[theta];​​A=DiagonalMatrix[{noise,1}];​​A=rot.A.Inverse[rot]//Simplify;​​Print[A//MatrixForm];​​a=h.A.h;​​b=h.A.Inverse[h]+Inverse[h].A.h;​​(*equivalentwaysofsolvingtheproblem*);​​radius=R/.SemidefiniteOptimization[R,{cons},{R}];​​cons=VectorLessEqual[{H.A.H,R(A.H+H.A)},{"SemidefiniteCone",d}];​​radius=R/.SemidefiniteOptimization[R,{cons},{R}];​​​​Min@Eigenvalues[
-1
H
+
-1
A
.
-1
H
.A](*sameasMin[Eigenvalues[(H.F+F.H).Inverse[H.F.H]]]*)​​,​​1radius​​;​​getLR2[Pi/4,4]​​getLR2[Pi/4,9]//N​​getLR2[Pi/4,1/9]//N​​
H=
4
0
0
1

5
2
3
2
3
2
5
2
SemidefiniteOptimization
:Lack of progress.
Out[]=

5
16
,0.3125
H=
4
0
0
1


5
4
4
5

SemidefiniteOptimization
:There are no points that satisfy the constraints.
Out[]=
{0.,Indeterminate}
H=
4
0
0
1

5
9
-
4
9
-
4
9
5
9
SemidefiniteOptimization
:There are no points that satisfy the constraints.
SemidefiniteOptimization
:Lack of progress.
Out[]=
{0.,0.00192203}

Animate section

In[]:=
exportAnim[fn_,images_,is_]:=Module[{},​​rasters=Rasterize[Style[#,AntialiasingTrue],ImageSizeis,RasterSize2is]&/@images;​​SetDirectory[NotebookDirectory[]];​​SetDirectory["export"];​​Export[fn,rasters,"AnimationRepetitions"Infinity]​​];​​​​makePlot[scale_,alpha_]:=(​​H=DiagonalMatrix[{4,1}];​​Adiag=DiagonalMatrix[{scale,1}];​​A0=With[{rot=RotationMatrix[Pi/4]},rot.Adiag.Inverse[rot]];​​ii=IdentityMatrix[2];​​A1=(ii-alphaH).A0.(ii-alphaH);​​bound=2.5;​​ContourPlot[{​​{x,y}.
-1
A0
.{x,y}==1,​​{x,y}.
-1
A1
.{x,y}==1},{x,-bound,bound},{y,-bound,bound},PlotLegends->{"A0","A1"},​​PlotLabel->StringForm["H=``, Σ=R `` R, α=``",MatrixForm[H],MatrixForm@Adiag,​​NumberForm[alpha,{2,2}]]]​​);​​plots=Table[makePlot[9,alpha+0.00001],{alpha,0,0.5,.05}];​​ListAnimate[plots]
Out[]=
A0
A1
In[]:=
plots=Table[makePlot[1/9,alpha+.0001],{alpha,0,0.5,.05}];​​ListAnimate[plots]
Out[]=
A0
A1
In[]:=
exportAnim[fn_,images_,is_]:=Module[{},​​rasters=Rasterize[Style[#,AntialiasingTrue],ImageSizeis,RasterSize2is]&/@images;​​SetDirectory[NotebookDirectory[]];​​SetDirectory["export"];​​Export[fn,rasters,"AnimationRepetitions"Infinity]​​];​​​​​​curve[x_]:=(Cos[Pi(x-1)]+1)/2;(*startsat0,reachesmaximumat1,secondminat2*)​​(*likecurve,butaddsextra1/2intervalsatmaximumandminimum,secondminat3*)​​flattenedCos[x_]:=Piecewise[{​​{0,x<1/2},​​{curve[x-1/2],1/2<=x<1+1/2},​​{1,1+1/2<=x<2},​​{curve[x-1],2<=x<2+1}​​}];​​​​maxAlpha=0.5;​​plots=Table[makePlot[9,maxAlpha*curve[x]+.0001],{x,0,2,.02}];​​exportAnim["semidef-shrinkage.gif",plots,200]

Failing example

Connection to numerical range