In[]:=
CompoundExpression[
]
​​deploy
Fri 6 Jun 2025 16:07:14
Wolfram Three Domains notability.
​
Formulas for mapping between three domains.
​
Suppose we have a quadratic with coefficients 1, 1/4, 1/9, 1/16, ....,
1/
2
n
and we minimize it using gradient descent with learning rate 1. How does the loss behave over time?
​
We can unlock analytic tools by studying a closely related quadratic with random coefficients.
​
​
1. Random Variable (expectations)
2. Density (integral transforms)
3. Sequence of moments (generating functions)
​
Related:
https://www.wolframcloud.com/obj/yaroslavvb/newton/wolfram-integrals-post.nb
​
https://www.wolframcloud.com/obj/yaroslavvb/nn-linear/forum-erfc-formula.nb
​
root: linear-estimation-scratch-2025
chris-desa-scratch.nb
chris-desa: Mapping between three domains for Beta(1/2,1)
​

Random quadratic realization

In[]:=
Table[1
2
i
,{i,1,20}]
Out[]=
1,
1
4
,
1
9
,
1
16
,
1
25
,
1
36
,
1
49
,
1
64
,
1
81
,
1
100
,
1
121
,
1
144
,
1
169
,
1
196
,
1
225
,
1
256
,
1
289
,
1
324
,
1
361
,
1
400

In[]:=
SeedRandom[1];​​Needs["MaTeX`"];​​pretty[expr_]:=MaTeX[expr];​​SetAttributes[pretty,HoldAll];​​numCoefs=1000;​​z=Integrate
-3/2
x
,x,
1
2
numCoefs
,1;​​(*thiscorrespondstotruncatedBetaDistribution[-1/2,1]*)​​dist=ProbabilityDistribution
1
z
-3/2
x
,x,
1
2
numCoefs
,1;​​randomCoefs=Reverse@Sort@RandomVariate[dist,numCoefs];​​randomCoefs=randomCoefs/First@randomCoefs;​​myCoefs=Table[
-2
i
,{i,1,numCoefs}];​​​​dec[list_]:=With[{n=Length@list},{Range[n],list}];​​sf={"Log","Log"};​​coefPlot=ListPlot{dec@myCoefs,dec@randomCoefs},ScalingFunctions->sf,PlotLegends->pretty@
1
2
i
,"sorted sample",PlotStyle->{PointSize[Medium],Automatic};​​​​expectedPlot=Plot[InverseSurvivalFunction[dist,x/numCoefs],{x,0,numCoefs},ScalingFunctions->{"Log","Log"},PlotStyle->{Directive[Gray,Thickness[.01],Opacity[.2]]},PlotLegends->{"Inverse Survival Function"}];​​SF=StringForm;​​Show[coefPlot,expectedPlot,AxesLabel->{"i","value"},PlotLabel->SF["Coefficients, sample of Beta(-1/2,1)"]]
Out[]=
sorted sample
Inverse Survival Function
Check loss (from wolfram-integrals-post.nb *)
In[]:=
dist=BetaDistribution[1/2,1];​​truthContinuous=Simplify[MomentGeneratingFunction[dist,t]/.t->-t]​​​​lossTruePrecision=20;​​lossTrue[t0_]:=With[{t=SetPrecision[t0,lossTruePrecision],h=
-2
(1+i)
},​​nsumMethod={"EulerMaclaurin",Method{"NIntegrate","MaxRecursion"14,Method"DoubleExponential"}};​​NSum[h
t
(1-h)
,{i,0,∞},WorkingPrecision->lossTruePrecision-1,MethodnsumMethod]];​​​​maxT=20;​​momentExpr=
t
4
2
(t!)
(1+2t)!
;​​momentExpr=
(2t)!!
(1+2t)!!
;​​actualPlot=DiscretePlot[{lossTrue[t],momentExpr},{t,1,maxT},PlotLegends->{"true loss",pretty@momentExpr},PlotStyle->{PointSize[Large],Automatic}];​​SF=StringForm;​​ShowactualPlot,​​Plot[{None,truthContinuous},{t,1,maxT},PlotLegends->{pretty@truthContinuous}],PlotLabel->SF"Minimizing
​, p=``, for t steps",2,AxesLabel->{"t","loss"}
Out[]=
π
Erf[
t
]
2
t
Out[]=
true loss

Using difference equation vs differential equation

Difference equation

In[]:=
expr1=Moment[TransformedDistribution[1-
2
x
,xUniformDistribution[]],n];​​expr2=
1
2
Beta1+n,
1
2
;​​Column[{expr1==expr2,​​Table[Simplify[expr1==expr2],{n,1,3}]}]
Out[]=
π
Gamma[1+n]
2Gamma
3
2
+n

1
2
Beta1+n,
1
2

{True,True,True}
In[]:=
expr1=RSolveValuea[n]==1-
1
2n+1
a[n-1],a[0]==1,a[n],n​​expr2=
1
2
Beta1+n,
1
2
;​​Column[{expr1==expr2,​​Table[Simplify[expr1==expr2],{n,1,3}]}]​​
Out[]=
Pochhammer[1,n]
Pochhammer
3
2
,n
Out[]=
Pochhammer[1,n]
Pochhammer
3
2
,n

1
2
Beta1+n,
1
2

{True,True,True}

Differential equation

In[]:=
ClearAll["Global`*"];​​case1=-1/(2*t);​​DSolveValuef'[t]==-
1
2t
f[t],f[1]==
Pi
2
,f[t],t​​
Out[]=
π
2
t
In[]:=
sol=MomentGeneratingFunction[BetaDistribution[1/2,1],t]/.t->-t​​Asymptotic[sol,t->∞]
Out[]=
π
Erfi[
-t
]
2
-t

Unit tests

Density column

Expectation column

Moment column

Beta(1/2,1) Stieltjes and OGF

Uniform Distribution