ABSTRACT (original article): This work introduces a method for determining the energy spectrum of lattice quantum chromodynamics (LQCD) by applying the Lanczos algorithm to the transfer matrix and using a boot strap generalization of the Cullum-Willoughby method to filter out spurious eigenvalues. Proof-of principle analyses of the simple harmonic oscillator and the LQCD proton mass demonstrate that this method provides faster ground-state convergence than the “effective mass,” which is related to the power-iteration algorithm. Lanczos provides more accurate energy estimates than multi-state fits to correlation functions with small imaginary times while achieving comparable statistical precision. Two-sided error bounds are computed for Lanczos results and guarantee that excited-state effects cannot shift Lanczos results far outside their statistical uncertainties. CITATION (original article): Wagman, M. L. (2024). Lanczos, the transfer matrix, and the signal-to-noise problem. arXiv:2406.20009. https://doi.org/10.48550/arXiv.2406.20009

Lanczos Formalism


Noiseless data

20 state toy model

Fake data for a 20-state model with E_n = (n+1)Efake = (n+1)/10
In[]:=
Efake=1/10;
In[]:=
states=20;
In[]:=
fakeCorr[t_]:=Block[{norm},​​norm=Sum[k^2,{k,1,states}];​​Sum[k^2Exp[-Efakekt],{k,1,states}]/norm];
In[]:=
fakeLen=2states-1;
In[]:=
fakeData=Table[fakeCorr[t],{t,0,fakeLen}];
Compute effective mass and relative error, plot vs largest time included
In[]:=
fakeEM=Table[{t+1,Log[fakeData[[t]]/fakeData[[t+1]]]},{t,1,fakeLen-1}];
In[]:=
fakeEMerr=Table[{t+1,(Log[fakeData[[t]]/fakeData[[t+1]]]-Efake)/Efake},{t,1,fakeLen-1}];
Compute Lanczos energy estimate and relative error
In[]:=
LanczosGroundState[data_]:=Block[{λ},​​λ=Quiet@ObliqueLanczosEvalsErrors[data,data,data][[1]];​​-Log@Table[Max[Select[λ[[m]],#<1&]],{m,Length[λ]}]]
In[]:=
LanczosGroundStateResiduals[data_]:=Block[{out,λ,gs},​​out=Quiet@ObliqueLanczosEvalsErrors[data,data,data];​​λ=out[[1]];​​gs=Table[FirstPosition[λ[[m]],Max[Select[λ[[m]],#<1&]]][[1]],{m,Length[λ]-1}];​​Table[out[[3,m,gs[[m]]]],{m,Length[λ]-1}]]
In[]:=
LanczosEs=LanczosGroundState[fakeData]
Plot vs largest time included
In[]:=
LanczosEM=Table[{2m-1,LanczosEs[[m]]},{m,1,Length[LanczosEs]}];
In[]:=
LanczosEMerr=Table[{2m-1,(LanczosEs[[m]]-Efake)/Efake},{m,1,Length[LanczosEs]}]
Plot relative errors
mockDataError=ShowListLogPlotfakeEMerr,
,ListLogPlotLanczosEMerr,
,

Out[]=
Compute Lanczos residual bounds
In[]:=
LanczosResidualBounds=Sqrt[LanczosGroundStateResiduals[fakeData]]/Exp[-LanczosEs[[;;-2]]]/LanczosEs[[;;-2]]
In[]:=
LanczosResidualErrors=Sqrt[LanczosGroundStateResiduals[fakeData]]/Exp[-LanczosEs[[;;-2]]]/LanczosEs[[;;-2]];
Compute KPS bounds
In[]:=
KPS[m_,δ_,Z0_]:=Tan[ArcCos[Z0]]^2/ChebyshevT[m-1,2Exp[δ]-1]^2
In[]:=
Z0=1/Sqrt[Sum[k^2,{k,1,states}]];
In[]:=
δ=Efake;
Plot residual and KPS bounds alongside exact errors
mockDataErrorBounds=ShowmockDataError,LogPlotKPS[(t+1)/2,δ,Z0]Efake,{t,1,43},
,ListLogPlotTable[{2m-1,LanczosResidualErrors[[m]]/LanczosEs[[m]]},{m,6,Length[LanczosResidualErrors]}],

Out[]=

Thermal boson toy model

Fake data for a finite-temperature 50-state / β=100 model with E_n = (n+1)Efake = (n+1)/10
In[]:=
β=100;δ=Efake;
In[]:=
statesTherm=β/2;
In[]:=
fakeCorrTherm[t_]:=Block[{norm},​​norm=Sum[k^2(1+Exp[-δk(β)])/2,{k,1,statesTherm}];​​Sum[k^2(Exp[-δkt]+Exp[-δk(β-t)])/2,{k,1,statesTherm}]/norm];
In[]:=
fakeDataTherm=Table[fakeCorrTherm[t],{t,0,β-1}];
Compute effective mass and relative error, plot vs largest time included
In[]:=
fakeEMTherm=Table[{t+1,ArcCosh[(fakeDataTherm[[t+2]]+fakeDataTherm[[t]])/(2fakeDataTherm[[t+1]])]},{t,1,β-2}];
In[]:=
fakeEMerrTherm=Table[{t+1,Abs[ArcCosh[(fakeDataTherm[[t+2]]+fakeDataTherm[[t]])/(2fakeDataTherm[[t+1]])]-Efake]/Efake},{t,1,β-2}];
Compute Lanczos energy estimate and relative error
In[]:=
LanczosEsTherm=LanczosGroundState[fakeDataTherm];
Plot vs largest time included
In[]:=
LanczosEMTherm=Table[{2m-1,LanczosEsTherm[[m]]},{m,1,Length[LanczosEsTherm]}];
In[]:=
LanczosEMerrTherm=Table[{2m-1,Abs[LanczosEsTherm[[m]]-Efake]/Efake},{m,1,Length[LanczosEsTherm]}];
Plot relative errors
mockDataErrorTherm=ShowListLogPlotfakeEMerrTherm,
,ListLogPlotLanczosEMerrTherm,
,

Out[]=
Compute Lanczos residual bounds
In[]:=
LanczosResidualErrorsTherm=Sqrt[LanczosGroundStateResiduals[fakeDataTherm]]/Exp[-LanczosEsTherm[[;;-2]]]/LanczosEsTherm[[;;-2]];
In[]:=
LanczosEMerrG=Table[{2m-1,Around[LanczosEsTherm[[m]],LanczosResidualErrorsTherm[[m]]LanczosEsTherm[[m]]]},{m,1,Length[LanczosResidualErrorsTherm]}];
Plot residual and KPS bounds alongside exact errors
In[]:=
Z0Therm=1/Sqrt[Sum[k^2(1+Exp[-βδk]),{k,1,statesTherm}]];
mockDataErrorBoundsTherm=ShowmockDataErrorTherm,LogPlotKPS[(t+1)/2,δ,Z0Therm]Efake,{t,1,100},

,ListLogPlotTable[{2k-1,LanczosResidualErrorsTherm[[k]]/LanczosEsTherm[[k]]},{k,3,Length[LanczosResidualErrorsTherm]}],
,

Out[]=
Plot Lanczos energies with width of allowed region from residual bounds as error bars
mockDataEMPLog=ShowListLogPlotfakeEMTherm,
,ListLogPlotLanczosEMerrG,
,ListLogPlotLanczosEM,
,

Out[]=

Ritz values = Prony roots

Compute ground-state energies using Prony's method
Compute ground-state energies using PGEVM method
After m=1 iteration ground-state energy from all three methods coincide with the t=1 effective mass
Ground-state energies from all three methods always coincide with each other

1+1D free boson

Reduce to run faster
Load complex scalar data
Compute effective mass
Compute Lanczos results
Plot effective mass
Plot multi-state fit results
Plot Lanczos energies and residual bound Bs
Plot comparison of Lanczos energies with exact results and other estimates
Plot correlations between Lanczos estimates vs iteration count
Last iteration result
Residual bound window using last iteration
Residual bound window using minimum residual iteration
Cross-check with naive uncorrelated error propagation
Compute residual bound for first iteration (effective mass)
Compute correlated uncertainties from bootstrap eigenvalues and Bs
Cross-check with naive uncorrelated error propagation

LQCD pion and nucleon

Reduce to run faster

Pion

Compute effective mass
Compute Lanczos results
Plot effective mass
Plot multi-state fit results
Plot Lanczos energies and residual bound Bs
Plot comparison of Lanczos energies with exact results and other estimates
Plot correlations between Lanczos estimates vs iteration count
Last iteration result
Residual bound window using last iteration
Residual bound window using minimum residual iteration
Cross-check with naive uncorrelated error propagation

Nucleon

Compute effective mass
Compute Lanczos results
Plot effective mass
Plot multi-state fit results
Check multi-state fit convergence
Plot Lanczos energies and residual bound Bs
Plot comparison of Lanczos energies with exact results and other estimates
Plot correlations between Lanczos estimates vs iteration count
Variance vs iteration
Last iteration result
Residual bound window using last iteration
Residual bound window using minimum residual iteration
Cross-check with naive uncorrelated error propagation

Related articles

◼
  • Abbott, R., Hackett, D. C., Fleming, G. T., Pefkou, D. A., & Wagman, M. L. (2025). Filtered Rayleigh-Ritz is all you need (Version 1). arXiv:2503.17357. https://doi.org/10.48550/arXiv.2503.17357
    • ◼
  • Hackett, D. C., & Wagman, M. L. (2024). Block Lanczos for lattice QCD spectroscopy and matrix elements (Version 2). arXiv:2412.04444. https://doi.org/10.48550/arXiv.2412.04444
    • ◼
  • Chakraborty, D., Sood, D., Radhakrishnan, A., & Mathur, N. (2024). Estimating energy levels from lattice QCD correlation functions using a transfer matrix formalism (Version 1). arXiv:2412.01900. https://doi.org/10.48550/arXiv.2412.01900
    • ◼
  • Ostmeyer, J., Sen, A., & Urbach, C. (2025). On the equivalence of Prony and Lanczos methods for Euclidean correlation functions. The European Physical Journal A, 61(2). https://doi.org/10.1140/epja/s10050-025-01495-8
    • ◼
  • Hackett, D. C., & Wagman, M. L. (2024). Lanczos for lattice QCD matrix elements (Version 1). arXiv:2407.21777. https://doi.org/10.48550/arXiv.2407.21777

    • CITE THIS NOTEBOOK

    Lanczos, the transfer matrix, and the signal-to-noise problem​
    by Michael Wagman​
    Wolfram Community, STAFF PICKS, May 15, 2025
    ​https://community.wolfram.com/groups/-/m/t/3461332