Linear Least Squares
Authors: Housam Binous and Ahmed Bellagi
Linear Least Squares
Authors: Housam Binous and Ahmed Bellagi
Authors: Housam Binous and Ahmed Bellagi
Hypothetical data generated by using the function a1 x^4+a2 x^3+a3 x^2+a4 x+a5
with a1=0.25, a2=-2.42, a3=-1.35, a4=3.65 and a5=1 and a random number generator
with a1=0.25, a2=-2.42, a3=-1.35, a4=3.65 and a5=1 and a random number generator
Finding the values of a1, a2, a3, a4 and a5 using linear least square There is a closed-form expression: a={a1,a2,a3,a4,a5}= y
-1
(X)
T
X
T
X
In[]:=
data=Table[0.25x^4-2.42x^3-1.35x^2+3.65x+1+0.1Random[],{x,0,10,0.1}]
Out[]=
{1.04506,1.35245,1.71349,1.94957,2.14712,2.27097,2.27536,2.21881,1.92886,1.60379,1.21956,0.613685,-0.159483,-1.08974,-2.18452,-3.4195,-4.82141,-6.4859,-8.24501,-10.2126,-12.4532,-14.7693,-17.3341,-20.1642,-23.1139,-26.2939,-29.7203,-33.2423,-37.1113,-41.0128,-45.2265,-49.569,-54.2275,-58.8934,-63.8293,-68.9958,-74.2403,-79.6696,-85.2434,-90.9508,-96.8157,-102.852,-108.89,-115.106,-121.461,-127.867,-134.38,-140.86,-147.411,-154.032,-160.663,-167.308,-173.919,-180.505,-187.119,-193.545,-199.939,-206.314,-212.453,-218.47,-224.368,-230.04,-235.602,-240.869,-245.806,-250.585,-255.075,-259.204,-262.965,-266.4,-269.41,-271.947,-274.083,-275.746,-276.881,-277.416,-277.454,-276.897,-275.614,-273.732,-271.176,-267.913,-263.811,-258.955,-253.249,-246.641,-239.16,-230.75,-221.323,-210.834,-199.412,-186.747,-173.032,-158.149,-142.099,-124.689,-106.011,-85.9875,-64.6434,-41.7254,-17.4897}
In[]:=
xvalues=Table[x,{x,0,10,0.1}]
Out[]=
{0.,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.,1.1,1.2,1.3,1.4,1.5,1.6,1.7,1.8,1.9,2.,2.1,2.2,2.3,2.4,2.5,2.6,2.7,2.8,2.9,3.,3.1,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.,4.1,4.2,4.3,4.4,4.5,4.6,4.7,4.8,4.9,5.,5.1,5.2,5.3,5.4,5.5,5.6,5.7,5.8,5.9,6.,6.1,6.2,6.3,6.4,6.5,6.6,6.7,6.8,6.9,7.,7.1,7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,8.,8.1,8.2,8.3,8.4,8.5,8.6,8.7,8.8,8.9,9.,9.1,9.2,9.3,9.4,9.5,9.6,9.7,9.8,9.9,10.}
In[]:=
xvalues^2
Out[]=
{0.,0.01,0.04,0.09,0.16,0.25,0.36,0.49,0.64,0.81,1.,1.21,1.44,1.69,1.96,2.25,2.56,2.89,3.24,3.61,4.,4.41,4.84,5.29,5.76,6.25,6.76,7.29,7.84,8.41,9.,9.61,10.24,10.89,11.56,12.25,12.96,13.69,14.44,15.21,16.,16.81,17.64,18.49,19.36,20.25,21.16,22.09,23.04,24.01,25.,26.01,27.04,28.09,29.16,30.25,31.36,32.49,33.64,34.81,36.,37.21,38.44,39.69,40.96,42.25,43.56,44.89,46.24,47.61,49.,50.41,51.84,53.29,54.76,56.25,57.76,59.29,60.84,62.41,64.,65.61,67.24,68.89,70.56,72.25,73.96,75.69,77.44,79.21,81.,82.81,84.64,86.49,88.36,90.25,92.16,94.09,96.04,98.01,100.}
In[]:=
xvalues^3
Out[]=
{0.,0.001,0.008,0.027,0.064,0.125,0.216,0.343,0.512,0.729,1.,1.331,1.728,2.197,2.744,3.375,4.096,4.913,5.832,6.859,8.,9.261,10.648,12.167,13.824,15.625,17.576,19.683,21.952,24.389,27.,29.791,32.768,35.937,39.304,42.875,46.656,50.653,54.872,59.319,64.,68.921,74.088,79.507,85.184,91.125,97.336,103.823,110.592,117.649,125.,132.651,140.608,148.877,157.464,166.375,175.616,185.193,195.112,205.379,216.,226.981,238.328,250.047,262.144,274.625,287.496,300.763,314.432,328.509,343.,357.911,373.248,389.017,405.224,421.875,438.976,456.533,474.552,493.039,512.,531.441,551.368,571.787,592.704,614.125,636.056,658.503,681.472,704.969,729.,753.571,778.688,804.357,830.584,857.375,884.736,912.673,941.192,970.299,1000.}
In[]:=
xvalues^4
Out[]=
{0.,0.0001,0.0016,0.0081,0.0256,0.0625,0.1296,0.2401,0.4096,0.6561,1.,1.4641,2.0736,2.8561,3.8416,5.0625,6.5536,8.3521,10.4976,13.0321,16.,19.4481,23.4256,27.9841,33.1776,39.0625,45.6976,53.1441,61.4656,70.7281,81.,92.3521,104.858,118.592,133.634,150.063,167.962,187.416,208.514,231.344,256.,282.576,311.17,341.88,374.81,410.063,447.746,487.968,530.842,576.48,625.,676.52,731.162,789.048,850.306,915.063,983.45,1055.6,1131.65,1211.74,1296.,1384.58,1477.63,1575.3,1677.72,1785.06,1897.47,2015.11,2138.14,2266.71,2401.,2541.17,2687.39,2839.82,2998.66,3164.06,3336.22,3515.3,3701.51,3895.01,4096.,4304.67,4521.22,4745.83,4978.71,5220.06,5470.08,5728.98,5996.95,6274.22,6561.,6857.5,7163.93,7480.52,7807.49,8145.06,8493.47,8852.93,9223.68,9605.96,10000.}
In[]:=
A=Transpose[{Table[1,{i,0,10,0.1}],xvalues,xvalues^2,xvalues^3,xvalues^4}];
In[]:=
sol=Inverse[Transpose[A].A].Transpose[A].data
Out[]=
{1.04573,3.65154,-1.34718,-2.42081,0.250053}
In[]:=
plt1=ListPlot[Table[{xvalues[[i]],data[[i]]},{i,1,Length[data]}],PlotStyle{Red,PointSize[0.015]},FrameTrue,FrameLabel{Style["x",16],Style["y",16]},ImageSize450{1,1},GridLinesAutomatic]
Out[]=
In[]:=
plt2=Plot[sol[[1]]+sol[[2]]x+sol[[3]]x^2+sol[[4]]x^3+sol[[5]]x^4,{x,0,10},PlotStyle{Blue,Thick},FrameTrue,FrameLabel{Style["x",16],Style["y",16]},ImageSize450{1,1},GridLinesAutomatic]
Out[]=
In[]:=
Show[plt1,plt2]
Out[]=
finding the values of a1, a2, a3, a4 and a5 using the built-in command FindFit
we recover the same values as those found above.
we recover the same values as those found above.
In[]:=
FindFit[Table[{xvalues[[i]],data[[i]]},{i,1,Length[data]}],a0+a1x+a2x^2+a3x^3+a4x^4,{a0,a1,a2,a3,a4},x]
Out[]=
{a01.04573,a13.65154,a2-1.34718,a3-2.42081,a40.250053}