Continuing Issues with LogitModelFit
Continuing Issues with LogitModelFit
The mystery of NominalVariables
Seth J. Chandler 2020
Here' s some relatively simple data that creates problems with LogitModelFit. Notice that the second part of each row is always 0 or 1. It is a nominal (a/k/a categorical) variable.
In[]:=
unpleasant={{-0.18765344042046503,1,1},{-0.7868417124501981,0,0},{0.5159877297060876,1,0},{0.5192552864411443,1,1},{-0.344034593110404,1,1},{0.4271541695794353,1,1},{-0.8105323274640571,0,1},{-0.2615840458685881,1,1},{0.17145086850545183,1,1},{-2.212384312432726,0,0},{0.5688136764148906,1,1},{0.32765406179511536,1,1},{1.2853090964313258,1,1},{-0.4141158288462166,1,0},{-0.1998832888320074,1,1},{0.22233926060893827,1,1},{-1.3950479727543563,1,0},{-0.19610182024695302,0,0},{0.7957424118817192,1,1},{-0.7477080037535241,1,0},{-0.2398443950865801,1,1},{-0.014513991786248733,1,1},{0.7403747849770742,1,1},{0.0009430098138824265,0,1},{0.8067655004254857,1,1},{1.5727656192536221,1,1},{1.6949798505254474,1,1},{1.1406137792154913,1,1},{-0.5779852039553085,1,1},{-1.2722561308326152,0,1}};
Although I have used functions like LogitModelFit for many years. I have now realized that it may not quite use the same syntax as Fit. With Fit, in order to include a have a constant basis term, one writes it this way as there is no IncludeConstantBasis option.
In[]:=
Fit[unpleasant,{1,x,a},{x,a}]
Out[]=
0.699417+0.0733905a+0.226731x
With functions like LinearModelFit, LogitModelFit, and the other ... ModelFit functions there is nothing in the documentation indicating that one shouldn' t use a function basis specification such as {1, x, a}-- and indeed I had done so for many years without incident-- but there is the option IncludeConstantBasis, which has a default value of True. The examples in the documentation appear to prefer a formulation like the code immediately below. Unfortunately, as shown, it too does not execute successfully.
In[]:=
lomfFail=LogitModelFit[unpleasant,{x,a},{x,a},NominalVariables{a}]
7
10
8
10
7
10
Out[]=
LogitModelFit[{{-0.187653,1,1},{-0.786842,0,0},{0.515988,1,0},{0.519255,1,1},{-0.344035,1,1},{0.427154,1,1},{-0.810532,0,1},{-0.261584,1,1},{0.171451,1,1},{-2.21238,0,0},{0.568814,1,1},{0.327654,1,1},{1.28531,1,1},{-0.414116,1,0},{-0.199883,1,1},{0.222339,1,1},{-1.39505,1,0},{-0.196102,0,0},{0.795742,1,1},{-0.747708,1,0},{-0.239844,1,1},{-0.014514,1,1},{0.740375,1,1},{0.00094301,0,1},{0.806766,1,1},{1.57277,1,1},{1.69498,1,1},{1.14061,1,1},{-0.577985,1,1},{-1.27226,0,1}},{x,a},{x,a},NominalVariables{a}]
Despite the suggestion earlier in this thread that this formulation would cure my “overparameterization,” it does not in fact do so. Moreover, there is no automatic problem using more basis functions in a regression than there are parameters to be estimated. This code, for example, has 4 basis functions and only two parameters, yet it works just fine.
In[]:=
LogitModelFit[unpleasant,{x,,,a},{x,a},NominalVariables{a}]
2
x
3
x
Out[]=
FittedModel
Delete the NominalVariables option
Delete the NominalVariables option
Now, what is curious is that the following code does cure the problem. I simply neglect to tell LogitModelFit that a is a nominal variable.
In[]:=
lomfNoNominal=LogitModelFit[unpleasant,{x,a},{x,a}]
Out[]=
FittedModel
In[]:=
lomfNoNominal["ParameterTable"]
Out[]=
Estimate | Standard Error | z-Statistic | P-Value | |
1 | 1.49088 | 1.17006 | 1.27419 | 0.202597 |
x | 1.78503 | 0.890207 | 2.00518 | 0.0449433 |
a | 0.121082 | 1.23796 | 0.0978074 | 0.922085 |
Moreover, as I pointed out, permuting the order of the data does cure the problem; and I would agree with Mr. Baldwin that it seems strange. Here I run my same LogitModelFit with a permuted version of the data.
In[]:=
lomfPermuted=LogitModelFit[unpleasant[[{8,5,19,26,18,6,15,16,17,21,20,7,10,4,22,2,9,1,3,13,24,28,12,14,27,11,23,25,29,30}]],{1,x,a},{x,a},NominalVariables{a}]
Out[]=
FittedModel
In[]:=
lomfPermuted["ParameterTable"]
Out[]=
Estimate | Standard Error | z-Statistic | P-Value | |
1 | 1.61196 | 0.629668 | 2.56001 | 0.0104668 |
x | 1.78503 | 0.890207 | 2.00518 | 0.0449433 |
a[0] | -0.121082 | 1.23796 | -0.0978074 | 0.922085 |
Notice that the coefficient in front of the x - term in lomfPermuted is the same as in lomfNoNominal, but that the term in front of the constant basis (1) term is different. Nonetheless, the two models lomfNoNominal and lomfPermuted give the same result.
In[]:=
lomfNoNominal@@@(Most/@unpleasant)lomfPermuted@@@(Most/@unpleasant)
Out[]=
True
Of course it is also true that the code below suggested by Mr. Baldwin works, but it does not solve my problem because I want my function to have an offset (constant basis) term. There is nothing wrong with my desire.
In[]:=
lomfNoConstantBasis=LogitModelFit[unpleasant,{x,a},{x,a},NominalVariables{a},IncludeConstantBasisFalse]
Out[]=
FittedModel
In[]:=
lomfNoConstantBasis["ParameterTable"]
Out[]=
Estimate | Standard Error | z-Statistic | P-Value | |
x | 1.78503 | 0.890207 | 2.00518 | 0.0449433 |
a[0] | 1.49088 | 1.17006 | 1.27419 | 0.202597 |
a[1] | 1.61196 | 0.629668 | 2.56001 | 0.0104668 |
Conclusion
Conclusion
There is a need for clarification from Wolfram Developers and potentially a need for a fix. Some clarification would be useful on when one should use NominalVariables and when one should not. Perhaps there can sometimes be a problem in using both NominalVariables and a constant basis term. Indeed, this appears to be the thrust of Mr. Baldwin’s post where he uses a design matrix/response format. I may be able to solve my immediate problem simply by not telling LogitModelFit that the second feature is always 0 or 1 (hence nominal). But it does seem really strange that LogitModelFit works notwithstanding my use of both a NominalVariable and a constant basis term when I permute the data but fails in the initial instance when I do not.
I look forward to any constructive assistance.