You are using a browser not supported by the Wolfram Cloud
Supported browsers include recent versions of Chrome, Edge, Firefox and Safari.
I understand and wish to continue anyway »
WOLFRAM NOTEBOOK
Trying to Implement the LIME XAI algorithm in Wolfram Language
Trying to Implement the LIME XAI algorithm in Wolfram Language
Seth J. Chandler
A significant modern issue in artificial intelligence is providing models that provide explanations of their computations that humans can understand. One of the leading algorithms in this field of “XAI” goes by the name of LIME. This essay shows an effort to approximate LIME within the Wolfram Language. As shown, we can presently get close to LIME using the Fit function, the LinearModelFit function and the neural network set of functionality, but at present it does not appear possible to fully implement LIME within the Wolfram Language. The purpose of this essay is to solicit suggestions on how to implement the algorithm more fully within the existing Wolfram Language or to stimulate development of built-in functions, as part of a large suite of XAI capabilities, that would do so.
Introduction
Introduction
LIME as an XAI algorithm
LIME as an XAI algorithm
A significant modern issue in artificial intelligence is providing models that provide explanations of their computations that humans can understand. One of the leading algorithms in this field of “XAI” goes by the name of LIME. Basically, one takes the function produced by some complex algorithm (neural networks, boosted trees or whatever) and then explains its result on some new data by computing some simple predictive method in which the inputs are a weighted sample of the training data that is “close” to the new data and the outputs are not the actual outputs associated with that training data but rather the outputs produced by the more complex model. This way, one develops a local emulator of the more complex model that should be more explicable. The local emulator doesn’t directly predict reality; rather it indirectly explains reality by providing a simple explanation of the result produced by the complex model, which, one hopes, provides a decent prediction of reality.
The authors of the LIME paper sensibly recommend that at least in some settings, the simple predictive method used as a local emulator is LASSO, a variant of ordinary least squares linear regression which adds the absolute value of the regression parameters to the loss function. Adding this regularization penalty often results in the optimal parameters of less important components of the input data being optimized at zero and thus can provide a parsimonious and simple model. This essay shows an effort to approximate LIME within the Wolfram Language. As shown, we can presently get close to LIME using the Fit function, the LinearModelFit function and the neural network set of functionality, but at present it does not appear possible to fully implement LIME.
Getting Wolfram Language to implement LIME
Getting Wolfram Language to implement LIME
Unfortunately, at present I can not get LIME to quite work using native Wolfram Language code. Fit can handle regularization, including LASSO, but can not handle weighted data. LinearModelFit can handle weighted data but not regularization. Since the Neural Network framework can handle weighted data and can handle some forms of regularization, my hope was that one could build LIME using the neural network. To date, though, I can not figure out how to implement it. I am thus looking for help or someone to tell me it is not currently possible. And, if it is not possible, might there be some sort of new layer that would facilitate it.
The remainder of this essay tells the more elaborate story, which shows various attempts imperfectly to approximate what is going on with LIME. The methods described may be useful for a variety of data scientists despite the failure to succeed fully. Ultimately, it would be lovely if there were a Wolfram command ModelEmulate (with a Method option), in which the input was some complex model and the output was either a true emulator or a function, which when applied to new data, would produce an emulator for that piece of data.
The Complex Model
The Complex Model
Building the complex model
Building the complex model
Our first goal is to get the data in a form suitable for building our complex predictor.
Download the Boston data :
In[]:=
(bostonTraining=ExampleData[{"MachineLearning","BostonHomes"},"TrainingData"])//[5]
Out[]//Short=
{{0.02731,0,7.07,0,0.469,6.421,78.9,4.9671,2,242,17.8,396.9,9.14}21.6,{0.02729,0,7.07,0,0.469,7.185,61.1,4.9671,2,242,17.8,392.83,4.03}34.7,334,{0.10959,0,11.93,0,0.573,6.794,89.3,2.3889,1,273,21,393.45,6.48}22,{0.04741,0,11.93,0,0.573,6.03,80.8,2.505,1,273,21,396.9,7.88}11.9}
Create the complex predictor:
In[]:=
complex=Predict[bostonTraining,Method"GradientBoostedTrees"]
Out[]=
PredictorFunction
Testing the complex model
Testing the complex model
See how the predictor performs on the first item in the test data:
In[]:=
(bostonTest=ExampleData[{"MachineLearning","BostonHomes"},"TestData"])//[5]
Out[]//Short=
{{0.00632,18,2.31,0,0.538,6.575,65.2,4.09,1,296,15.3,396.9,4.98}24,166,{0.26838,0,9.69,0,0.585,5.794,70.6,2.8927,6,391,19.2,396.9,14.1}18.3}
In[]:=
complex[bostonTest[[1,1]]]
Out[]=
24.5223
Building an Emulator of the Complex Model
Building an Emulator of the Complex Model
The complex model gives a good prediction but it is basically impossible to explain how the algorithm reached that determination other than to say that it derived from the weighted vote of various decision trees, just as it will always be when the method used to train the predictor was gradient boosted trees.
Find the Neighborhood of new data
Find the Neighborhood of new data
Let' s try to build a local emulator for our first test input and explain the results. LIME tells us to find the neighbors of the new data. To do this, I need to come up with a proximity measure. I don’t want to use the Euclidean Distance on the raw data because the scales of each variable are quite different. To cure this problem, I am going to get the CDF function for each of the 13 features of the training data and then compute the CDF values of each of the 13 columns. This will normalize all the values to lie between 0 and 1.
Transpose the training data :
In[]:=
Dimensions[transposedTrainingInputs=Transpose[bostonTraining[[All,1]]]]
Out[]=
{13,338}
Compute the CDFs of the features (and make the function Listable) :
In[]:=
cdfFunctions=Map[Function[q,Evaluate@CDF[EmpiricalDistribution[#],q],Listable]&,transposedTrainingInputs]
Out[]=
Now I will apply the quantile functions to all of the training data and then to the test data. The result will be normalized data on which a Euclidean Distance (or Manhattan Distance) is suitable for computing proximity.
Plot the normalized training and test data :
RowMatrixPlotnormalizedTrainingInputs=Transpose[MapThread[Construct,{cdfFunctions,transposedTrainingInputs}]],,MatrixPlotnormalizedTestInputs=Transpose[MapThread[Construct,{cdfFunctions,transposedTestInputs=Transpose[bostonTest[[All,1]]]}]],
Out[]=
Compute the nearest function of the normalized training data:
In[]:=
nf=Nearest[normalizedTrainingInputs"Index"]
Out[]=
NearestFunction
I can test out our Nearest function by applying it to our normalized first test input, which I will call t1..
Find the 50 points in the normalized training data nearest our normalized first test input:
In[]:=
neighbors50=nf[t1=normalizedTestInputs[[1]],50]
Out[]=
{60,66,115,64,1,58,233,61,118,116,185,3,40,65,188,2,37,121,130,56,234,117,27,193,132,133,113,127,59,192,47,187,38,114,124,129,63,36,206,199,198,205,189,236,62,200,122,29,207,191}
Building a Local Emulator with Fit
Building a Local Emulator with Fit
Now I' m ready to build our simple model. I could try Fit, which would permit regularization and ignore the weights. First I get the data in the right form for Fit and then run Fit, outputting a list of "Best Fit Parameters." As it happens, the Wolfram Language represents this list as a SparseArray object, but this fact is inessential to the underlying problem. I compute this list of Best Fit Parameters for a variety of regularization penalties and thus with an Association between the regularization penalties and the computed best parameters (housed within a SparseArray object).
Use the PairMap ResourceFunction that is currently available in the Cloud but not yet an "official" resource function:
In[]:=
lassoAssociation50=Withdata=[complex][Append][bostonTraining[[neighbors50,1]]],AssociationMap[Fit[data,Prepend[Array[x,13],1],Array[x,13],"BestFitParameters",FitRegularization{"LASSO",#}]&,{0,1,2,4,8,16,32,64,128,256,512}]
Out[]=
0SparseArray,1SparseArray,2SparseArray,4SparseArray,8SparseArray,16SparseArray,32SparseArray,64SparseArray,128SparseArray,256SparseArray,512SparseArray
Write a function that lets me create this sort of Association for any row of data and for any list of regularization penalties:
Compute the correct answer for the first test input is 24 and the prediction of our complex model:
One I have the list of coefficients, I can use matrix multiplication to determine the predictions.I can now see how the amount of regularization affects both the complexity of the simplified local model and the accuracy of the result.
Create a dataset showing the regularization penalty, the complexity of the resulting LASSO emulator and the prediction made by that emulator:
So, what we can see is that none of the local models do a particularly good job in predicting either the truth or what the complex model predicts. But the accuracy of the local model does not deteriorate greatly as the complexity of the local model declines via heavier regularization coefficients. Perhaps the absence of data weighting is a significant problem.
Using Weighted Sampling to Fake Weights
Using Weighted Sampling to Fake Weights
Wrap a Histogram around the computed weights for the neighboring points:
Use the weights to create a dataset in which the nearest values are oversampled:
I run the same code as before but substitute weighted data for the original data. I also multiply the LASSO parameter by 100 to take account of the fact that we have 100 times more data than we did before:
Create a dataset showing, for the weighted data, the regularization penalty, the complexity of the resulting LASSO emulator and the prediction made by that emulator :
What we can see is that weighting does no real harm to the emulator, but it also doesn't do much good, at least for the particular test example selected. And the computations are much slower due to the greater number of data points involved.
LinearModelFit
LinearModelFit
I can also try using LinearModelFit, which can accommodate weights but which can not accommodate regularization.
Compute the best local emulator with weighted neighbors :
And compute the prediction of that emulator :
Using the Neural Network functionality to implement LIME
Using the Neural Network functionality to implement LIME
Wolfram already has an excellent tutorial on how to accommodate weighted data within the Neural Network framework. I can use this to conduct weighted linear regression.
Get the data in the right form: an Association in which the keys are "Input","Target", and "Weight":
Build a neural network:
Train the net:
Extract the optimized weights and biases of the linear regression and thus emulate linear regression:
Get the layer that represents the model :
And apply the model to the test data :
This works just fine, but there doesn't seem to be a way to do it using L1 regularization. (L2 regularization can be simulated using the WeightDecay option, but L2 regularization amounts to Tikhonov or Ridge regression and does not generally result in the sort of sparse models that LIME understandably suggests). I would want a layer that somehow transports the weights of the linear layer, computes the sum of their absolute value and adds that sum (multiplied by some coefficient) to the weighted loss. I don't see a way to do that. Any help appreciated.