CodeParser and CodeInspector
CodeParser and CodeInspector
Brenton Bostick
April 2020
April 2020
I am a developer at Wolfram Research and I have been working on parser and linter technology that I think would be nice to share publicly.
Setup
Setup
There are 2 paclets on the public paclet server that you can download and use for finding problems with WL code.
The two paclets you will need are CodeParser and CodeInspector:
In[]:=
PacletInstall["CodeParser"]PacletInstall["CodeInspector"]
Out[]=
PacletObject
Out[]=
PacletObject
CodeParser
CodeParser
The CodeParser paclet provides functions for parsing WL code and returning syntax trees with interesting metadata attached, such as file and line information.
Here is a demo of using the CodeParser paclet.
First, load the CodeParser` package:
In[]:=
Needs["CodeParser`"]
You can parse WL syntax:
In[]:=
CodeParse["a + b"]
Out[]=
ContainerNode[String,{CallNode[LeafNode[Symbol,Plus,],{LeafNode[Symbol,a,Source{{1,1},{1,2}}],LeafNode[Symbol,b,Source{{1,5},{1,6}}]},Source{{1,1},{1,6}}]},]
The metadata such as Source{{1,5},{1,6}} indicates that this node spans from line 1, column 5 to line 1, column 6.
Notice that there is no whitespace in the result. If you care about whitespace, you can use CodeConcreteParse:
In[]:=
CodeConcreteParse["a + b"]
Out[]=
ContainerNode[String,{InfixNode[Plus,{LeafNode[Symbol,a,Source{{1,1},{1,2}}],LeafNode[Whitespace, ,Source{{1,2},{1,3}}],LeafNode[Token`Plus,+,Source{{1,3},{1,4}}],LeafNode[Whitespace, ,Source{{1,4},{1,5}}],LeafNode[Symbol,b,Source{{1,5},{1,6}}]},Source{{1,1},{1,6}}]},]
CodeConcreteParse preserves all whitespaces and comments.
With CodeParser, it is possible to build tools that consume a symbolic representation of WL syntax.
It is possible to also parse files with by doing CodeParse[File[file]]
CodeInspector
CodeInspector
The CodeInspector paclet uses CodeParser and provides functions for scanning WL code and finding problems and reporting them.
Here is a demo of using the CodeInspector paclet.
First, load the CodeInspector` package:
In[]:=
Needs["CodeInspector`"]
Next, take a snippet of code that you are interested in linting.
In[]:=
code="Which[a, b, a, b]";
Use CodeInspect to get a List of InspectionObjects:
In[]:=
lints=CodeInspect[code]
Out[]=
The InspectionObjects are interesting by themselves, but it would be nice to see them formatted within lines of the source code.
In[]:=
CodeInspectSummarize[code]
Out[]=
It is possible to also scan files with by doing CodeInspect[File[file]]
Both the parser and linter are open-sourced here:
I hope you enjoy using them!