Neural Radiance Fields (Mildenhall et al., ECCV 2020) revolutionized computer vision and computer graphics by representing complex 3D scenes not as explicit polygonal meshes or voxel grids, but as continuous coordinate-based neural networks
f:(x,d)->(c,σ)
.
A common misconception is that training a NeRF requires complex custom CUDA kernels, external C++ build chains, or PyTorch. In this notebook, we demonstrate how to build, train, and render a complete NeRF entirely within native Wolfram Language using NetGraph, MXNet primitives, and the WLNeRFLite paclet.
Key technical highlights:
1
.
Direct camera ray generation from standard NeRF / OpenGL 4x4 camera-to-world (c2w) pose matrices (transforms.json).
2
.
GPU-embedded volumetric quadrature: computing continuous optical transmittance and alpha compositing entirely in-graph using lower-triangular matrix multiplication (trilStrict), eliminating CPU-GPU transfer bottlenecks.
3
.
Automated dataset ingestion of the classic synthetic 'Lego' bulldozer dataset via native Wolfram Language (DownloadNeRFDataset & LoadNeRFDataset).
4
.
Interactive 3D camera rig visualization with camera frustums and optical axes.
5
.
High-throughput ray pooling with stratified foreground sampling (>25,000 rays/sec).
6
.
Quantitative validation against held-out views (PSNR / MSE) and 360-degree novel-view turntable synthesis.
7
.
Continuous 3D density field extraction via Image3D and ListContourPlot3D.

1. Environment Setup & Package Loading

Make sure the WLNeRFLite paclet is in the same directory of this notebook, then evaluate the following to load it into the Wolfram kernel session and detect the compute target device (CUDA GPU or multi-core CPU).
In[]:=
SetDirectory[NotebookDirectory[]];
In[]:=
PacletDirectoryLoad[Directory[]];​​Needs["WLNeRFLite`"];
(*$TargetDevice="GPU";*)

2. Automated Dataset Ingestion: The Classic Synthetic Lego Bulldozer

The synthetic Lego bulldozer dataset from Mildenhall et al. contains 100 inward-facing hemispherical views for training and 100 validation views at 800x800 resolution with ground-truth camera matrices in transforms_train.json and transforms_val.json.
If the dataset is not yet present in data/lego, we can automatically fetch it directly in Wolfram Language using DownloadNeRFDataset:
In[]:=
dataDir=FileNameJoin[{Directory[],"data","lego"}];
Download the dataset if it’s not here already. This utility function goes to a repository with various NeRF datasets and downloads them.
In[]:=
DownloadNeRFDataset[dataDir,"Scene"->"lego","Splits"->{"train","val"}]
Out[]=
Directory/Users/fxpppr/GitHub/WLNeRFLite/data/lego,Scenelego,Splits{train,val},JSONFilestrain/Users/fxpppr/GitHub/WLNeRFLite/data/lego/transforms_train.json,val/Users/fxpppr/GitHub/WLNeRFLite/data/lego/transforms_val.json,TotalFrames200,DownloadedFiles0,CachedFiles200
Now load the training split into memory using LoadNeRFDataset. We downsample the training views to 200x200 for rapid interactive training and set WhiteBackground -> True to blend transparent rays against pure white.
In[]:=
views=LoadNeRFDataset[​​dataDir,​​"Split"->"train",​​"TargetDimensions"->{200,200},​​"WhiteBackground"->True​​];
In[]:=
Length[views]
Out[]=
100
The example dataset contains various useful bits and pieces for each view. Most important, the camera intrinsics and extrinsics, and respective images.
In[]:=
Keys[views[[1]]]
Out[]=
{CameraMatrix,FOV,Dimensions,RGB,Alpha,ForegroundPixels,BackgroundColor,File,FrameIndex}
So, for example, this is the first training view image:
In[]:=
{views[[1]]["Dimensions"],views[[1]]["FOV"]}
Out[]=
{{200,200},39.5978}
In[]:=
Image[views[[1]]["RGB"]]
Out[]=

3. Interactive 3D Camera Rig Visualization

WLNeRFLite automatically maps standard NeRF OpenGL poses (Z-backwards) to world camera frames. VisualizeNeRFCameraRig renders the 3D origin, camera orbital positions, and pyramidal viewing frustums pointing towards the Lego bulldozer:
In[]:=
VisualizeNeRFCameraRig[​​views[[1;;;;3]],​​"FrustumScale"->0.35,​​"PointSize"->0.015​​]
Out[]=

4. Differentiable Volumetric NetGraph Architecture

Standard volume rendering models radiant light transmission along ray
r(t)o+t*d
between bounds
[
t
near
,
t
far
]
. For
N
stratified samples:
τ
i

σ
i
*
δ
i
​​
T
i
exp-
∑
j<i
τ
j

​​
α
i
1-exp[-
τ
i
]
​​​
C(r)
N
∑
i1
T
i
*
α
i
*
c
i
+
T
N+1
*
c
bg
where τ is optical thickness,
T
is accumulated transmittance, α is the sample opacity, and
C(r)
the color.
In WLNeRFLite, this entire computation—harmonic positional encoding, coordinate MLP, lower-triangular optical accumulation (trilStrict), and alpha compositing—is encapsulated in an end-to-end differentiable NetGraph without external C++/CUDA custom kernels.
Initialize a coordinate MLP with 32 stratified samples per ray, 8 harmonic frequency bands, 96 hidden units, and 3 layers:
In[]:=
net=NetInitialize@CreateNeRFLiteNet[​​"NumSamples"->32,​​"HarmonicBands"->8,​​"HiddenDim"->96,​​"NumLayers"->3,​​"BackgroundColor"->{1.,1.,1.},​​"ColorActivation"->LogisticSigmoid​​]
Out[]=
NetGraph
Number of inputs:
2
Number of outputs:
2


5. High-Throughput Ray-Pool Training

Rather than evaluating entire images, WLNeRFLite pools candidate rays across all 100 views, sampling 1,024 random rays per batch with a 70% foreground bias (SampleForegroundRatio -> 0.7). This ensures the network concentrates on the intricate details of the Lego bulldozer rather than empty background.
Train for 3,000 rounds with real-time interactive progress reporting:
In[]:=
trainedModel=TrainNeRFLiteModel[​​net,​​views,​​"BatchSize"->1024,​​"MaxTrainingRounds"->3000,​​"LearningRate"->0.001,​​"Loss"->"L2",​​"RayBounds"->{2.0,6.0},​​"SampleForegroundRatio"->0.7,​​"TargetDevice"->Automatic,​​"TrainingProgressReporting"->"Panel"​​]
Out[]=
NetGraph
Number of inputs:
2
Number of outputs:
2

Save the trained model checkpoint to disk:
In[]:=
outputDir=FileNameJoin[{Directory[],"output"}];​​If[!DirectoryQ[outputDir],CreateDirectory[outputDir]];​​Export[FileNameJoin[{outputDir,"lego_nerflite.wlnet"}],trainedModel]

6. Novel View Synthesis & Validation Evaluation

Evaluate reconstruction fidelity against held-out validation poses from transforms_val.json (poses not seen during training):
First, we load the held-out validation views.
Here’s a sample ‘ground truth’ image:
Let’s look at a quantitative difference here:
Here’s a side by side comparison, along with an error map that is amplified by 3x.

7. 360-Degree Turntable Orbit Animation

RenderNeRFLiteTurntable generates a smooth circular camera trajectory orbiting the origin along the Z-axis (Z-up) and renders novel views from the trained field:
Optionally export the turntable animation to a self-contained GIF:

8. Continuous 3D Volumetric Field Sampling & Geometry Extraction

Sample 64x64x64 density grid in world space
Here’s the density image with the default color map:
It’s not too tough to add your own color map with ColorFunction:
The density volume can also be converted into a (somewhat messy looking) isosurface:

9. Summary & Forum Discussion Points

Summary of Key Concepts for Wolfram Community:
- Fully Native Architecture: Demonstrates that complex radiance fields do not require external CUDA kernels or PyTorch—Wolfram NetGraph cleanly handles end-to-end differentiable volumetric quadrature.
- Stratified Ray Pooling: Demonstrates efficient data generation with foreground sampling bias (>25,000 rays/sec).
- Standard NeRF Compatibility: Directly supports canonical transforms.json datasets (NeRF Synthetic, Instant-NGP, Nerfstudio) with native Wolfram Language dataset downloading via DownloadNeRFDataset.
Code Repository: https://github.com/ritmps/WLNeRFLite
Original Paper: NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12346. Springer, Cham. https://doi.org/10.1007/978-3-030-58452-8_24

CITE THIS NOTEBOOK

Training a neural radiance field (NeRF) from scratch in Wolfram Language: the classic Lego bulldozer​
by Flip Phillips​
Wolfram Community, STAFF PICKS, November 16, 2026
​https://community.wolfram.com/t/28600