In this project, we analyze Chess, a game which is notoriously complex and difficult to investigate. We analyze endgame positions specifically, which have fewer pieces and are therefore less complex. Our methodology involves creating multiway systems of endgame positions and experimenting with various classification methods for ensemble neural networks in order to extract the features of a position. We then show how these features are useful for determining the outcome, under optimal play, of a position, as well as driving more sophisticated human understanding of a position. These methods present insightful observations into the tactical and graph-theoretical structure of positions, as well as generating many paths for future study.
Introduction
Introduction
Chess is a zero-sum, 2-player, turn based game where each player starts with many pieces of different types (pawn, knight, bishop, rook, queen, king), and attempts to “checkmate”, or capture, the opposing king by moving their pieces and capturing opposing pieces. We will specifically study endgame positions, which are positions where there are only a few pieces left on the board.
To get started, let’s load Wolfram’s Chess Paclet, which contains many useful functions and constructs dealing with chess
Install Wolfram Chess
PacletInstall["Wolfram/Chess"]
Load Wolfram Chess
In[]:=
Needs["Wolfram`Chess`"]ParallelNeeds["Wolfram`Chess`"]
Load an internal package of Wolfram Chess, which contains fundamental functions such as move generation
In[]:=
Needs["Wolfram`Chess`MLE`ChessBoard`"]ParallelNeeds["Wolfram`Chess`MLE`ChessBoard`"]
Make a board (from the starting position in chess) where we can explore some of the functions we load
In[]:=
test=NewBoard["startpos"]
Out[]=
ChessBoard[1]
Display our board
In[]:=
test@print[]
Out[]=
Wolfram`Chess`MLE`ChessBoard`Private`PrintBoard
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,{ , , , , , , , },{ , , , , , , , },{ , , , , , , , },
, , , , , , , , ,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
In Chess, moves from a position are often notated in Standard Algebraic Notation (SAN), which includes the letter representing the moving piece (except for pawns, where the letter is omitted), and the destination square of the move (e.g. a1, e4). (There are other details to the notation which can be found here: https://rcc.fide.com/appendixc/)
Generate the legal moves from the starting position, in SAN
In[]:=
test@legalMovesText["SAN"]
Out[]=
{a3,a4,b3,b4,c3,c4,d3,d4,e3,e4,f3,f4,g3,g4,h3,h4,Na3,Nc3,Nf3,Nh3}
Make the move a3 in our starting position; we can see that the board got updated with this move
In[]:=
test@moveString["a3","SAN"];test@print[]
Out[]=
Wolfram`Chess`MLE`ChessBoard`Private`PrintBoard
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,{ , , , , , , , },{ , , , , , , , },{ , , , , , , , },
, , , , , , , , ,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Forsyth-Edwards Notation is a way to represent a chess position. It goes through the board from the top left corner (from white’s perspective) then goes to the right and down, writing down the letter representing the piece in that square (in uppercase for white, lowercase for black), and writing the number of consecutive empty squares when such squares appear. Between each rank, there is a forward slash. After this, there is some metadata about the position, the most important of which is the side to move, which is “b” if it is black’s turn, and “w” if it is white’s turn.
Get the FEN representation of the board
In[]:=
test@fen[]
Out[]=
rnbqkbnr/pppppppp/8/8/8/P7/1PPPPPPP/RNBQKBNR b KQkq - 0 1
Before we begin our analysis, it should be noted that many code blocks will take a long time to run, so easy to use progress bars will be helpful.
Create a wrapper around ProgressIndicator
In[]:=
$progressStatus=0;progressBar[]:=($progressStatus=0;ProgressIndicator[Dynamic[$progressStatus]])
Multiway Systems
Multiway Systems
A Multiway System is similar to a directed graph. For instance, in a multiway system of a chess position, the vertices are positions and the directed edges are moves. Graphing multiway systems often leads to insights, so we can implement one starting with some utility functions.
Define a function which takes in a FEN string and returns a list of the possible FENs after making each legal move in the position
In[]:=
generateFollowingFens[currBoard_]:=(new=NewBoard[currBoard];new@moveString[#,"SAN"];new@fen[])&/@(curr=NewBoard[currBoard];curr@legalMovesText["SAN"])
Define a function which takes in a board and returns a list of the possible boards after making each legal move in the position
In[]:=
generateFollowingBoards[currBoard_]:=(new=NewBoard[currBoard];new@moveString[#,"SAN"];new)&/@(currBoard@legalMovesText["SAN"])
Let’s use our functions to create a multiway system of an endgame. We’ll use the NestGraph function rather than the MultiwaySystem resource function because NestGraph allows us to input a generating function which generates the children of a node, while MultiwaySystem does not.
Create a graph of a King and Queen vs King endgame, up to depth 4
In[]:=
notReducedG=NestGraph[generateFollowingFens,"8/6Q1/8/8/8/5K2/7k/8 w - - 10 6",4,VertexLabels->None]
Out[]=
Symmetries
Symmetries
In order to reduce the size of our multiway system, we can try to exploit symmetries in the chess board.
A chess board is a square, so we can use the symmetries in the dihedral group of order 8. It contains the 4 rotations and 4 reflections. However, not all of these symmetries are always applicable. For example, pawns can only move forwards, so only a reflection across the vertical axis maintains the direction the pawns can move. So, we can consider different symmetries depending on whether or not pawns are on the board. Also, symmetries cannot be exploited when castling, a special move available only if the king has not moved and is on its starting square, is available, since the king would no longer be on its starting square after a non-identity transformation, however since we are analyzing endgames, where the king has likely moved already, this will probably not be an issue.
The Mailbox representation of the board is a 2D matrix, where mailbox[[rank]][[file]] tells you the piece on the corresponding rank and file. This will help us apply transformations on the board more easily.
Make a function which converts the part of FEN strings which describe the position (i.e. excluding the castling rights, en passant square, etc.) to a mailbox representation which
In[]:=
fenToMailbox[fen_]:=Block[{pieceData,rankStrings,digitReplace,expandedRanks},pieceData=First@StringSplit[fen];rankStrings=StringSplit[pieceData,"/"];digitReplace=Table[ToString[i]->StringRepeat["-",i],{i,8}];expandedRanks=StringReplace[rankStrings,digitReplace];Characters/@expandedRanks];
Make a function which converts the part of FEN strings which describe the position (i.e. excluding the castling rights, en passant square, etc.) to a “mailbag” representation, which is a list that looks like: {mailbox representation, side to move}
In[]:=
fenToMailbag[fen_]:={fenToMailbox[fen],StringSplit[fen][[2]]}
Inverse of fenToMailbox
In[]:=
mailboxToFen[mailbox_]:=StringRiffle[Module[{fen="",count=0},Table[If[sq=="-",count++,If[count>0,fen=fen<>ToString[count];count=0];fen=fen<>sq;],{sq,#}];If[count>0,fen=fen<>ToString[count]];fen]&/@mailbox,"/"]
Get symmetries which are unsafe/not symmetrical if castling/pawn movements are possible
In[]:=
getUnsafeSymmetries[mailbox_]:=Flatten[Table[curr=If[j==2,Reverse[mailbox],mailbox];curr=Nest[Reverse[Transpose[#]]&,curr,i];curr,{i,4},{j,2}],1]
Get symmetries which are safe if pawn movements are possible, but still unsafe if castling is possible
In[]:=
getSaferSymmetries[mailbox_]:={mailbox,Reverse[mailbox,2]}
For a list of symmetrical boards, choose a canonical representation based on the hash of each board
In[]:=
getCanonicalRep[symmetries_]:=MaximalBy[symmetries,Hash][[1]]
Same as generateFollowingFens and generateFollowingBoards except with mailbags instead
In[]:=
generateFollowingMailbags[mailbag_]:=fenToMailbag[#]&/@generateFollowingFens[StringJoin[mailboxToFen[mailbag[[1]]]," ",mailbag[[2]]," - - 0 1"]]generateFollowingMailbagsWithSymmetries[mailbag_]:=(mb=fenToMailbag[#];{getCanonicalRep[getUnsafeSymmetries[mb[[1]]]],mb[[2]]})&/@generateFollowingFens[StringJoin[mailboxToFen[getCanonicalRep[getUnsafeSymmetries[mailbag[[1]]]]]," ",mailbag[[2]]," - - 0 1"]]
Graph, with symmetry-based reduction, the same position graphed earlier to the same depth as earlier (depth 4)
In[]:=
egMailbag=fenToMailbag["8/6Q1/8/8/8/5K2/7k/8 w - - 10 6"];egMailbag={getCanonicalRep[getUnsafeSymmetries[egMailbag[[1]]]],egMailbag[[2]]};reducedG=NestGraph[generateFollowingMailbagsWithSymmetries,{egMailbag},4]
Out[]=
Check how much the graph size was reduced by
We can see that the number of vertices has decreased notably. Unfortunately, the graph is still extremely complex and hard to analyze.
To improve the visualization, we can color the vertices. One criteria to color the vertices is the distance between the two kings. This can be measured in Chebyshev distance, which represents the amount of king moves it would take for one king to reach the current position of the other king. The motivation for this choice is that in many endgames (our current King and Queen vs King being one), the king must protect/support the other pieces of its color to checkmate the opposing king.
We will mark vertices which represent checkmate with red, and use colors with descending wavelengths for vertices with greater Chebyshev distances
Color vertices based on Chebyshev distance between kings
Group together “communities” in the graph based on modularity, or identifying groups of nodes which have many edges within themselves but few edges with other groups
Create helper functions for grouping vertices based on Chebyshev distance between kings
Graph with grouped vertices based on Chebyshev distance between kings
These graphs show the connectedness of positions in a more readable way. Unfortunately, it is outside the scope of the project to show what chess concept these communities and other visually interesting patterns reflect.
Classifications
Classifications
Let’s try to classify endgame positions based on their properties. Our goal is to use properties which are generalizable and not specific to the game of chess, since this way the methods could potentially be used in other domains too. The number of legal moves in a position, or the degree, satisfies this criterion.
Get the number of pieces on the board, which is a proxy for whether the game is in an endgame phase or opening/midgame phase
Generate many random endgame positions and their corresponding degrees
Make histogram of the degrees of the positions we generated
Find the quartiles of the degrees of the positions we generated
Now that we can classify positions based on which quartile its degree falls under, we can try a couple more methods. Next, let’s try to classify based on the average score(1 for a win, 0 for a loss, 0.5 for a draw) of many random walks in a position. This approach similarly satisfies the goal of a generalizable method, since random walks are applicable to any multiway system. Additionally, the idea of random walks in chess has been explored by https://community.wolfram.com/groups/-/m/t/2575537, although in a different context. A drawback of this approach is that it can be slow, so in order to speed up the code, we end the random walk and do not use it in the average if it has lasted for a long time.
Make function to do a random walk from a given position
Do random walks for the positions we generated and take the average for each position
Create a histogram for our random walk averages
Get quartiles for our random walk averages
We see that the most of the random walks are ending in draws, so much so that all the quartile cutoffs are at 0.5. This is potentially because when there are not many pieces left, playing random moves leads to pointless shuffling of pieces (which eventually ends the game in a draw due to the fifty-move rule) even in a completely winning position. To solve this, we can adjudicate a position as a win if a position has a high chance of being winning, such as when the opponent only has a king and no other pieces and the side to move has enough material to checkmate.
Find if a side only has a king left and no other pieces
Find if a side has enough material to checkmate
Find if the side to move is probably going to win the endgame because of material
Adjudicate and end random walks if the side to move is in a winning endgame (as seen above)
Run our original random walk function and our random walk with adjudication function on the same position 300 times each to compare/contrast the results
We see that the position, which is a forced mate for white, receives a much more accurate average score when there are adjudications in the random walks. Also, we see that many more random walks are ending and being considered in the average before we stop them for taking too long to complete.
Do random walks with adjudication for the positions we generated and take the average for each position
Create a histogram for our random walk with adjudication averages
Get quartiles for our random walk with adjudication averages
We see that adjudication in random walks gives us a much different distribution, leading to many more wins/losses, and provides us with quartiles we can use to classify positions with, unlike before.
Neural Networks
Neural Networks
Now, let’s try to evaluate if our classifications actually capture relevant information. We hypothesize that the classifications capture information which is useful to determining whether a position is won/lost/drawn.To determine this, we can create an ensemble model which predicts the score of a game. The ensemble model is composed of several member neural networks, each of which specializes in predicting the score in positions with a specific classification. We can compare the performance of this ensemble model with a control model, which is a single, unspecialized neural network.
First, let’s collect data which we will use for training and testing the neural network. We will use the Lichess database, find endgame positions from it, and then label those positions with the endgame tablebase (which store the result of optimal play for endgame positions) evaluations. Since endgame tablebases for positions with 6 and 7 pieces take a large amount of storage, we will only use positions with 5 or less pieces in our data. The labelling was done in C++ for speed reasons and the final dataset is on the Wolfram Cloud.
Get labelled data from the cloud
-Data from `lichess_db _standard _rated _ 2014 - 09. pgn`
-Fens extracted from pgn with `pgn - extract - Wl - o fen_output . txt lichess_db _standard _rated _ 2014 - 09. pgn`
-Endgames with <= 5 pieces evaled with 3 - 4 - 5 Syzygy TBS
-Data from `lichess_db _standard _rated _ 2014 - 09. pgn`
-Fens extracted from pgn with `pgn - extract - Wl - o fen_output . txt lichess_db _standard _rated _ 2014 - 09. pgn`
-Endgames with <= 5 pieces evaled with 3 - 4 - 5 Syzygy TBS
Neural Networks are essentially parameterized functions. We will make our neural network take a board position as input and output the expected score of the position, with the parameters being tuned during the training process to produce outputs which are as accurate as possible. On the specifics of the architecture of the neural network, the input to the network will be 768 booleans, one for each (piece, color, square) combination (6 piece types * 2 colors * 64 squares = 768). There are two hidden layers of 64 neurons with ReLU activation, and a single output neuron describing the expected score of the position, from 0 to 1, with sigmoid activation. All layers are fully connected.
Create the architecture for our neural network
Transform a FEN string into numerical inputs for the NNUE
Get the training data for our model
Train our neural network
In order to evaluate the accuracy of our neural network, let’s used the mean squared error (MSE), or the mean of the squared differences between the neural network output and the actual score for many positions outside of the neural networks training data (lower MSE is better).
Evaluate, with MSE loss, the neural network on positions outside of the training data
Now, we can implement our ensemble model. For time reasons, it was only feasible to use a classifier based on degree, and not one based on random walks. First, let’s check the quartiles of our training data.
Get data on the degrees of the positions in our training data
Make a histogram on the degrees of our training data
Get the quartiles of the degrees of our training data
Interestingly, the distribution of degrees here is quite different from the distribution of degrees found earlier. A hypothesis for why this might be is that in human endgames, the proportion of pieces which are pawns is higher. Since pawns have at most 3 legal moves per position, this would explain why the distribution is more concentrated to positions with lower degree. However, further analysis is needed to test this hypothesis. Let’s continue with processing our data.
Classify a position based on the quartiles we found
Group our training data based on their degree
Turn the FENs in our data to numerical inputs
Train each of the member neural networks in our ensemble model
Evaluate a fen using the ensemble model
Test the MSE of the ensemble model
We see that the ensemble model performed slightly worse than the single model, but this may simply be due to ~25,000 positions for each individual model in the ensemble not being not enough to capture a good idea of how chess works. Let’s try providing each neural network in the ensemble model with ~100,000 positions. This means that if this new ensemble model does better than the single model, the improvement must be due to the classifications providing useful information which allows the individual neural networks to specialize and improve their predictions.
Get our larger training dataset
Turn the FENs in our data to numerical inputs
A For loop is needed here or else the kernel stalls from not having enough memory
A For loop is needed here or else the kernel stalls from not having enough memory
Train the member neural networks of our ensemble model
Evaluate a fen using the ensemble model
Evaluate the MSE loss of our new ensemble model
We see that the ensemble model has indeed performed better compared to the single model, confirming our original hypothesis that classifying by degree does give useful, underlying information about the position.
Conclusion & Future Directions
Conclusion & Future Directions
The work in this project has shown promising results in analyzing a specific endgame position and proven the usefulness of specific methods for classification. By visualizing endgame positions with multiway systems, we uncovered ways for humans to better understand a position, and by classifying endgame positions by degree, we uncovered a way for neural networks to also better understand a position. Our work on classification also incidentally led to many intriguing observations, such as a completely different distribution of random walk averages by simply adding adjudication, a significant difference in the distribution of the degree of positions in human endgames versus the distribution of the degree of positions from random moves. The result of the analysis in this project helps lead future research direction, including deeply analyzing more endgame positions, testing a greater variety of classification methods, identifying and studying differences between positions generated from random moves and human positions, and more.
Acknowledgements
Acknowledgements
First and foremost, I would like to thank Adam Millar, my mentor, for his patience and willingness to help me develop ideas, discuss observations, and teaching me when I encountered unfamiliar topics. Annie Zhu was also instrumental in helping me debug my code, peer review my essay, and implement speedups that made it possible to run the code in a reasonable amount of time. Additionally, I am grateful to Stephen Wolfram, who helped synthesize my interests and pointed me towards working on this project. Finally, I’d like to thank the open-source Lichess and Syzygy Tablebase projects, which helped me compile a large amount of data in a short period of time.
References
References
◼
Multiway graphs for minimal chess endgames by Pavel Arkhipov, Wolfram Community, STAFF PICKS, July 19, 2022
https://community.wolfram.com/groups/-/m/t/2575537
https://community.wolfram.com/groups/-/m/t/2575537
◼
Stephen Wolfram (2022), “Games and Puzzles as Multicomputational Systems,” Stephen Wolfram Writings. https://writings.stephenwolfram.com/2022/06/games-and-puzzles-as-multicomputational-systems/
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
Analyzing chess endgames with multiway systems and ensemble neural networks
by Winston Cai
Wolfram Community, STAFF PICKS, July 11, 2025
https://community.wolfram.com/groups/-/m/t/3503047
by Winston Cai
Wolfram Community, STAFF PICKS, July 11, 2025
https://community.wolfram.com/groups/-/m/t/3503047