Abstract: In a simulated reality, how could real player characters get in to experience it as a conscious being? I use a well-known cellular automaton, the game of life, to reason about this. The initial idea was to use AI to interface with a glider so that an external player could control the glider by resetting the initial conditions in real time. But there isn’t much time for me to implement it, so here’s a prototype in case anyone wants to move on.
Video Game Inspirations
Video Game Inspirations
There is growing interest in the question of whether we are living in a simulated reality. The growing interest in this topic goes with the increasing realistic capabilities of video game engines [1]. A fundamental problem is how to simulate consciousness, an arguably hard problem not really addressed in current discussions of the simulation hypothesis. One hypothesis about the nature of consciousness is that it involves a multilayered neural network encompassing strange loops within strange loops [2,3] expanding into reality itself in the self-simulation hypothesis [4]. In other words consciousness would need to emerge through a mechanism that must involve many strata, from a hypothetical Planck scale code to the neural network code to the large scale network of (dark) galaxies and dark energy. In this scenario to get consciousness one would need to simulate the underlying Planck scale code in a large scale to get emergence of matter and spacetime and all the necessary multi-strata chains in between them.
Suppose we can do that in the future, the emergent consciousness would be more like a non-player character (NPC) from our video games, but ones over which we would have no control. Like in a dream. If we think about our dreams at least, the people that appear there almost never do what we ask them to do, in fact it’s quite the opposite. But what about real players characters (PC) from our video games? Once the multi-strata neural network conditions for the emergence of consciousness are in place from this underlying simulation, would it be possible to connect our emerging consciousness to the simulation? We can do this with dreams. When we dream most of the time we participate as a PC in our dream, even though we don’t have control of many things there including the NPC that we can meet. This scenario we can only speculate about it right now. But we can suggest here how we can approach in practice some of the principles involved.
In computational frameworks, this type of emergence can be seen in a simple way in codes called cellular automaton (CA) [5]. A special case of interest is the game of life (GoL) [6], where from simple rules complicated patterns with their own dynamics can emerge. A simple example of such a pattern that propagates in some direction until it interacts with another pattern, given some initial condition, is called a glider. One idea would be to code one of this games in a modern video game engine, like the Unreal Engine 5 (UE5) such that we can see a 2 dimensional GoL toy world emerge and such that we can play the game by connecting to this toy world as one of the allowed emerging patterns such as the glider. Another idea in Mathematica would be to use tools from machine learning to track the glider in real time and allow us to control it by setting new initial conditions. The UE5 and machine learning will be let for a future exploration and here I present a prototype of the idea with a 2 dimensional GoL on the Mathematica software.
Suppose we can do that in the future, the emergent consciousness would be more like a non-player character (NPC) from our video games, but ones over which we would have no control. Like in a dream. If we think about our dreams at least, the people that appear there almost never do what we ask them to do, in fact it’s quite the opposite. But what about real players characters (PC) from our video games? Once the multi-strata neural network conditions for the emergence of consciousness are in place from this underlying simulation, would it be possible to connect our emerging consciousness to the simulation? We can do this with dreams. When we dream most of the time we participate as a PC in our dream, even though we don’t have control of many things there including the NPC that we can meet. This scenario we can only speculate about it right now. But we can suggest here how we can approach in practice some of the principles involved.
In computational frameworks, this type of emergence can be seen in a simple way in codes called cellular automaton (CA) [5]. A special case of interest is the game of life (GoL) [6], where from simple rules complicated patterns with their own dynamics can emerge. A simple example of such a pattern that propagates in some direction until it interacts with another pattern, given some initial condition, is called a glider. One idea would be to code one of this games in a modern video game engine, like the Unreal Engine 5 (UE5) such that we can see a 2 dimensional GoL toy world emerge and such that we can play the game by connecting to this toy world as one of the allowed emerging patterns such as the glider. Another idea in Mathematica would be to use tools from machine learning to track the glider in real time and allow us to control it by setting new initial conditions. The UE5 and machine learning will be let for a future exploration and here I present a prototype of the idea with a 2 dimensional GoL on the Mathematica software.
The Interactive GoL Game
The Interactive GoL Game
Usual game of life with simple external interaction
Usual game of life with simple external interaction
Usually when we run a CA or GoL we have control of the coding part to get some specific game and then control of the initial conditions. Then we run the game and watch what happens. Below one example where we start with some initial condition where we have a pattern, the so-called glider, above and some random configuration below, in the case two oscillators. A glider is one of many emergent patterns in a GoL that propagates in some direction. The idea will be to change the glider orientation in some key frames such that the glider don't merge with the oscillators. We want it to pass in between the oscillators without interaction.
GoL initialization
GoL initialization
The GoL initial condition
The GoL initial condition
In[]:=
MatrixPlot[grid2DGlider,Mesh->True]
Out[]=
The GoL initial evolution
The GoL initial evolution
In[]:=
golEvolution=CellularAutomaton["GameOfLife",grid2DGlider,50];Row[Table[MatrixPlot[CellularAutomaton["GameOfLife",grid2DGlider,50][[i]]],{i,1,50,3}]]
Out[]=
Deterministic evolution will led the glider to merge
Deterministic evolution will led the glider to merge
In[]:=
Manipulate[ golEvolution=CellularAutomaton["GameOfLife",grid2DGlider,300]; MatrixPlot[golEvolution[[index]],Mesh->True],{{index,1,"Walking"},1,300,1,AnimationRate->5,RefreshRate->60},SaveDefinitions->True,TrackedSymbols:>{index}]
Out[]=
GoL initial evolution in three dimension
GoL initial evolution in three dimension
The 3D view of the evolution with the possible final fate of the glider:
Manipulate[ golEvolution=CellularAutomaton["GameOfLife",grid2DGlider,100]; Graphics3D[Sphere/@Position[golEvolution[[1;;index+2]],1],ViewVertical->{-1,0,0}],{{index,1,"Walking"},1,98,1,AnimationRate->5,RefreshRate->60},SaveDefinitions->True,TrackedSymbols:>{index}]
Out[]=
Changing GoL initial evolution
Changing GoL initial evolution
Now we want to be able to join the game while it is playing or interact with it in different ways. Here it would be good to quote the book dune by Frank Herbert:
"She said the mystery of life isn’t a problem to solve, but a reality to experience. So I quoted the First Law of Mentat at her: `A process cannot be understood by stopping it. Understanding must move with the flow of the process, must join it and flow with it’."
Today’s video games already have this idea of a world out there where the PCs can enter and interact with it. The world and NPCs are rendered as needed, in a massively multiplayer online role-playing game (MMORPG) for example. But as far as we know, everything in some rendering of the game needs to be computed, NPCs are programmed to do what they do, there is a physics engine that handles physics simulation and so on. The prototype sought here is to have a scenario where there is emergence from some underlying rules, those of the GoL in our case. When the PC enters the game, what it is really doing is setting new initial conditions on the flow, similar to observations in real world. In other words, for this game to work, the PC needs to take turns with the underlying processor doing the calculation of the underlying rules.
For example let us say that in the frame 17 we have the state of the game with the glider in a new position
going down as presented below:
"She said the mystery of life isn’t a problem to solve, but a reality to experience. So I quoted the First Law of Mentat at her: `A process cannot be understood by stopping it. Understanding must move with the flow of the process, must join it and flow with it’."
Today’s video games already have this idea of a world out there where the PCs can enter and interact with it. The world and NPCs are rendered as needed, in a massively multiplayer online role-playing game (MMORPG) for example. But as far as we know, everything in some rendering of the game needs to be computed, NPCs are programmed to do what they do, there is a physics engine that handles physics simulation and so on. The prototype sought here is to have a scenario where there is emergence from some underlying rules, those of the GoL in our case. When the PC enters the game, what it is really doing is setting new initial conditions on the flow, similar to observations in real world. In other words, for this game to work, the PC needs to take turns with the underlying processor doing the calculation of the underlying rules.
For example let us say that in the frame 17 we have the state of the game with the glider in a new position
going down as presented below:
In[]:=
MatrixPlot[golEvolution[[17]],Mesh->True]
Out[]=
The idea is that if we want to play as this glider to prevent it from merging with the oscillator, we should be able to define a new initial condition for the system by affecting the local glider configuration such as switching ON/OFF cells (or a rotation of the full structure):
In[]:=
golEvolutionFull=golEvolution[[1;;17]];
Current glider state:
In[]:=
grid2DGliderCurrent=golEvolution[[17]];MatrixPlot[grid2DGliderCurrent[[5;;10,5;;10]],Mesh->True]
Out[]=
GoL glider state updated externally:
In[]:=
grid2DGlider2=grid2DGliderCurrent;grid2DGlider2[[7,9]]=1;grid2DGlider2[[8,7]]=1;grid2DGlider2[[9,7]]=0;grid2DGlider2[[9,8]]=0;Row[{MatrixPlot[grid2DGliderCurrent[[5;;10,5;;10]]],"->",MatrixPlot[grid2DGlider2[[5;;10,5;;10]]]}]
And then the new state becomes the new initial conditions for the GoL future evolution:
The new initial condition leads to new deterministic evolution with the glider moving up:
The new initial condition leads to new deterministic evolution with the glider moving up:
GoL new evolution with the glider moving up
GoL new evolution with the glider moving up
GoL new evolution with the glider moving up in three dimension
GoL new evolution with the glider moving up in three dimension
Repeating the process
Repeating the process
Here we can observe a side effect of our simulation, we have just obtained the behavior of the observer in the real world. When a PC enters the game or when consciousness emerges in reality, it starts to take turns with the underlying processor making observations, which is a kind of resetting of initial conditions for GoL.
Now we can continue playing this game by changing the glider’s direction up and down until it can get in between the two oscillators without interaction.
So the general idea is to first observe the dynamics of the game and get to know the rules. We can set the frame update to be slow so that we have time to make our moves, then we can join in a position where the rules will allow us to stay alive. Now we can change our perspective to the two-dimensional game and we can make moves taking turns with the underlying frame rules updating process and the idea is to try to stay alive! We don’t have a working game but we outline it with a few more examples of how it would work. A few frames ahead of where we made the first external change to define new initial conditions we can update the state of the glider again so that it goes down:
Now we can continue playing this game by changing the glider’s direction up and down until it can get in between the two oscillators without interaction.
So the general idea is to first observe the dynamics of the game and get to know the rules. We can set the frame update to be slow so that we have time to make our moves, then we can join in a position where the rules will allow us to stay alive. Now we can change our perspective to the two-dimensional game and we can make moves taking turns with the underlying frame rules updating process and the idea is to try to stay alive! We don’t have a working game but we outline it with a few more examples of how it would work. A few frames ahead of where we made the first external change to define new initial conditions we can update the state of the glider again so that it goes down:
So, repeating this process, eventually the glider passes through the oscillators:
So, repeating this process, eventually the glider passes through the oscillators:
The full evolution after the external updates
The full evolution after the external updates
Thought on emergence within emergence
Thought on emergence within emergence
As is well known from GoL, if we zoom out, new large independent patterns emerge, we could play the game at this level by telling the
engine to update the local neighborhood of some pattern in a specific way or we could tell the lower instances like
gliders to rotate or change their local ON/OFF structure and their instances would tell the engine to set new initial conditions.
engine to update the local neighborhood of some pattern in a specific way or we could tell the lower instances like
gliders to rotate or change their local ON/OFF structure and their instances would tell the engine to set new initial conditions.
References
References
◼
[1] The Simulation Hypothesis: An MIT Computer Scientist Shows Why AI, Quantum Physics and Eastern Mystics All Agree We Are In a Video Game; Bayview Books (2019).
◼
[2] Deleuze, G.; Guattari, G.: A Thousand Plateaus: Capitalism and Schizophrenia; University of Minnesota Press: Minneapolis, MN, USA, 1987.
◼
[3] Hofstadter, D.: I Am a Strange Loop; Basic Books: New York, NY, USA, 2007.
◼
[4] Irwin, K.: Amaral, M.; Chester, D.: The Self-Simulation Hypothesis Interpretation of Quantum Mechanics. Entropy 2020, 22, 247.
◼
[5] Wolfram, S.: A New Kind of Science; Wolfram Media: Champaign, IL, USA, 2002.
◼
[6] Eugene M. Izhikevich et al. (2015), Scholarpedia, 10(6):1816. http://www.scholarpedia.org/article/Game_of_Life. (Visited April 15, 2022).