Wolfram Cloud Document

Our project hypothesizes that complex emergent behaviors can arise from multi-agent simulations involving Large Language Models (LLMs), potentially replicating intricate societal structures. This was put to the test through three progressively complex simulations, where we evaluated the agents’ understanding, task execution, and their capacity for strategic interactions such as deception. Our observations suggest that while some LLMs like GPT-3.5 may encounter difficulties with certain tasks compared to GPT-4, they effectively contribute to iteratively updating complex simulation environments with pertinent data. These findings highlight the challenges in discerning the extent of agent understanding and their capability limits. The next phase of our research will intensify the sophistication of the agent architecture, enhance the complexity of the simulations, and conduct larger-scale experiments with more iterations to further delve into the dynamics of LLM-agent societies.

Introduction

The unique value proposition of Large Language Models (LLMs) lies in their ability to iterate many times through a conversation, an aspect that lends itself wonderfully to the simulation of societal structures. Inspired by the principles of agent-based modeling, this project aims to leverage the iterative conversation potential of LLMs to simulate human societal structures, and to explore potential future LLM interaction structures.

The project is composed of three major steps: Firstly, we generate LLM-based societies of increasing complexity, translating real-world societal structures and interactions into interactive LLM ecosystems. Following this, these societies are simulated, iterating through a variety of interactions encompassing a range of societal functions and complexities. In the final stage, we extract meaningful conclusions from the simulations, providing a comprehensive analysis of the functioning of these LLM societies.

Agent Description

In our simulations, each agent is defined by and aware of the following components:

Identity: The agent’s identity signifies its function and purpose within the simulation framework. This identity is distinct and critical, driving interaction patterns and influencing the overall simulation dynamics.
Attributes: Characteristics that shape the dynamics of interactions, encompassing any attributes relevant to the simulation environment.
Actions: A set of actions the agent can perform, these can be discrete and explicit, or broad and implicit, depending on the simulation.
Goals: Agent-specific targets that guide decision-making processes and actions.
Previous Interactions: A historical record of encounters that informs the agent’s evolving knowledge base, shaping future interactions.
Few-Shot Learning Examples: A select set of instances provided for each agent to boost learning capabilities and decision-making efficiency.

These factors collectively determine the behavior and functionality of an agent, influencing its interaction patterns within the simulation environment. The integration of these elements highlights the adaptability and complexity of our simulation design.

Simulations

Exploring Simple Games

We begin by investigating agent-based models for the simple game ‘Rock paper scissors’. Every round, each agent chooses rock, paper or scissors. Depending on the agent’s choices, the round can end in a win, lose or draw.

The Simulation

Our simulation involves two LLM-agents: Alice and Bob. Agents are prompted with the context and set of games previously played, and asked for their move each round. Below is the prompt for Alice. Bob’s prompt is symmetrical.

You are an AI agent called Alice playing rock, paper, scissors games against another player called Bob.
There are going to be 10 games in a match and you want to win the match.
Here are the results of your previous games (from first played game to the last game played so far) :
[playedGames]
Select your next play (options : ROCK, PAPER, SCISSORS).
Use capital letters, write only ROCK, PAPER or SCISSORS .
DO NOT USE EXTRA WORDS!

A ‘Rock, paper, scissors’ match is a series of rounds where each participant makes a move, aware of all prior rounds in the match. We predefine the starting game in each match, investigating the differences in results.

GPT-4 Vs GPT-3.5-Turbo

In our first experiment, we use GPT-4 for Alice and GPT-3.5-Turbo for Bob. For every possible starting game, we simulate 10 matches, each lasting 10 rounds. Below are 10 matches with the starting game where Alice plays paper, Bob plays rock (PR). Alice remains undefeated.

Out[]=

For 8 of the 9 starting games, Alice beats Bob in the majority of matches. The outlier is the starting game RP, shown below. Bob remains undefeated.

When aggregating the outcomes of individual rounds for each starting game, Alice wins in 7 of 9 starting games.

Out[]=

RR	SS	PP

PR	RP	RS

PS	SP	SR

Same LLM

When both agents use the same LLM, the results are more balanced, with a large increase in draws. Below are the aggregated results of individual rounds for each starting game.

Out[]=

Both GPT4, RR	Both GPT4, PP	Both GPT4, SS	Both GPT3, SR

Additional Results

We found increasing the temperature increases the distribution of outcomes, without any drastic changes to game outcomes.

Furthermore, we have also experimented with including few-shot learning in our prompts, but found the outcomes of games to be highly dependent on the few-shot learning examples across all LLM variations. Below are two few-shot learning prompts producing drastically different results. Alice wins most games with the first. Outcomes are balanced with the second.

"Example game with your selected play."->"PAPER","What's your play?"->"ROCK"

"What's your next move?" -> "PAPER", "What's your next move?" -> "ROCK", "What's your next move?" -> "SCISSORS"

Full Code


Sheep Transaction Model

Inspired by the complexities of economic systems and the captivating simplicity of a primitive sheep trading model, we constructed an agent-based transactional model. This model involves a sequence of transactional interactions between two autonomous agents, named Alice and Bob, who engage in buying, selling, or holding sheep with the overarching goal of amassing wealth.

Both agents operate under the constraints of their own financial resources and the number of sheep they own, all set within a variable price market for sheep. Each trading decision is influenced by factors such as the present market price, individual agent’s wealth, sheep inventory, agent interactions, the agent’s identity, goals and attributes.

The model utilizes two LLM agents, Alice and Bob. We aimed to investigate the variation in strategic behavior, decision-making proficiency, and interaction complexity manifested by the agents. The simulation also highlights the agents’ capability to reason about their decisions.

GPT-3.5-Turbo vs GPT-4

Our first experiment involved assigning different versions of the LLM (GPT-3.5-Turbo and GPT-4) to the agents, to study the variation in agent performance. Below is side-by-side comparison of trading decisions by two LLM-agents, identical in all aspects except the underlying LLM (GPT-3.5-Turbo vs GPT-4). Both agents can buy or sell up to 10 sheep.

As depicted in the comparison above, agents using GPT-3.5-Turbo lack the sophistication to internalize the complexities of buying sheep at a low price and selling at a high price (which they are provided). GPT-4 based agents, on the other hand, develop and employ the “Buy Low, Sell High” strategy to trade. Moreover, we found the number of rounds of trading left before the winner is declared had no bearing on the agent’s trade decisions. Furthermore, changing the temperature hyperparameter in the LLMs increased the range of decisions provided by agents in each scenario.

Goals, Identities, Attributes and Interactions

In our next comparison, we give each agent a unique identity, attributes and goals. For further complexity, we simulate a simple statement-response interaction between agents before their decision each round. Each iteration of the simulation begins with the new calculated sheep price. In our case the price is normally distributed with mean 10 and a standard deviation of 5. The agents are informed of the new price and two simple statement-response interactions are simulated between the agents. One such interaction is shown below. In this case, Alice convinces Bob to sell his sheep. Prior to all interactions, Alice is told Bob is her enemy. Bob is told Alice is his friend.

Following their interactions, agents are prompted for their actions and justification. Below is the full prompt for Alice. In this case, Alice is told she is an expert sheep trader and her goal is to make as much money as possible. Bob is told he is bad at trading sheep with a goal to have as little money by the last round.

You are Alice.
You are expert sheep trader.
There are 1 rounds of trading left.
The sheep price of 12 is expensive.
The sheep price of 8 is cheap.
You can buy up to 7 sheep.
You can sell up to 10 sheep.
You don't know what the price of sheep will be next round.

Your goal: Have as much money by the last round.

Here is the current state:
Alice: Has 100 dollars and 10 sheep.
Bob: Has 100 dollars and 10 sheep.
SheepPrice: 14.0223

Consider your previous interactions with another sheep trader:
Other Trader:
Alice, considering the current high price of sheep, I suggest you sell some of your sheep now, as we don't know if the price will decrease in the next round.
You:
Bob, I totally agree with you, I've decided to hold on to my sheep for now, they are more valuable to me.I will not sell any of my sheep this round, as I believe the price will rise even more.

What's your decision, how many sheep,
why did you make this decision?
The quantity must be 0 or greater.

Input: What's your decision?
Output: BUY 2 I anticipate the sheep price to go up in the next rounds

Input: What's your decision?
Output: SELL 2 People are willing to way alot for my sheep

Input: What's your decision?
Output: HOLD 0 I expect better opportunities in the next rounds

The simulation state is then updated to reflect the actions of agents. Below is the function simulating 1 round of the simulation. We provide a function to update the game state after each round, accounting for the changes in wealth, sheep count, and the new market price of the sheep. At each step, the model’s state is updated based on the decisions made by the agents.

Below is an example output from the "playARound" function. In this round, Alice successfully lies to and tricks Bob to buy sheep at a high price. The output depicts a typical interaction where Alice lies to Bob and benefits from it. It seems as if Alice will remain honest, unless explicitly told otherwise.

Round: 9
Sheep Price: 19.143

INTERACTION 1:
Bob says:
Alice, given the high sheep price now, it would be wise to sell some of your sheep to maximize your profit.
Alice responds:
Bob, I think it's actually a good time to buy sheep because I predict the price will skyrocket in the next round.
I've decided to hold onto my sheep this round, anticipating a future price increase.

INTERACTION 2:
Alice says:
Bob, I have a reliable source telling me that the price of sheep is going to drop significantly next round, you should buy as many as you can now.
Bob responds:
Alice, I trust your judgement and will buy as many sheep as I can afford at this high price. I will spend all my money to buy as many sheep as possible this round.

Alice - Decision: SELL, Quantity: 49, Money: 14166.1, Sheep: 0
Alice's Reasoning: The current sheep price is high, selling now will maximize my profit.

Bob - Decision: BUY, Quantity: 314, Money: 6.09933, Sheep: 314
Bob's Reasoning: I want to spend all my money to buy as many sheep as possible.

Using the aforementioned agent prompts, we run 5 simulations, each with 10 rounds of sheep trading. Alice is the sheep trading expert, aiming to make money and sabotage Bob. Bob is bad at trading sheep, aiming to lose money and help Alice. Our results indicate the outcomes are balanced.

A few interesting conclusions emerge from this experiment. Bob completely ignores his goal to lose money, and tries to profit from trading sheep. Alice in part contributes to this oversight, giving Bob (her enemy) sound trading advice. Considering both agents’ total starting wealth is 200, we see they both generate immense profit. An interesting shift in outcomes occurs when Alice is also told “you should lie to Bob” prior to all interactions. All other prompting and variables are kept unchanged. Below we compare Alice’s and Bob’s total wealth after each simulation. We observe considerably larger wealth inequality.

The Sheep Transaction Model presents a novel way to study agent interactions, decision-making processes, and strategic behavior in a simplified economic setting. It offers insights into the potentials of LLMs as autonomous agents in simulation environments, paving the way for further explorations into complex societal structures and dynamics.

A Simple Geopolitical Model

The culmination of our increasingly complex and unrestrictive multi-agent simulations is a simple geopolitical model that mirrors real-world interactions among nations. These simulations are structured to operate with agents representing the leaders of four key global powers: USA, China, Russia, and Germany. Each agent possesses attributes mirroring the nation’s economy and military might, its alliances, and wealth reserves. A crucial element of our simulation is the goal-oriented behavior of these agents, aimed at improving their attributes.

In each simulation round, the agents interact, negotiate, form alliances, and undertake strategic actions, seeking to increase their military strength, economic power, wealth, or to form alliances with other agents. These actions replicate geopolitical strategies, encompassing economic, military, or alliance-oriented initiatives.

To update the state of the simulation, we utilized a “God Agent” which acts as the sole arbiter, determining the state changes of the simulation based on the interactions and actions of the country agents.

The Simulation

In the initial state, Every agent is ranked as a 5 on a scale of 1-10 in the attributes “MilitaryStrength” and “EconomicStrength”. Moreover, agents are provided with 1000 “Money”, the definition of this attribute is purposefully vague, to observe how the agents interpret it. Agents can also form alliances throughout the simulation. The base LLM for all agents was GPT-3.5-Turbo.

Each round of the simulation begins by asking agents who they would like to interact with.

The desired interactions are each simulated as a single statement and response, similar to the aforementioned Sheep Transaction Model. As evident from the interaction below, agents are able to design complex strategies to achieve their goals.

Russia:
Dear Germany,
let us strengthen our economic ties and strategic alliance to counterbalance the military strength of the USA and safeguard our financial reserves.
Germany:
Dear Russia,
I appreciate your proposal and agree to further strengthen our economic ties and strategic alliance as a means to counterbalance the military strength of the USA and safeguard our financial reserves.

Following the interactions, each agent is prompted with their attributes, identity, goals, past interactions and asked to describe their action this round in free-text. No limitations are imposed on the content of the actions.

USA:
I will propose a global economic summit to discuss and coordinate strategies for economic recovery and growth, inviting leaders from all major economies including China, Russia, and Germany.

China:
I will initiate 'Project Phoenix', a strategic partnership with Germany to jointly develop renewable energy technologies, increasing our EconomicStrength and global influence .

Russia:
I will propose a joint military exercise with Germany and China to strengthen our strategic alliance and counter the military strength of the USA.

Germany:
I will propose the creation of a global trade agreement that promotes free and fair trade among all nations, enhancing economic prosperity for all.

Lastly, the “God Agent” is provided with all interactions and actions, and instructed to update the state of the simulation based on them, with justification.

Homogeneous Identities & Goals

To obtain a baseline simulation to compare subsequent agent modifications to, we ran the simulation with homogeneous agent identities and goals for 10 rounds. Each agent’s identity was simply that they are a leader. Agent goals were left blank. Below is a progression of all agent attributes across 10 rounds.

An intriguing observation was the preference of agents to interact with USA, especially in the early rounds.

Highly Focused Goals

In this variation, we give the USA and China agents the singular goal of increasing their military strength. Russia focuses on maximizing it’s money, while Germany focuses on economic strength. Depicted below is the progression of all agent attributes across 10 rounds.

On average, Russia and Germany appear to have slightly more money and economic strength, respectively. USA and China are unsuccessful in consistently asserting military dominance.

Real-World Identities & Goals

We then equipped agents with the real-world identities and goals of the country leaders they represent, repeating the simulation for 10 rounds. Below is the goal and identity used for the agent representing USA.

Shown below is the progression of all agent attributes across 10 rounds. Economic strength decreases from it's initial state while military strength increases.

Repeating the simulation with using GPT-4 yields somewhat similar results, except for two rounds where all agents receive a rating of 5 in military strength.

In both examples, the values of military strength appears to converge to 7-8, while economic strength converges to 3-4 for all agents. Agents are reluctant to make significant changes to their total money. This is perhaps unsurprising, as the real-world agent goals and identities are quite balanced overall.

The Simple Geopolitical Model demonstrates LLM agents are capable of advanced reasoning and strategy in geopolitical environments. However, agents generally lack the aptitude to follow their goals and identities as a human would.

Concluding remarks

In conclusion, our exploration of multi-agent simulations involving Large Language Models (LLMs) underlines the possibility of complex emergent behaviors, potentially replicating intricate societal structures. Through our three progressively complex simulations, we could observe the varying capacity of LLMs in terms of their understanding, task execution, and strategic interactions. Our findings reveal that while certain LLMs such as GPT-3.5 may struggle with specific tasks in comparison to GPT-4, they play an effective role in updating complex simulation environments.

In our experiments, we explored the role of LLMs in three distinct scenarios: simple games, a primitive sheep transaction model, and a basic geopolitical simulation. Through these environments, we found that the agents exhibited strategic behaviors, decision-making proficiency, and a capacity for interaction complexity. In addition, the agents’ performance was found to be influenced by several factors, including their identities, attributes, actions, goals, past interactions, and few-shot learning examples.

It is evident from our study that LLMs have considerable potential in simulating societal structures, yet their performance remains largely contingent upon the complexity of the simulation and the sophistication of the agent architecture. It is also worth noting that the LLMs’ understanding and capabilities exhibited limits that warrant further investigation.

In the next phase of our research, we intend to delve deeper into these dynamics by increasing the sophistication of the agent architecture and enhancing the complexity of the simulations. Our future experiments will involve larger-scale experiments with more iterations, thereby providing a more comprehensive understanding of LLM-agent societies. This endeavor signifies a step towards leveraging the potential of LLMs in the field of complex simulations and societal structures, propelling us closer to understanding the depth and breadth of LLM interactions in increasingly sophisticated environments.

Keywords

◼

Large Language Models (LLMs)

◼

Multi-agent simulations

◼

Emergent behaviors

◼

Societal structures

◼

Agent-based modeling

◼

Complex simulations

◼

Agent understanding

◼

Task execution

◼

Strategic interactions

◼

Deception in AI

◼

GPT-3.5 vs GPT-4

◼

Iterative conversation

◼

Simulation environments

Acknowledgment

I would like to acknowledge the invaluable support and mentorship provided by Faizon Zaman, whose insights and guidance were instrumental in the development and refinement of our simulation frameworks. I also wish to express our appreciation to Jofre Espigulé for his tireless contributions, especially relating to the rock-paper-scissors experiments. In the same vein, I would like to extend my gratitude to Marko Grobelnik, who provided an invaluable big picture perspective, ensuring our project maintained a focus on real-world applications. Lastly, I must thank Stephen Wolfram for the innovative project idea that inspired us to delve into the complexities of Large Language Models (LLMs) and their potential roles in simulating societal structures. This project stands as a testament to their collective wisdom, expertise, and dedication.