Multi-agent LLM coordination in a Pokémon 2v2 environment

Oliver Grabner introduces an open environment where two LLM agents share a side in 4-player Pokémon Showdown battles and can talk before every turn, measuring whether natural-language coordination wins more battles and what it costs.

October 10, 2026

https://github.com/OliverGrabner/multi-agent-poke-showdown

Watch a battle with team chat↗

Two AI teammates planning their moves in a Pokémon-style double battle

The future is in LLM collaborations, and it is important to be able to understand how good agents are at working together, communicating and negotiating actions. Most multi-agent research is about workflows, where agents pass tasks down a pipeline (Hong et al., 2024; Qian et al., 2024). To test this, I built a multi-agent environment on Pokémon Showdown's 2v2 format to evaluate these abilities.

Qwen3.8-27B - Player 1

"Both opps are Water types … so my Fire moves are resisted on both. … I plan to Wisp Dondozo … can you open with Fake Out to flinch one of them this turn?"

Qwen3.8-27B - Player 2

"Good call on Wisp-ing Dondozo. Quick correction though - Fighting isn't actually resisted by Water or Ground, so my Close Combat is neutral (1x) on both. … Locking in Fake Out → Whiscash."

Qwen3.8-27B - Player 1 chose Will-O-Wisp → opposing Dondozo (p4)

Qwen3.8-27B - Player 2 chose Fake Out → opposing Whiscash (p2)

Turn 1 of a real battle between two Qwen3.8-27B teammates (excerpts). They split targets, and one corrects the other's type-matchup mistake before committing. Watch the full battle with team chat.

Why Pokémon?

Pokémon doubles is a game-theory environment without full information. The opponents' sets are hidden and damage is random, so winning takes real strategy: reading the other team, splitting targets and planning around your partner. Recent work like the PokéAgent Challenge (Karten et al., 2026) benchmarks Pokémon battling, but each side is played by a single agent, and specialist RL agents still outplay general LLMs there. It doesn't test teammates who have to coordinate.

The goal is a controlled environment where I can measure how LLM agents coordinate, and that can later be used to train them with RL to encourage these collaborative behaviors.

Methodology

Environment

The official Pokémon Showdown simulator, wrapped as a Gym-style environment. Four players each control one Pokémon from a team of three, two per side.

Agents

Each turn, an agent reads a plain-English summary of the battle and a numbered list of its legal moves. It can say a private message to its partner or choose a move, and once it chooses, its partner is told what it picked.

Talk modes

Free talk (keep talking until both have chosen), one message each, no talk, and one model controlling both Pokémon.

Evaluation

Qwen3.8-27B plays both teammates against a scripted max-damage bot. Battles are seeded and every team draw is played from both sides, so each talk mode faces exactly the same battles.

Results

Win rate by talk condition vs. max-damage bot baseline

Condition		Tokens per battle
Free talk	64%	978k
One model controls both	54%	450k
One message each	52%	654k
No talk	42%	259k

Talking helps. Teammates who could talk freely won 64% of their battles against the max-damage bot, while teammates who couldn't talk at all won 42%. Even a single message each (52%) beat no talk. One model controlling both Pokémon (54%) did better than no talk but worse than two agents talking freely, so I think the gain comes from the conversation itself, not just from one mind planning both moves.

Talking also costs. Free talk used 978k prompt tokens per battle, 3.8x the 259k of no talk, since every message means another model call that re-reads the battle and the conversation so far.

Qwen3.8-27B team with free talk, against

Opponent	Win rate
Qwen3.5-2B team		96%
Qwen3.5-9B team		74%
Max-damage bot		64%

I also checked that the environment can tell models apart. A Qwen3.8-27B team won 96% of its battles against a team of 2B models and 74% against a team of 9B models, compared with 64% against the max-damage bot. The stronger the opponent, the closer the games, which is what makes the environment usable as a benchmark for comparing models.

I'm separating the effect of talking from the effect of seeing your partner's move. I'm also looking into turning the environment into a benchmark with a leaderboard across models, and using it for RL training to encourage collaborative behaviors.

Thanks

Through this project I wanted to learn more about agent collaboration and RL environments, and thought I'd share the results! I ran these experiments on Texas A&M's Grace cluster. Thank you to Maria Teleki and to my advisor, Prof. James Caverlee, for the guidance and feedback along the way.