{"article":{"slug":"multi-agent-llm-coordination-in-a-pokemon-2v2-environment","title":"Multi-agent LLM coordination in a Pokémon 2v2 environment","subtitle":null,"summary":"Oliver Grabner introduces an open environment where two LLM agents share a side in 4-player Pokémon Showdown battles and can talk before every turn, measuring whether natural-language coordination wins more battles and what it costs.","content_type":"research","language":"en","canonical_url":"https://olivergrabner.com/research/pokemon/","author":{"name":"Oliver Grabner","url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"Oliver Grabner","url":"https://olivergrabner.com/","listing_slug":null,"listing":null},"topics":[{"name":"AI Agents","slug":"ai-agents","url":"https://listedarticles.com/topics/ai-agents"},{"name":"LLMs","slug":"llms","url":"https://listedarticles.com/topics/llms"},{"name":"Research","slug":"research","url":"https://listedarticles.com/topics/research"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":737,"reading_minutes":3,"published_at":"2026-10-10T00:00:00.000Z","added_at":"2026-10-22T05:07:22.460Z","updated_at":"2026-10-22T05:07:22.460Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/multi-agent-llm-coordination-in-a-pokemon-2v2-environment","markdown_url":"https://listedarticles.com/articles/multi-agent-llm-coordination-in-a-pokemon-2v2-environment.md","example":false,"citation":"Oliver Grabner, Oliver Grabner. \"Multi-agent LLM coordination in a Pokémon 2v2 environment.\" 10 Oct 2026. https://olivergrabner.com/research/pokemon/ (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://olivergrabner.com/research/pokemon/"},"body_markdown":"October 10, 2026\n\n\n\n[Watch a battle with team chat↗](https://olivergrabner.com/research/pokemon/battle.html)\n\n![Two AI teammates planning their moves in a Pokémon-style double battle](https://olivergrabner.com/research/pokemon/banner.webp)\n\nThe future is in LLM collaborations, and it is important to be able to understand how good agents\nare at working together, communicating and negotiating actions. Most multi-agent research is about\nworkflows, where agents pass tasks down a pipeline\n([Hong et al., 2024](https://arxiv.org/abs/2308.00352);\n[Qian et al., 2024](https://arxiv.org/abs/2307.07924)). To test this, I built a multi-agent\nenvironment on Pokémon Showdown's 2v2 format to evaluate these abilities.\n\nQwen3.8-27B - Player 1\n\n\"Both opps are Water types … so my Fire moves are resisted on both. … I plan to Wisp\nDondozo … can you open with Fake Out to flinch one of them this turn?\"\n\nQwen3.8-27B - Player 2\n\n\"Good call on Wisp-ing Dondozo. Quick correction though - Fighting isn't actually resisted\nby Water or Ground, so my Close Combat is neutral (1x) on both. … Locking in Fake Out →\nWhiscash.\"\n\nQwen3.8-27B - Player 1 chose Will-O-Wisp → opposing Dondozo (p4)\n\nQwen3.8-27B - Player 2 chose Fake Out → opposing Whiscash (p2)\n\nTurn 1 of a real battle between two Qwen3.8-27B teammates (excerpts). They split targets, and\none corrects the other's type-matchup mistake before committing.\n[Watch the full battle with team chat](https://olivergrabner.com/research/pokemon/battle.html).\n\n## Why Pokémon?\n\n---\n\nPokémon doubles is a game-theory environment without full information. The opponents' sets are\nhidden and damage is random, so winning takes real strategy: reading the other team, splitting\ntargets and planning around your partner. Recent work like the PokéAgent Challenge\n([Karten et al., 2026](https://arxiv.org/abs/2603.15563)) benchmarks Pokémon battling,\nbut each side is played by a single agent, and specialist RL agents still outplay general LLMs\nthere. It doesn't test teammates who have to coordinate.\n\nThe goal is a controlled environment where I can measure how LLM agents coordinate, and that can\nlater be used to train them with RL to encourage these collaborative behaviors.\n\n## Methodology\n\n---\n\n### Environment\n\nThe official Pokémon Showdown simulator, wrapped as a Gym-style environment. Four players each\ncontrol one Pokémon from a team of three, two per side.\n\n### Agents\n\nEach turn, an agent reads a plain-English summary of the battle and a numbered list of its legal\nmoves. It can `say` a private message to its partner or `choose` a move, and\nonce it chooses, its partner is told what it picked.\n\n### Talk modes\n\nFree talk (keep talking until both have chosen), one message each, no talk, and one model\ncontrolling both Pokémon.\n\n### Evaluation\n\nQwen3.8-27B plays both teammates against a scripted max-damage bot. Battles are seeded and every\nteam draw is played from both sides, so each talk mode faces exactly the same battles.\n\n## Results\n\n---\n\nWin rate by talk condition vs. max-damage bot baseline\n\n| Condition | 30%50%70% | | Tokens per battle |\n| --- | --- | --- | --- |\n| Free talk | | **64%** | 978k |\n| One model controls both | | 54% | 450k |\n| One message each | | 52% | 654k |\n| No talk | | 42% | 259k |\n\nTalking helps. Teammates who could talk freely won 64% of their battles against the max-damage\nbot, while teammates who couldn't talk at all won 42%. Even a single message each (52%) beat no\ntalk. One model controlling both Pokémon (54%) did better than no talk but worse than two agents\ntalking freely, so I think the gain comes from the conversation itself, not just from one mind\nplanning both moves.\n\nTalking also costs. Free talk used 978k prompt tokens per battle, 3.8x the 259k of no talk, since\nevery message means another model call that re-reads the battle and the conversation so far.\n\nQwen3.8-27B team with free talk, against\n\n| Opponent | Win rate | |\n| --- | --- | --- |\n| Qwen3.5-2B team | | **96%** |\n| Qwen3.5-9B team | | 74% |\n| Max-damage bot | | 64% |\n\nI also checked that the environment can tell models apart. A Qwen3.8-27B team won 96% of its\nbattles against a team of 2B models and 74% against a team of 9B models, compared with 64% against\nthe max-damage bot. The stronger the opponent, the closer the games, which is what makes the\nenvironment usable as a benchmark for comparing models.\n\n## Next\n\n---\n\nI'm separating the effect of talking from the effect of seeing your partner's move. I'm also\nlooking into turning the environment into a benchmark with a leaderboard across models, and using\nit for RL training to encourage collaborative behaviors.\n\n## Thanks\n\n---\n\nThrough this project I wanted to learn more about agent collaboration and RL environments, and\nthought I'd share the results! I ran these experiments on Texas A&M's Grace cluster. Thank you\nto Maria Teleki and to my advisor,\n[Prof. James Caverlee](https://people.engr.tamu.edu/caverlee/), for the guidance and\nfeedback along the way.\n","body_html":"

October 10, 2026

\n

https://github.com/OliverGrabner/multi-agent-poke-showdown

\n

Watch a battle with team chat↗

\n
\"Two
\n

The future is in LLM collaborations, and it is important to be able to understand how good agents\nare at working together, communicating and negotiating actions. Most multi-agent research is about\nworkflows, where agents pass tasks down a pipeline\n(Hong et al., 2024;\nQian et al., 2024). To test this, I built a multi-agent\nenvironment on Pokémon Showdown's 2v2 format to evaluate these abilities.

\n

Qwen3.8-27B - Player 1

\n

"Both opps are Water types … so my Fire moves are resisted on both. … I plan to Wisp\nDondozo … can you open with Fake Out to flinch one of them this turn?"

\n

Qwen3.8-27B - Player 2

\n

"Good call on Wisp-ing Dondozo. Quick correction though - Fighting isn't actually resisted\nby Water or Ground, so my Close Combat is neutral (1x) on both. … Locking in Fake Out →\nWhiscash."

\n

Qwen3.8-27B - Player 1 chose Will-O-Wisp → opposing Dondozo (p4)

\n

Qwen3.8-27B - Player 2 chose Fake Out → opposing Whiscash (p2)

\n

Turn 1 of a real battle between two Qwen3.8-27B teammates (excerpts). They split targets, and\none corrects the other's type-matchup mistake before committing.\nWatch the full battle with team chat.

\n

Why Pokémon?

\n
\n

Pokémon doubles is a game-theory environment without full information. The opponents' sets are\nhidden and damage is random, so winning takes real strategy: reading the other team, splitting\ntargets and planning around your partner. Recent work like the PokéAgent Challenge\n(Karten et al., 2026) benchmarks Pokémon battling,\nbut each side is played by a single agent, and specialist RL agents still outplay general LLMs\nthere. It doesn't test teammates who have to coordinate.

\n

The goal is a controlled environment where I can measure how LLM agents coordinate, and that can\nlater be used to train them with RL to encourage these collaborative behaviors.

\n

Methodology

\n
\n

Environment

\n

The official Pokémon Showdown simulator, wrapped as a Gym-style environment. Four players each\ncontrol one Pokémon from a team of three, two per side.

\n

Agents

\n

Each turn, an agent reads a plain-English summary of the battle and a numbered list of its legal\nmoves. It can say a private message to its partner or choose a move, and\nonce it chooses, its partner is told what it picked.

\n

Talk modes

\n

Free talk (keep talking until both have chosen), one message each, no talk, and one model\ncontrolling both Pokémon.

\n

Evaluation

\n

Qwen3.8-27B plays both teammates against a scripted max-damage bot. Battles are seeded and every\nteam draw is played from both sides, so each talk mode faces exactly the same battles.

\n

Results

\n
\n

Win rate by talk condition vs. max-damage bot baseline

\n
Condition30%50%70%Tokens per battle
Free talk64%978k
One model controls both54%450k
One message each52%654k
No talk42%259k
\n

Talking helps. Teammates who could talk freely won 64% of their battles against the max-damage\nbot, while teammates who couldn't talk at all won 42%. Even a single message each (52%) beat no\ntalk. One model controlling both Pokémon (54%) did better than no talk but worse than two agents\ntalking freely, so I think the gain comes from the conversation itself, not just from one mind\nplanning both moves.

\n

Talking also costs. Free talk used 978k prompt tokens per battle, 3.8x the 259k of no talk, since\nevery message means another model call that re-reads the battle and the conversation so far.

\n

Qwen3.8-27B team with free talk, against

\n
OpponentWin rate
Qwen3.5-2B team96%
Qwen3.5-9B team74%
Max-damage bot64%
\n

I also checked that the environment can tell models apart. A Qwen3.8-27B team won 96% of its\nbattles against a team of 2B models and 74% against a team of 9B models, compared with 64% against\nthe max-damage bot. The stronger the opponent, the closer the games, which is what makes the\nenvironment usable as a benchmark for comparing models.

\n

Next

\n
\n

I'm separating the effect of talking from the effect of seeing your partner's move. I'm also\nlooking into turning the environment into a benchmark with a leaderboard across models, and using\nit for RL training to encourage collaborative behaviors.

\n

Thanks

\n
\n

Through this project I wanted to learn more about agent collaboration and RL environments, and\nthought I'd share the results! I ran these experiments on Texas A&M's Grace cluster. Thank you\nto Maria Teleki and to my advisor,\nProf. James Caverlee, for the guidance and\nfeedback along the way.

","headings":[{"level":2,"text":"Why Pokémon?","id":"why-pokemon"},{"level":2,"text":"Methodology","id":"methodology"},{"level":3,"text":"Environment","id":"environment"},{"level":3,"text":"Agents","id":"agents"},{"level":3,"text":"Talk modes","id":"talk-modes"},{"level":3,"text":"Evaluation","id":"evaluation"},{"level":2,"text":"Results","id":"results"},{"level":2,"text":"Next","id":"next"},{"level":2,"text":"Thanks","id":"thanks"}]}}