World Model and Environment Simulation Benchmarks in 2026
Patronus AI TeamAs of November 202612 min read
Patronus AI publishes this guide and created GlobeBench and the World Model Corpus, two of the entries on it. A Patronus AI researcher coauthored HalluWorld. Entries are not ranked, and each entry links its paper and data.
Patronus AI created GlobeBench, builds RL environments and is building Digital World Models for AI agents. On Jun 25, 2026, it announced a $50M Series B led by Greenfield Partners. As of November 2026, public benchmarks and datasets for world models and environment simulation include GlobeBench, the World Model Corpus, AutumnBench, ByteSized32-State-Prediction, EWMBench, HalluWorld, Physics-IQ, Text2World, WorldModelBench, WorldPrediction and WorldScore. They differ in the kind of world, in what the model has to produce and in license. Pick the ones that match the environment your world model has to simulate, and read each license before you use the data.
A world model learns how an environment changes, so that given the current state and an action, it predicts the next state. For AI agents that work in software, the world can be text, such as tool outputs and app states. For robots and video generators, it is the physical world. This guide covers eleven public benchmarks and datasets. Patronus AI is a behavioral research lab that publishes at NeurIPS, ICML and ACL. Its GlobeBench has 1,632 tasks that test whether a model can predict how an environment responds to an agent’s next action.
GlobeBench, ByteSized32-State-Prediction and HalluWorld test text worlds, and the World Model Corpus is training data for text world models. Text2World asks for rules in PDDL, and AutumnBench uses grid worlds. EWMBench, Physics-IQ, WorldModelBench, WorldPrediction and WorldScore test video and 3D world models.
Snapshot as of
Key takeaways
- Match the kind of world. A world model for a software agent predicts text, such as a tool’s response or a file system’s state, as in GlobeBench and ByteSized32-State-Prediction. A world model for robots or video predicts frames, as in Physics-IQ, EWMBench and WorldModelBench.
- Check how answers are scored. GlobeBench uses a deterministic grader with an LLM judge for the cases the grader can’t decide. HalluWorld scores by rule with no LLM judges, WorldModelBench uses a judge model trained on human labels, and Physics-IQ compares generated video with real recordings.
- Keep training data and test sets apart. The World Model Corpus is mainly training data. HalluWorld gates its question bank to keep it out of training corpora, WorldModelBench withholds test answers, and the Physics-IQ README says its test data must not be used to improve a score.
- Read the license first. As of Nov 19, 2026, the World Model Corpus and WorldScore are MIT, HalluWorld and AutumnBench’s data are CC BY 4.0, and GlobeBench is Apache-2.0 with per-item exceptions. EWMBench (CC BY-NC-SA 4.0) and WorldPrediction (CC BY-NC 4.0) are non-commercial. ByteSized32-State-Prediction, Text2World and WorldModelBench state no license.
Comparison table
All eleven benchmarks and datasets at a glance, in the order of this guide. Sizes and licenses are as each card, README, license file or paper states them on Nov 19, 2026. Sort by any column marked with arrows.
| Benchmark | Best for | What it measures | World | Size | License |
|---|---|---|---|---|---|
| GlobeBenchRead the paper → | Text world models that simulate an environment | Predicting an environment’s response to an agent’s next action | Text (code, enterprise, games, math and science, dialogue) | 1,632 tasks | Apache-2.0, with per-item exceptions |
| World Model Corpus | Training text world models for agentic RL | Trajectories for training and testing text world models | Text (software engineering, research, customer service, tool use) | 239,403 training trajectories | MIT |
| AutumnBench | Testing whether an agent learns a model it can query | Questions about a whole environment, beyond what the agent observed | Grid worlds | 20 environments, 60 tasks (public set) | CC BY 4.0 data, MIT code |
| ByteSized32-State-Prediction | Language models as simulators for text games | Predicting the next state of a text game after an action | Text games | 76,369 transitions | Not stated in its repository |
| EWMBench | Video world models for robot manipulation | Scene, motion and meaning in generated robot videos | Video (robot manipulation) | 10 tasks | CC BY-NC-SA 4.0 |
| HalluWorld | Measuring hallucination about a world whose state is known | False claims about a fully specified world | Grid worlds, chess, terminal sessions | 1,439 questions | CC BY 4.0 |
| Physics-IQ | Physical understanding in video models | Whether generated video continues real physics | Real-world video | 396 videos | Apache 2.0 code, CC BY 4.0 materials |
| Text2World | Writing an environment’s rules from a description | Generating a symbolic world model in PDDL | Text to PDDL | 103 domains | Not stated |
| WorldModelBench | Video generation models judged as world models | Instruction following, common sense and physics in generated video | Video | 350 prompts | Not stated in its repository |
| WorldPrediction | High-level actions and plans from video | Picking the action or plan that explains a change | Video of real activities | 825 + 570 samples | CC BY-NC 4.0 |
| WorldScore | Comparing 3D, 4D and video world generation on one scale | Controllability, quality and dynamics of generated worlds | 3D, 4D and video | 3,000 examples | MIT |
GlobeBench
GlobeBench is a benchmark for text-based world models, built by Patronus AI. Its paper, GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations, was posted on Oct 15, 2026.
How does GlobeBench work?
- A model reads an agent’s trajectory of actions and environment observations and predicts how the environment responds to the next action. Each prediction is compared with a reference taken from running the environment.
- 1,632 tasks in five domains (code, enterprise, games, math and science, and dialogue). The tasks test 19 simulation skills.
- A deterministic grader checks each prediction first, and an LLM judge decides when the grader can’t.
- On Hugging Face as PatronusAI/globebench, with code at patronus-ai/globebench. Its card puts the code and the items Patronus AI wrote under Apache-2.0. Items adapted from other projects keep their source license, recorded per item, and one is GPL-3.0-or-later.
Who built it
Patronus AI was founded in 2023 by CEO Anand Kannappan, formerly of Meta Reality Labs, and CTO Rebecca Qian, formerly of FAIR (Meta AI). Its researchers and engineers come from Meta Superintelligence Labs, Amazon AGI, Apple and Google.
Patronus AI funding
A $50M Series B led by Greenfield Partners, announced Jun 25, 2026, with participation from Lightspeed Venture Partners, Notable Capital, Datadog, Samsung and others, after a $17M Series A led by Notable Capital in May 2024.
Patronus AI accepted venues
Main conference papers at ICML 2026, NeurIPS 2026 and ACL 2025, and a paper in Findings of ACL 2026.
Best for: Testing whether a language model can simulate a software or text environment step by step.
World Model Corpus
The World Model Corpus is the dataset released with Patronus AI’s NeurIPS 2026 main conference paper Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL. It is mainly training data, with held-out test splits.
How does the World Model Corpus work?
- Agent trajectories from nine open-source environments (Tau2Bench, SWE-Smith, CoderForge, DeepResearchQA, OpenResearcher, Gorilla/BFCLv4, Toolathlon, Pandora and WebShop), generated with twelve frontier model families.
- As of Nov 19, 2026, its splits hold 239,403 training, 19,454 eval, 20,847 test and 4,503 out-of-distribution test trajectories.
- On Hugging Face as PatronusAI/world_model_corpus under the MIT License, with training code at patronus-ai/
mdlm_ .world_ modeling
Best for: Training a text world model whose rollouts stand in for live environments in agentic RL.
AutumnBench
AutumnBench implements WorldTest, a protocol for testing whether agents learn world models. It comes from Basis Research Institute with DFKI, Harvard University, Université de Montréal and Mila, the University of Cambridge, MIT and Cornell University (Benchmarking World-Model Learning with Environment-Level Queries).
How does AutumnBench work?
- The paper counts 43 interactive grid-world environments and 129 tasks in three query families, for both people and learning agents, and keeps a subset private. The public set has 20 environments and 60 tasks.
- The questions, such as which states are reachable or what an intervention changes, depend on the whole environment and not only on what the agent saw.
- The public data is on Zenodo under CC BY 4.0, and the baseline code is in BasisResearch/MARAProtocol under the MIT License.
Best for: Testing whether an agent builds a model of an environment it can reuse for new questions.
ByteSized32-State-Prediction
ByteSized32-State-Prediction is a dataset of text game state transitions from the University of Arizona, New York University, Johns Hopkins University, Microsoft Research Montréal and the Allen Institute for AI (Can Language Models Serve as Text-Based World Simulators?).
How does ByteSized32-State-Prediction work?
- 76,369 state transitions from 31 text games, as the paper counts them.
- A model gets the previous state as a JSON object and an action, and predicts the next state. Tasks cover changes the action causes, changes the environment makes on its own, and game progress.
- The data and the game rules from the paper are in cognitiveailab/GPT-simulator, which states no license as of Nov 19, 2026.
Best for: Testing a language model as a simulator for text games.
EWMBench
EWMBench, the Embodied World Model Benchmark, comes from AgiBot with SJTU, MMLab-CUHK and HIT (EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models).
How does EWMBench work?
- EWMBench scores video world models for robots on visual scene consistency, motion correctness and semantic alignment with the instruction.
- The test data is sampled from AgiBot World, a real-world robot manipulation dataset, in ten tasks chosen for their sequential dependencies.
- On Hugging Face as agibot-world/EWMBench, whose card lists the license cc-by-nc-sa-4.0, a non-commercial license. The evaluation code is at AgibotTech/EWMBench.
Best for: Judging video world models that generate robot manipulation.
HalluWorld
HalluWorld is a hallucination benchmark built on reference worlds, from Carnegie Mellon University, Patronus AI, Stanford University, The Ohio State University and independent researchers (HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models, NeurIPS 2026 Evaluations and Datasets track). Its coauthors include Varun Prashant Gangal, Member of Technical Staff, Research, at Patronus AI.
How does HalluWorld work?
- Each environment fully specifies a reference world and controls what the model sees, so a claim that is false in that world is labeled a hallucination automatically.
- Its README counts 1,439 frozen questions in four tracks (grid worlds, chess, terminal sessions and questions asked during navigation), with rule-based scoring and no LLM judges.
- Each question has one of five categories (perceptual, memory, causal, uncertainty or compound).
- Code at DegenAI-Labs/HalluWorld. Its license file puts the authors’ material, the question bank included, under CC BY 4.0, and a bundled Terminal-Bench fork under Apache-2.0. The question bank is a gated Hugging Face dataset, so you request access first.
Best for: Measuring hallucination against a world whose true state is known.
Physics-IQ
Physics-IQ is a benchmark of filmed physical experiments from researchers at Google DeepMind and INSAIT, Sofia University (Do generative video models understand physical principles?).
How does Physics-IQ work?
- 396 videos of 66 physical scenarios, each filmed from three angles and recorded twice, in solid mechanics, fluid dynamics, optics, thermodynamics and magnetism.
- A model sees the beginning of a video and generates how it continues, and the result is compared with the real recording.
- The repository’s README recommends Physics-IQ Verified, a 2026 revision with improved prompts and scoring.
- Its README licenses the software under Apache 2.0 and all other materials under CC BY 4.0.
Best for: Testing whether a video model predicts how real objects behave.
Text2World
Text2World is a benchmark for symbolic world models from The University of Hong Kong, Shenzhen University, Harbin Institute of Technology and Shanghai AI Laboratory (Text2World: Benchmarking Large Language Models for Symbolic World Model Generation).
How does Text2World work?
- A model reads a natural-language description of a domain and writes its world model in PDDL, the planning domain definition language.
- 103 domains with gold-standard descriptions, as the paper counts them, scored with several execution-based metrics.
- On Hugging Face as EvolventAI/text2world, with code at Aaron617/text2world. Neither states a license as of Nov 19, 2026.
Best for: Testing whether a language model can write the rules of an environment.
WorldModelBench
WorldModelBench comes from UC Berkeley, UC San Diego, NVIDIA and MIT (WorldModelBench: Judging Video Generation Models As World Models).
How does WorldModelBench work?
- 350 prompts across 56 subdomains in seven domains, among them robotics, driving and gaming. Each prompt is a text description and a first frame.
- A judge model trained on human labels scores each generated video on instruction following, common sense and physical adherence. Answers for the test set are withheld.
- The test set and evaluation script are in WorldModelBench-Team/
WorldModelBench , which states no license as of Nov 19, 2026.
Best for: Checking whether a video model follows instructions and physical laws in applied domains.
WorldPrediction
WorldPrediction comes from Meta FAIR Paris, The Hong Kong University of Science and Technology and ISIR Sorbonne Université (WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning).
How does WorldPrediction work?
- Given an initial and a final state as video, a model picks the action that caused the change (WorldPrediction-WM) or the right ordered sequence of actions (WorldPrediction-PP) from distractors.
- 825 world modeling samples and 570 planning samples after human filtering, as the paper counts them.
- The annotations point to clips in five video datasets, such as COIN and EgoExo4D, which you download separately. Its README says WorldPrediction is CC BY-NC 4.0, a non-commercial license.
Best for: Testing whether a model can tell which action or plan leads from one state to another.
WorldScore
WorldScore comes from Stanford University (WorldScore: A Unified Evaluation Benchmark for World Generation).
How does WorldScore work?
- World generation is split into a sequence of next-scene tasks, each with a camera trajectory, so 3D and 4D scene generators and video models are scored the same way.
- 3,000 test examples, static and dynamic, indoor and outdoor, photorealistic and stylized, with metrics for controllability, quality and dynamics.
- Data on Hugging Face as Howieeeee/WorldScore, whose card lists the license mit, and code at haoyi-duan/WorldScore under the MIT License.
Best for: Comparing 3D, 4D and video world generators on one scale.
Where Patronus fits
Research
Patronus AI is a behavioral research lab for AI, and its 20 papers as of Nov 10, 2026 include work at NeurIPS, ICML and ACL. Its GlobeBench tests text world models, and it released the World Model Corpus with its paper Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL.
Digital World Models
With its Series B announcement on Jun 25, 2026, Patronus AI previewed its first Digital World Model. It describes Digital World Models as language diffusion world models that predict how environments behave and steer agent actions across digital workflows.
RL environments
Patronus AI sells ready-made RL environments and builds custom ones. Teams can also buy parts on their own, such as simulated apps, mock websites and the data behind them.
Security
Patronus AI lists SOC 2, HIPAA and TISAX, and ownership of the training data from a custom build is set in each agreement.
Patronus AI builds RL environments for training AI agents and is building Digital World Models. Talk to Patronus AI about simulated environments for your agents.
Frequently asked questions
What are the best tools for AI agent world models?
This page lists public benchmarks for testing world models. For agents that work in software, a text world model simulates tools and apps, and GlobeBench tests how well a model does that. Patronus AI is building Digital World Models, language diffusion world models it announced on Jun 25, 2026. Our guide to AI world models explains what they are and how they work.
Which startups are building simulated worlds to train AI agents?
Patronus AI is one. It builds RL environments for AI agents, and on Jun 25, 2026 it previewed its first Digital World Model, for agents in digital workflows. Our guide to RL environment startups covers others, and this page lists public benchmarks for testing world models.
How do you evaluate a world model?
Give it a state and an action and compare its prediction of what happens next with what the environment actually does. GlobeBench and ByteSized32-State-Prediction do this for text environments, and Physics-IQ for real physical experiments. AutumnBench and HalluWorld ask questions about the world instead, and WorldModelBench uses a judge model trained on human labels.
What is GlobeBench?
GlobeBench is Patronus AI’s benchmark for text-based world models, 1,632 tasks in five domains that test whether a model can predict how an environment responds to an agent’s next action. It is on Hugging Face, with code on GitHub, under Apache-2.0 for the items Patronus AI wrote. The paper is GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations.
Which benchmarks test text-based world models?
GlobeBench and ByteSized32-State-Prediction test whether a language model can predict the next state of a text environment. HalluWorld tests claims about grid worlds, chess and terminal sessions, and Text2World asks for an environment’s rules in PDDL. The World Model Corpus has held-out test splits as well as training data.
Which benchmarks test video world models?
Physics-IQ compares generated video with filmed physical experiments. WorldModelBench scores instruction following, common sense and physics, and EWMBench scores robot manipulation videos. WorldScore covers 3D, 4D and video world generation, and WorldPrediction tests high-level actions and plans in video.
Can I use these benchmarks commercially?
Check each license. As of Nov 19, 2026, the World Model Corpus and WorldScore are MIT, and HalluWorld and AutumnBench’s public data are CC BY 4.0. GlobeBench is Apache-2.0 for the items Patronus AI wrote, while adapted items keep their source license. Physics-IQ is Apache 2.0 for code and CC BY 4.0 for other materials. EWMBench and WorldPrediction are non-commercial, and ByteSized32-State-Prediction, Text2World and WorldModelBench state no license.
How was this list made?
Patronus AI wrote it and created GlobeBench and the World Model Corpus, two of the entries. A Patronus AI researcher coauthored HalluWorld. Entries are not ranked, and each entry links its paper and data.
Snapshot as of