Florent Delgrange won the Best Blue Sky Paper Award at AAMAS 2026 for his work Foundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments. We caught up with him to find out more about his vision for agent learning.
What is the topic of your Blue Sky Ideas paper and why is it an interesting area for study?
My Blue Sky Ideas paper asks a simple but difficult question: how can an autonomous agent keep learning as its world changes without quietly losing the guarantees that made its behavior trustworthy?
Reinforcement learning and formal methods address complementary parts of this problem. Reinforcement learning allows an agent to learn by trial and error and can scale to environments for which we could never write down every rule. However, the agent is usually asked to maximize a reward. A poorly specified reward can be exploited, and a high reward does not by itself tell us that a safety or coordination requirement has been satisfied. Reactive synthesis starts from the other end: given a model of the environment and a logical description of the intended behavior, it can construct a policy that is correct by design. The difficulty is that it normally needs an explicit, fixed model, which is precisely what an agent lacks in an open and changing world.
The paper proposes a research agenda that brings these traditions into one loop. A foundation world model would be learned from experience, but structured so that a verifier can reason about it. As the agent learns a policy, it would update its model, measure the reliability of its abstraction, and check whether the policy still satisfies its specification. The verifier’s feedback could reject an unsafe update, request data from an uncertain region, or trigger a revision of the model.
This matters because real environments do not politely remain as they were during training. Goals evolve, conditions change, and, in a multi-agent system, every adapting agent changes the environment perceived by the others. Reliability therefore cannot be a certificate obtained once at deployment. It has to be maintained as the agent continues to learn.
What is your vision for foundation world models?
To me, foundation refers first to reuse, not simply to size. I do not envision a larger video predictor trained on more trajectories. I envision a persistent and structured model that an agent can carry across tasks, policies, and changing populations of agents.
One way to think about it is as a map that records more than roads. It should also tell the agent which areas have been surveyed, which conclusions depend on those areas, and when a change in the world has made an old route unreliable. A useful foundation world model should do the same for decision-making: predict what may happen, expose the structure needed for reasoning, and quantify when those predictions are trustworthy enough to support a guarantee.
I’d want such a model to have three key properties. First, it should be calibrated: every learned abstraction should come with a measure of its error or coverage, so that the agent knows where formal conclusions remain valid. Second, it should be compositional: verified local dynamics, behaviors, and certificates should be reusable when a new task is assembled. Third, it should be semantically queryable: a formal requirement or high-level instruction should help the agent derive a suitable reward model, task-specific abstraction, or policy prior with little additional experience.
The durable idea is that the world model becomes a common substrate for learning, planning, and verification. It should help an agent act, but also identify the limits of its competence, gather evidence where those limits matter, and explain why a particular behavior can or cannot currently be certified.
How does this vision differ from current models?
Most current model-based approaches focus on learning for a particular task, environment, policy, or training distribution. Their main model learning objective is predictive accuracy: reconstruct an observation, forecast the next (latent) state, or generate imagined trajectories for planning. Those are important capabilities, but average predictive metric is not the same as fitness for a particular guarantee.
Consider a model for a scenario involving an agent interacting in a warehouse, where the world model predicts almost every transition correctly but misses a rare, dangerous interaction with a forklift. Its average error may be excellent while that transition is exactly the one that determines whether a collision-avoidance claim is valid. The reverse is also possible: a compact model may ignore colors and textures yet preserve everything needed to reason about routes and collisions. For verification, the relevant question is therefore not only “How accurate is the model?” but “Which conclusions does this accuracy justify, for this policy and this requirement?”
Foundation world models would make that connection an explicit design objective. Their abstractions would carry reliability information tied to the behavior being analyzed, and this information would be revised online as the data distribution changes. Previously verified components could be reused and composed, while specifications expressed in logic or language could guide which representation and policy the agent needs for a new task.
The main difference is therefore a change in role. Current models are primarily prediction tools. The model I envisage is a persistent, analyzable basis for learning, adaptation, and formal reasoning, with language models providing a complementary semantic interface. They could translate high-level instructions into candidate specifications or propose structured model updates, while the world model grounds these proposals in experience and the verifier checks their validity. Foundation world models may benefit from scale through broader task and environment coverage, but scale alone does not provide the structure or certificates required for reliability.
Could you give an example of how such a model might work?
Consider a delivery robot in a busy warehouse. Its task could be stated as: “eventually deliver the package while always avoiding collisions.” Instead of manually combining many bonuses and penalties, the system would translate that requirement into a reward model. Learning and verification would then start from the same description of the intended behavior.
As the robot receives observations, it would learn a compact representation of the warehouse and a policy that acts on that representation. A verifier would ask two related questions. Does the policy satisfy the delivery and collision-avoidance requirement in the learned world model? And is the learned abstraction accurate enough, along the routes that matter, for that conclusion to be trusted? A certificate is meaningful only when both answers are supported.
Now suppose a new forklift begins using a shortcut that was rarely visited during training, cluttering the way. Rather than treating an old prediction as a guarantee, the model should lower its confidence in that region. The verifier would detect this, withdraw the affected certificate, reject a risky policy update, direct exploration toward the shortcut, and reinstate a guarantee only after the world model has been recalibrated.
The paper also considers a more ambitious test-time loop. A language model could propose one or more small formal program describing the new dynamics. A model checker would test them, learn to compose with them, and return a counterexample or structural inconsistency when they are wrong. The language model could revise its hypothesis, the robot could collect targeted experience, and the cycle would repeat. In this division of labor, the language model proposes and the formal verifier checks.
We already have pieces of the loop, including formal reward translations, verifiable abstractions, safe policy-improvement methods, and program generation. Building an efficient end-to-end agent that keeps all of these pieces calibrated while it learns remains the research challenge.
What do you think the impact of such a framework on multi-agent systems could be?
Multi-agent systems make the problem both more urgent and more difficult. Each learning agent is a moving part of every other agent’s environment. Even when the physical world is unchanged, the effective dynamics evolve as agents update their policies, join or leave the system, share information, or pursue new objectives.
A foundation world model could retain reusable descriptions of these interaction patterns, while formal specifications state what must hold for the group. A fleet of warehouse robots, for example, may need to avoid collisions and complete deliveries while respecting shared capacity constraints. The model could connect each local policy to the assumptions on which its certificate depends. If one robot changes its route, the system could identify which assumptions and guarantees are affected, collect new data where needed, and revise only the relevant components rather than relearning and re-verifying the entire fleet.
Composition is the main source of potential leverage. Verified local dynamics or coordination behaviors could serve as building blocks for new teams and tasks, and previous agents could provide useful priors for new participants. This could support faster adaptation while making failures easier to diagnose: the system should report which interaction invalidated a certificate and provide a counterexample, rather than only revealing that the joint reward has fallen.
The central obstacle for multi-agent systems is scalability. The joint state space grows very quickly with the number of agents, and local guarantees do not automatically compose into a global one. We will need principled ways to expose dependencies, preserve soundness under composition, and run verification quickly enough to influence learning online. If we can solve those problems, the field could move from learning coordination strategies and checking them afterwards to learning new strategies while continuously tracking which global properties remain guaranteed.
About Florent
Florent Delgrange is a postdoctoral researcher in computer science at the Artificial Intelligence Lab of Vrije Universiteit Brussel (VUB). His research lies at the intersection of reinforcement learning, world models, and formal verification. He develops methods for agents that can learn and adapt while justifying and certifying the behavior they adopt. He completed a joint PhD at VUB and the University of Antwerp in 2024 on the formal verification of deep reinforcement learning policies. His paper Foundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments received the Best Blue Sky Paper Award at AAMAS 2026.