LeRobot Teaches Robots to Imagine

Teaching a robot to pick up a red cube sounds simple until you realize the robot has no idea whether it succeeded, no sense of what happens next, and no easy way to learn from its own mistakes. Hugging Face's LeRobot project, an open toolkit for robot learning, just shipped version 0.6.0 with a clear theme: close that loop. The update focuses on robots that imagine, evaluate, and improve.

Policies that imagine the future

The headline additions are three "world model" policies. A world model is software that learns to predict what a scene will look like a moment from now, so the robot can, in effect, rehearse before it moves.

The clever part is cost control. VLA-JEPA trains a small model to anticipate upcoming frames, then throws that machinery away at inference time, so you get the benefit of imagination without paying for it when the robot actually runs. LingBot-VA predicts future video and actions together and can even save what it imagined so you can compare it with reality; its inference fits on a single 24 to 32 GB GPU. FastWAM pairs a video-generation expert with a compact action expert, learns to "dream" its own rollouts during training, and then skips the dreaming entirely when it acts.

A growing zoo of action models

LeRobot also expanded its lineup of VLAs, short for vision-language-action models, which turn what a camera sees plus a plain-English instruction into robot movement. NVIDIA's GR00T integration jumps to the newest N1.7 generation, tested against NVIDIA's original code to produce identical outputs. The Allen Institute for AI's MolmoAct2 arrives ready to run zero-shot on hobbyist SO-100 arms in about 12 GB of memory. And EVO1 makes the case that VLAs need not be enormous, packing everything into 0.77 billion parameters that run in real time on modest hardware.

Knowing when the robot actually succeeds

Here is a gap most people never think about: robots are bad at grading their own homework. v0.6.0 adds a unified reward models API, where a reward model watches a task and scores progress and success. Robometer, built on Qwen3-VL-4B and trained on more than a million robot trajectories, scores any dataset from raw video and a language instruction with no task-specific tuning. TOPReward goes further and uses no special weights at all. It simply asks an off-the-shelf language model how likely the word "True" is given the video and the task. Any capable model becomes a success detector.

Turning failures into fuel

The new lerobot-rollout command handles deployment and supports DAgger-style corrections, meaning a human can step in when the robot fumbles and those corrections flow straight back into the training data. Combined with the reward models, this is the actual learning loop: act, judge, fix, repeat.

Supporting all this is a batch of practical upgrades. Datasets can now record depth from an Intel RealSense camera, store rich timestamped language annotations generated automatically by a language model, and use whatever video codec you prefer. Video training loads up to twice as fast, and pulling a subset of a large dataset dropped from around 275 seconds to a fraction of a second in Hugging Face's own benchmark. You can also train models bigger than a single GPU using a technique called FSDP, or offload the whole job to the cloud.

Measuring it all

To check whether any of this actually helps, v0.6.0 unifies six new simulation benchmarks under one command. They range from LIBERO-plus, which throws roughly 10,000 perturbed scenarios at a policy to find where it breaks, to RoboCasa365's 365 kitchen tasks across 2,500 generated kitchens, to memory and long-horizon reasoning exams. Each ships with a baseline checkpoint that is smoke-tested automatically.

What's next

The open question the release keeps returning to is whether imagining the future genuinely makes robots better, or just more expensive to train. By putting world models, reward models, and shared benchmarks in one place, LeRobot is less interested in declaring a winner than in making the experiment easy to run. That is the quieter kind of progress worth watching: not a single breakthrough, but the plumbing that lets everyone test their own.