Retriever: Taming the Multi-Speed Beast of Robotic Control

Robots think slow and move fast. This fundamental mismatch has long been the bane of roboticists, forcing them to cobble together systems where millisecond-level control loops coexist uneasily with planning algorithms that ponder for seconds. Retriever, a new programming framework, argues that we've been doing it wrong all along. Instead of hiding timing in the dark corners of callback functions and middleware, it drags it into the spotlight.

At the heart of Retriever is a deceptively simple idea: every part of a robot's 'brain' should have its own explicit clock. This isn't just an implementation detail, it's the core abstraction. Retriever introduces 'Flow', a component that's essentially a stateful function with a schedule attached.

planner = VLMPlanner().attach(clock=Rate(hz=0.2))  # Thinks every 5 seconds
skill_policy = VLASkillPolicy().attach(clock=Rate(hz=2))  # Acts twice a second
controller = JointController().attach(clock=Rate(hz=200))  # Twitches 200 times a second

pipeline = (
    planner.then(skill_policy, sync=Latest())
    .then(controller, sync=Latest())
)

This isn't just syntactic sugar. By making timing explicit, Retriever forces developers to confront the reality of how information flows through a robotic system. No more pretending that everything happens in perfect lockstep or that there's some magical global 'tick' that synchronizes the world.

To prove this isn't just theoretical navel-gazing, the Retriever team built Retriever-0, a two-armed robot that can rummage through drawers looking for spices. It doesn't sound impressive until you realize what's happening under the hood: a language model is pondering its next move every few seconds, while a control loop keeps the robot's joints from going haywire 200 times a second. All in one coherent program.

Retriever takes aim at three perennial headaches in robotics:

1. Feedback isn't an afterthought: In the real world, robots need to constantly update their understanding and plans. Retriever makes these feedback loops explicit in the code structure.

2. Not all thinking happens at the same speed: From glacial planning to lightning-fast control, Retriever embraces the multi-speed nature of robotic cognition.

3. Timing is everything: Instead of pretending timing doesn't matter until it breaks something, Retriever puts it front and center.

The holy grail here is determinism. Given the same inputs and starting state, a Retriever program should always do the same thing. This might sound obvious, but anyone who's debugged a complex robot system knows it's anything but. The promise of reproducible behavior could be a major shift for testing and debugging.

But let's not get carried away. Retriever is still in its infancy. It needs to prove itself on diverse real-world robots tackling messy, unpredictable tasks. There's also the not-insignificant hurdle of retraining a generation of roboticists to think explicitly about timing in their code.

Retriever isn't a silver bullet. It won't magically make robot programming easy. What it does do is force developers to grapple with a fundamental truth: in robotics, when something happens is often just as important as what happens. By dragging timing out of the shadows and into the core of the programming model, Retriever might just help us build robots that can think and act in harmony, across all timescales.