"We measured how well 15 AI models can program a cheap ($129) off-the-shelf drone to find and follow a person in our office. Today, no model completes this task. Fable 5 is the best model, coming within 84% of our baseline, followed by Opus 4.8 and GPT 5.6 Sol."
Drone-Bench's task is based on Project Pilot, our recent work with Anthropic exploring AI's impact on the physical world. In Project Pilot, a drone autonomously navigates our office to find and follow a targeted human.
No lab has access to Drone-Bench. This demo spans five capabilities, each reproduced in simulation as its own benchmark task. The baseline is our human+AI code used for the demo. A model that surpasses it on all tasks can thus autonomously recreate a demo at least as capable as ours. Task 1, Reconstruct: Turn videos of the office into a 3D model, find each frame’s position in that model, and provide a function that slices the model into a 2D obstacle map. Task 2, Localize: Locate the drone by matching a frame from its camera against the office videos, using each video frame’s known position from Reconstruct (task 1) to estimate the drone’s own position. Task 3, Navigate: Plan a path between rooms on the obstacle map and fly it, continuously calling the solution from Localize (task 2) during flight to track the drone’s position and correct for noisy controls. — Andon Labs