Seeing the 4th Dimension: D4RT and CVPR 2026's Best Paper, Explored in FiftyOne
Jun 11, 2026
Article content
In this article
D4RT (Dynamic 4D Reconstruction and Tracking) won best paper
D4RT (Dynamic 4D Reconstruction and Tracking) won best paper at the most competitive venue in computer vision by replacing an entire pipeline of specialized models with a single elegant query interface. D4RT, is a 4D scene reconstruction model from Google DeepMind, University College London, and the University of Oxford that recovers the geometry and motion of a dynamic scene from video using a single query interface, and it won the best paper award at CVPR 2026.
We built a notebook using the open source FiftyOne toolkit to make that idea tangible — loading real street footage, layering simulated depth heatmaps and point tracks directly onto video frames, and making the paper's core claim that dynamic objects are no longer a special case, something you can watch rather than just read. This post explains the paper, what makes it groundbreaking, and what the notebook built using FiftyOne demonstrates that a static paper figure cannot.
Key takeaways
- D4RT is a unified, efficient model that won the best paper award at CVPR 2026.
- It replaces the traditional multi-model 4D reconstruction pipeline with a single query interface, answering depth, point-tracking, and camera-pose queries from the same latent representation.
- D4RT treats dynamic objects the same way as static ones and sets a new state of the art across 4D reconstruction benchmarks.
Congratulations to the D4RT Team
At CVPR 2026, over 16,000 submitted papers were reviewed, with the top honors going to "Efficiently Reconstructing Dynamic Scenes One D4RT at a Time" by a team from Google DeepMind, University College London, and the University of Oxford.
What is D4RT and how does it work?
4D scene reconstruction has been traditionally treated as a pipeline problem. D4RT replaces this whole pipeline with a single question, encoding a video into a compact representation. It allows users to query this representation, returning various outputs without separate fusion steps. This efficient method treats dynamic and static objects uniformly.
Why D4RT won CVPR 2026's best paper
D4RT's conceptual contribution lies in its refusal to treat dynamic objects as exceptions, unlike previous methods. It also features a unified query interface that enables efficient computation and geometric consistency.
Exploring D4RT outputs in FiftyOne
Before describing the purpose of the notebook, it is essential to clarify its functionality. The notebook is a reading companion to visualize D4RT’s capabilities but does not perform inference itself.
The transition from simulations to real inference is minimal once model weights are public, making the notebook a valuable reproduction tool.
| D4RT Concept | FiftyOne Representation | Key Insight |
|---|---|---|
| Unified query interface | Single dataset, multiple label types per frame | One model -> depth + tracks + cameras |
| Feedforward depth decoding | fo.Heatmap per frame | Objects form near-depth spike |
| Dynamic point tracking | fo.Keypoints grounded in COCO detections | 10x more displacement than static |
| Static/dynamic separation | fo.Segmentation motion mask | Emerges from depth, no seg model needed |
| Method comparison | Per-clip AJ/AbsRel fields + scatter plot | Gap grows with dynamic content |
| Camera recovery | Per-frame rotation/translation + 3D plots | No test-time optimisation required |
This table outlines the core capabilities of D4RT and their representation in the FiftyOne companion notebook.
Next steps
When D4RT weights are released, the notebook becomes a genuine reproduction tool. Users can download PointOdyssey or Sintel to interactively validate D4RT’s results on real datasets.
FAQ
What is D4RT?
D4RT is a 4D scene reconstruction model that won the best paper award at CVPR 2026.
How does D4RT work?
D4RT encodes video into a compact representation and answers queries with a pixel location, timestamps, and camera frame to return a 3D position.
Are the D4RT model weights available?
Not yet, but the notebook visualizes simulated outputs until real inference is possible.