The reconstruction pipeline, three to four times faster and more accurate.
Release 0.7.0rc1 of the proxy reconstruction pipeline against the previous version, measured on the same scenes with the same ground-truth boxes. A meeting room that took 29 minutes now takes 6 to 8, and the fraction of objects placed within IoU 0.5 of the ground truth rises from 0.18 to 0.71–0.77.
Headline numbers
Each row is one cold-start run: model loading included, nothing cached between runs. Recall is measured against ground-truth tight boxes of the visible, non-structural objects (walls, floors, ceilings, curtains, ceiling fixtures and small clutter are excluded from the targets); mIoU and the centre error are over matched objects. The office and the tabletop have no ground-truth boxes, so only their time is compared.
| scene | pipeline | wall clock | edit rounds | R@0.25 | R@0.5 | mIoU | centre | objects |
|---|---|---|---|---|---|---|---|---|
| Meeting room (Imaginarium official_04) | old pipeline | 1731 s | 2 | 0.88 | 0.18 | 0.39 | 17 cm | 29 |
| Meeting room (Imaginarium official_04) | release 0.7.0rc1 | 359 s | 1 | 0.94 | 0.77 | 0.60 | 10 cm | 29 |
| Bedroom (Imaginarium bedroom_01) | old pipeline | 1230 s | 2 | 0.77 | 0.15 | 0.39 | 22 cm | 17 |
| Bedroom (Imaginarium bedroom_01) | release 0.7.0rc1 | 354 s | 1 | 0.92 | 0.85 | 0.68 | 7 cm | 23 |
| Office (ScanNet++ 09bced689e) | old pipeline | 1630 s | 2 | — | — | — | — | 18 |
| Office (ScanNet++ 09bced689e) | release 0.7.0rc1 | 490 s | 1 | — | — | — | — | 26 |
| Tabletop, two stereo views (robot head camera) | release 0.7.0rc1 | 255 s | 1 | — | — | — | — | 7 |
| Meeting room (Imaginarium official_04) | release, 4RC backend | 597 s | 1 | 0.94 | 0.77 | 0.60 | 10 cm | 29 |
| Meeting room (Imaginarium official_04) | release, second run | 473 s | 1 | 0.94 | 0.71 | 0.60 | 13 cm | 29 |
Accuracy varies between runs of the same scene by about ±0.05 to ±0.1 in recall, which comes from the global build; the release runs listed for the meeting room are two independent samples plus a run with the 4RC geometry backend.
The scenes in 3D
Each run's exported scene over its own predicted point cloud: the old pipeline on 4RC geometry, the release on Pi3X. Drag to orbit, scroll to zoom, and pick an object from the list to highlight its box.
old pipeline
release
Where the time goes
The front end now runs the geometry model, the vocabulary planning call, the SAM3 segmentation and the Boxer detector at the same time, and the agent stage does one editing round with a numeric check before it instead of two blind rounds.
- Front end: 132 s → 67 s on the meeting room. The old version computed a SHA-256 of the 6 GB checkpoint, wrote the geometry archive compressed twice and ran geometry, detection and planning one after another. The release reads weights with parallel range requests straight onto the GPU, skips random initialisation, writes once, and starts the three model workers together.
- Editing: 981 s → 124 s. One round instead of two. Before each round the host projects every box into the instance masks of its own object and lists the few that disagree; the session may end as soon as those are handled. A second round measured on the same scene (run “two rounds (v2)”) changed nothing within the evaluation.
- Review: 344 s → 58 s. The reviewer assesses the flagged and visibly wrong objects; objects that pass the numeric check and draw no objection are recorded by the host with the numbers as the reason.
- Blender: 13.0 s → 2.1 s per code edit. One Blender process serves every edit and export of a run instead of starting for each; a job that fails there is repeated in a fresh process, so the result is the same.
Accuracy stage by stage
The same ground truth, scored after each stage of the run: the proposal seeds, the global build, each review round and the exported scene.
Meeting room
| run | stage | boxes | R@0.25 | R@0.5 | mIoU | centre |
|---|---|---|---|---|---|---|
| old pipeline | seeds | 9 | 0.29 | 0.12 | 0.44 | 23 cm |
| build | 29 | 0.23 | 0.00 | 0.28 | 37 cm | |
| round 1 | 29 | 0.82 | 0.18 | 0.41 | 20 cm | |
| round 2 | 29 | 0.88 | 0.18 | 0.39 | 17 cm | |
| final | 29 | 0.88 | 0.18 | 0.39 | 17 cm | |
| release | seeds | 41 | 0.94 | 0.71 | 0.55 | 11 cm |
| build | 29 | 0.94 | 0.77 | 0.60 | 10 cm | |
| round 1 | 29 | 0.94 | 0.77 | 0.60 | 10 cm | |
| final | 29 | 0.94 | 0.77 | 0.60 | 10 cm | |
| release, second sample | seeds | 34 | 0.88 | 0.53 | 0.55 | 17 cm |
| build | 29 | 0.94 | 0.71 | 0.60 | 13 cm | |
| round 1 | 29 | 0.94 | 0.71 | 0.60 | 13 cm | |
| final | 29 | 0.94 | 0.71 | 0.60 | 13 cm | |
| release, 4RC backend | seeds | 41 | 0.88 | 0.35 | 0.46 | 21 cm |
| build | 29 | 0.94 | 0.71 | 0.58 | 11 cm | |
| round 1 | 29 | 0.94 | 0.77 | 0.60 | 10 cm | |
| final | 29 | 0.94 | 0.77 | 0.60 | 10 cm | |
| release, two rounds (v2) | seeds | 34 | 0.94 | 0.59 | 0.57 | 15 cm |
| build | 29 | 0.94 | 0.71 | 0.59 | 13 cm | |
| round 1 | 29 | 0.94 | 0.71 | 0.59 | 13 cm | |
| round 2 | 29 | 0.94 | 0.71 | 0.59 | 13 cm | |
| final | 29 | 0.94 | 0.71 | 0.59 | 13 cm |
Bedroom
| run | stage | boxes | R@0.25 | R@0.5 | mIoU | centre |
|---|---|---|---|---|---|---|
| old pipeline | seeds | 4 | 0.23 | 0.15 | 0.56 | 17 cm |
| build | 17 | 0.23 | 0.15 | 0.34 | 30 cm | |
| round 1 | 17 | 0.39 | 0.15 | 0.38 | 27 cm | |
| round 2 | 17 | 0.77 | 0.15 | 0.39 | 22 cm | |
| final | 17 | 0.77 | 0.15 | 0.39 | 22 cm | |
| release | seeds | 24 | 0.92 | 0.85 | 0.68 | 7 cm |
| build | 23 | 0.92 | 0.77 | 0.65 | 8 cm | |
| round 1 | 23 | 0.92 | 0.85 | 0.68 | 7 cm | |
| final | 23 | 0.92 | 0.85 | 0.68 | 7 cm |
The old pipeline started from 4RC geometry whose scale was 2.5 to 5 times off, so its lifted detections were already wrong and two editing rounds spent most of their time repairing them. The release starts from near-metric Pi3X geometry and from proposals that are already close, and editing only touches a few objects.
Proposal sources
Three ways of proposing object boxes on the same Pi3X point cloud, before any agent is involved. Boxer lifts 2D detections with its own depth head; SpatialLM reads the point cloud and writes a layout; the SAM3 route segments every frame with the planned vocabulary, lifts the masks through the predicted depth, links them across views by surface visibility and fits an upright box to each instance. The release merges the Boxer boxes with the mask instances they do not cover.
Meeting room (Imaginarium official_04)
| proposal source | boxes | R@0.25 | P@0.25 | R@0.5 | mIoU |
|---|---|---|---|---|---|
| Boxer (lifted 2D detections) | 15 | 0.71 | 1.00 | 0.65 | 0.63 |
| SpatialLM 1.1 (0.5B) | 6 | 0.00 | 0.00 | 0.00 | 0.20 |
| SAM3 mask fusion | 33 | 0.82 | 0.58 | 0.23 | 0.45 |
| Merged (release default) | 34 | 0.94 | 0.64 | 0.65 | 0.55 |
Bedroom (Imaginarium bedroom_01)
| proposal source | boxes | R@0.25 | P@0.25 | R@0.5 | mIoU |
|---|---|---|---|---|---|
| Boxer (lifted 2D detections) | 16 | 0.92 | 1.00 | 0.85 | 0.66 |
| SpatialLM 1.1 (0.5B) | 13 | 0.23 | 0.27 | 0.15 | 0.45 |
| SAM3 mask fusion | 25 | 1.00 | 0.77 | 0.69 | 0.60 |
| Merged (release default) | 25 | 1.00 | 0.77 | 0.92 | 0.68 |
SpatialLM ran 56 s (official_04) and 78 s (bedroom_01) on the same point cloud, measured 2026-11-05; its output is dominated by wall and room boxes that the box definition excludes.
Geometry backends
Pi3X against 4RC on six synthetic scenes with ground-truth cameras and depth. Points are lifted from the predicted depth, aligned to the ground truth by a trimmed similarity, and the median point error and the recovered scale are reported. Pi3X is near metric, which is what lets the lifted detections and the merge rule work without a camera-height assumption; 4RC remains available as a backend, where the release uses the scale-free mask fusion alone.
| scene | Pi3X point error (median) | Pi3X scale | 4RC point error (median) | 4RC scale |
|---|---|---|---|---|
| bathroom_05 | 2.8 cm | 1.00 | 6.8 cm | 2.52 |
| bedroom_01 | 4.3 cm | 1.02 | 19.7 cm | 3.89 |
| diningroom_08 | 1.9 cm | 1.07 | 4.5 cm | 2.64 |
| kitchen_03 | 4.2 cm | 0.99 | 7.8 cm | 4.99 |
| livingroom_10 | 4.5 cm | 1.15 | 16.7 cm | 4.12 |
| official_04 | 7.3 cm | 1.16 | 8.1 cm | 3.42 |
Point-cloud fitting, where it helps and where it does not
The release adds a 3D term to the numeric check (the share of an object's own lifted points that its box holds) and a fit_to_points tool that refits a box to those points. Measured against ground truth, refitting every box lowers accuracy: visible-surface points do not bound the unseen side of a body, and depth bleeds at mask borders, so a tight fit shrinks a table to a sliver and a containing fit swells a laptop to several times its size. The tool therefore acts only on a box that fails the check, and keeps the fit only when it agrees clearly better without a large volume change. In the runs above it was applied to 6 objects and kept 2: area_rug (20% → 44% of points inside), dark_laptop (19% → 99% of points inside).
Final scenes, old and new
Instance overlays rendered from each run's exported scene on the same input frames.
Meeting room (Imaginarium official_04)
old pipeline · frame 0
release · frame 0
old pipeline · frame 24
release · frame 24
old pipeline · frame 47
release · frame 47
Office (ScanNet++ 09bced689e)
old pipeline · frame 2
release · frame 2
old pipeline · frame 26
release · frame 26
old pipeline · frame 40
release · frame 40
Bedroom (Imaginarium bedroom_01)
old pipeline · frame 0
release · frame 0
old pipeline · frame 25
release · frame 25
old pipeline · frame 47
release · frame 47
Tabletop, two stereo views (robot head camera)
old pipeline · frame 0
release · frame 0
old pipeline · frame 1
release · frame 1
Notes
- Runs use the same agent model and budgets (300 s and 30 actions per editing session, 6 review actions); the old pipeline ran its default two sessions of 480 s.
- Scale for the synthetic scenes is the geometry model's own; the ground-truth alignment is a similarity, so a global scale error does not count against the boxes.
- The meeting room and bedroom frames are Imaginarium renders; the office is a ScanNet++ capture; the tabletop is a single stereo pair from a robot head camera.
- Source numbers:
experiments/recon_release_20261105/results.json, recomputed from the run folders.