Abstract
Multi-camera bird’s-eye-view (BEV) perception models are commonly trained and evaluated on a fixed camera rig. However, real deployment may involve camera-mounting changes on the same vehicle or transfer to different vehicle platforms. Existing evaluations rely on proxies rather than reproducing such camera-geometry changes. Extrinsic-only perturbation leaves the input images unchanged, whereas cross-domain generalization entangles camera geometry with domain shift. We introduce RoboGeo, a CARLA-based benchmark for camera-geometry robustness in multi-camera BEV perception. RoboGeo re-renders the same scenes under changed camera geometry, producing geometrically consistent image–extrinsic pairs. We define three diagnostic conditions that separate extrinsic-only perturbation, viewpoint change with base extrinsics, and viewpoint change with the corresponding extrinsics. RoboGeo provides two protocols, viewpoint perturbation and cross-platform transfer to the SUV and Bus layouts. Robustness under each is quantified by the mean Viewpoint Robustness Score (mVRS) and the Cross-Platform Transfer Score (CTS), respectively. We benchmark representative BEV segmentation and 3D object detection models, providing insights into the relationship between view transformation and camera-geometry robustness.
Viewpoint Perturbation
This protocol rotates cameras of the base rig about roll, pitch, or yaw to model camera-rig changes such as vibration or mount shift. Drag the video — horizontally for yaw, vertically for pitch — or use the dials (one axis at a time, ±20° in 4° steps; front camera shown). All pose videos share a single timeline: the scene never stops, only the camera geometry changes.
Cross-Platform Transfer
This protocol deploys a Sedan-trained model on vehicles whose cameras are mounted at different heights and orientations. The same scene is captured from the camera layouts of three platforms — switch the platform while the scene plays (front camera shown): the timeline continues, only the camera layout changes.
Leaderboard
Normalized robustness scores in percent (higher is better): mVRS under viewpoint perturbation and CTS under cross-platform transfer. Shaded columns mark the primary condition of each protocol (IMG / CAL); EXT is reported for analysis only. Click a column header to sort.
PNORMAL is each model’s unperturbed performance and Poracle the performance of a model trained and evaluated on that platform (CDS for detection, IoU for segmentation). Bold / underlined: best / second-best per shaded (primary) column and PNORMAL. † trained with explicit depth supervision. ‡ explicitly designed to be robust to extrinsic viewpoint changes.
Want your model on the leaderboard? Open an issue with your results on the RoboGeo evaluation split.
Key Findings
Extrinsic-only perturbation is a poor proxy
Perturbing only the extrinsics (EXT) leaves the input images unchanged and underestimates the degradation from a real viewpoint change. Under changed images (IMG), every benchmarked model degrades — even the most robust model in each task loses more than 13% of its unperturbed performance.
View transformation drives robustness
Robustness varies systematically with how a model maps image features into BEV space. Backward-projection models transfer most robustly across platforms, recovering up to 76.9% (detection) and 67.5% (segmentation) of the oracle on the SUV.
Cross-platform transfer is far from solved
The Bus differs from the Sedan the most in camera height, spacing, and orientation. Even with the correct target extrinsics supplied (CAL), the best models reach only 48.1% (detection) and 36.8% (segmentation) of the oracle, with most entries below half.
Existing methods narrow but do not close the gap
Extrinsic augmentation changes the calibration but never the images: it adds little robustness and can lower clean performance. Cross-domain methods help more — PD-BEV lifts BEVDepth’s Bus transfer from 16.6% to 41.8% CTS — yet no method closes the gap on either protocol. RoboGeo remains far from saturated.
Dataset
Every camera geometry is rendered over the same underlying scenes, so images and extrinsics stay geometrically consistent across the whole benchmark. Annotations follow the nuScenes schema, extended with object attributes and a four-level visibility computed per platform.
Scenes & frames
- 268 driving scenes — 220 train / 48 held-out eval
- Rendered at 5 Hz for all three platforms
- 17,380 train + 3,792 eval frames per platform
- 6 surround cameras per frame, fixed intrinsics
Camera geometries
- 212 changed geometries beyond the base rig
- Viewpoint perturbation: 210 — roll / pitch / yaw at ±4°, ±8°, ±12°, ±16°, ±20°
- Per-camera (6 × 30) and all-camera (30) modes
- Cross-platform: 2 — SUV and Bus layouts
Structure
RoboGeo/
├── viewpoint_perturbation/ # rotated rigs, one tar per camera and angle
│ ├── RGB-CAM_FRONT_yaw8pitch0roll0.tar # 6 cameras x 31 settings = 186 tars
│ ├── RGB-CAM_BACK_LEFT_yaw0pitch0roll-20.tar
│ └── viewpoint_metadata.json
├── platform_layouts/ # one tar per camera and platform
│ ├── RGB-CAM_FRONT.tar # Sedan (base rig, no prefix)
│ ├── RGB-suv-CAM_FRONT.tar # SUV
│ └── RGB-bus-CAM_FRONT.tar # Bus
└── metadata/ # nuScenes-schema annotations,
└── carla_geobev_meta.tar.zst # extrinsics, attributes, visibility
Perturbation tars are named RGB-<CAMERA>_yaw<Y>pitch<P>roll<R>.tar
with angles in degrees, so RGB-CAM_FRONT_yaw8pitch0roll0.tar is the front camera
rotated by yaw +8°. Cameras use the nuScenes names
(CAM_FRONT, CAM_FRONT_LEFT, CAM_FRONT_RIGHT,
CAM_BACK, CAM_BACK_LEFT, CAM_BACK_RIGHT), and each has
31 settings: 30 single-axis rotations plus the unperturbed
yaw0pitch0roll0. Because renders are stored per camera, both the
per-camera and the all-camera mode are assembled from the same tars: rotate one camera and
read the other five unperturbed, or take the same angle for all six. The Sedan base rig in
platform_layouts/ provides the NORMAL condition and the training data for
both protocols.
Getting started
The dataset is gated — request access on the dataset page first, then authenticate with
huggingface_hub. The full release is large, so download only the geometries you
need with allow_patterns.
from huggingface_hub import snapshot_download
# One perturbation setting: front camera, yaw +8 degrees
snapshot_download(
"shdragron/RoboGeo",
repo_type="dataset",
allow_patterns=[
"viewpoint_perturbation/RGB-CAM_FRONT_yaw8pitch0roll0.tar",
"viewpoint_perturbation/viewpoint_metadata.json",
"metadata/*",
],
local_dir="RoboGeo",
)
# Or a whole platform layout: every camera of the Bus
snapshot_download(
"shdragron/RoboGeo",
repo_type="dataset",
allow_patterns=["platform_layouts/RGB-bus-*.tar", "metadata/*"],
local_dir="RoboGeo",
)Acknowledgements & License
RoboGeo is built on SimBEV (Mehr and Eskandarian, 2025), a synthetic multi-task multi-sensor driving data generation tool based on the CARLA simulator. We sincerely thank Dr. Goodarz Mehr for making SimBEV available and for supporting its use in this benchmark, and the CARLA team for the open-source simulator that made this work possible.
The RoboGeo dataset is released under the CC BY-NC-SA 4.0 license (Attribution–NonCommercial–ShareAlike).
BibTeX
@inproceedings{robogeo2027,
title = {RoboGeo: Benchmarking Camera-Geometry Robustness in Multi-Camera BEV Perception},
author = {Jeongtae Kim and Giyong Moon and Chanyoung Lee and Soonmin Hwang},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
year = {2027}
}