R&D advanced 3 min read Apr 29, 2026 · Updated May 1, 2026
Public Preview Sign in free for the full digest →

CARLA plus AirSim in one simulation

“They merged CARLA and AirSim by changing 35 lines of upstream code — and got a sub-millisecond physics tick where bridge-based co-sim costs 1–5 ms per frame.”

CARLA plus AirSim in one simulation
2 Views
0 Likes
0 Bookmarks
Source · huggingface.co

You know that feeling when you want to train a drone to land on a moving car, but the car simulator and the drone simulator are two different programs talking over a network, and your sensor timestamps are always slightly off? Every paired aerial-ground frame you collect carries an interpolation error, and the synchronization tax grows with every extra sensor you bolt on. Bridge-based co-simulation between CARLA and AirSim adds 1,000–5,000 µs of cross-process sync per frame, and weather/lighting drift between the two renderers corrupts cross-view perception data.

simulationroboticsdronescarlaairsimembodied-airesearch

Unreal Engine 4 enforces one rule the authors had to dance around: each world can have exactly one active GameMode, and CARLA's GameMode is welded to its traffic and weather subsystems. AirSim's flight controller, fortunately, is just a regular Actor — not a GameMode. So the authors write a new class CARLAAirGameMode that inherits CARLA's GameMode (keeping all its ground machinery), then spawns AirSim's flight actor at BeginPlay as a normal world entity. Two RPC servers run side-by-side in the same process — CARLA on TCP 2000, AirSim on TCP 41451 — so existing CARLA and AirSim Python clients connect without modification. Every sensor read happens on the same physics tick, so all 18 sensor streams share one timestamp.

01
Single-process integration — you avoid the 1,000–5,000 µs/frame penalty bridge-based stacks pay for cross-process state sync
02
Zero-code-migration APIs — your existing CARLA and AirSim Python scripts run unchanged because both native RPC servers stay live
03
18 synchronized sensor modalities per tick — RGB, depth, semantic seg, LiDAR, radar, IMU, GNSS, barometry across air and ground, no interpolation needed
04
3-hour stability run with 0 crashes and 0 API errors over 357 actor spawn/destroy cycles — verified for RL reset patterns
05
MIT-licensed open source with prebuilt v0.1.7 binaries (~19 GB) and source (~651 MB) — drop-in for academic teams
06
63 ROS 2 topics published out of the box (43 CARLA + 14 AirSim + 6 generic) — usable from existing ROS 2 stacks
07
All 14 official CARLA weather presets verified to propagate consistently across both sensor layers in W4 cross-view perception
Who it’s for

If you are a robotics or embodied-AI researcher who needs paired aerial/ground sensor data — VLN/VLA dataset construction, cross-view perception, cooperative landing or escort policies — and you have been duct-taping CARLA and AirSim across processes, this gives you back your timestamps. Not useful yet if you need GPU-parallel multi-environment RL throughput (Isaac Lab / OmniDrones still win on sample efficiency), if you need >2 drones in one scene (functional but not validated), or if you can't tolerate ~20 FPS under joint load.

Worth exploring

Yes, if your work specifically needs spatially and temporally co-registered aerial-ground sensor data — there is no other open-source single-process option in this niche today. Treat it as beta: the authors validate exactly five workflows (precision landing, VLN data, 12-stream capture, cross-view perception, RL env), explicitly flag high-density traffic and >2-drone scaling as not-yet-validated (§6), and the engine is locked to UE 4.26 while Microsoft's Project AirSim has moved to UE5. A reasonable bet for a research team, a risky one for a production roadmap.

Developer playbook
Tech stack, code snippet, sentiment, alternatives.
PM playbook
Adoption angles, user fit, positioning.
CEO playbook
Traction signals, ROI, build vs buy.
Deep-dive insight
Full long-form analysis, no fluff.
Easy mode
Core idea, fast — when you need the gist.
Pro mode
Technical nuance, edge cases, tradeoffs.
Read the full digest
Go beyond the preview

Deep-dive insight, Easy and Pro modes, plus action playbooks — the full breakdown is one tap away.

Underrated tools. Unfiltered takes.

Read the full digest in the Snaplyze app for deep-dive insight, Easy and Pro modes, and the playbooks you can actually use.

Install Snaplyze →