Designing Scenarios with the Synthex Simulation Engine
How configurable virtual environments could support reproducible data generation and testing: scenario configuration, sensors, frameworks and sim-to-real challenges.
Page type: Explains a workflow step by step.
Overview
Some conditions are rare, dangerous or expensive to capture: night rain at a junction, fog on a rural road, dense weeds at dusk. Configurable virtual environments can produce such conditions on demand, with exact labels. Because each run is described by a versioned configuration and a seed, the same data can be regenerated and the same test repeated.
Simulation workflow
- 01Scenario requirements
- 02Environment configuration
- 03Object and agent configuration
- 04Sensor configuration
- 05Simulation execution
- 06Data capture
- 07Annotation and metadata
- 08Evaluation
Scenario configuration
- Environment
- The scene type and its static content: road layout, field rows, warehouse shelving.
- Lighting
- Time of day, sun angle and artificial lights, all of which change object appearance.
- Weather
- Rain, fog and their intensities, plus side effects such as wet surfaces.
- Object placement
- Which objects appear, how many, and how they are distributed or randomised.
- Agent behaviour
- How moving agents such as pedestrians, vehicles or machinery behave over time.
- Sensors
- Sensor type, mounting pose, resolution, field of view and frame rate.
- Random seed
- Fixes every random choice so the same configuration regenerates the same data.
- Output requirements
- Frame count, label types and file formats the run must produce.
Scenario configuration explorer
Illustrative Scenario Configuration
No simulation runs hereScenario description
A urban scene at dusk in rain conditions, with vehicles, pedestrians, cyclists, traffic signs and scripted pedestrians crossing at junctions, captured by a rgb camera.
{
"scenario_id": "example-urban-dusk-rain",
"version": "0.1.0",
"environment": {
"type": "urban",
"objects": [
"vehicles",
"pedestrians",
"cyclists",
"traffic signs"
]
},
"lighting": {
"time_of_day": "dusk"
},
"weather": {
"condition": "rain",
"intensity": 0.6,
"wet_surfaces": true
},
"agents": {
"behaviour": "scripted pedestrians crossing at junctions"
},
"sensors": [
{
"id": "sensor_0",
"type": "rgb_camera",
"resolution": [
1280,
720
],
"fov_deg": 90,
"frame_rate_hz": 10
}
],
"random_seed": 1234,
"output": {
"frames": 200,
"task": "object-detection",
"annotations": [
"bbox_2d",
"class_label"
]
}
}Potential outputs
- RGB frames (PNG)
- 2D boxes or masks
- camera intrinsics and pose
Evaluation objectives
- Measure detection recall and precision under rain dusk conditions.
- Compare against the same scenario under clear daylight to isolate the effect of the condition.
- Confirm any finding on real-world data before acting on it.
Technical considerations
- Rain adds droplets, reflections and wet-road glare; real rain varies more than most simulations capture.
- Low sun angles create glare and long shadows that change object appearance.
- Fix the random seed and record the configuration version so the dataset can be regenerated.
Sensor data concepts
- RGB camera frames
- Colour images, the most common input for perception models.
- Depth maps
- Per-pixel distance from the camera, exact in simulation and noisy in reality.
- Semantic segmentation
- Per-pixel class labels rendered directly from scene knowledge.
- Object detection labels
- 2D or 3D boxes for each visible object, with occlusion and truncation flags.
- Sensor metadata
- Intrinsics, exposure, timestamps and noise parameters for each frame.
- Pose and coordinates
- Sensor and object positions in a declared coordinate frame, so data from several sensors can be aligned.
Simulation frameworks
CARLAExploratory
Open-source autonomous driving simulator (code MIT, assets CC-BY), built on Unreal Engine.
Possible pattern: Under exploration for research scenarios: a scenario configuration would be translated into simulator settings and sensor definitions.
Unreal EngineProposed
Real-time 3D engine.
Possible pattern: Possible rendering backend for photorealistic scenes, directly or via simulators built on it.
UnityProposed
Real-time 3D engine.
Possible pattern: Possible environment for custom scenes such as agricultural fields or warehouses.
Other compatible frameworksProposed
Any simulator that can be driven by configuration and export sensor data.
Possible pattern: An adapter boundary keeps scenario definitions independent of a single simulator.
Sim-to-real challenges
- Visual domain gap
- Rendered textures, lighting and materials differ from reality in ways models can exploit.
- Sensor fidelity
- Real sensors add noise, blur, distortion and artefacts that simple models omit.
- Scenario representativeness
- Generated conditions must resemble the real distribution, not just look plausible.
- Physics limitations
- Simplified physics affects motion, deformation, rain and fog behaviour.
- Environmental variation
- Real environments vary in countless small ways that a finite scenario set may miss.
- Real-world validation requirements
- Synthetic results count only after confirmation on held-out real data.