Skip to content

File formats

This page documents the data files the repo reads and writes: scenarios and suites, the scenario catalog, the output of a CARLA render, and the files produced by the calibration, evaluation and latency tools.

Scenario

A scenario is a Python dict, or the equivalent JSON object. The testbench reads four keys and ignores the rest.

{
    'name': 'Fast Approach',
    'expected_dominant': 'alert',
    'vehicle': 'ebike',
    'agents': [
        {'type': 'person', 'id': 1,
         'path': [{'x': 5, 'y': -4, 't': 0}, {'x': 5, 'y': 4, 't': 6}]},
        {'type': 'bike', 'id': 2,
         'path': [{'x': 25, 'y': 0.5, 't': 2}, {'x': 1, 'y': -0.1, 't': 4}]},
    ],
}
Key Type Required Meaning
name string yes Unique display name.
expected_dominant idle, safe, warning, alert yes Expected dominant state: the most severe state (alert > warning > safe > idle) that the pipeline holds for at least 0.3 s in total over the run.
vehicle bicycle, ebike, aashto no Braking profile. Defaults to bicycle.
agents list yes Pedestrians and cyclists.
agents[].type person, bike yes Agent class seen by the decision pipeline.
agents[].id integer yes Track id, unique within the scenario.
agents[].path list yes Waypoints in time order.
agents[].path[].x metres yes Position along the road.
agents[].path[].y metres yes Position across the road.
agents[].path[].t seconds yes Time of the waypoint. Positions between waypoints are linear in time.

The crosswalk centre is at (5, 0). The frame is drawn in Scenarios and the path rules are listed in Authoring.

Metadata keys

bikeped.scenarios, the CARLA renderer and the docs read these. The testbench ignores them.

Key Type Meaning
id string Slug, unique across all suites.
suite string Suite name.
family string Family key.
description string One or two plain sentences.
tags list of strings Free-form labels.
render object Optional rendering hints, such as {'weather': 'wet_night'}.
agents[].model string Optional appearance hint for the renderer.
agents[].label string Optional short label.

Suite file

A suite file is a JSON file written by python -m bikeped.scenarios generate --out suite.json. The scenarios inside it use the format above. See Generators.

Scenario catalog

The catalog is the machine-readable index of every registered scenario. python -m bikeped.scenarios export --out docs_src/assets/scenarios writes it as catalog.json, next to one SVG thumbnail per scenario named <id>.svg. The catalog tables in Conformance suite and Extended suites are written by tools/build_docs.py from the registry itself and use the same thumbnails.

Render output

One run of python -m bikeped.sim render writes one directory. Its name is <engine>_<scenario-id>_<weather>.

renders/<run_id>/
  manifest.json                 run metadata, per-frame records, camera models
  video/<view>.mp4              clean view, H.264
  video/<view>_overlay.mp4      view with the HUD
  video/composite.mp4           chosen layout
  frames/<view>/000000.jpg      only with --save-frames
  gt/<view>.jsonl               one line per frame
  stills/                       first frame, first alert frame, closest approach

The directory is described from the user's side in CARLA output.

manifest.json

The manifest holds the run metadata, the camera model of every view and one record per frame. The exporter in bikeped/sim/export.py reads these top-level keys:

Key Meaning
run_id Name of the run. Falls back to the directory name when absent.
run Run metadata, described below.
views Names of the rendered views.
cameras Camera model per view name.
videos Relative path of the clean video per view name. The default is video/<view>.mp4.
gt Relative path of the ground-truth file per view name. The default is gt/<view>.jsonl.
frames_dir, frame_ext Relative directory of the saved frames per view name and their file extension, present when frames were saved.

Run metadata (RunInfo in bikeped/sim/types.py):

Field Type Meaning
scenario_name, scenario_id, suite string Which scenario was rendered.
fps integer Simulation and video frame rate. Default 30.
duration_s float Scenario duration in seconds.
engine ue4, ue5 Unreal generation of the CARLA server.
server_version, client_version string CARLA versions reported at connect time.
map string CARLA map name.
crosswalk_id integer or null Entry of crosswalks.json the scenario was placed on.
weather, quality, drive, ambient string Presets and modes the run used.
seed integer Random seed.
views list of strings Names of the rendered views.
site object Site geometry derived from the crosswalk.
pipeline_config string Key of PIPELINE_CONFIGS used for the decision states.
extra object Anything else the runner recorded.

Frame record (FrameRecord.to_dict):

Field Type Meaning
index integer Frame number. The first frame is 0.
t float Scenario time in seconds, rounded to four decimals.
state string Decision state of the frame: idle, safe, warning or alert.
states object Decision state per named pipeline configuration.
alert_pair [bike_id, ped_id] or null The pair that triggered the alert.
danger object or null Ground-truth danger assessment of the frame.
agents list One agent record per agent present in the frame.

Images and auxiliary depth or instance buffers are not written to the manifest.

Agent record (AgentState):

Field Type Meaning
id integer Agent id from the scenario.
kind person, bike Agent class.
sx, sy metres Scripted position in the scenario frame. The decision pipeline and the danger model work on these values.
heading radians Scripted heading in the scenario frame.
speed m/s Scripted speed.
location [x, y, z] metres Centre of the actor's 3D box in the CARLA world frame.
rotation [pitch, yaw, roll] degrees Orientation in the CARLA world frame.
extent [forward, right, up] metres Half sizes of the box.
role scripted, ambient Whether the agent comes from the scenario or from ambient traffic.
model string Appearance hint from the scenario.
blueprint string CARLA blueprint that was spawned.
track_error_m metres Ground-plane distance between the scripted and the simulated position.

Camera model (CameraModel):

Field Type Meaning
name string View name.
kind pinhole, fisheye Projection.
width, height pixels Image size.
position [x, y, z] metres Camera position in the world frame.
forward, right, up unit vectors Camera basis in the world frame. Image u grows with right, image v grows against up.
fov_deg degrees Horizontal field of view for a pinhole view, full field of view of the image circle for a fisheye view.
cx, cy pixels or null Principal point. Null means the image centre.
f_px_per_rad pixels per radian Fisheye focal length.
k1, k2 number Fisheye polynomial terms: r = f (θ + k1 θ³ + k2 θ⁵).
circle_radius_px pixels Radius of the fisheye image circle.

gt/<view>.jsonl

The file has one JSON object per line and one line per frame. Line k belongs to frame k of the clean video of the same view.

Field Meaning
index Frame number, the same as in the manifest.
t Scenario time in seconds.
width, height Image size in pixels that the boxes refer to. When absent, the size of the view's camera model applies.
objects One entry per agent, with the fields below.
Object field Meaning
id, kind, role As in the agent record.
bbox Axis-aligned 2D box [x1, y1, x2, y2] in pixels, origin at the top-left corner.
corners_image The eight projected corners of the 3D box.
distance_m Distance from the camera in metres.
truncation Fraction of the box that falls outside the image.
visible_fraction Fraction of the agent that is not occluded, available when depth or instance buffers were rendered.

To turn one view into a detection dataset, run python -m bikeped.sim export RUN_DIR --format coco|yolo. The exported classes are person and bicycle; a scenario agent of type bike becomes bicycle. See Ground truth.

Tool output files

File Written by Read by Contents
camera_calibration.json calibration/calibrate_fisheye.py testbench, visualizer, renderer Calibrated fisheye intrinsics. Every key is listed in Configuration.
eval_cache.json tools/eval_models.py decision_testbench.py --eval-cache One entry per model name. Under merged, the classes pedestrian, cyclist and vehicle each carry ap5095 and recall, and optionally per_size and ar_curve.
latency_data.npy the bridge, when latency profiling is on tools/latency_report.py A 2D array with one row per processed frame and ten columns: frame_acquire, preprocess, yolo_preprocess, yolo_inference, yolo_postprocess, bev_tracking, decision, viz_submit, total, pipeline_only.

How the detector statistics enter the testbench is covered in Uncertainty models, and the latency columns in Latency.