Skip to content

bikeped.sim

The CARLA renderer. Modules that talk to the simulator import carla only when first used, so everything below imports without it. CARLA rendering explains how the parts fit together.

types

Data types shared by the CARLA runtime and the imaging code.

Nothing in this module imports carla. The runtime (session, actors, rigs, runner) fills these records from the simulator, and the imaging code (projection, ground truth, overlay, recorder, export) consumes them, so both halves can be developed and tested without a server.

Frames of reference

Scenario frame The frame of the testbench scenarios: metres, right-handed, x along the road (cyclists travel along it), y across the road (pedestrians cross along it), crosswalk centre at (5, 0).

World frame The CARLA world frame: metres, left-handed, x forward, y right, z up. Rotations are (pitch, yaw, roll) in degrees with the Unreal convention used by carla.Rotation.

Camera basis A camera is described by its position and by three orthonormal world-frame vectors forward, right and up. A world point P has camera coordinates (f, r, u) = (forward . d, right . d, up . d) with d = P - position. Image u grows with r and image v grows with -u.

CameraModel dataclass

CameraModel(
    name: str,
    kind: str,
    width: int,
    height: int,
    position: Vec3,
    forward: Vec3,
    right: Vec3,
    up: Vec3,
    fov_deg: float = 90.0,
    cx: Optional[float] = None,
    cy: Optional[float] = None,
    f_px_per_rad: Optional[float] = None,
    k1: float = 0.0,
    k2: float = 0.0,
    circle_radius_px: Optional[float] = None,
)

Intrinsics and world pose of one rendered view.

kind is "pinhole" or "fisheye".

Pinhole: fov_deg is the horizontal field of view, and the focal length in pixels is width / (2 tan(fov / 2)).

Fisheye: the optical axis is forward and the projection is the equidistant model with odd polynomial terms, r = f_px_per_rad * (theta + k1 theta^3 + k2 theta^5), where theta is the angle between the ray and the optical axis. fov_deg is the full field of view of the image circle and circle_radius_px its radius. cx and cy default to the image centre.

focal_px property

focal_px: float

Pinhole focal length in pixels.

basis

basis() -> np.ndarray

3x3 array whose rows are forward, right and up.

Source code in bikeped/sim/types.py
def basis(self) -> np.ndarray:
    """3x3 array whose rows are ``forward``, ``right`` and ``up``."""
    return np.array([self.forward, self.right, self.up], dtype=np.float64)

AgentState dataclass

AgentState(
    id: int,
    kind: str,
    sx: float,
    sy: float,
    heading: float,
    speed: float,
    location: Vec3,
    rotation: Vec3,
    extent: Vec3,
    role: str = "scripted",
    model: str = "",
    blueprint: str = "",
    track_error_m: float = 0.0,
    instance_id: Optional[int] = None,
)

One agent at one frame.

sx, sy and heading are the scripted pose in the scenario frame and are what the decision pipeline and the ground-truth danger model see. location, rotation and extent describe the oriented 3D box of the simulated actor in the world frame: location is the box centre, extent the half sizes along the actor's forward, right and up axes. track_error_m is the ground-plane distance between the scripted and the simulated position.

FrameRecord dataclass

FrameRecord(
    index: int,
    t: float,
    agents: List[AgentState],
    state: str = "idle",
    states: Dict[str, str] = dict(),
    alert_pair: Optional[Tuple[int, int]] = None,
    danger: Optional[Dict[str, Any]] = None,
    cameras: Dict[str, CameraModel] = dict(),
    images: Dict[str, ndarray] = dict(),
    aux: Dict[str, ndarray] = dict(),
)

Everything known about one simulation step.

images maps a view name to a BGR uint8 array. aux holds optional per-view buffers under "<view>/depth" (float32 metres) and "<view>/instance" (integer ids). Neither is serialised to the manifest.

RunInfo dataclass

RunInfo(
    scenario_name: str,
    scenario_id: str,
    suite: str = "conformance",
    fps: int = 30,
    duration_s: float = 0.0,
    engine: str = "ue4",
    server_version: str = "",
    client_version: str = "",
    map: str = "",
    crosswalk_id: Optional[int] = None,
    weather: str = "clear_noon",
    quality: str = "standard",
    drive: str = "velocity",
    ambient: str = "none",
    seed: int = 0,
    views: List[str] = list(),
    site: Dict[str, Any] = dict(),
    pipeline_config: str = "default",
    extra: Dict[str, Any] = dict(),
)

Metadata of one rendered scenario, written at the top of the manifest.

engine

Engine profiles, server detection and capability probing.

Two CARLA generations are supported: CARLA 0.9.14 to 0.9.16 on Unreal Engine 4 (ue4) and CARLA 0.10.0 on Unreal Engine 5 (ue5). The rest of the package asks :func:probe what a server offers instead of branching on version numbers, so a feature missing from one build degrades the same way on any engine.

carla is imported lazily through :func:carla_module. Tests replace it with :func:set_carla_module.

EngineProfile dataclass

EngineProfile(
    name: str,
    launcher: str,
    default_map: str,
    default_drive: str,
    default_port: int,
    launch_flags: Tuple[str, ...],
    supports_quality_flag: bool,
    root_env: str,
    wheel_hint: str,
    search_paths: Tuple[str, ...] = (),
)

Static facts about one CARLA generation.

Capabilities dataclass

Capabilities(
    engine: str,
    server_version: str,
    client_version: str,
    map: str,
    maps: List[str] = list(),
    two_wheelers: List[str] = list(),
    bicycles: List[str] = list(),
    walkers: int = 0,
    wheelchair_walkers: List[str] = list(),
    child_walkers: List[str] = list(),
    vehicles: int = 0,
    depth_camera: bool = False,
    instance_camera: bool = False,
    ground_projection: bool = False,
    light_manager: bool = False,
    crosswalks: int = 0,
)

What one running server offers to the renderer.

set_carla_module

set_carla_module(module: Any) -> None

Use module in place of the real carla package (None restores it).

Source code in bikeped/sim/engine.py
def set_carla_module(module: Any) -> None:
    """Use ``module`` in place of the real ``carla`` package (``None`` restores it)."""
    global _CARLA_OVERRIDE
    _CARLA_OVERRIDE = module

carla_module

carla_module() -> Any

Return the carla module, importing it on first use.

Raises ImportError with installation hints when the client is missing.

Source code in bikeped/sim/engine.py
def carla_module() -> Any:
    """Return the ``carla`` module, importing it on first use.

    Raises ``ImportError`` with installation hints when the client is missing.
    """
    if _CARLA_OVERRIDE is not None:
        return _CARLA_OVERRIDE
    try:
        return importlib.import_module("carla")
    except ImportError as exc:
        raise ImportError(
            "The CARLA Python client is not installed. Install the wheel shipped with the "
            "server, e.g. PythonAPI/carla/dist/carla-0.9.16-*.whl for Unreal Engine 4 or "
            "carla-0.10.0-cp310-*.whl (Python 3.10) for Unreal Engine 5."
        ) from exc

engine_for_version

engine_for_version(version: str) -> str

Map a CARLA version string to ue4 (0.9.x) or ue5 (0.10.x and later).

Source code in bikeped/sim/engine.py
def engine_for_version(version: str) -> str:
    """Map a CARLA version string to ``ue4`` (0.9.x) or ``ue5`` (0.10.x and later)."""
    parts = version.strip().lstrip("v").split(".")
    try:
        major, minor = int(parts[0]), int(parts[1])
    except (IndexError, ValueError):
        # Development builds report a git hash; only 0.9.x builds do that on this API.
        return "ue4"
    return "ue4" if (major, minor) < (0, 10) else "ue5"

detect_engine

detect_engine(client: Any) -> str

Return ue4 or ue5 for the connected server and warn on a client mismatch.

Source code in bikeped/sim/engine.py
def detect_engine(client: Any) -> str:
    """Return ``ue4`` or ``ue5`` for the connected server and warn on a client mismatch."""
    server = str(client.get_server_version())
    engine = engine_for_version(server)
    client_version = str(getattr(client, "get_client_version", lambda: "")())
    if client_version and engine_for_version(client_version) != engine:
        log.warning(
            "CARLA client %s does not match server %s (%s); install %s",
            client_version, server, engine, PROFILES[engine].wheel_hint,
        )
    return engine

attribute_text

attribute_text(blueprint: Any, name: str) -> Optional[str]

Value of a blueprint attribute as lower-case text, or None when absent.

The real client refuses as_str() on numeric attributes, so the typed accessors are tried in turn.

Source code in bikeped/sim/engine.py
def attribute_text(blueprint: Any, name: str) -> Optional[str]:
    """Value of a blueprint attribute as lower-case text, or ``None`` when absent.

    The real client refuses ``as_str()`` on numeric attributes, so the typed
    accessors are tried in turn.
    """
    if not blueprint.has_attribute(name):
        return None
    attribute = blueprint.get_attribute(name)
    for getter in ("as_str", "as_int", "as_float", "as_bool"):
        fn = getattr(attribute, getter, None)
        if fn is None:
            continue
        try:
            return str(fn()).lower()
        except (RuntimeError, ValueError, TypeError):
            continue
    return str(attribute).lower()

is_two_wheeler

is_two_wheeler(blueprint: Any) -> bool

True for bicycles, scooters and motorcycles in any CARLA build.

Source code in bikeped/sim/engine.py
def is_two_wheeler(blueprint: Any) -> bool:
    """True for bicycles, scooters and motorcycles in any CARLA build."""
    if _attr(blueprint, "number_of_wheels") == "2":
        return True
    return (_attr(blueprint, "base_type") or "").lower() in ("bicycle", "motorcycle")

probe

probe(client: Any) -> Capabilities

Query a connected server for maps, blueprints, sensors and world features.

Source code in bikeped/sim/engine.py
def probe(client: Any) -> Capabilities:
    """Query a connected server for maps, blueprints, sensors and world features."""
    engine = detect_engine(client)
    world = client.get_world()
    library = world.get_blueprint_library()
    caps = Capabilities(
        engine=engine,
        server_version=str(client.get_server_version()),
        client_version=str(getattr(client, "get_client_version", lambda: "")()),
        map=world.get_map().name.split("/")[-1],
        maps=sorted({m.split("/")[-1] for m in client.get_available_maps()}),
    )
    for bp in library.filter("vehicle.*"):
        if is_two_wheeler(bp):
            caps.two_wheelers.append(bp.id)
            if (_attr(bp, "base_type") or "").lower() == "bicycle":
                caps.bicycles.append(bp.id)
        else:
            caps.vehicles += 1
    for bp in library.filter("walker.pedestrian.*"):
        caps.walkers += 1
        if (_attr(bp, "can_use_wheelchair") or "").lower() == "true":
            caps.wheelchair_walkers.append(bp.id)
        if (_attr(bp, "age") or "").lower() == "child":
            caps.child_walkers.append(bp.id)
    ids = {bp.id for bp in library.filter("sensor.camera.*")}
    caps.depth_camera = "sensor.camera.depth" in ids
    caps.instance_camera = "sensor.camera.instance_segmentation" in ids
    caps.ground_projection = hasattr(world, "ground_projection")
    caps.light_manager = hasattr(world, "get_lightmanager")
    try:
        caps.crosswalks = len(world.get_map().get_crosswalks()) // 5
    except (AttributeError, RuntimeError):
        caps.crosswalks = 0
    return caps

format_capabilities

format_capabilities(caps: Capabilities) -> str

Render capabilities as an aligned two-column table.

Source code in bikeped/sim/engine.py
def format_capabilities(caps: Capabilities) -> str:
    """Render capabilities as an aligned two-column table."""
    def yes(flag: bool) -> str:
        return "yes" if flag else "no"

    rows = [
        ("engine", caps.engine),
        ("server version", caps.server_version),
        ("client version", caps.client_version),
        ("current map", caps.map),
        ("maps", ", ".join(caps.maps)),
        ("two-wheelers", ", ".join(caps.two_wheelers) or "none (cyclists need --cyclist-proxy)"),
        ("bicycles", ", ".join(caps.bicycles) or "none"),
        ("walkers", f"{caps.walkers} ({len(caps.child_walkers)} child, "
                    f"{len(caps.wheelchair_walkers)} wheelchair-capable)"),
        ("four-wheeled vehicles", str(caps.vehicles)),
        ("depth camera", yes(caps.depth_camera)),
        ("instance segmentation", yes(caps.instance_camera)),
        ("ground projection", yes(caps.ground_projection)),
        ("light manager", yes(caps.light_manager)),
        ("crosswalks on map", str(caps.crosswalks)),
    ]
    width = max(len(k) for k, _ in rows)
    return "\n".join(f"{k.ljust(width)}  {v}" for k, v in rows)

root_from_env

root_from_env(engine: str) -> Optional[str]

Install root named by CARLA_UE4_ROOT or CARLA_UE5_ROOT, if set.

Source code in bikeped/sim/engine.py
def root_from_env(engine: str) -> Optional[str]:
    """Install root named by ``CARLA_UE4_ROOT`` or ``CARLA_UE5_ROOT``, if set."""
    return os.environ.get(PROFILES[engine].root_env) or None

server

Locate, launch and stop a CARLA server process.

A CARLA release ships a launcher script (CarlaUE4.sh in the Unreal Engine 4 releases, CarlaUnreal.sh in the Unreal Engine 5 release) that starts the simulator binary. This module finds an install, builds the launch command for either engine, runs it in its own process group, waits until a client can read the server version and stops the whole group again.

carla is only needed by :meth:ServerProcess.wait_ready, which asks the server for its version through a client.

ServerProcess

ServerProcess(
    engine: str,
    root: Optional[PathLike] = None,
    port: int = 2000,
    quality: str = "Epic",
    windowed: bool = False,
    host: str = "127.0.0.1",
    log_path: Optional[PathLike] = None,
    extra_args: Sequence[str] = (),
)

One CARLA server started from a local install.

The launcher script starts the simulator binary as a child process, so the server gets its own session and process group and :meth:stop signals the whole group. A terminal interrupt therefore reaches only the Python process, which is expected to call :meth:stop on its way out (:func:managed_server does).

Parameters:

Name Type Description Default
engine str

"ue4" or "ue5".

required
root Optional[PathLike]

install folder; looked up with :func:find_install when omitted.

None
port int

RPC port of the server.

2000
quality str

UE4 quality level, "Epic" or "Low".

'Epic'
windowed bool

open a window instead of rendering off screen.

False
host str

address used to check that the server answers.

'127.0.0.1'
log_path Optional[PathLike]

file that receives the server's output; a file in the system temporary folder by default.

None
extra_args Sequence[str]

further launcher arguments, appended as they are.

()
Source code in bikeped/sim/server.py
def __init__(
    self,
    engine: str,
    root: Optional[PathLike] = None,
    port: int = 2000,
    quality: str = "Epic",
    windowed: bool = False,
    host: str = "127.0.0.1",
    log_path: Optional[PathLike] = None,
    extra_args: Sequence[str] = (),
) -> None:
    self.engine = engine
    self.profile = _profile(engine)
    self.root = root
    self.port = int(port)
    self.quality = _quality(quality)
    self.windowed = windowed
    self.host = host
    self.extra_args = tuple(extra_args)
    default_log = Path(tempfile.gettempdir()) / f"carla_{engine}_{self.port}.log"
    self.log_path = Path(log_path) if log_path is not None else default_log
    self.command: List[str] = []
    self.version: Optional[str] = None
    self._process: Optional["subprocess.Popen[bytes]"] = None
    self._pgid: Optional[int] = None
    self._log: Optional[IO[bytes]] = None

pid property

pid: Optional[int]

Process id of the launcher, or None when no server is running.

running property

running: bool

Whether any process of the server's group is still alive.

start

start() -> 'ServerProcess'

Launch the server and return immediately.

Raises:

Type Description
RuntimeError

a server was already started from this object, or something already listens on the RPC port.

FileNotFoundError

no install with the engine's launcher was found.

Source code in bikeped/sim/server.py
def start(self) -> "ServerProcess":
    """Launch the server and return immediately.

    Raises:
        RuntimeError: a server was already started from this object, or
            something already listens on the RPC port.
        FileNotFoundError: no install with the engine's launcher was found.
    """
    if self._process is not None:
        raise RuntimeError("this server was already started")
    if _port_open(self.host, self.port):
        raise RuntimeError(
            f"port {self.port} on {self.host} is already in use; stop the server that "
            "listens there or choose another --port"
        )
    root = find_install(self.engine, self.root)
    self.command = build_command(
        self.engine, root, self.port, self.quality, self.windowed, self.extra_args
    )
    self.log_path.parent.mkdir(parents=True, exist_ok=True)
    log = open(self.log_path, "wb")
    try:
        self._process = subprocess.Popen(
            self.command,
            cwd=str(Path(self.command[0]).parent),
            stdin=subprocess.DEVNULL,
            stdout=log,
            stderr=subprocess.STDOUT,
            start_new_session=True,
        )
    except OSError:
        log.close()
        raise
    self._log = log
    # start_new_session makes the launcher the leader of a new process group.
    self._pgid = self._process.pid
    logger.info("started CARLA %s server (pid %d): %s", self.engine, self._process.pid,
                " ".join(self.command))
    return self

wait_ready

wait_ready(timeout: float = 180.0, poll_interval: float = 1.0) -> str

Block until a client can read the server version, and return it.

Raises:

Type Description
RuntimeError

the server was not started or exited while starting up.

TimeoutError

the server did not answer within timeout seconds.

Source code in bikeped/sim/server.py
def wait_ready(self, timeout: float = 180.0, poll_interval: float = 1.0) -> str:
    """Block until a client can read the server version, and return it.

    Raises:
        RuntimeError: the server was not started or exited while starting up.
        TimeoutError: the server did not answer within ``timeout`` seconds.
    """
    if self._process is None:
        raise RuntimeError("the server has not been started")
    carla = carla_module()
    deadline = time.monotonic() + timeout
    last_error: Optional[Exception] = None
    while True:
        if not self.running:
            raise RuntimeError(
                f"the CARLA server exited with code {self._process.returncode} before it "
                f"accepted connections; its output is in {self.log_path}\n{self.log_tail()}"
            )
        try:
            client = carla.Client(self.host, self.port)
            client.set_timeout(2.0)
            self.version = str(client.get_server_version())
            logger.info("CARLA server %s is ready on port %d", self.version, self.port)
            return self.version
        except (RuntimeError, OSError) as error:
            last_error = error
        remaining = deadline - time.monotonic()
        if remaining <= 0.0:
            raise TimeoutError(
                f"no answer from the CARLA server on {self.host}:{self.port} after "
                f"{timeout:.0f} s (last error: {last_error}); its output is in {self.log_path}"
            )
        time.sleep(min(poll_interval, remaining))

stop

stop(grace: float = 10.0) -> None

Stop the server and every process it started.

The process group receives SIGTERM and, if it is still alive after grace seconds, SIGKILL. Calling this on a server that is not running does nothing.

Source code in bikeped/sim/server.py
def stop(self, grace: float = 10.0) -> None:
    """Stop the server and every process it started.

    The process group receives ``SIGTERM`` and, if it is still alive after
    ``grace`` seconds, ``SIGKILL``. Calling this on a server that is not
    running does nothing.
    """
    if self._process is None:
        return
    try:
        if self.running:
            self._signal_group(signal.SIGTERM)
            if not self._wait_exit(grace):
                logger.warning("CARLA server ignored SIGTERM for %.0f s, killing it", grace)
                self._signal_group(signal.SIGKILL)
                if not self._wait_exit(grace):
                    logger.error("CARLA server process group %s is still alive", self._pgid)
    finally:
        if self._log is not None:
            self._log.close()
            self._log = None
        self._process = None
        self._pgid = None

log_tail

log_tail(lines: int = 20) -> str

Last lines lines of the server's output, empty if unreadable.

Source code in bikeped/sim/server.py
def log_tail(self, lines: int = 20) -> str:
    """Last ``lines`` lines of the server's output, empty if unreadable."""
    try:
        text = self.log_path.read_text(errors="replace")
    except OSError:
        return ""
    return "\n".join(text.splitlines()[-lines:])

launcher_path

launcher_path(engine: str, root: PathLike) -> Path

Path of the launcher script inside the install at root.

The UE4 archives keep the launcher at the top of the install and the UE5 archive keeps it under Linux/, so both places are checked. When the script is in neither, the top-level path is returned so that callers can name it in their error message.

Source code in bikeped/sim/server.py
def launcher_path(engine: str, root: PathLike) -> Path:
    """Path of the launcher script inside the install at ``root``.

    The UE4 archives keep the launcher at the top of the install and the UE5
    archive keeps it under ``Linux/``, so both places are checked. When the
    script is in neither, the top-level path is returned so that callers can
    name it in their error message.
    """
    profile = _profile(engine)
    base = Path(root).expanduser()
    candidates = (base / profile.launcher, base / "Linux" / profile.launcher)
    for candidate in candidates:
        if candidate.is_file():
            return candidate
    return candidates[0]

find_install

find_install(engine: str, root: Optional[PathLike] = None) -> Path

Return the root folder of a CARLA install for engine.

An explicit root is the only place searched when it is given. Without it, the engine's environment variable (CARLA_UE4_ROOT or CARLA_UE5_ROOT) is tried first and then the profile's list of common install paths.

Raises:

Type Description
FileNotFoundError

no candidate contains the engine's launcher script.

Source code in bikeped/sim/server.py
def find_install(engine: str, root: Optional[PathLike] = None) -> Path:
    """Return the root folder of a CARLA install for ``engine``.

    An explicit ``root`` is the only place searched when it is given. Without
    it, the engine's environment variable (``CARLA_UE4_ROOT`` or
    ``CARLA_UE5_ROOT``) is tried first and then the profile's list of common
    install paths.

    Raises:
        FileNotFoundError: no candidate contains the engine's launcher script.
    """
    profile = _profile(engine)
    if root is not None:
        base = Path(root).expanduser()
        if launcher_path(engine, base).is_file():
            return base
        raise FileNotFoundError(
            f"{profile.launcher} not found in {base} or {base / 'Linux'}. Point --root or "
            f"{profile.root_env} at the folder of the extracted CARLA release."
        )

    candidates: List[str] = []
    env_root = os.environ.get(profile.root_env)
    if env_root:
        candidates.append(env_root)
    candidates.extend(profile.search_paths)

    tried: List[str] = []
    for candidate in candidates:
        base = Path(os.path.expandvars(candidate)).expanduser()
        if launcher_path(engine, base).is_file():
            return base
        tried.append(str(base))
    looked = ", ".join(tried) if tried else "no candidate paths"
    raise FileNotFoundError(
        f"No CARLA {engine} install found (looked for {profile.launcher} in: {looked}). "
        f"Set {profile.root_env} or pass --root with the folder of the extracted CARLA release."
    )

build_command

build_command(
    engine: str,
    root: Optional[PathLike] = None,
    port: int = 2000,
    quality: str = "Epic",
    windowed: bool = False,
    extra_args: Sequence[str] = (),
) -> List[str]

Build the argument list that launches a server.

The server renders off screen unless windowed is set. port is the RPC port; the server also uses the two ports after it for streaming. quality is passed as -quality-level= on UE4 and ignored on UE5, whose launcher has no such flag. With root left out the install is looked up with :func:find_install.

Source code in bikeped/sim/server.py
def build_command(
    engine: str,
    root: Optional[PathLike] = None,
    port: int = 2000,
    quality: str = "Epic",
    windowed: bool = False,
    extra_args: Sequence[str] = (),
) -> List[str]:
    """Build the argument list that launches a server.

    The server renders off screen unless ``windowed`` is set. ``port`` is the
    RPC port; the server also uses the two ports after it for streaming.
    ``quality`` is passed as ``-quality-level=`` on UE4 and ignored on UE5,
    whose launcher has no such flag. With ``root`` left out the install is
    looked up with :func:`find_install`.
    """
    profile = _profile(engine)
    level = _quality(quality)
    base = find_install(engine) if root is None else root
    command = [str(launcher_path(engine, base))]
    for flag in profile.launch_flags:
        if windowed and flag.lower() == "-renderoffscreen":
            continue
        command.append(flag)
    if windowed:
        command.append("-windowed")
    command.append(f"-carla-rpc-port={int(port)}")
    if profile.supports_quality_flag:
        command.append(f"-quality-level={level}")
    command.extend(extra_args)
    return command

managed_server

managed_server(
    engine: str,
    root: Optional[PathLike] = None,
    port: int = 2000,
    quality: str = "Epic",
    windowed: bool = False,
    timeout: float = 180.0,
    host: str = "127.0.0.1",
    log_path: Optional[PathLike] = None,
) -> Iterator[ServerProcess]

Run a CARLA server for the duration of a with block.

The server is started, the block is entered once a client can reach it, and the server is stopped when the block exits for any reason, including an exception or a keyboard interrupt.

Source code in bikeped/sim/server.py
@contextlib.contextmanager
def managed_server(
    engine: str,
    root: Optional[PathLike] = None,
    port: int = 2000,
    quality: str = "Epic",
    windowed: bool = False,
    timeout: float = 180.0,
    host: str = "127.0.0.1",
    log_path: Optional[PathLike] = None,
) -> Iterator[ServerProcess]:
    """Run a CARLA server for the duration of a ``with`` block.

    The server is started, the block is entered once a client can reach it,
    and the server is stopped when the block exits for any reason, including
    an exception or a keyboard interrupt.
    """
    server = ServerProcess(
        engine, root=root, port=port, quality=quality, windowed=windowed, host=host,
        log_path=log_path,
    )
    try:
        server.start()
        server.wait_ready(timeout=timeout)
        yield server
    finally:
        server.stop()

session

Connection to a CARLA server in synchronous mode with guaranteed cleanup.

:class:Session connects (with retries), loads the requested map only when it is not already loaded, switches the world and the Traffic Manager to synchronous mode with a fixed time step, tracks every actor spawned through it and, on exit (including KeyboardInterrupt), destroys those actors and restores the previous world settings. :class:SensorHub returns, for each world.tick(), the measurement of every sensor for exactly that frame.

SensorHub

SensorHub(timeout: float = 10.0)

Frame-synchronised collection of sensor data.

Each registered sensor gets its own queue; :meth:collect blocks until every sensor has delivered the measurement of the requested frame and discards older ones.

Source code in bikeped/sim/session.py
def __init__(self, timeout: float = 10.0) -> None:
    self.timeout = timeout
    self._queues: Dict[str, "queue.Queue[Any]"] = {}
    self._decoders: Dict[str, Callable[[Any], Any]] = {}

collect

collect(frame: int) -> Dict[str, Any]

Decoded data of every sensor for frame.

Source code in bikeped/sim/session.py
def collect(self, frame: int) -> Dict[str, Any]:
    """Decoded data of every sensor for ``frame``."""
    out: Dict[str, Any] = {}
    deadline = time.monotonic() + self.timeout
    for key, q in self._queues.items():
        while True:
            remaining = deadline - time.monotonic()
            try:
                data = q.get(timeout=max(remaining, 0.001))
            except queue.Empty:
                raise TimeoutError(
                    f"sensor '{key}' delivered nothing for frame {frame} within "
                    f"{self.timeout:.0f} s; the server may be overloaded (try --quality draft "
                    "or fewer views)") from None
            if data.frame == frame:
                out[key] = self._decoders[key](data)
                break
            if data.frame > frame:
                raise RuntimeError(f"sensor '{key}' skipped frame {frame} (got {data.frame})")
    return out

Session

Session(
    host: str = "127.0.0.1",
    port: int = 2000,
    *,
    map_name: Optional[str] = None,
    fixed_delta: float = 1.0 / 30.0,
    seed: int = 0,
    timeout: float = 30.0,
    retries: int = 5,
    tm_port: Optional[int] = None,
)

Synchronous-mode session on one CARLA server.

Use as a context manager. fixed_delta is the simulation step in seconds (1 / fps). map_name is loaded only when the current map differs.

Source code in bikeped/sim/session.py
def __init__(self, host: str = "127.0.0.1", port: int = 2000, *, map_name: Optional[str] = None,
             fixed_delta: float = 1.0 / 30.0, seed: int = 0, timeout: float = 30.0,
             retries: int = 5, tm_port: Optional[int] = None) -> None:
    self.host, self.port = host, port
    self.map_name = map_name
    self.fixed_delta = fixed_delta
    self.seed = seed
    self.timeout = timeout
    self.retries = retries
    self.tm_port = tm_port if tm_port is not None else port + 6000
    self.carla: Any = None
    self.client: Any = None
    self.world: Any = None
    self.engine = ""
    self.actors: List[Any] = []
    self.sensors: List[Any] = []
    self._original_settings: Any = None
    self._tm: Any = None
    self.frame = 0
    #: Ids of map objects disabled for this session, enabled again on close.
    self.hidden_objects: set = set()

traffic_manager

traffic_manager() -> Any

Traffic Manager in synchronous mode and seeded, or None when unavailable.

Source code in bikeped/sim/session.py
def traffic_manager(self) -> Any:
    """Traffic Manager in synchronous mode and seeded, or ``None`` when unavailable."""
    if self._tm is None:
        try:
            tm = self.client.get_trafficmanager(self.tm_port)
            tm.set_synchronous_mode(True)
            if hasattr(tm, "set_random_device_seed"):
                tm.set_random_device_seed(self.seed)
            self._tm = tm
        except RuntimeError as exc:
            log.warning("Traffic Manager unavailable: %s", exc)
            return None
    return self._tm

spawn

spawn(blueprint: Any, transform: Any, attach_to: Any = None) -> Any

Spawn and track an actor; raises RuntimeError when the server refuses.

Source code in bikeped/sim/session.py
def spawn(self, blueprint: Any, transform: Any, attach_to: Any = None) -> Any:
    """Spawn and track an actor; raises ``RuntimeError`` when the server refuses."""
    if attach_to is not None:
        actor = self.world.spawn_actor(blueprint, transform, attach_to=attach_to)
    else:
        actor = self.world.spawn_actor(blueprint, transform)
    self._track(actor)
    return actor

try_spawn

try_spawn(blueprint: Any, transform: Any, attach_to: Any = None) -> Any

Like :meth:spawn but returns None on failure.

Source code in bikeped/sim/session.py
def try_spawn(self, blueprint: Any, transform: Any, attach_to: Any = None) -> Any:
    """Like :meth:`spawn` but returns ``None`` on failure."""
    if attach_to is not None:
        actor = self.world.try_spawn_actor(blueprint, transform, attach_to=attach_to)
    else:
        actor = self.world.try_spawn_actor(blueprint, transform)
    if actor is not None:
        self._track(actor)
    return actor

destroy

destroy(actor: Any) -> None

Destroy one tracked actor now and stop tracking it.

Source code in bikeped/sim/session.py
def destroy(self, actor: Any) -> None:
    """Destroy one tracked actor now and stop tracking it."""
    for group in (self.actors, self.sensors):
        if actor in group:
            group.remove(actor)
    if getattr(actor, "is_alive", True):
        actor.destroy()

close

close() -> None

Stop sensors, destroy tracked actors, restore settings. Safe to call twice.

Source code in bikeped/sim/session.py
def close(self) -> None:
    """Stop sensors, destroy tracked actors, restore settings. Safe to call twice."""
    for sensor in reversed(self.sensors):
        try:
            if getattr(sensor, "is_listening", False):
                sensor.stop()
        except RuntimeError:
            pass
    for actor in reversed(self.sensors + self.actors):
        try:
            if getattr(actor, "is_alive", True):
                actor.destroy()
        except RuntimeError as exc:
            log.debug("destroy %s failed: %s", getattr(actor, "id", "?"), exc)
    self.sensors, self.actors = [], []
    if self.hidden_objects and self.world is not None:
        try:
            self.world.enable_environment_objects(set(self.hidden_objects), True)
        except RuntimeError as exc:
            log.warning("could not restore hidden map objects: %s", exc)
        self.hidden_objects = set()
    if self._tm is not None:
        try:
            self._tm.set_synchronous_mode(False)
        except RuntimeError:
            pass
        self._tm = None
    if self.world is not None and self._original_settings is not None:
        try:
            self.world.apply_settings(self._original_settings)
        except RuntimeError as exc:
            log.warning("could not restore world settings: %s", exc)
        self._original_settings = None

image_to_bgr

image_to_bgr(image: Any) -> np.ndarray

carla.Image (BGRA bytes) to a contiguous uint8 BGR array.

Source code in bikeped/sim/session.py
def image_to_bgr(image: Any) -> np.ndarray:
    """``carla.Image`` (BGRA bytes) to a contiguous ``uint8`` BGR array."""
    buf = np.frombuffer(image.raw_data, dtype=np.uint8).reshape(image.height, image.width, 4)
    return np.ascontiguousarray(buf[:, :, :3])

depth_to_metres

depth_to_metres(image: Any) -> np.ndarray

Decode a sensor.camera.depth image to float32 metres (1000 m far plane).

Source code in bikeped/sim/session.py
def depth_to_metres(image: Any) -> np.ndarray:
    """Decode a ``sensor.camera.depth`` image to ``float32`` metres (1000 m far plane)."""
    buf = np.frombuffer(image.raw_data, dtype=np.uint8).reshape(image.height, image.width, 4)
    b, g, r = (buf[:, :, i].astype(np.float32) for i in range(3))
    return ((r + g * 256.0 + b * 65536.0) / (256.0 ** 3 - 1.0) * 1000.0).astype(np.float32)

instance_ids

instance_ids(image: Any) -> np.ndarray

Decode sensor.camera.instance_segmentation to int32 object ids (G + B * 256).

Source code in bikeped/sim/session.py
def instance_ids(image: Any) -> np.ndarray:
    """Decode ``sensor.camera.instance_segmentation`` to ``int32`` object ids (G + B * 256)."""
    buf = np.frombuffer(image.raw_data, dtype=np.uint8).reshape(image.height, image.width, 4)
    return buf[:, :, 1].astype(np.int32) + buf[:, :, 0].astype(np.int32) * 256

site

Placement of the scenario frame on a CARLA crosswalk.

The scenario frame (see :mod:bikeped.sim.types) is anchored on one crosswalk: scenario +y runs along the crosswalk (the direction pedestrians cross), scenario +x runs along the road, and the crosswalk centre is scenario (5, 0). As in the original carla_scenario.py, the reference short edge of the crosswalk polygon defines -y (the pole stands there). The road axis is the cross axis turned by (a, b) -> (-b, a) in world coordinates: the world frame is left-handed, so this is the choice that keeps the scene unmirrored (seen from above, turning from scenario +x to +y is a counter-clockwise turn in both frames, and a cyclist approaching from scenario +x enters the pole view from the right, as in the testbench camera).

Everything here is plain numpy. :meth:Site.from_world is the only method that touches a CARLA world, and only through world.get_map().get_crosswalks().

Site dataclass

Site(
    origin: Tuple[float, float],
    road_axis: Tuple[float, float],
    cross_axis: Tuple[float, float],
    ground_z: float,
    road_half_width: float,
    crosswalk_id: Any = None,
    pole_xy: Tuple[float, float] = (5.0, -13.0),
    pole_height: float = POLE_HEIGHT_M,
)

A crosswalk with the scenario frame attached to it.

origin is the world (X, Y) of scenario (0, 0); road_axis and cross_axis are the world unit vectors of scenario +x and +y. ground_z is the world height of the crosswalk surface and road_half_width half the crosswalk length (kerb to kerb), in metres.

pole_world property

pole_world: Tuple[float, float, float]

World position of the pole camera (the testbench camera position).

from_crosswalk classmethod

from_crosswalk(entry: Dict[str, Any]) -> 'Site'

Build a site from one crosswalks.json entry (corners, center).

Source code in bikeped/sim/site.py
@classmethod
def from_crosswalk(cls, entry: Dict[str, Any]) -> "Site":
    """Build a site from one ``crosswalks.json`` entry (``corners``, ``center``)."""
    corners = [np.asarray(c[:2], dtype=np.float64) for c in entry["corners"]]
    edges = []
    for i in range(4):
        a, b = corners[i], corners[(i + 1) % 4]
        edges.append((float(np.linalg.norm(b - a)), (a + b) / 2.0))
    edges.sort(key=lambda e: e[0])
    cam_edge = edges[0][1]
    center3 = list(entry["center"])
    center = np.asarray(center3[:2], dtype=np.float64)
    to_center = center - cam_edge
    half = float(np.linalg.norm(to_center))
    cross = to_center / half
    road = np.array([-cross[1], cross[0]])
    origin = center - 5.0 * road
    px, py, ph = _testbench_pole()
    return cls(
        origin=(float(origin[0]), float(origin[1])),
        road_axis=(float(road[0]), float(road[1])),
        cross_axis=(float(cross[0]), float(cross[1])),
        ground_z=float(center3[2]) if len(center3) > 2 else 0.0,
        road_half_width=half,
        crosswalk_id=entry.get("id"),
        pole_xy=(px, py),
        pole_height=ph,
    )

from_json classmethod

from_json(path: Union[str, Path], index: int) -> 'Site'

Load entry index (list position or id) of a crosswalks.json file.

Source code in bikeped/sim/site.py
@classmethod
def from_json(cls, path: Union[str, Path], index: int) -> "Site":
    """Load entry ``index`` (list position or ``id``) of a ``crosswalks.json`` file."""
    entries = json.loads(Path(path).read_text())
    for entry in entries:
        if entry.get("id") == index:
            return cls.from_crosswalk(entry)
    return cls.from_crosswalk(entries[index])

from_world classmethod

from_world(world: Any, index: int) -> 'Site'

Build a site from crosswalk index of the loaded map.

Map.get_crosswalks() returns closed polygons as a flat list of locations; rectangles come with their first point repeated (five points).

Source code in bikeped/sim/site.py
@classmethod
def from_world(cls, world: Any, index: int) -> "Site":
    """Build a site from crosswalk ``index`` of the loaded map.

    ``Map.get_crosswalks()`` returns closed polygons as a flat list of
    locations; rectangles come with their first point repeated (five points).
    """
    entries = crosswalks_from_points(
        [(p.x, p.y, p.z) for p in world.get_map().get_crosswalks()])
    if not entries:
        raise RuntimeError("the loaded map exposes no crosswalk polygons")
    if not 0 <= index < len(entries):
        raise IndexError(f"crosswalk {index} not found, the map has {len(entries)}")
    return cls.from_crosswalk(entries[index])

to_world

to_world(x: float, y: float, z: float = 0.0) -> Tuple[float, float, float]

Scenario (x, y) and height above ground to world (X, Y, Z).

Source code in bikeped/sim/site.py
def to_world(self, x: float, y: float, z: float = 0.0) -> Tuple[float, float, float]:
    """Scenario ``(x, y)`` and height above ground to world ``(X, Y, Z)``."""
    X = self.origin[0] + x * self.road_axis[0] + y * self.cross_axis[0]
    Y = self.origin[1] + x * self.road_axis[1] + y * self.cross_axis[1]
    return float(X), float(Y), float(self.ground_z + z)

to_scenario

to_scenario(X: float, Y: float) -> Tuple[float, float]

World (X, Y) to scenario (x, y).

Source code in bikeped/sim/site.py
def to_scenario(self, X: float, Y: float) -> Tuple[float, float]:
    """World ``(X, Y)`` to scenario ``(x, y)``."""
    dx, dy = X - self.origin[0], Y - self.origin[1]
    return (float(dx * self.road_axis[0] + dy * self.road_axis[1]),
            float(dx * self.cross_axis[0] + dy * self.cross_axis[1]))

heading_to_yaw

heading_to_yaw(heading: float) -> float

Scenario heading (radians, 0 = +x) to CARLA yaw (degrees, clockwise from above).

Source code in bikeped/sim/site.py
def heading_to_yaw(self, heading: float) -> float:
    """Scenario heading (radians, 0 = +x) to CARLA yaw (degrees, clockwise from above)."""
    dx = math.cos(heading) * self.road_axis[0] + math.sin(heading) * self.cross_axis[0]
    dy = math.cos(heading) * self.road_axis[1] + math.sin(heading) * self.cross_axis[1]
    return math.degrees(math.atan2(dy, dx))

yaw_to_heading

yaw_to_heading(yaw_deg: float) -> float

Inverse of :meth:heading_to_yaw.

Source code in bikeped/sim/site.py
def yaw_to_heading(self, yaw_deg: float) -> float:
    """Inverse of :meth:`heading_to_yaw`."""
    c, s = math.cos(math.radians(yaw_deg)), math.sin(math.radians(yaw_deg))
    x = c * self.road_axis[0] + s * self.road_axis[1]
    y = c * self.cross_axis[0] + s * self.cross_axis[1]
    return math.atan2(y, x)

direction

direction(heading: float) -> np.ndarray

World-frame unit vector (X, Y, 0) of a scenario heading.

Source code in bikeped/sim/site.py
def direction(self, heading: float) -> np.ndarray:
    """World-frame unit vector (X, Y, 0) of a scenario heading."""
    yaw = math.radians(self.heading_to_yaw(heading))
    return np.array([math.cos(yaw), math.sin(yaw), 0.0])

crosswalks_from_points

crosswalks_from_points(points: Sequence[Tuple[float, float, float]]) -> list

Split CARLA's flat crosswalk point list into crosswalks.json-like entries.

A polygon ends where a point repeats the polygon's first point. Polygons with other than four distinct corners are approximated by their minimum-area rectangle so the edge logic of :meth:Site.from_crosswalk applies.

Source code in bikeped/sim/site.py
def crosswalks_from_points(points: Sequence[Tuple[float, float, float]]) -> list:
    """Split CARLA's flat crosswalk point list into ``crosswalks.json``-like entries.

    A polygon ends where a point repeats the polygon's first point. Polygons with
    other than four distinct corners are approximated by their minimum-area
    rectangle so the edge logic of :meth:`Site.from_crosswalk` applies.
    """
    entries = []
    current: list = []
    for p in points:
        if current and np.allclose(p[:2], current[0][:2], atol=1e-3):
            entries.append(current)
            current = []
        else:
            current.append(tuple(float(v) for v in p))
    out = []
    for poly in entries:
        if len(poly) < 3:
            continue
        pts = np.asarray(poly, dtype=np.float64)
        corners = pts[:, :2] if len(poly) == 4 else _min_area_rect(pts[:, :2])
        z = float(pts[:, 2].mean())
        center = corners.mean(axis=0)
        out.append({
            "id": len(out),
            "center": [float(center[0]), float(center[1]), z],
            "corners": [[float(c[0]), float(c[1]), z] for c in corners],
        })
    return out

motion

Smooth rendering poses for scripted scenario agents.

Positions come straight from the scripted piecewise-linear path with the same semantics as decision_testbench.interpolate_agent, so the ground truth stays exact. Only quantities that the testbench does not use are smoothed: the heading (paths turn instantly at waypoints, people and bicycles do not) and the lean of a two-wheeler, derived from speed and yaw rate. All angles are in the scenario frame, radians, counter-clockwise from +x.

Pose dataclass

Pose(
    t: float,
    active: bool,
    x: float,
    y: float,
    vx: float,
    vy: float,
    speed: float,
    path_heading: float,
    heading: float,
    yaw_rate: float,
    lean: float,
)

Pose of one agent at one instant (scenario frame).

Trajectory

Trajectory(
    agent: Dict[str, Any],
    fps: int = 30,
    time_constant: Optional[float] = None,
    max_turn_rate: Optional[float] = None,
)

Rendering trajectory of one scenario agent sampled on a fixed time grid.

time_constant is the time constant (s) of the critically damped heading filter and max_turn_rate its slew limit (rad/s). Headings are computed once over the whole active window at fps so that :meth:sample is pure and repeatable.

Source code in bikeped/sim/motion.py
def __init__(self, agent: Dict[str, Any], fps: int = 30, time_constant: Optional[float] = None,
             max_turn_rate: Optional[float] = None) -> None:
    self.agent = agent
    self.id = int(agent["id"])
    self.kind = str(agent["type"])
    path = agent["path"]
    self.t = np.array([float(p["t"]) for p in path])
    self.xy = np.array([[float(p["x"]), float(p["y"])] for p in path])
    self.fps = int(fps)
    bike = self.kind == "bike"
    self.time_constant = time_constant if time_constant is not None else (0.35 if bike else 0.2)
    self.max_turn_rate = max_turn_rate if max_turn_rate is not None else (1.6 if bike else 4.0)
    self._grid, self._heading, self._rate = self._filter_headings()

active

active(t: float) -> bool

True inside the scripted window, where interpolate_agent returns a pose.

Source code in bikeped/sim/motion.py
def active(self, t: float) -> bool:
    """True inside the scripted window, where ``interpolate_agent`` returns a pose."""
    return self.start <= t <= self.end

position

position(t: float) -> Tuple[float, float]

Scripted position, clamped to the path ends outside the active window.

Source code in bikeped/sim/motion.py
def position(self, t: float) -> Tuple[float, float]:
    """Scripted position, clamped to the path ends outside the active window."""
    if len(self.t) == 1:
        return float(self.xy[0, 0]), float(self.xy[0, 1])
    tc = min(max(t, self.start), self.end)
    i = self._segment(tc)
    span = self.t[i + 1] - self.t[i]
    frac = 0.0 if span <= 0 else (tc - self.t[i]) / span
    p = self.xy[i] + (self.xy[i + 1] - self.xy[i]) * frac
    return float(p[0]), float(p[1])

velocity

velocity(t: float) -> Tuple[float, float]

Velocity (m/s) of the segment containing t; zero outside the window.

Source code in bikeped/sim/motion.py
def velocity(self, t: float) -> Tuple[float, float]:
    """Velocity (m/s) of the segment containing ``t``; zero outside the window."""
    if len(self.t) == 1 or not self.active(t):
        return 0.0, 0.0
    i = self._segment(t)
    span = self.t[i + 1] - self.t[i]
    if span <= 0:
        return 0.0, 0.0
    v = (self.xy[i + 1] - self.xy[i]) / span
    return float(v[0]), float(v[1])

path_heading

path_heading(t: float) -> float

Heading of the segment containing t (what interpolate_agent reports).

Source code in bikeped/sim/motion.py
def path_heading(self, t: float) -> float:
    """Heading of the segment containing ``t`` (what ``interpolate_agent`` reports)."""
    if len(self.t) == 1:
        return 0.0
    i = self._segment(min(max(t, self.start), self.end))
    d = self.xy[i + 1] - self.xy[i]
    if not np.any(d):
        for j in list(range(i, -1, -1)) + list(range(i + 1, len(self.t) - 1)):
            d = self.xy[j + 1] - self.xy[j]
            if np.any(d):
                break
    return float(math.atan2(d[1], d[0]))

heading

heading(t: float) -> Tuple[float, float]

Smoothed heading (rad) and yaw rate (rad/s) at t.

Source code in bikeped/sim/motion.py
def heading(self, t: float) -> Tuple[float, float]:
    """Smoothed heading (rad) and yaw rate (rad/s) at ``t``."""
    k = int(round((min(max(t, self.start), self._grid[-1]) - self.start) * self.fps))
    k = max(0, min(k, len(self._grid) - 1))
    return float(self._heading[k]), float(self._rate[k])

sample

sample(t: float) -> Pose

Full pose at time t.

Source code in bikeped/sim/motion.py
def sample(self, t: float) -> Pose:
    """Full pose at time ``t``."""
    x, y = self.position(t)
    vx, vy = self.velocity(t)
    speed = math.hypot(vx, vy)
    heading, rate = self.heading(t)
    lean = 0.0
    if self.kind == "bike":
        lean = float(np.clip(math.atan(speed * rate / GRAVITY), -MAX_LEAN_RAD, MAX_LEAN_RAD))
    return Pose(t=t, active=self.active(t), x=x, y=y, vx=vx, vy=vy, speed=speed,
                path_heading=self.path_heading(t), heading=heading, yaw_rate=rate, lean=lean)

wrap_angle

wrap_angle(a: float) -> float

Wrap an angle to (-pi, pi].

Source code in bikeped/sim/motion.py
def wrap_angle(a: float) -> float:
    """Wrap an angle to ``(-pi, pi]``."""
    return math.atan2(math.sin(a), math.cos(a))

trajectories

trajectories(scenario: Dict[str, Any], fps: int = 30) -> List[Trajectory]

One :class:Trajectory per scenario agent, in scenario order.

Source code in bikeped/sim/motion.py
def trajectories(scenario: Dict[str, Any], fps: int = 30) -> List[Trajectory]:
    """One :class:`Trajectory` per scenario agent, in scenario order."""
    return [Trajectory(a, fps) for a in scenario["agents"]]

scripted_agents

scripted_agents(scenario: Dict[str, Any], t: float) -> List[Dict[str, Any]]

Agents active at t exactly as interpolate_agent reports them.

Source code in bikeped/sim/motion.py
def scripted_agents(scenario: Dict[str, Any], t: float) -> List[Dict[str, Any]]:
    """Agents active at ``t`` exactly as ``interpolate_agent`` reports them."""
    import decision_testbench as tb

    out = []
    for agent in scenario["agents"]:
        pos = tb.interpolate_agent(agent, t)
        if pos is not None:
            out.append(pos)
    return out

actors

Spawning and driving the scripted scenario agents.

Each scenario agent becomes one CARLA actor (a walker for person agents, a two-wheeled vehicle for bike agents, or a walker standing in for a cyclist when the engine has no rideable two-wheeler and cyclist_proxy='walker').

Two drive modes are available:

kinematic The actor is placed on the scripted pose every tick. Height comes from a ground profile sampled with world.ground_projection before any actor exists (so the ray never hits an agent), yaw from the smoothed heading and, for two-wheelers, roll from the lean angle. Walkers keep physics and also receive a WalkerControl so their walk cycle plays.

velocity Walkers are steered with WalkerControl (scripted velocity plus a proportional correction on position error) and riders with target linear and angular velocities plus a light throttle and steer so wheels, pedals and handlebar move. An actor is re-placed on its scripted pose only when the position error exceeds a tolerance.

Positions reported to the decision pipeline and the ground truth always come from the script, never from the simulated actor.

Agent dataclass

Agent(
    trajectory: Trajectory,
    kind: str,
    model: str,
    actor: Any = None,
    bp: Any = None,
    blueprint: str = "",
    walker: bool = True,
    half_height: float = 0.9,
    visible: bool = False,
    ground: List[Tuple[float, float, float]] = list(),
)

One scripted agent and the actor that renders it.

Cast

Cast(
    session: Any,
    site: Site,
    scenario: Dict[str, Any],
    trajectories: Sequence[Trajectory],
    caps: Capabilities,
    *,
    drive: str = "kinematic",
    seed: int = 0,
    cyclist_proxy: Optional[str] = None,
    walker_drive: Optional[str] = None,
)

All scripted actors of one scenario.

drive is the drive mode for riders and walkers (see module docstring). cyclist_proxy is None or 'walker'.

Source code in bikeped/sim/actors.py
def __init__(self, session: Any, site: Site, scenario: Dict[str, Any],
             trajectories: Sequence[Trajectory], caps: Capabilities, *,
             drive: str = "kinematic",
             seed: int = 0, cyclist_proxy: Optional[str] = None,
             walker_drive: Optional[str] = None) -> None:
    if drive not in DRIVE_MODES:
        raise ValueError(f"drive must be one of {DRIVE_MODES}")
    self.session = session
    self.site = site
    self.scenario = scenario
    self.caps = caps
    self.drive = drive
    self.walker_drive = walker_drive or drive
    self.rng = random.Random(seed)
    self.cyclist_proxy = cyclist_proxy
    self.agents: List[Agent] = []
    self.dt = session.fixed_delta
    self.time = 0.0
    if needs_rideable(scenario) and not caps.has_rideable and cyclist_proxy != "walker":
        raise RuntimeError(
            f"{caps.engine} server {caps.server_version} has no rideable two-wheeler "
            "blueprint, so cyclists cannot be rendered. Pass --cyclist-proxy walker to "
            "render them as walkers.")
    defs = {int(a["id"]): a for a in scenario["agents"]}
    for traj in trajectories:
        a = defs[traj.id]
        model = str(a.get("model") or ("bicycle" if a["type"] == "bike" else "adult"))
        self.agents.append(Agent(trajectory=traj, kind=a["type"], model=model))

sample_ground

sample_ground(step: float = 0.5) -> None

Sample ground heights along every path before any agent is spawned.

Source code in bikeped/sim/actors.py
def sample_ground(self, step: float = 0.5) -> None:
    """Sample ground heights along every path before any agent is spawned."""
    world = self.session.world
    carla = self.session.carla
    for agent in self.agents:
        traj = agent.trajectory
        pts: List[Tuple[float, float, float]] = []
        t = traj.start
        while t <= traj.end + 1e-9:
            x, y = traj.position(t)
            pts.append((x, y, self._project(world, carla, x, y)))
            t += step / max(1.0, math.hypot(*traj.velocity(t)) or 1.0)
        agent.ground = pts

spawn

spawn() -> None

Choose the blueprint of every agent; actors appear when their window starts.

Actors are spawned on their first active tick and destroyed after their last one. Parking inactive actors under the map is not an option: walkers keep falling with physics disabled and the engine deletes them at the kill height.

Source code in bikeped/sim/actors.py
def spawn(self) -> None:
    """Choose the blueprint of every agent; actors appear when their window starts.

    Actors are spawned on their first active tick and destroyed after their
    last one. Parking inactive actors under the map is not an option: walkers
    keep falling with physics disabled and the engine deletes them at the
    kill height.
    """
    library = self.session.world.get_blueprint_library()
    for agent in self.agents:
        bp = None
        if agent.kind == "bike" and self.caps.has_rideable:
            bp = choose_rider(library, agent.model, self.rng)
        if bp is None:
            walker_model = agent.model if agent.model in WALKER_MODELS else "adult"
            bp, wheelchair = choose_walker(library, walker_model, self.rng)
            if wheelchair and bp.has_attribute("use_wheelchair"):
                bp.set_attribute("use_wheelchair", "true")
            if bp.has_attribute("is_invincible"):
                bp.set_attribute("is_invincible", "false")
            agent.walker = True
            if agent.kind == "bike":
                agent.model = f"walker-proxy:{agent.model}"
        else:
            agent.walker = False
        if bp.has_attribute("role_name"):
            bp.set_attribute("role_name", f"scripted_{agent.trajectory.id}")
        agent.bp, agent.blueprint = bp, bp.id

drive_tick

drive_tick(t: float) -> None

Command every actor for the step that ends at t.

Source code in bikeped/sim/actors.py
def drive_tick(self, t: float) -> None:
    """Command every actor for the step that ends at ``t``."""
    self.time = t
    for agent in self.agents:
        traj = agent.trajectory
        active = traj.active(t)
        if not active:
            if agent.visible:
                self._hide(agent)
            continue
        if not agent.visible:
            self._show(agent, t)
            continue
        mode = self.walker_drive if agent.walker else self.drive
        if mode == "kinematic":
            self._drive_kinematic(agent, t)
        else:
            self._drive_velocity(agent, t)

states

states(t: float, scripted: Dict[int, Dict[str, Any]]) -> List[AgentState]

AgentState of every agent active at t from the actual actor pose.

Source code in bikeped/sim/actors.py
def states(self, t: float, scripted: Dict[int, Dict[str, Any]]) -> List[AgentState]:
    """``AgentState`` of every agent active at ``t`` from the actual actor pose."""
    import decision_testbench as tb

    out: List[AgentState] = []
    for agent in self.agents:
        pos = scripted.get(agent.trajectory.id)
        if pos is None or agent.actor is None:
            continue
        tf = agent.actor.get_transform()
        bb = agent.actor.bounding_box
        ext = (float(bb.extent.x), float(bb.extent.y), float(bb.extent.z))
        centre = tf.transform(bb.location)
        loc = (float(centre.x), float(centre.y), float(centre.z))
        if agent.kind == "bike" and not agent.walker and min(ext[0], ext[2]) < 0.3:
            obj = tb.OBJECT_TYPES["cyclist"]
            ext = (obj.length / 2.0, obj.width / 2.0, obj.height / 2.0)
            gz = self.ground_z(agent, pos["x"], pos["y"])
            loc = (float(tf.location.x), float(tf.location.y), gz + obj.height / 2.0)
        rot = (float(tf.rotation.pitch), float(tf.rotation.yaw), float(tf.rotation.roll))
        sx, sy = self.site.to_scenario(tf.location.x, tf.location.y)
        vx, vy = agent.trajectory.velocity(t)
        out.append(AgentState(
            id=agent.trajectory.id, kind=agent.kind, sx=float(pos["x"]), sy=float(pos["y"]),
            heading=float(pos["heading"]), speed=math.hypot(vx, vy), location=loc,
            rotation=rot, extent=ext, model=agent.model, blueprint=agent.blueprint,
            track_error_m=math.hypot(sx - pos["x"], sy - pos["y"]),
            instance_id=int(agent.actor.id)))
    return out

choose_walker

choose_walker(library: Any, model: str, rng: Random) -> Tuple[Any, bool]

Pick a walker blueprint for a hint; returns (blueprint, use_wheelchair).

Source code in bikeped/sim/actors.py
def choose_walker(library: Any, model: str, rng: random.Random) -> Tuple[Any, bool]:
    """Pick a walker blueprint for a hint; returns (blueprint, use_wheelchair)."""
    walkers = sorted(library.filter("walker.pedestrian.*"), key=lambda b: b.id)
    if not walkers:
        raise RuntimeError("the server has no walker blueprints")
    adults = [b for b in walkers if _attr(b, "age") not in ("child",)]
    pool = adults or walkers
    wheelchair = False
    if model == "child":
        pool = [b for b in walkers if _attr(b, "age") == "child"] or pool
    elif model == "wheelchair":
        capable = [b for b in walkers if _attr(b, "can_use_wheelchair") == "true"]
        if capable:
            pool, wheelchair = capable, True
    return rng.choice(pool), wheelchair

choose_rider

choose_rider(library: Any, model: str, rng: Random) -> Optional[Any]

Pick a two-wheeler blueprint for a hint, or None when the engine has none.

Source code in bikeped/sim/actors.py
def choose_rider(library: Any, model: str, rng: random.Random) -> Optional[Any]:
    """Pick a two-wheeler blueprint for a hint, or ``None`` when the engine has none."""
    rideable = sorted((b for b in library.filter("vehicle.*") if is_two_wheeler(b)),
                      key=lambda b: b.id)
    if not rideable:
        return None
    by_id = {b.id: b for b in rideable}
    for bp_id in RIDER_BLUEPRINTS.get(model or "bicycle", RIDER_BLUEPRINTS["bicycle"]):
        if bp_id in by_id:
            return by_id[bp_id]
    bicycles = [b for b in rideable if _attr(b, "base_type") == "bicycle"]
    return rng.choice(bicycles or rideable)

environment

Weather and lighting presets.

Each preset is a plain dict of carla.WeatherParameters fields. Fields a server does not know are skipped, so the same table serves CARLA 0.9.x and 0.10.0. The sun azimuth is not stored as an absolute angle: sun_offset_deg is measured from the pole camera's viewing direction (CARLA gives the azimuth the light travels from, hence the negative offsets), so the pole view is lit from behind or from the side whatever the orientation of the crosswalk.

camera_overrides

camera_overrides(preset: str) -> Dict[str, float]

Camera attributes a preset needs, such as extra exposure for night scenes.

Source code in bikeped/sim/environment.py
def camera_overrides(preset: str) -> Dict[str, float]:
    """Camera attributes a preset needs, such as extra exposure for night scenes."""
    value = WEATHER_PRESETS[preset].get("exposure_compensation")
    return {} if value is None else {"exposure_compensation": float(value)}

settle_ticks

settle_ticks(preset: str, fps: int) -> int

Ticks to run after a weather change so auto exposure and wet surfaces settle.

Source code in bikeped/sim/environment.py
def settle_ticks(preset: str, fps: int) -> int:
    """Ticks to run after a weather change so auto exposure and wet surfaces settle."""
    preset_values = WEATHER_PRESETS[preset]
    seconds = 2.0 + (1.5 if preset_values.get("wetness", 0.0) > 30.0 or preset_values["night"]
                     else 0.0)
    return int(round(seconds * fps))

weather_values

weather_values(preset: str, view_yaw_deg: float) -> Dict[str, float]

WeatherParameters fields for preset with an absolute sun azimuth.

Source code in bikeped/sim/environment.py
def weather_values(preset: str, view_yaw_deg: float) -> Dict[str, float]:
    """``WeatherParameters`` fields for ``preset`` with an absolute sun azimuth."""
    if preset not in WEATHER_PRESETS:
        raise KeyError(f"unknown weather preset '{preset}', choose from "
                       f"{', '.join(WEATHER_PRESETS)}")
    values = {k: float(v) for k, v in WEATHER_PRESETS[preset].items() if k not in _META_KEYS}
    values["sun_azimuth_angle"] = (view_yaw_deg + WEATHER_PRESETS[preset]["sun_offset_deg"]) % 360.0
    return values

apply_weather

apply_weather(world: Any, preset: str, view_yaw_deg: float) -> Dict[str, float]

Apply a preset field by field; returns the fields that were set.

Source code in bikeped/sim/environment.py
def apply_weather(world: Any, preset: str, view_yaw_deg: float) -> Dict[str, float]:
    """Apply a preset field by field; returns the fields that were set."""
    weather = world.get_weather()
    applied: Dict[str, float] = {}
    for key, value in weather_values(preset, view_yaw_deg).items():
        if hasattr(weather, key):
            setattr(weather, key, value)
            applied[key] = value
    world.set_weather(weather)
    set_street_lights(world, is_night(preset))
    return applied

set_street_lights

set_street_lights(world: Any, on: bool) -> bool

Switch street lights through the light manager when the server has one.

Source code in bikeped/sim/environment.py
def set_street_lights(world: Any, on: bool) -> bool:
    """Switch street lights through the light manager when the server has one."""
    if not hasattr(world, "get_lightmanager"):
        return False
    try:
        carla = carla_module()
        manager = world.get_lightmanager()
        group = getattr(getattr(carla, "LightGroup", None), "Street", None)
        lights = manager.get_all_lights(group) if group is not None else manager.get_all_lights()
        if on:
            manager.turn_on(lights)
        else:
            manager.turn_off(lights)
        return True
    except (RuntimeError, AttributeError, TypeError) as exc:
        log.warning("street lights unavailable: %s", exc)
        return False

ambient

Background traffic and pedestrians around a scripted scenario.

Ambient actors make the rendered street look inhabited without getting in the way of the scripted agents. The scripted corridor is the bounding box of all scripted paths in the scenario frame plus a margin. Vehicles are spawned outside it, drive under the Traffic Manager and are stopped when they approach it. Walkers are spawned on the navigation mesh outside it, do not cross the road, and are sent back to where they started if they wander in.

The corridor helpers are plain numpy and need no simulator.

Ambient

Ambient(
    session: Any,
    site: Any,
    scenario: Dict[str, Any],
    level: str = "none",
    seed: int = 0,
    night: bool = False,
)

Ambient vehicles and walkers for one render.

Every actor is spawned through the session, which tracks and destroys it. Spawning is best effort: a map without spawn points, a missing Traffic Manager or a full navigation mesh leaves the scene with fewer ambient actors and a log message, and never stops the render.

Parameters:

Name Type Description Default
session Any

the running Session.

required
site Any

the Site that maps between the world and the scenario frame.

required
scenario Dict[str, Any]

the scripted scenario; its paths define the corridor.

required
level str

a key of :data:AMBIENT_LEVELS.

'none'
seed int

seed of the blueprint, colour and spawn point choices.

0
night bool

switch vehicle lights on.

False
Source code in bikeped/sim/ambient.py
def __init__(
    self,
    session: Any,
    site: Any,
    scenario: Dict[str, Any],
    level: str = "none",
    seed: int = 0,
    night: bool = False,
) -> None:
    if level not in AMBIENT_LEVELS:
        raise ValueError(
            f"unknown ambient level {level!r}, expected one of: {', '.join(AMBIENT_LEVELS)}"
        )
    self.session = session
    self.site = site
    self.level = level
    self.seed = int(seed)
    self.night = night
    self.bounds: Bounds = corridor_bounds(scenario)
    self.hold_bounds: Bounds = expand_bounds(self.bounds, HOLD_BACK_M)
    self._rng = random.Random(self.seed)
    self._tm: Optional[Any] = None
    self._tm_port = DEFAULT_TM_PORT
    self._vehicles: List[Any] = []
    self._walkers: List[_Walker] = []
    self._held: Set[int] = set()
    self._braked: Set[int] = set()
    self._ticks = 0
    self._controllers_started = False

counts property

counts: Dict[str, int]

Number of ambient vehicles and walkers alive, and of vehicles held back.

spawn

spawn() -> None

Spawn the vehicles and walkers of the chosen level.

Source code in bikeped/sim/ambient.py
def spawn(self) -> None:
    """Spawn the vehicles and walkers of the chosen level."""
    budget = AMBIENT_LEVELS[self.level]
    if budget["vehicles"]:
        try:
            self._spawn_vehicles(budget["vehicles"])
        except Exception:
            logger.warning("ambient vehicles could not be spawned", exc_info=True)
    if budget["walkers"]:
        try:
            self._spawn_walkers(budget["walkers"])
        except Exception:
            logger.warning("ambient walkers could not be spawned", exc_info=True)
    if budget["vehicles"] or budget["walkers"]:
        logger.info("ambient %s: %d vehicles, %d walkers", self.level,
                    len(self._vehicles), len(self._walkers))

tick

tick() -> None

Update the ambient actors; call once before every world.tick().

Source code in bikeped/sim/ambient.py
def tick(self) -> None:
    """Update the ambient actors; call once before every ``world.tick()``."""
    # A walker controller can only be started once the server has ticked
    # after the spawn, so the first call leaves the controllers alone.
    if self._ticks >= 1 and not self._controllers_started:
        self._start_controllers()
    self._ticks += 1
    self._hold_vehicles()
    self._turn_back_walkers()

stop

stop() -> None

Stop the walker controllers. The session destroys the actors.

Source code in bikeped/sim/ambient.py
def stop(self) -> None:
    """Stop the walker controllers. The session destroys the actors."""
    if self._controllers_started:
        for walker in self._walkers:
            if walker.controller is None:
                continue
            try:
                walker.controller.stop()
            except RuntimeError:
                logger.debug("walker controller was already gone", exc_info=True)
    self._vehicles.clear()
    self._walkers.clear()
    self._held.clear()
    self._braked.clear()

corridor_bounds

corridor_bounds(scenario: Dict[str, Any], margin: float = 6.0) -> Bounds

Bounding box of every scripted path point, grown by margin metres.

Returns (xmin, ymin, xmax, ymax) in the scenario frame.

Source code in bikeped/sim/ambient.py
def corridor_bounds(scenario: Dict[str, Any], margin: float = 6.0) -> Bounds:
    """Bounding box of every scripted path point, grown by ``margin`` metres.

    Returns ``(xmin, ymin, xmax, ymax)`` in the scenario frame.
    """
    points = np.array(
        [(p["x"], p["y"]) for agent in scenario.get("agents", []) for p in agent["path"]],
        dtype=np.float64,
    )
    if points.size == 0:
        raise ValueError("the scenario has no path points to build a corridor from")
    low = points.min(axis=0) - margin
    high = points.max(axis=0) + margin
    return float(low[0]), float(low[1]), float(high[0]), float(high[1])

expand_bounds

expand_bounds(bounds: Bounds, distance: float) -> Bounds

Grow bounds by distance metres on every side.

Source code in bikeped/sim/ambient.py
def expand_bounds(bounds: Bounds, distance: float) -> Bounds:
    """Grow ``bounds`` by ``distance`` metres on every side."""
    xmin, ymin, xmax, ymax = bounds
    return xmin - distance, ymin - distance, xmax + distance, ymax + distance

in_corridor

in_corridor(x: float, y: float, bounds: Bounds) -> bool

Whether the scenario-frame point (x, y) lies inside bounds.

Source code in bikeped/sim/ambient.py
def in_corridor(x: float, y: float, bounds: Bounds) -> bool:
    """Whether the scenario-frame point ``(x, y)`` lies inside ``bounds``."""
    xmin, ymin, xmax, ymax = bounds
    return bool(xmin <= x <= xmax and ymin <= y <= ymax)

distance_to_bounds

distance_to_bounds(x: float, y: float, bounds: Bounds) -> float

Distance in metres from (x, y) to bounds, zero inside.

Source code in bikeped/sim/ambient.py
def distance_to_bounds(x: float, y: float, bounds: Bounds) -> float:
    """Distance in metres from ``(x, y)`` to ``bounds``, zero inside."""
    xmin, ymin, xmax, ymax = bounds
    dx = max(xmin - x, 0.0, x - xmax)
    dy = max(ymin - y, 0.0, y - ymax)
    return float(np.hypot(dx, dy))

rigs

Camera rigs: named views, their camera models and the CARLA sensors behind them.

A rig yields one :class:~bikeped.sim.types.CameraModel per tick (world frame, see :mod:bikeped.sim.types) and owns the sensors that render it. Static rigs (pole, street, bev, fisheye) never move; chase and orbit move their sensor before every tick.

Views:

pole Pinhole camera on the pole at the testbench camera position (12 ft), aimed so the crosswalk and every scripted path fit in the frame. fisheye The deployed fisheye lens, rendered as cube faces and resampled with :class:~bikeped.sim.fisheye.FisheyeRemap. chase Follows the first cyclist (or pedestrian) from behind and above. orbit Slow eye-level arc around the conflict point with a longer lens. street Static eye-level tripod on the kerb. bev Straight down from 60 m with a narrow lens, image up along scenario +y.

Rig

Rig(site: Site, scenario: Dict[str, Any], quality: str, ground_truth: bool = False)

One named view: the camera model(s) per tick and their sensors.

Source code in bikeped/sim/rigs.py
def __init__(self, site: Site, scenario: Dict[str, Any], quality: str,
             ground_truth: bool = False) -> None:
    self.post_overrides: Dict[str, Any] = {}
    self.site = site
    self.scenario = scenario
    self.quality = QUALITY_PRESETS[quality]
    self.ground_truth = ground_truth
    self.sensors: Dict[str, Any] = {}
    self.specs: List[SensorSpec] = []

sensor_models

sensor_models(t: float) -> Dict[str, CameraModel]

Models of the sensors to render, keyed by sensor key.

Source code in bikeped/sim/rigs.py
def sensor_models(self, t: float) -> Dict[str, CameraModel]:
    """Models of the sensors to render, keyed by sensor key."""
    return {self.name: self.camera(t)}

move

move(session: Any, t: float) -> None

Place moving sensors for the frame rendered at t.

Source code in bikeped/sim/rigs.py
def move(self, session: Any, t: float) -> None:
    """Place moving sensors for the frame rendered at ``t``."""
    if not self.moving:
        return
    models = self.sensor_models(t)
    for key, actor in self.sensors.items():
        base = key.split("/")[0] if key.split("/")[-1] in ("depth", "instance") else key
        actor.set_transform(self.transform(session.carla, models[base]))

frame

frame(data: Dict[str, Any], t: float) -> Tuple[np.ndarray, Dict[str, np.ndarray]]

BGR image of this view and its auxiliary buffers from collected sensor data.

Source code in bikeped/sim/rigs.py
def frame(self, data: Dict[str, Any], t: float) -> Tuple[np.ndarray, Dict[str, np.ndarray]]:
    """BGR image of this view and its auxiliary buffers from collected sensor data."""
    aux = {k: v for k, v in data.items()
           if k.startswith(self.name + "/") and k.split("/")[-1] in ("depth", "instance")}
    return data[self.name], aux

FollowRig

FollowRig(
    site: Site,
    scenario: Dict[str, Any],
    quality: str,
    ground_truth: bool = False,
    trajectories: Sequence[Trajectory] = (),
    fps: int = 30,
)

Bases: Rig

Base of the moving views: an exponentially smoothed camera path.

Source code in bikeped/sim/rigs.py
def __init__(self, site: Site, scenario: Dict[str, Any], quality: str,
             ground_truth: bool = False, trajectories: Sequence[Trajectory] = (),
             fps: int = 30) -> None:
    super().__init__(site, scenario, quality, ground_truth)
    bikes = [t for t in trajectories if t.kind == "bike"]
    self.target = (bikes or list(trajectories))[0] if trajectories else None
    self.fps = fps
    self._cache: Dict[int, CameraModel] = {}

rotation_to_basis

rotation_to_basis(
    pitch: float, yaw: float, roll: float = 0.0
) -> Tuple[Tuple[float, ...], ...]

Unreal (pitch, yaw, roll) degrees to world (forward, right, up).

Source code in bikeped/sim/rigs.py
def rotation_to_basis(pitch: float, yaw: float, roll: float = 0.0) -> Tuple[Tuple[float, ...], ...]:
    """Unreal ``(pitch, yaw, roll)`` degrees to world ``(forward, right, up)``."""
    from .projection import carla_rotation_to_basis

    return carla_rotation_to_basis(pitch, yaw, roll)

look_at

look_at(position: Sequence[float], target: Sequence[float]) -> Tuple[float, float]

CARLA (pitch, yaw) in degrees of a level-horizon camera aimed at target.

Source code in bikeped/sim/rigs.py
def look_at(position: Sequence[float], target: Sequence[float]) -> Tuple[float, float]:
    """CARLA ``(pitch, yaw)`` in degrees of a level-horizon camera aimed at ``target``."""
    d = np.asarray(target, dtype=float) - np.asarray(position, dtype=float)
    yaw = math.degrees(math.atan2(d[1], d[0]))
    pitch = math.degrees(math.atan2(d[2], math.hypot(d[0], d[1])))
    return pitch, yaw

pole_framing

pole_framing(
    site: Site, scenario: Dict[str, Any], aspect: float = 16 / 9, reach_m: float = 40.0
) -> Tuple[float, float, float]

Heading (rad, scenario frame), pitch (deg) and horizontal FOV (deg) of the pole view.

The view always contains the crosswalk (scenario x = 5, |y| <= 4) and every path point within reach_m of the pole.

Source code in bikeped/sim/rigs.py
def pole_framing(site: Site, scenario: Dict[str, Any], aspect: float = 16 / 9,
                 reach_m: float = 40.0) -> Tuple[float, float, float]:
    """Heading (rad, scenario frame), pitch (deg) and horizontal FOV (deg) of the pole view.

    The view always contains the crosswalk (scenario ``x = 5``, ``|y| <= 4``) and
    every path point within ``reach_m`` of the pole.
    """
    px, py = site.pole_xy
    pts = scenario_points(scenario)
    pts = np.vstack([pts, [[5.0, -4.0], [5.0, 4.0], [3.0, 0.0], [7.0, 0.0]]])
    d = pts - np.array([px, py])
    dist = np.hypot(d[:, 0], d[:, 1])
    d = d[(dist < reach_m) & (dist > 2.0)]
    centre = math.pi / 2.0
    angles = np.arctan2(d[:, 1], d[:, 0]) - centre
    angles = (angles + math.pi) % (2 * math.pi) - math.pi
    lo, hi = float(angles.min()), float(angles.max())
    heading = centre + (lo + hi) / 2.0
    fov = float(np.clip(math.degrees(hi - lo) + 14.0, 60.0, 100.0))
    near = float(np.percentile(np.hypot(d[:, 0], d[:, 1]), 50))
    pitch = -math.degrees(math.atan2(site.pole_height - 1.0, max(near, 4.0)))
    return heading, pitch, fov

make_rig

make_rig(
    name: str,
    site: Site,
    scenario: Dict[str, Any],
    quality: str,
    *,
    ground_truth: bool = False,
    trajectories: Sequence[Trajectory] = (),
    fps: int = 30,
) -> Rig

Build the rig called name (one of :data:RIG_NAMES).

Source code in bikeped/sim/rigs.py
def make_rig(name: str, site: Site, scenario: Dict[str, Any], quality: str, *,
             ground_truth: bool = False, trajectories: Sequence[Trajectory] = (),
             fps: int = 30) -> Rig:
    """Build the rig called ``name`` (one of :data:`RIG_NAMES`)."""
    if name not in RIG_CLASSES:
        raise KeyError(f"unknown rig '{name}', choose from {', '.join(RIG_NAMES)}")
    cls = RIG_CLASSES[name]
    if issubclass(cls, FollowRig):
        return cls(site, scenario, quality, ground_truth, trajectories, fps)
    return cls(site, scenario, quality, ground_truth)

describe_rigs

describe_rigs() -> List[Tuple[str, str]]

Name and first docstring line of every rig, for presets.

Source code in bikeped/sim/rigs.py
def describe_rigs() -> List[Tuple[str, str]]:
    """Name and first docstring line of every rig, for ``presets``."""
    docs = {
        "pole": "pinhole on the 12 ft pole at the testbench camera position",
        "fisheye": "deployed fisheye lens, cube faces remapped",
        "chase": "smoothed camera behind the first cyclist",
        "orbit": "slow eye-level arc around the conflict point",
        "street": "static eye-level tripod on the kerb",
        "bev": "straight down from 60 m, image up along scenario +y",
    }
    return [(n, docs[n]) for n in RIG_NAMES]

runner

Render one scenario end to end.

:func:render_scenario builds the trajectories, connects a :class:Session, places the site, applies weather, spawns the cast and the camera rigs, warms up, and then for every tick drives the actors, steps the world, collects the frame-synchronised images, reads the simulated agent states, runs the decision pipeline on the scripted positions exactly as the testbench does, attaches the ground-truth danger and hands a :class:FrameRecord to the recorder.

RenderOptions dataclass

RenderOptions(
    host: str = "127.0.0.1",
    port: int = 2000,
    engine: str = "auto",
    map: Optional[str] = None,
    crosswalk: int = 4,
    crosswalks_file: Optional[str] = None,
    rigs: Sequence[str] = ("pole",),
    weather: Optional[str] = None,
    drive: Optional[str] = None,
    ambient: str = "none",
    quality: str = "standard",
    layout: str = "single",
    overlay: bool = True,
    save_frames: str = "none",
    out: str = "renders",
    seed: int = 0,
    fps: int = 30,
    pipeline_config: str = "default",
    cyclist_proxy: Optional[str] = None,
    ground_truth: bool = True,
    warmup_s: float = 1.0,
    max_frames: Optional[int] = None,
    extra: Dict[str, Any] = dict(),
)

Everything render accepts besides the scenario itself.

resolve_scenario

resolve_scenario(ref: str, suite: Optional[str] = None) -> Dict[str, Any]

Look a scenario up by slug, name, suite:index or conformance index.

Source code in bikeped/sim/runner.py
def resolve_scenario(ref: str, suite: Optional[str] = None) -> Dict[str, Any]:
    """Look a scenario up by slug, name, ``suite:index`` or conformance index."""
    try:
        from bikeped.scenarios import get_scenario
    except ImportError:
        get_scenario = None
    if get_scenario is not None:
        key = f"{suite}:{ref}" if suite and ref.isdigit() else ref
        try:
            return get_scenario(key)
        except (KeyError, ValueError, IndexError):
            if not ref.isdigit():
                raise
    import decision_testbench as tb

    if ref.split(":")[-1].isdigit():
        return tb.SCENARIOS[int(ref.split(":")[-1])]
    for sc in tb.SCENARIOS:
        if sc["name"] == ref:
            return sc
    raise KeyError(f"unknown scenario '{ref}'")

danger_for

danger_for(scenario: Dict[str, Any], t: float) -> Optional[Dict[str, Any]]

Ground-truth danger at t as a JSON-friendly dict.

Source code in bikeped/sim/runner.py
def danger_for(scenario: Dict[str, Any], t: float) -> Optional[Dict[str, Any]]:
    """Ground-truth danger at ``t`` as a JSON-friendly dict."""
    try:
        from .groundtruth import danger_record
        return danger_record(scenario, t)
    except ImportError:
        import decision_testbench as tb

        d = tb.compute_ground_truth_danger(scenario, t)
        return {k: (float(v) if isinstance(v, (float, np.floating)) else v)
                for k, v in vars(d).items()}

clear_near_field

clear_near_field(world: Any, carla: Any, cam: Any, radius_m: float = 7.5) -> list

Hide street furniture poles standing right in front of the pole camera.

The camera is mounted on a pole of its own; a lamp post a few metres in front of it would cover a large part of the image. Returns the ids of the disabled objects so they can be enabled again.

Source code in bikeped/sim/runner.py
def clear_near_field(world: Any, carla: Any, cam: Any, radius_m: float = 7.5) -> list:
    """Hide street furniture poles standing right in front of the pole camera.

    The camera is mounted on a pole of its own; a lamp post a few metres in
    front of it would cover a large part of the image. Returns the ids of the
    disabled objects so they can be enabled again.
    """
    label = getattr(getattr(carla, "CityObjectLabel", None), "Poles", None)
    if label is None or not hasattr(world, "get_environment_objects"):
        return []
    px, py, _ = cam.position
    fx, fy = cam.forward[0], cam.forward[1]
    ids = []
    for obj in world.get_environment_objects(label):
        loc = obj.transform.location
        dx, dy = loc.x - px, loc.y - py
        if math.hypot(dx, dy) < radius_m and dx * fx + dy * fy > -0.5:
            ids.append(obj.id)
    if ids:
        world.enable_environment_objects(set(ids), False)
    return ids

render_scenario

render_scenario(
    scenario: Dict[str, Any],
    opts: RenderOptions,
    recorder_factory: Optional[Callable[..., Any]] = None,
) -> Dict[str, Any]

Render scenario on a running server; returns the recorder's summary.

Source code in bikeped/sim/runner.py
def render_scenario(scenario: Dict[str, Any], opts: RenderOptions,
                    recorder_factory: Optional[Callable[..., Any]] = None) -> Dict[str, Any]:
    """Render ``scenario`` on a running server; returns the recorder's summary."""
    import decision_testbench as tb

    fps = opts.fps
    dt = 1.0 / fps
    trajs = trajectories(scenario, fps)
    duration = tb.get_scenario_duration(scenario)
    n_frames = int(math.floor(duration * fps + 1e-9)) + 1
    if opts.max_frames is not None:
        n_frames = min(n_frames, opts.max_frames)
    weather = opts.weather or scenario.get("render", {}).get("weather") or "clear_noon"
    recorder_factory = recorder_factory or default_recorder_factory()

    with Session(opts.host, opts.port, map_name=opts.map, fixed_delta=dt,
                 seed=opts.seed) as session:
        if opts.engine != "auto" and opts.engine != session.engine:
            raise RuntimeError(f"--engine {opts.engine} requested but the server is "
                               f"{session.engine}")
        caps = probe(session.client)
        drive = opts.drive or PROFILES[session.engine].default_drive
        site = _load_site(session, opts)
        cast = Cast(session, site, scenario, trajs, caps, drive=drive, seed=opts.seed,
                    cyclist_proxy=opts.cyclist_proxy, walker_drive=opts.extra.get("walker_drive"))
        cast.sample_ground()
        view_yaw = site.heading_to_yaw(math.pi / 2.0)
        apply_weather(session.world, weather, view_yaw)
        ambient = None
        if opts.ambient != "none":
            from .ambient import Ambient
            ambient = Ambient(session, site, scenario, level=opts.ambient, seed=opts.seed,
                              night=is_night(weather))
            ambient.spawn()
        cast.spawn()
        hub = SensorHub(timeout=30.0)
        rigs = [make_rig(name, site, scenario, opts.quality,
                         ground_truth=opts.ground_truth and name == "pole",
                         trajectories=trajs, fps=fps) for name in opts.rigs]
        for rig in rigs:
            rig.post_overrides = camera_overrides(weather)
            rig.spawn(session, hub, caps)
        if "pole" in opts.rigs:
            pole_cam = next(r for r in rigs if r.name == "pole").camera(0.0)
            session.hidden_objects.update(
                clear_near_field(session.world, session.carla, pole_cam))
        warm = max(int(opts.warmup_s * fps), settle_ticks(weather, fps))
        for _ in range(warm):
            if ambient is not None:
                ambient.tick()
            frame_id = session.tick()
            hub.collect(frame_id)

        run = RunInfo(
            scenario_name=scenario["name"], scenario_id=scenario_slug(scenario),
            suite=str(scenario.get("suite", "conformance")), fps=fps, duration_s=duration,
            engine=session.engine, server_version=caps.server_version,
            client_version=caps.client_version, map=caps.map, crosswalk_id=opts.crosswalk,
            weather=weather, quality=opts.quality, drive=drive, ambient=opts.ambient,
            seed=opts.seed, views=list(opts.rigs), site=site.to_dict(),
            pipeline_config=opts.pipeline_config,
            extra={"cyclist_proxy": opts.cyclist_proxy, **opts.extra})
        recorder = recorder_factory(Path(opts.out), run, views=list(opts.rigs),
                                    overlay=opts.overlay,
                                    layout=opts.layout, save_frames=opts.save_frames,
                                    scenario=scenario)
        pipeline = tb.make_pipeline(tb.PIPELINE_CONFIGS[opts.pipeline_config], fps,
                                    profile=tb.get_braking_profile(scenario))
        cam_x, cam_y = site.pole_xy
        started = time.monotonic()
        try:
            for index in range(n_frames):
                t = index * dt
                cast.drive_tick(t)
                for rig in rigs:
                    rig.move(session, t)
                if ambient is not None:
                    ambient.tick()
                frame_id = session.tick()
                data = hub.collect(frame_id)
                agents_now = scripted_agents(scenario, t)
                state = tb.run_pipeline_on_agents(pipeline, agents_now, t, cam_x=cam_x,
                                                  cam_y=cam_y)
                images: Dict[str, np.ndarray] = {}
                aux: Dict[str, np.ndarray] = {}
                cameras = {}
                for rig in rigs:
                    img, extra_aux = rig.frame(data, t)
                    images[rig.name] = img
                    aux.update(extra_aux)
                    cameras[rig.name] = rig.camera(t)
                frame = FrameRecord(
                    index=index, t=t,
                    agents=cast.states(t, {a["id"]: a for a in agents_now}),
                    state=state, states={opts.pipeline_config: state},
                    alert_pair=_alert_pair(pipeline, state), danger=danger_for(scenario, t),
                    cameras=cameras, images=images, aux=aux)
                recorder.write(frame)
            summary = recorder.close()
        except BaseException:
            try:
                recorder.close()
            except Exception as exc:  # noqa: BLE001
                log.debug("recorder close after failure: %s", exc)
            raise
        finally:
            if ambient is not None:
                ambient.stop()
        summary["seconds"] = round(time.monotonic() - started, 1)
        summary["run"] = run.to_dict()
        return summary

projection

Rotation conventions and camera projection for rendered views.

All world quantities are in the CARLA world frame (metres, left-handed, z up) and rotations are (pitch, yaw, roll) in degrees with the Unreal convention of carla.Rotation. A camera is a :class:~bikeped.sim.types.CameraModel; its camera coordinates are (f, r, u) along forward, right and up.

Pixel conventions follow the source of the intrinsics. Pinhole models use the CARLA convention, principal point at (width / 2, height / 2). Fisheye models use the calibration convention of camera_calibration.json and decision_testbench.FisheyeCamera, in which pixel centres sit at integer coordinates.

carla_rotation_to_basis

carla_rotation_to_basis(pitch: float, yaw: float, roll: float) -> Tuple[Vec, Vec, Vec]

Return the world-frame (forward, right, up) unit vectors of a rotation.

Angles are in degrees. The result equals carla.Transform(rotation=...) .get_forward_vector(), .get_right_vector() and .get_up_vector().

Source code in bikeped/sim/projection.py
def carla_rotation_to_basis(pitch: float, yaw: float, roll: float) -> Tuple[Vec, Vec, Vec]:
    """Return the world-frame ``(forward, right, up)`` unit vectors of a rotation.

    Angles are in degrees. The result equals ``carla.Transform(rotation=...)``
    ``.get_forward_vector()``, ``.get_right_vector()`` and ``.get_up_vector()``.
    """
    p, y, r = np.radians([pitch, yaw, roll])
    cp, sp = math.cos(p), math.sin(p)
    cy, sy = math.cos(y), math.sin(y)
    cr, sr = math.cos(r), math.sin(r)
    forward = (cp * cy, cp * sy, sp)
    right = (sr * sp * cy - cr * sy, sr * sp * sy + cr * cy, -sr * cp)
    up = (-(cr * sp * cy + sr * sy), cy * sr - cr * sp * sy, cr * cp)
    return forward, right, up

basis_to_carla_rotation

basis_to_carla_rotation(
    forward: Sequence[float], right: Sequence[float], up: Sequence[float]
) -> Tuple[float, float, float]

Return (pitch, yaw, roll) in degrees for an orthonormal camera basis.

Inverse of :func:carla_rotation_to_basis. At pitch of plus or minus 90 degrees only yaw + roll (or yaw - roll) is defined; roll is then set to zero and the whole in-plane rotation goes into yaw.

Source code in bikeped/sim/projection.py
def basis_to_carla_rotation(forward: Sequence[float], right: Sequence[float],
                            up: Sequence[float]) -> Tuple[float, float, float]:
    """Return ``(pitch, yaw, roll)`` in degrees for an orthonormal camera basis.

    Inverse of :func:`carla_rotation_to_basis`. At pitch of plus or minus 90
    degrees only ``yaw + roll`` (or ``yaw - roll``) is defined; roll is then set
    to zero and the whole in-plane rotation goes into yaw.
    """
    f = np.asarray(forward, dtype=np.float64)
    r = np.asarray(right, dtype=np.float64)
    u = np.asarray(up, dtype=np.float64)
    pitch = math.degrees(math.asin(float(np.clip(f[2], -1.0, 1.0))))
    if math.hypot(f[0], f[1]) > 1e-9:
        yaw = math.degrees(math.atan2(f[1], f[0]))
        roll = math.degrees(math.atan2(-r[2], u[2]))
    else:
        # With roll = 0 the right vector is (-sin yaw, cos yaw, 0) at any pitch.
        yaw = math.degrees(math.atan2(-r[0], r[1]))
        roll = 0.0
    return pitch, yaw, roll

box_corners_world

box_corners_world(
    location: Sequence[float], rotation: Sequence[float], extent: Sequence[float]
) -> np.ndarray

Return the 8 world-frame corners of an oriented box as an (8, 3) array.

location is the box centre, rotation (pitch, yaw, roll) in degrees, extent the half sizes along the box forward, right and up axes. Corner 4*ix + 2*iy + iz lies on the minus (0) or plus (1) side of each axis, so corners 0, 2, 6, 4 form the bottom face.

Source code in bikeped/sim/projection.py
def box_corners_world(location: Sequence[float], rotation: Sequence[float],
                      extent: Sequence[float]) -> np.ndarray:
    """Return the 8 world-frame corners of an oriented box as an ``(8, 3)`` array.

    ``location`` is the box centre, ``rotation`` ``(pitch, yaw, roll)`` in
    degrees, ``extent`` the half sizes along the box forward, right and up axes.
    Corner ``4*ix + 2*iy + iz`` lies on the minus (0) or plus (1) side of each
    axis, so corners 0, 2, 6, 4 form the bottom face.
    """
    basis = np.array(carla_rotation_to_basis(*rotation), dtype=np.float64)
    ext = np.asarray(extent, dtype=np.float64)
    signs = np.array([[1 if i & 4 else -1, 1 if i & 2 else -1, 1 if i & 1 else -1]
                      for i in range(8)], dtype=np.float64)
    return np.asarray(location, dtype=np.float64) + (signs * ext) @ basis

world_to_camera

world_to_camera(points_w: ndarray, cam: CameraModel) -> np.ndarray

Return camera coordinates (f, r, u) in metres of (N, 3) world points.

Source code in bikeped/sim/projection.py
def world_to_camera(points_w: np.ndarray, cam: CameraModel) -> np.ndarray:
    """Return camera coordinates ``(f, r, u)`` in metres of ``(N, 3)`` world points."""
    pts = np.asarray(points_w, dtype=np.float64).reshape(-1, 3)
    return (pts - np.asarray(cam.position, dtype=np.float64)) @ cam.basis().T

fisheye_focal

fisheye_focal(cam: CameraModel) -> float

Equidistant focal length in pixels per radian of a fisheye model.

Source code in bikeped/sim/projection.py
def fisheye_focal(cam: CameraModel) -> float:
    """Equidistant focal length in pixels per radian of a fisheye model."""
    if cam.f_px_per_rad is not None:
        return float(cam.f_px_per_rad)
    if cam.circle_radius_px is None:
        raise ValueError(f"fisheye camera {cam.name!r} needs f_px_per_rad or circle_radius_px")
    return float(cam.circle_radius_px) / math.radians(cam.fov_deg / 2.0)

camera_to_image

camera_to_image(
    fru: ndarray, cam: CameraModel
) -> Tuple[np.ndarray, np.ndarray, np.ndarray]

Project camera coordinates (N, 3) to pixels; see :func:project_points.

Source code in bikeped/sim/projection.py
def camera_to_image(fru: np.ndarray, cam: CameraModel) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
    """Project camera coordinates ``(N, 3)`` to pixels; see :func:`project_points`."""
    fru = np.asarray(fru, dtype=np.float64).reshape(-1, 3)
    f, r, u = fru[:, 0], fru[:, 1], fru[:, 2]
    cx, cy = cam.principal_point
    if cam.kind == "pinhole":
        valid = f > NEAR_PLANE_M
        safe = np.where(valid, f, 1.0)
        fpx = cam.focal_px
        uv = np.stack([cx + fpx * r / safe, cy - fpx * u / safe], axis=1)
        return uv, valid, f.copy()
    if cam.kind == "fisheye":
        rng = np.linalg.norm(fru, axis=1)
        lateral = np.hypot(r, u)
        theta = np.arctan2(lateral, f)
        rad = _fisheye_radius(theta, cam)
        scale = np.where(lateral > 1e-12, rad / np.maximum(lateral, 1e-12), 0.0)
        uv = np.stack([cx + scale * r, cy - scale * u], axis=1)
        valid = (theta <= math.radians(cam.fov_deg / 2.0)) & (rng > 1e-9)
        return uv, valid, rng
    raise ValueError(f"unknown camera kind {cam.kind!r}")

project_points

project_points(
    points_w: ndarray, cam: CameraModel
) -> Tuple[np.ndarray, np.ndarray, np.ndarray]

Project world points to pixels.

Returns uv (N, 2), valid (N,) and depth (N,). For a pinhole camera a point is valid in front of the near plane and depth is the distance along the optical axis, as in a CARLA depth image. For a fisheye a point is valid inside the field of view and depth is the Euclidean range. Validity does not test the image bounds.

Source code in bikeped/sim/projection.py
def project_points(points_w: np.ndarray,
                   cam: CameraModel) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
    """Project world points to pixels.

    Returns ``uv`` ``(N, 2)``, ``valid`` ``(N,)`` and ``depth`` ``(N,)``.
    For a pinhole camera a point is valid in front of the near plane and depth
    is the distance along the optical axis, as in a CARLA depth image. For a
    fisheye a point is valid inside the field of view and depth is the
    Euclidean range. Validity does not test the image bounds.
    """
    return camera_to_image(world_to_camera(points_w, cam), cam)

project_polyline

project_polyline(
    points_w: ndarray,
    cam: CameraModel,
    *,
    max_step_px: float = 6.0,
    closed: bool = False,
) -> List[np.ndarray]

Project a world polyline to a list of image polylines (M, 2), M >= 2.

Pinhole segments are clipped exactly at the near plane and stay straight. Fisheye segments are resampled so that consecutive samples are at most about max_step_px apart, since straight lines map to curves, and the polyline is split where it leaves the field of view.

Source code in bikeped/sim/projection.py
def project_polyline(points_w: np.ndarray, cam: CameraModel, *, max_step_px: float = 6.0,
                     closed: bool = False) -> List[np.ndarray]:
    """Project a world polyline to a list of image polylines ``(M, 2)``, ``M >= 2``.

    Pinhole segments are clipped exactly at the near plane and stay straight.
    Fisheye segments are resampled so that consecutive samples are at most
    about ``max_step_px`` apart, since straight lines map to curves, and the
    polyline is split where it leaves the field of view.
    """
    pts = np.asarray(points_w, dtype=np.float64).reshape(-1, 3)
    if closed and len(pts) > 2:
        pts = np.vstack([pts, pts[:1]])
    if len(pts) < 2:
        return []
    fru = world_to_camera(pts, cam)
    pieces: List[np.ndarray] = []
    current: List[np.ndarray] = []

    def flush() -> None:
        if len(current) >= 2:
            pieces.append(np.vstack(current))
        current.clear()

    for a, b in zip(fru[:-1], fru[1:]):
        if cam.kind == "pinhole":
            clipped = _clip_near(a, b)
            if clipped is None:
                flush()
                continue
            ca, cb = clipped
            uv, _, _ = camera_to_image(np.vstack([ca, cb]), cam)
            if current and ca is not a:
                flush()
            if not current:
                current.append(uv[:1])
            current.append(uv[1:])
            if cb is not b:
                flush()
        else:
            n = _segment_samples(a, b, cam, max_step_px)
            s = np.linspace(0.0, 1.0, n + 1)[:, None]
            uv, valid, _ = camera_to_image(a + (b - a) * s, cam)
            start = 1 if current else 0
            for k in range(start, len(uv)):
                if valid[k]:
                    current.append(uv[k:k + 1])
                else:
                    flush()
    flush()
    return pieces

project_box

project_box(agent: AgentState, cam: CameraModel) -> Optional[Dict[str, object]]

Project an agent's oriented 3D box into a view.

Returns None when no part of the box is in front of the camera or inside the image. Otherwise a dict with bbox [x1, y1, x2, y2] clipped to the image, corners_image (8, 2) with NaN rows for corners behind the near plane or outside the field of view, truncation (fraction of the unclipped box area outside the image) and distance_m (camera to box centre). Pinhole edges are clipped against the near plane one by one, so a box that straddles the camera keeps its true extent; fisheye edges are sampled densely because they curve.

Source code in bikeped/sim/projection.py
def project_box(agent: AgentState, cam: CameraModel) -> Optional[Dict[str, object]]:
    """Project an agent's oriented 3D box into a view.

    Returns ``None`` when no part of the box is in front of the camera or
    inside the image. Otherwise a dict with ``bbox`` ``[x1, y1, x2, y2]``
    clipped to the image, ``corners_image`` ``(8, 2)`` with NaN rows for
    corners behind the near plane or outside the field of view,
    ``truncation`` (fraction of the unclipped box area outside the image) and
    ``distance_m`` (camera to box centre). Pinhole edges are clipped against
    the near plane one by one, so a box that straddles the camera keeps its
    true extent; fisheye edges are sampled densely because they curve.
    """
    corners = box_corners_world(agent.location, agent.rotation, agent.extent)
    uv, valid, _ = project_points(corners, cam)
    corners_image = np.where(valid[:, None], uv, np.nan)
    samples: List[np.ndarray] = []
    for i, j in BOX_EDGES:
        for piece in project_polyline(corners[[i, j]], cam, max_step_px=4.0):
            samples.append(piece)
    if not samples:
        return None
    allpts = np.vstack(samples)
    x1, y1 = allpts.min(axis=0)
    x2, y2 = allpts.max(axis=0)
    cx1, cy1 = max(x1, 0.0), max(y1, 0.0)
    cx2, cy2 = min(x2, float(cam.width)), min(y2, float(cam.height))
    if cx2 <= cx1 or cy2 <= cy1:
        return None
    full = max((x2 - x1) * (y2 - y1), 1e-12)
    inside = (cx2 - cx1) * (cy2 - cy1)
    distance = float(np.linalg.norm(np.asarray(agent.location) - np.asarray(cam.position)))
    return {
        "bbox": [float(cx1), float(cy1), float(cx2), float(cy2)],
        "corners_image": corners_image,
        "truncation": float(np.clip(1.0 - inside / full, 0.0, 1.0)),
        "distance_m": distance,
    }

pixel_rays

pixel_rays(uv: ndarray, cam: CameraModel) -> Tuple[np.ndarray, np.ndarray]

Return world-frame unit ray directions (N, 3) and a validity mask for pixels.

Source code in bikeped/sim/projection.py
def pixel_rays(uv: np.ndarray, cam: CameraModel) -> Tuple[np.ndarray, np.ndarray]:
    """Return world-frame unit ray directions ``(N, 3)`` and a validity mask for pixels."""
    pix = np.asarray(uv, dtype=np.float64).reshape(-1, 2)
    cx, cy = cam.principal_point
    du, dv = pix[:, 0] - cx, pix[:, 1] - cy
    if cam.kind == "pinhole":
        fpx = cam.focal_px
        fru = np.stack([np.ones(len(pix)), du / fpx, -dv / fpx], axis=1)
        valid = np.ones(len(pix), dtype=bool)
    elif cam.kind == "fisheye":
        rad = np.hypot(du, dv)
        theta = _fisheye_theta(rad, cam)
        phi = np.arctan2(dv, du)
        st = np.sin(theta)
        fru = np.stack([np.cos(theta), st * np.cos(phi), -st * np.sin(phi)], axis=1)
        valid = theta <= math.radians(cam.fov_deg / 2.0)
    else:
        raise ValueError(f"unknown camera kind {cam.kind!r}")
    rays = fru @ cam.basis()
    rays /= np.linalg.norm(rays, axis=1, keepdims=True)
    return rays, valid

pixel_to_ground

pixel_to_ground(
    uv: ndarray, cam: CameraModel, ground_z: float = 0.0
) -> Tuple[np.ndarray, np.ndarray]

Intersect pixel rays with the horizontal plane z = ground_z.

Returns world points (N, 3) (NaN where invalid) and valid (N,), false for rays outside the field of view or not pointing at the plane.

Source code in bikeped/sim/projection.py
def pixel_to_ground(uv: np.ndarray, cam: CameraModel,
                    ground_z: float = 0.0) -> Tuple[np.ndarray, np.ndarray]:
    """Intersect pixel rays with the horizontal plane ``z = ground_z``.

    Returns world points ``(N, 3)`` (NaN where invalid) and ``valid`` ``(N,)``,
    false for rays outside the field of view or not pointing at the plane.
    """
    rays, valid = pixel_rays(uv, cam)
    pos = np.asarray(cam.position, dtype=np.float64)
    rz = rays[:, 2]
    with np.errstate(divide="ignore", invalid="ignore"):
        t = (ground_z - pos[2]) / rz
    valid = valid & (np.abs(rz) > 1e-9) & (t > 0)
    pts = pos + rays * np.where(valid, t, 0.0)[:, None]
    pts[~valid] = np.nan
    return pts, valid

fisheye

The deployed fisheye lens as a sim camera, and cube-face to fisheye remapping.

CARLA has no equidistant fisheye sensor, so the fisheye view is rendered as a set of 90 degree pinhole cameras sharing one position (the faces of a cube around the fisheye's optical axis) and resampled into the fisheye image with a lookup table built once by :class:FisheyeRemap.

:func:deployed_fisheye_model reproduces the lens of the field deployment. The calibration describes a 3500 x 3500 frame centred on the image circle; the physical sensor is 3840 x 2160, which is that frame padded to 3840 x 3840 and cropped to the middle 2160 rows, so the top and bottom of the circle are cut (rows 670 to 2829 of the 3500 px frame are on the sensor). The sim camera renders the sensor image, so its principal point is the calibrated one shifted by (+170, -670) pixels. With the same pose it agrees with decision_testbench.FisheyeCamera.world_to_pixel after that shift.

FisheyeRemap

FisheyeRemap(
    model: CameraModel,
    face_size: int = 1024,
    supersample: int = 1,
    interpolation: int = cv2.INTER_LINEAR,
    feather_px: float = 1.5,
)

Lookup table from cube-face pinhole images to a fisheye image.

Every face is a square face_size pinhole camera with a 90 degree field of view at the fisheye position. Faces are oriented relative to the fisheye's optical axis (front, right, back, left, up, down); :attr:faces lists only those that receive pixels inside the image circle and on the sensor. supersample renders the lookup at an integer multiple of the output resolution and averages down, which anti-aliases fine texture near the rim of the circle.

Source code in bikeped/sim/fisheye.py
def __init__(self, model: CameraModel, face_size: int = 1024, supersample: int = 1,
             interpolation: int = cv2.INTER_LINEAR, feather_px: float = 1.5) -> None:
    if model.kind != "fisheye":
        raise ValueError("FisheyeRemap needs a fisheye CameraModel")
    self.model = model
    self.face_size = int(face_size)
    self.supersample = max(1, int(supersample))
    self.interpolation = interpolation
    gx, gy = _output_grid(model, self.supersample)
    shape = gx.shape
    rays, valid = pixel_rays(np.stack([gx.ravel(), gy.ravel()], axis=1), model)
    fru = rays @ model.basis().T
    signed = np.concatenate([fru, -fru], axis=1)
    choice = np.argmax(signed, axis=1)
    names = {(0, 1): "front", (1, 1): "right", (2, 1): "up",
             (0, -1): "back", (1, -1): "left", (2, -1): "down"}
    self._lookup: Dict[str, Tuple[np.ndarray, np.ndarray, np.ndarray]] = {}
    half = self.face_size / 2.0
    for k in range(6):
        face = names[(k % 3, 1 if k < 3 else -1)]
        idx = np.flatnonzero(valid & (choice == k))
        if idx.size == 0:
            continue
        axes = _FACE_AXES[face]
        comp = [fru[idx, a] * s for a, s in axes]
        f = np.maximum(comp[0], 1e-9)
        # Face cameras follow the CARLA pinhole convention: principal point
        # at the image centre and pixel i covering [i, i + 1).
        mx = half + half * comp[1] / f - 0.5
        my = half - half * comp[2] / f - 0.5
        self._lookup[face] = (idx, _as_map(mx), _as_map(my))
    order = ("down", "front", "right", "back", "left", "up")
    self.faces: Tuple[str, ...] = tuple(f for f in order if f in self._lookup)
    self._shape = shape
    cx, cy = model.principal_point
    ox, oy = _output_grid(model, 1)
    radius = np.hypot(ox - cx, oy - cy)
    edge = float(model.circle_radius_px) if model.circle_radius_px else float(radius.max())
    feather = max(float(feather_px), 1e-3)
    self.mask = np.clip((edge - radius) / feather + 0.5, 0.0, 1.0).astype(np.float32)

face_camera

face_camera(face: str) -> CameraModel

Return the 90 degree square pinhole :class:CameraModel of a cube face.

Its basis converts to a CARLA rotation with :func:~bikeped.sim.projection.basis_to_carla_rotation.

Source code in bikeped/sim/fisheye.py
def face_camera(self, face: str) -> CameraModel:
    """Return the 90 degree square pinhole :class:`CameraModel` of a cube face.

    Its basis converts to a CARLA rotation with
    :func:`~bikeped.sim.projection.basis_to_carla_rotation`.
    """
    if face not in _FACE_AXES:
        raise KeyError(f"unknown face {face!r}")
    b = self.model.basis()
    f, r, u = (b[a] * s for a, s in _FACE_AXES[face])
    return CameraModel(name=f"{self.model.name}/{face}", kind="pinhole",
                       width=self.face_size, height=self.face_size,
                       position=self.model.position, forward=tuple(f), right=tuple(r),
                       up=tuple(u), fov_deg=90.0)

face_rotation

face_rotation(face: str) -> Tuple[float, float, float]

CARLA (pitch, yaw, roll) in degrees of a face camera.

Source code in bikeped/sim/fisheye.py
def face_rotation(self, face: str) -> Tuple[float, float, float]:
    """CARLA ``(pitch, yaw, roll)`` in degrees of a face camera."""
    cam = self.face_camera(face)
    return basis_to_carla_rotation(cam.forward, cam.right, cam.up)

remap

remap(face_images: Dict[str, ndarray]) -> np.ndarray

Resample BGR face images into the fisheye image (uint8, black outside the circle).

face_images must hold every face in :attr:faces. Images of a different size than face_size are accepted and sampled in proportion.

Source code in bikeped/sim/fisheye.py
def remap(self, face_images: Dict[str, np.ndarray]) -> np.ndarray:
    """Resample BGR face images into the fisheye image (``uint8``, black outside the circle).

    ``face_images`` must hold every face in :attr:`faces`. Images of a
    different size than ``face_size`` are accepted and sampled in proportion.
    """
    h, w = self._shape
    out = np.zeros((h * w, 3), dtype=np.float32)
    for face in self.faces:
        if face not in face_images:
            raise KeyError(f"missing face image {face!r}; need {self.faces}")
        img = np.asarray(face_images[face])
        if img.ndim == 2:
            img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
        img = img[..., :3]
        idx, mx, my = self._lookup[face]
        sx, sy = img.shape[1] / self.face_size, img.shape[0] / self.face_size
        if sx != 1.0 or sy != 1.0:
            mx = ((mx + 0.5) * sx - 0.5).astype(np.float32)
            my = ((my + 0.5) * sy - 0.5).astype(np.float32)
        sampled = cv2.remap(img, mx, my, self.interpolation, borderMode=cv2.BORDER_REPLICATE)
        out[idx] = sampled.reshape(-1, 3)[:idx.size]
    out = out.reshape(h, w, 3)
    if self.supersample > 1:
        out = cv2.resize(out, (self.model.width, self.model.height),
                         interpolation=cv2.INTER_AREA)
    out *= self.mask[..., None]
    return np.clip(out + 0.5, 0, 255).astype(np.uint8)

deployed_fisheye_model

deployed_fisheye_model(
    position: Sequence[float],
    yaw_deg: float = 0.0,
    *,
    pitch_deg: Optional[float] = None,
    calibration: Optional[PathLike] = "camera_calibration.json",
    config: Optional[PathLike] = "config.yaml",
    scale: float = 1.0,
    name: str = "fisheye",
) -> CameraModel

Return a :class:CameraModel of the deployed fisheye at position.

position is in world metres and yaw_deg is the CARLA yaw of the optical axis. pitch_deg defaults to camera.pitch_deg in the config (0, a level camera, as in the deployment and in the testbench). Intrinsics come from the calibration file, falling back to the config; relative paths are resolved against the repository root when not found in the working directory. The projection is equidistant with optional k1/k2 terms and f = radius / (fov / 2), the same focal length the testbench uses. scale resamples the 3840 x 2160 sensor (0.5 gives 1920 x 1080) and scales the intrinsics to match.

Mapping to the testbench: a testbench heading h (scenario frame, counter-clockwise from +x) is CARLA yaw -h once the scenario y axis is flipped into the left-handed world, so the testbench default (heading 90 degrees) is yaw_deg=-90.

Source code in bikeped/sim/fisheye.py
def deployed_fisheye_model(position: Sequence[float], yaw_deg: float = 0.0, *,
                           pitch_deg: Optional[float] = None,
                           calibration: Optional[PathLike] = "camera_calibration.json",
                           config: Optional[PathLike] = "config.yaml",
                           scale: float = 1.0, name: str = "fisheye") -> CameraModel:
    """Return a :class:`CameraModel` of the deployed fisheye at ``position``.

    ``position`` is in world metres and ``yaw_deg`` is the CARLA yaw of the
    optical axis. ``pitch_deg`` defaults to ``camera.pitch_deg`` in the config
    (0, a level camera, as in the deployment and in the testbench). Intrinsics
    come from the calibration file, falling back to the config; relative paths
    are resolved against the repository root when not found in the working
    directory. The projection is equidistant with optional ``k1``/``k2``
    terms and ``f = radius / (fov / 2)``, the same focal length the testbench
    uses. ``scale`` resamples the 3840 x 2160 sensor (0.5 gives 1920 x 1080)
    and scales the intrinsics to match.

    Mapping to the testbench: a testbench heading ``h`` (scenario frame,
    counter-clockwise from ``+x``) is CARLA yaw ``-h`` once the scenario
    ``y`` axis is flipped into the left-handed world, so the testbench default
    (heading 90 degrees) is ``yaw_deg=-90``.
    """
    p = _load_camera_parameters(calibration, config)
    pitch = p["pitch_deg"] if pitch_deg is None else float(pitch_deg)
    forward, right, up = carla_rotation_to_basis(pitch, yaw_deg, 0.0)
    x_off = (PAD_SIZE - CROP_SIZE) // 2
    y_off = (PAD_SIZE - CROP_SIZE) // 2 - (PAD_SIZE - SENSOR_SIZE[1]) // 2
    width = max(2, int(round(SENSOR_SIZE[0] * scale / 2.0)) * 2)
    height = max(2, int(round(SENSOR_SIZE[1] * scale / 2.0)) * 2)
    sx, sy = width / SENSOR_SIZE[0], height / SENSOR_SIZE[1]
    # Pixel centres sit at integer coordinates, so scaling acts on (x + 0.5).
    cx = (p["cx"] + x_off + 0.5) * sx - 0.5
    cy = (p["cy"] + y_off + 0.5) * sy - 0.5
    radius = p["radius_px"] * sx
    return CameraModel(
        name=name, kind="fisheye", width=width, height=height,
        position=tuple(float(v) for v in position), forward=forward, right=right, up=up,
        fov_deg=p["fov_deg"], cx=cx, cy=cy,
        f_px_per_rad=radius / math.radians(p["fov_deg"] / 2.0),
        k1=p["k1"], k2=p["k2"], circle_radius_px=radius,
    )

synthetic_faces

synthetic_faces(
    remap: FisheyeRemap,
    pattern: str = "grid",
    *,
    ground_z: float = 0.0,
    cell_m: float = 1.0,
    markers: Sequence[Sequence[float]] = (),
) -> Dict[str, np.ndarray]

Render face images of an analytic scene, for tests and demos.

The scene is the plane z = ground_z (world metres) painted as a checkerboard (pattern='checker') or a light grid on asphalt grey (pattern='grid') with cell_m cells, under a vertical sky gradient. Each entry of markers is a world (x, y) drawn as a pure red (BGR (0, 0, 255)) disc of 0.15 m radius.

Source code in bikeped/sim/fisheye.py
def synthetic_faces(remap: FisheyeRemap, pattern: str = "grid", *, ground_z: float = 0.0,
                    cell_m: float = 1.0,
                    markers: Sequence[Sequence[float]] = ()) -> Dict[str, np.ndarray]:
    """Render face images of an analytic scene, for tests and demos.

    The scene is the plane ``z = ground_z`` (world metres) painted as a
    checkerboard (``pattern='checker'``) or a light grid on asphalt grey
    (``pattern='grid'``) with ``cell_m`` cells, under a vertical sky gradient.
    Each entry of ``markers`` is a world ``(x, y)`` drawn as a pure red
    (BGR ``(0, 0, 255)``) disc of 0.15 m radius.
    """
    if pattern not in ("grid", "checker"):
        raise ValueError(f"unknown pattern {pattern!r}")
    images: Dict[str, np.ndarray] = {}
    n = remap.face_size
    xs = np.arange(n, dtype=np.float64) + 0.5
    gu, gv = np.meshgrid(xs, xs)
    uv = np.stack([gu.ravel(), gv.ravel()], axis=1)
    pos = np.asarray(remap.model.position, dtype=np.float64)
    for face in remap.faces:
        cam = remap.face_camera(face)
        rays, _ = pixel_rays(uv, cam)
        elev = np.clip(rays[:, 2], -1.0, 1.0)
        sky = np.stack([235 - 60 * elev, 205 - 70 * elev, 170 - 90 * elev], axis=1)
        img = np.clip(sky, 0, 255)
        with np.errstate(divide="ignore", invalid="ignore"):
            t = (ground_z - pos[2]) / rays[:, 2]
        hit = np.isfinite(t) & (t > 0)
        gx = pos[0] + rays[:, 0] * np.where(hit, t, 0.0)
        gy = pos[1] + rays[:, 1] * np.where(hit, t, 0.0)
        if pattern == "checker":
            parity = (np.floor(gx / cell_m) + np.floor(gy / cell_m)) % 2
            ground = np.where(parity[:, None] > 0, [[200, 200, 200]], [[60, 60, 60]])
        else:
            fx = np.abs(gx / cell_m - np.round(gx / cell_m))
            fy = np.abs(gy / cell_m - np.round(gy / cell_m))
            line = np.minimum(fx, fy) * cell_m < 0.03
            ground = np.where(line[:, None], [[215, 215, 210]], [[78, 76, 74]])
        for mx_, my_ in markers:
            disc = np.hypot(gx - mx_, gy - my_) < 0.15
            ground = np.where(disc[:, None], [[0, 0, 255]], ground)
        img = np.where(hit[:, None], ground, img)
        images[face] = img.reshape(n, n, 3).astype(np.uint8)
    return images

groundtruth

Per-frame ground truth: 2D boxes, visibility, danger labels and the closest pair.

Boxes come from :func:~bikeped.sim.projection.project_box on each agent's oriented 3D box in the world frame. Danger labels come from decision_testbench.compute_ground_truth_danger evaluated on the scripted scenario, so the rendered runs carry the same labels as the paper's evaluation. The synthetic helpers place testbench boxes in a world without a simulator, for tests and demos.

danger_record

danger_record(scenario: Mapping[str, Any], t: float) -> Dict[str, Any]

Ground-truth danger at time t (seconds) as a JSON-safe dict.

Wraps decision_testbench.compute_ground_truth_danger; the keys are the fields of its DangerAssessment (distances in metres, speeds in m/s, times in seconds) and infinite values become None.

Source code in bikeped/sim/groundtruth.py
def danger_record(scenario: Mapping[str, Any], t: float) -> Dict[str, Any]:
    """Ground-truth danger at time ``t`` (seconds) as a JSON-safe dict.

    Wraps ``decision_testbench.compute_ground_truth_danger``; the keys are the
    fields of its ``DangerAssessment`` (distances in metres, speeds in m/s,
    times in seconds) and infinite values become ``None``.
    """
    assessment = _testbench().compute_ground_truth_danger(dict(scenario), float(t))
    return {k: _json_number(v) for k, v in asdict(assessment).items()}

closest_pair

closest_pair(frame: FrameRecord) -> Optional[Tuple[int, int, float]]

Nearest scripted (bike_id, person_id, distance_m) in the scenario frame, or None.

Source code in bikeped/sim/groundtruth.py
def closest_pair(frame: FrameRecord) -> Optional[Tuple[int, int, float]]:
    """Nearest scripted ``(bike_id, person_id, distance_m)`` in the scenario frame, or ``None``."""
    bikes = [a for a in frame.agents if a.role == "scripted" and a.kind == "bike"]
    people = [a for a in frame.agents if a.role == "scripted" and a.kind == "person"]
    best: Optional[Tuple[int, int, float]] = None
    for b in bikes:
        for p in people:
            d = math.hypot(b.sx - p.sx, b.sy - p.sy)
            if best is None or d < best[2]:
                best = (b.id, p.id, d)
    return best

scenario_to_world

scenario_to_world(x: float, y: float, z: float = 0.0) -> Tuple[float, float, float]

Default embedding of the right-handed scenario frame in the left-handed world.

Flips y. Used by the synthetic helpers; the CARLA runtime places scenarios with its own site transform. A scenario heading h (radians) becomes the world yaw -degrees(h).

Source code in bikeped/sim/groundtruth.py
def scenario_to_world(x: float, y: float, z: float = 0.0) -> Tuple[float, float, float]:
    """Default embedding of the right-handed scenario frame in the left-handed world.

    Flips ``y``. Used by the synthetic helpers; the CARLA runtime places
    scenarios with its own site transform. A scenario heading ``h`` (radians)
    becomes the world yaw ``-degrees(h)``.
    """
    return float(x), float(-y), float(z)

synthetic_agents

synthetic_agents(
    scenario: Mapping[str, Any], t: float, ground_z: float = 0.0
) -> List[AgentState]

Scripted agents at time t with testbench box sizes.

Positions are placed with :func:scenario_to_world; speeds are in m/s.

Source code in bikeped/sim/groundtruth.py
def synthetic_agents(scenario: Mapping[str, Any], t: float,
                     ground_z: float = 0.0) -> List[AgentState]:
    """Scripted agents at time ``t`` with testbench box sizes.

    Positions are placed with :func:`scenario_to_world`; speeds are in m/s.
    """
    tb = _testbench()
    agents: List[AgentState] = []
    for agent_def in scenario["agents"]:
        pose = tb.interpolate_agent(agent_def, t)
        if pose is None:
            continue
        obj = tb.OBJECT_TYPES[KIND_TO_OBJECT.get(agent_def["type"], "pedestrian")]
        agents.append(AgentState(
            id=int(agent_def["id"]), kind=agent_def["type"], sx=float(pose["x"]),
            sy=float(pose["y"]), heading=float(pose["heading"]),
            speed=float(tb.estimate_agent_speed(agent_def, t)),
            location=scenario_to_world(pose["x"], pose["y"], ground_z + obj.height / 2.0),
            rotation=(0.0, -math.degrees(pose["heading"]), 0.0),
            extent=(obj.length / 2.0, obj.width / 2.0, obj.height / 2.0),
            model=str(agent_def.get("model", "")),
        ))
    return agents

synthetic_frame

synthetic_frame(
    scenario: Mapping[str, Any],
    index: int,
    fps: int,
    cameras: Optional[Dict[str, CameraModel]] = None,
    states: Optional[Sequence[str]] = None,
) -> FrameRecord

A :class:FrameRecord at t = index / fps built from the scenario alone, without images.

state is states[index] when given, else "idle"; in the alert state alert_pair is the closest bike and pedestrian.

Source code in bikeped/sim/groundtruth.py
def synthetic_frame(scenario: Mapping[str, Any], index: int, fps: int,
                    cameras: Optional[Dict[str, CameraModel]] = None,
                    states: Optional[Sequence[str]] = None) -> FrameRecord:
    """A :class:`FrameRecord` at ``t = index / fps`` built from the scenario alone, without images.

    ``state`` is ``states[index]`` when given, else ``"idle"``; in the alert
    state ``alert_pair`` is the closest bike and pedestrian.
    """
    t = index / float(fps)
    state = states[index] if states is not None and index < len(states) else "idle"
    frame = FrameRecord(index=index, t=t, agents=synthetic_agents(scenario, t), state=state,
                        states={"default": state}, danger=danger_record(scenario, t),
                        cameras=dict(cameras or {}))
    if state == "alert":
        pair = closest_pair(frame)
        if pair is not None:
            frame.alert_pair = (pair[0], pair[1])
    return frame

frame_ground_truth

frame_ground_truth(frame: FrameRecord, view: str) -> List[Dict[str, Any]]

Per-agent ground truth of one view, JSON-safe.

Each entry has id, kind, role, bbox (pixels, clipped to the image), corners_image (8 [u, v] or None), distance_m, truncation and visible_fraction. The visible fraction is the share of the agent's projected silhouette carrying its id in aux['<view>/instance'] when that buffer and AgentState.instance_id exist, else the share of camera-facing box surface samples not occluded in aux['<view>/depth'] (planar depth for pinhole views, range for fisheye views), else None. Agents outside the view are omitted.

Source code in bikeped/sim/groundtruth.py
def frame_ground_truth(frame: FrameRecord, view: str) -> List[Dict[str, Any]]:
    """Per-agent ground truth of one view, JSON-safe.

    Each entry has ``id``, ``kind``, ``role``, ``bbox`` (pixels, clipped to
    the image), ``corners_image`` (8 ``[u, v]`` or ``None``), ``distance_m``,
    ``truncation`` and ``visible_fraction``. The visible fraction is the share
    of the agent's projected silhouette carrying its id in
    ``aux['<view>/instance']`` when that buffer and ``AgentState.instance_id``
    exist, else the share of camera-facing box surface samples not occluded in
    ``aux['<view>/depth']`` (planar depth for pinhole views, range for
    fisheye views), else ``None``. Agents outside the view are omitted.
    """
    from .projection import project_box

    cam = frame.cameras.get(view)
    if cam is None:
        return []
    depth = frame.aux.get(f"{view}/depth")
    instance = frame.aux.get(f"{view}/instance")
    objects: List[Dict[str, Any]] = []
    for agent in frame.agents:
        box = project_box(agent, cam)
        if box is None:
            continue
        visible: Optional[float] = None
        if instance is not None:
            visible = _visible_from_instance(agent, cam, instance)
        if visible is None and depth is not None:
            visible = _visible_from_depth(agent, cam, depth)
        corners = [None if not np.all(np.isfinite(c)) else [round(float(c[0]), 2),
                                                             round(float(c[1]), 2)]
                   for c in box["corners_image"]]
        objects.append({
            "id": agent.id, "kind": agent.kind, "role": agent.role,
            "bbox": [round(float(v), 2) for v in box["bbox"]],
            "corners_image": corners,
            "distance_m": round(float(box["distance_m"]), 3),
            "truncation": round(float(box["truncation"]), 4),
            "visible_fraction": None if visible is None else round(visible, 4),
        })
    return objects

overlay

HUD overlay and schematic top-down panel for rendered scenario frames.

draw_overlay annotates one camera view with 3D boxes, id chips, ground trails, the bike-pedestrian connector, the decision state badge and a timeline of ground-truth danger against the system's alerts. draw_bev renders the same frame as a plan view in the scenario frame. Both work on BGR uint8 arrays, return new arrays and scale every size with the image height, so 720p and 4K output look the same.

Pixel coordinates follow bikeped.sim.projection: continuous, with pixel i covering [i, i + 1).

Run python -m bikeped.sim.overlay --demo OUT_DIR to render a still and a short clip over a synthetic backdrop without a simulator.

OverlayStyle dataclass

OverlayStyle(
    background: str = "#0a0b0d",
    text: str = "#ecebe8",
    muted: str = "#9b9ca3",
    cyclist: str = "#ffb547",
    pedestrian: str = "#6fe3ff",
    alert: str = "#ff5a4f",
    warning: str = "#ffb547",
    safe: str = "#5ad19a",
    idle: str = "#6c6e76",
    font_path: Optional[str] = None,
    boxes: bool = True,
    labels: bool = True,
    trails: bool = True,
    connector: bool = True,
    rings: bool = False,
    ring_views: Tuple[str, ...] = ("fisheye",),
    badge: bool = True,
    timeline: bool = True,
    caption: bool = True,
    trail_seconds: float = 1.5,
    connector_max_m: float = 30.0,
    scale: float = 1.0,
)

Appearance of the HUD and of the top-down panel.

Colours are #rrggbb strings and match the project site. The boolean fields switch single elements. rings forces the range rings on for any view; without it they are drawn only on the views named in ring_views, where the camera footprint is the sensor position. scale multiplies every size on top of the automatic scaling with image height, which is useful when a view is shown small inside a composite.

bgr

bgr(name: str) -> BGR

BGR tuple of the palette entry name.

Source code in bikeped/sim/overlay.py
def bgr(self, name: str) -> BGR:
    """BGR tuple of the palette entry ``name``."""
    return _hex_to_bgr(getattr(self, name))

state_color

state_color(state: str) -> BGR

Colour of a decision state; unknown states use the idle colour.

Source code in bikeped/sim/overlay.py
def state_color(self, state: str) -> BGR:
    """Colour of a decision state; unknown states use the idle colour."""
    return self.bgr(state if state in _STATE_LABELS else "idle")

agent_color

agent_color(kind: str, role: str = 'scripted') -> BGR

Colour of an agent: cyan pedestrians, amber cyclists, muted ambient traffic.

Source code in bikeped/sim/overlay.py
def agent_color(self, kind: str, role: str = "scripted") -> BGR:
    """Colour of an agent: cyan pedestrians, amber cyclists, muted ambient traffic."""
    if role != "scripted":
        return self.bgr("muted")
    return self.bgr("pedestrian" if kind == "person" else "cyclist")

build_timeline

build_timeline(
    scenario: Dict[str, Any], fps: int, states: Optional[Sequence[str]] = None
) -> Dict[str, Any]

Precompute what the timeline strip shows for one scenario.

Frame i is at t = i / fps. Ground-truth danger comes from decision_testbench.compute_ground_truth_danger. states is the decision state per frame as recorded by a run; when it is None the states come from decision_testbench.simulate_scenario with the default pipeline configuration and no detection noise, and the result is cached per scenario and frame rate.

Returns a dict with duration (s), fps, n (frames), states, the interval lists danger, actionable, alert and warning ([t0, t1] pairs in seconds, t1 exclusive), and first_alert_t and first_danger_t (seconds or None).

Source code in bikeped/sim/overlay.py
def build_timeline(scenario: Dict[str, Any], fps: int,
                   states: Optional[Sequence[str]] = None) -> Dict[str, Any]:
    """Precompute what the timeline strip shows for one scenario.

    Frame ``i`` is at ``t = i / fps``. Ground-truth danger comes from
    ``decision_testbench.compute_ground_truth_danger``. ``states`` is the
    decision state per frame as recorded by a run; when it is ``None`` the
    states come from ``decision_testbench.simulate_scenario`` with the default
    pipeline configuration and no detection noise, and the result is cached
    per scenario and frame rate.

    Returns a dict with ``duration`` (s), ``fps``, ``n`` (frames), ``states``,
    the interval lists ``danger``, ``actionable``, ``alert`` and ``warning``
    (``[t0, t1]`` pairs in seconds, ``t1`` exclusive), and ``first_alert_t``
    and ``first_danger_t`` (seconds or ``None``).
    """
    tb = _testbench()
    fps = int(fps)
    key = (_scenario_key(scenario), fps)
    if states is None and key in _TIMELINES:
        return _TIMELINES[key]
    dt = 1.0 / fps
    duration = float(tb.get_scenario_duration(scenario))
    if states is None:
        result = tb.simulate_scenario(scenario, tb.PIPELINE_CONFIGS["default"], fps=fps)
        state_list = [str(s) for s in result["states"]]
        danger = [bool(d) for d in result["dangers"]]
        actionable = [bool(a.actionable) for a in result["assessments"]]
        _DANGER[(key[0], fps, len(state_list))] = (danger, actionable)
    else:
        state_list = [str(s) for s in states]
        cached = _DANGER.get((key[0], fps, len(state_list)))
        if cached is None:
            assessments = [tb.compute_ground_truth_danger(scenario, i * dt)
                           for i in range(len(state_list))]
            cached = ([bool(a.is_danger) for a in assessments],
                      [bool(a.actionable) for a in assessments])
            _DANGER[(key[0], fps, len(state_list))] = cached
        danger, actionable = cached
    n = len(state_list)
    alert = [s == "alert" for s in state_list]
    timeline = {
        "duration": max(duration, n * dt),
        "fps": fps,
        "n": n,
        "states": state_list,
        "danger": _intervals(danger, dt),
        "actionable": _intervals(actionable, dt),
        "alert": _intervals(alert, dt),
        "warning": _intervals([s == "warning" for s in state_list], dt),
        "first_alert_t": round(alert.index(True) * dt, 4) if any(alert) else None,
        "first_danger_t": round(danger.index(True) * dt, 4) if any(danger) else None,
    }
    if states is None:
        _TIMELINES[key] = timeline
    return timeline

draw_overlay

draw_overlay(
    image: ndarray,
    frame: FrameRecord,
    view: str,
    *,
    run: RunInfo,
    scenario: Optional[Dict[str, Any]] = None,
    style: Optional[OverlayStyle] = None,
    history: Optional[Sequence[FrameRecord]] = None,
    timeline: Optional[Dict[str, Any]] = None,
) -> np.ndarray

Draw the HUD over one rendered view and return the annotated copy.

image is the BGR uint8 frame of view and is not modified. The camera model is frame.cameras[view]; without one only the screen-space elements (badge, caption, timeline) are drawn. history holds earlier frames, oldest first, and feeds the ground trails from the bottom centre of each agent's box in the world frame. timeline is the result of build_timeline; when it is None and scenario is given it is built from the scenario at run.fps.

Source code in bikeped/sim/overlay.py
def draw_overlay(image: np.ndarray, frame: FrameRecord, view: str, *, run: RunInfo,
                 scenario: Optional[Dict[str, Any]] = None,
                 style: Optional[OverlayStyle] = None,
                 history: Optional[Sequence[FrameRecord]] = None,
                 timeline: Optional[Dict[str, Any]] = None) -> np.ndarray:
    """Draw the HUD over one rendered view and return the annotated copy.

    ``image`` is the BGR ``uint8`` frame of ``view`` and is not modified. The
    camera model is ``frame.cameras[view]``; without one only the screen-space
    elements (badge, caption, timeline) are drawn. ``history`` holds earlier
    frames, oldest first, and feeds the ground trails from the bottom centre of
    each agent's box in the world frame. ``timeline`` is the result of
    ``build_timeline``; when it is ``None`` and ``scenario`` is given it is
    built from the scenario at ``run.fps``.
    """
    style = style or DEFAULT_STYLE
    out = np.ascontiguousarray(image).copy()
    if out.ndim == 2:
        out = cv2.cvtColor(out, cv2.COLOR_GRAY2BGR)
    canvas = _Canvas(out, style)
    cam = frame.cameras.get(view)
    if timeline is None and scenario is not None and style.timeline:
        timeline = build_timeline(scenario, run.fps)
    show_timeline = style.timeline and timeline is not None

    margin = canvas.px(28)
    lanes_h = 2 * canvas.px(6, 3) + canvas.px(5, 2)
    timeline_y = canvas.h - margin - lanes_h
    caption_font = canvas.px(16, 9)
    caption_h = canvas.text_size("Ag", caption_font)[1]
    caption_y = (timeline_y - canvas.px(14) - caption_h) if show_timeline else (
        canvas.h - margin - caption_h)
    hud_top = caption_y if style.caption else (timeline_y if show_timeline else canvas.h)
    if style.caption or show_timeline:
        canvas.scrim(int(hud_top - canvas.px(70)), canvas.h, 0.0, 0.66)

    occupied: List[Rect] = []
    if style.caption or show_timeline:
        occupied.append((0.0, hud_top - canvas.px(6), float(canvas.w), float(canvas.h)))
    badge_h = canvas.px(42, 20)
    if style.badge:
        occupied.append((0.0, 0.0, margin + canvas.px(250), margin + badge_h + canvas.px(40)))

    scripted = [a for a in frame.agents if a.role == "scripted"]
    if cam is not None:
        ground_z = run.site.get("ground_z")
        if ground_z is None:
            ground_z = min((_bottom_center(a)[2] for a in frame.agents), default=0.0)
        if style.rings or view in style.ring_views:
            boxes = [project_box(a, cam) for a in frame.agents]
            _draw_rings(canvas, cam, float(ground_z), occupied,
                        [tuple(b["bbox"]) for b in boxes if b is not None])
        if style.trails and history:
            _draw_trails(canvas, frame, cam, history)

        pair = _pair(frame) if style.connector else None
        highlighted = set()
        if pair is not None and pair[2] <= style.connector_max_m:
            a, b, distance = pair
            color = _connector_color(style, frame.state)
            if frame.state in ("alert", "warning"):
                highlighted = {a.id, b.id}
            ends = np.stack([_bottom_center(a), _bottom_center(b)])
            pieces = project_polyline(ends, cam)
            canvas.lines(pieces, style.bgr("background"), canvas.px(5), alpha=0.45)
            canvas.lines(pieces, color, canvas.px(2))
            uv, valid, _ = project_points(ends.mean(axis=0, keepdims=True), cam)
            if valid[0] and 0 <= uv[0, 0] < canvas.w and 0 <= uv[0, 1] < canvas.h:
                label = f"{distance:.1f} m"
                font = canvas.px(15, 9)
                tw, th = canvas.text_size(label, font, bold=True)
                pad = canvas.px(9)
                rect = (uv[0, 0] - tw / 2.0 - pad, uv[0, 1] - th / 2.0 - canvas.px(4),
                        uv[0, 0] + tw / 2.0 + pad, uv[0, 1] + th / 2.0 + canvas.px(4))
                canvas.plate(rect, style.bgr("background"), alpha=0.86)
                canvas.text_centered(rect, label, font, color, bold=True)
                occupied.append(rect)

        cam_pos = np.asarray(cam.position, dtype=np.float64)
        ordered = sorted(frame.agents, reverse=True,
                         key=lambda ag: float(np.linalg.norm(np.asarray(ag.location) - cam_pos)))
        chips: List[Tuple[AgentState, Sequence[float]]] = []
        for agent in ordered:
            color = style.agent_color(agent.kind, agent.role)
            hot = agent.id in highlighted and agent.role == "scripted"
            if style.boxes:
                _draw_box(canvas, agent, cam, color,
                          style.state_color(frame.state) if hot else color, 0.34 if hot else 0.2)
            if style.labels and agent.role == "scripted":
                box = project_box(agent, cam)
                if box is not None:
                    chips.append((agent, box["bbox"]))
        font = canvas.px(14, 9)
        for agent, bbox in reversed(chips):
            text = _chip_text(agent)
            tw, _ = canvas.text_size(text, font, bold=True)
            size = (tw + 2 * canvas.px(8), canvas.px(22, 13))
            rect = _place_chip(canvas, size, bbox, occupied)
            occupied.append(rect)
            color = style.agent_color(agent.kind)
            anchor = ((bbox[0] + bbox[2]) / 2.0, bbox[1])
            foot = ((rect[0] + rect[2]) / 2.0, rect[3])
            if math.hypot(anchor[0] - foot[0], anchor[1] - foot[1]) > canvas.px(10):
                canvas.lines([np.array([foot, anchor])], color, canvas.px(1.25), alpha=0.6)
            outline = style.state_color(frame.state) if agent.id in highlighted else None
            _draw_chip(canvas, rect, text, color, outline)

    if style.badge:
        canvas.scrim(0, margin + badge_h + canvas.px(50), 0.3, 0.0)
        badge = _draw_badge(canvas, margin, margin, frame.state)
        danger = frame.danger or {}
        if danger.get("is_danger"):
            sep = " \u00b7 " if canvas.fonts.truetype else " | "
            note = "Ground truth: danger"
            if danger.get("ttc") is not None and math.isfinite(float(danger["ttc"])):
                note += f"{sep}TTC {float(danger['ttc']):.1f} s"
            font = canvas.px(13, 9)
            tw, th = canvas.text_size(note, font)
            pad = canvas.px(12)
            top = badge[3] + canvas.px(8)
            rect = (margin, top, margin + tw + 2 * pad, top + th + canvas.px(10))
            canvas.plate(rect, style.bgr("background"), alpha=0.7)
            canvas.text_centered(rect, note, font, style.bgr("alert"))

    if style.caption:
        sep = "  \u00b7  " if canvas.fonts.truetype else "  |  "
        title, details = _caption_parts(frame, run, sep)
        tw, _ = canvas.text(margin, caption_y, title, caption_font, style.bgr("text"), bold=True)
        if details:
            canvas.text(margin + tw, caption_y, sep + details, caption_font, style.bgr("muted"))
        total = timeline["duration"] if timeline else run.duration_s
        clock = f"{frame.t:5.2f} s" + (f" / {total:.2f} s" if total else "")
        cw, _ = canvas.text_size(clock, caption_font)
        canvas.text(canvas.w - margin - cw, caption_y, clock, caption_font, style.bgr("text"))
        tag = view.upper()
        vw, _ = canvas.text_size(tag, canvas.px(12, 8), bold=True)
        canvas.text(canvas.w - margin - cw - canvas.px(18) - vw,
                    caption_y + (caption_h - canvas.text_size(tag, canvas.px(12, 8), True)[1]) / 2,
                    tag, canvas.px(12, 8), style.bgr("muted"), bold=True)
    if show_timeline:
        _draw_timeline(canvas, timeline, frame.t, margin, canvas.w - margin, timeline_y)
    if scripted and cam is None:
        _LOG.debug("view %s has no camera model; drew screen-space HUD only", view)
    return out

draw_bev

draw_bev(
    frame: FrameRecord,
    *,
    run: RunInfo,
    scenario: Optional[Dict[str, Any]],
    size: Tuple[int, int] = (1080, 1080),
    style: Optional[OverlayStyle] = None,
    history: Optional[Sequence[FrameRecord]] = None,
    timeline: Optional[Dict[str, Any]] = None,
) -> np.ndarray

Render the frame as a schematic plan view in the scenario frame.

size is (width, height) in pixels. The panel shows the road, the crosswalk at x = 5, the camera marker at run.site['camera_xy'] (the testbench camera position when absent) with the proximity rings of the default pipeline, the scripted paths, and every agent as an oriented glyph with a short trail. Scenario x points right and y up. Trails come from the scenario paths when scenario is given and from history otherwise. timeline adds the timeline strip at the bottom.

Source code in bikeped/sim/overlay.py
def draw_bev(frame: FrameRecord, *, run: RunInfo, scenario: Optional[Dict[str, Any]],
             size: Tuple[int, int] = (1080, 1080), style: Optional[OverlayStyle] = None,
             history: Optional[Sequence[FrameRecord]] = None,
             timeline: Optional[Dict[str, Any]] = None) -> np.ndarray:
    """Render the frame as a schematic plan view in the scenario frame.

    ``size`` is ``(width, height)`` in pixels. The panel shows the road, the
    crosswalk at ``x = 5``, the camera marker at ``run.site['camera_xy']``
    (the testbench camera position when absent) with the proximity rings of
    the default pipeline, the scripted paths, and every agent as an oriented
    glyph with a short trail. Scenario ``x`` points right and ``y`` up. Trails
    come from the scenario paths when ``scenario`` is given and from
    ``history`` otherwise. ``timeline`` adds the timeline strip at the bottom.
    """
    style = style or DEFAULT_STYLE
    width, height = int(size[0]), int(size[1])
    cam_xy = _camera_xy(run)
    window = _bev_window(scenario, frame, cam_xy)
    key = (width, height, style, _scenario_key(scenario) if scenario else None,
           tuple(round(v, 3) for v in window), cam_xy)
    static = _BEV_STATIC.get(key)
    if static is None:
        static = _BEV_STATIC[key] = _bev_static((width, height), style, scenario, window, cam_xy)
        if len(_BEV_STATIC) > 8:
            _BEV_STATIC.popitem(last=False)
    out = static.copy()
    canvas = _Canvas(out, style)
    canvas.s = style.scale * min(width, height) / 1080.0
    to_px = _bev_mapper((width, height), window)
    metre = float(to_px([[1.0, 0.0]])[0][0] - to_px([[0.0, 0.0]])[0][0])
    bg = style.bgr("background")

    pair = _pair(frame) if style.connector else None
    highlighted = set()
    if pair is not None and pair[2] <= style.connector_max_m:
        a, b, distance = pair
        color = _connector_color(style, frame.state)
        if frame.state in ("alert", "warning"):
            highlighted = {a.id, b.id}
        ends = to_px([[a.sx, a.sy], [b.sx, b.sy]])
        canvas.lines([ends], color, canvas.px(2), alpha=0.9)
        mid = ends.mean(axis=0)
        label = f"{distance:.1f} m"
        font = canvas.px(15, 9)
        tw, th = canvas.text_size(label, font, bold=True)
        normal = np.array([-(ends[1][1] - ends[0][1]), ends[1][0] - ends[0][0]])
        length = float(np.linalg.norm(normal))
        if length > 1e-6:
            normal = normal / length
            if normal[1] > 0:
                normal = -normal
            mid = mid + normal * (th / 2.0 + canvas.px(12))
        rect = (mid[0] - tw / 2.0 - canvas.px(9), mid[1] - th / 2.0 - canvas.px(4),
                mid[0] + tw / 2.0 + canvas.px(9), mid[1] + th / 2.0 + canvas.px(4))
        canvas.plate(rect, bg, alpha=0.9)
        canvas.text_centered(rect, label, font, color, bold=True)

    for agent in frame.agents:
        color = style.agent_color(agent.kind, agent.role)
        if style.trails and agent.role == "scripted":
            trail = to_px(_bev_trail(frame, agent, scenario, history, style.trail_seconds))
            edges = np.linspace(0, len(trail) - 1, 6).round().astype(int)
            for band in range(5):
                chunk = trail[edges[band]:edges[band + 1] + 1]
                canvas.lines([chunk], color, canvas.px(4), alpha=0.18 + 0.62 * (band + 1) / 5)

    font = canvas.px(14, 9)
    for agent in frame.agents:
        color = style.agent_color(agent.kind, agent.role)
        center = to_px([[agent.sx, agent.sy]])[0]
        direction = np.array([math.cos(agent.heading), -math.sin(agent.heading)])
        side = np.array([-direction[1], direction[0]])
        if agent.kind == "person":
            radius = max(canvas.px(8), 0.3 * metre)
            reach = radius
            if agent.id in highlighted:
                canvas.circle((center[0], center[1]), radius * 2.1,
                              style.state_color(frame.state), thickness=canvas.px(2))
            canvas.circle((center[0], center[1]), radius + canvas.px(2), bg)
            canvas.circle((center[0], center[1]), radius, color)
            if agent.speed > 0.05:
                tip = center + direction * radius * 1.9
                canvas.lines([np.stack([center + direction * radius, tip])], color, canvas.px(3))
        else:
            length = max(canvas.px(30), 2.0 * agent.extent[0] * metre)
            half_w = max(canvas.px(7), agent.extent[1] * metre)
            reach = length / 2.0
            glyph = np.stack([center + direction * length / 2.0,
                              center - direction * length / 2.0 + side * half_w,
                              center - direction * length / 4.0,
                              center - direction * length / 2.0 - side * half_w])
            if agent.id in highlighted:
                canvas.circle((center[0], center[1]), length * 0.78,
                              style.state_color(frame.state), thickness=canvas.px(2))
            canvas.lines([glyph], bg, canvas.px(5), closed=True)
            canvas.polygon(glyph, color)
        if style.labels and agent.role == "scripted":
            text = _chip_text(agent)
            tw, _ = canvas.text_size(text, font, bold=True)
            cw, ch = tw + 2 * canvas.px(8), canvas.px(22, 13)
            x = center[0] + reach * 0.75 + canvas.px(8)
            y = center[1] - reach * 0.75 - canvas.px(8) - ch
            x = min(max(x, canvas.px(6)), width - cw - canvas.px(6))
            y = min(max(y, canvas.px(6)), height - ch - canvas.px(6))
            _draw_chip(canvas, (x, y, x + cw, y + ch), text, color)

    pad = canvas.px(28)
    if style.badge:
        _draw_badge(canvas, width - pad, pad, frame.state, align_right=True)
    if timeline is not None and style.timeline:
        lanes_h = 2 * canvas.px(6, 3) + canvas.px(5, 2)
        bar = float(to_px([[5.0, 0.0]])[0][0] - to_px([[0.0, 0.0]])[0][0])
        _draw_timeline(canvas, timeline, frame.t, pad, width - pad - bar - canvas.px(28),
                       height - pad - lanes_h)
    return out

render_demo

render_demo(
    out_dir: Any,
    scenario_id: str = "Bike Approaches Ped",
    fps: int = 30,
    size: Tuple[int, int] = (1920, 1080),
) -> Dict[str, Any]

Render the HUD and the plan view for one scenario without a simulator.

A pole-height pinhole camera at the testbench camera position looks at the crosswalk over an analytic street backdrop, with agents placed from the scenario paths. Writes demo_view.png (the camera view with the HUD), demo_bev.png, demo_still.png (split layout), demo_fisheye.png when the fisheye module is available, and demo.mp4. Returns the paths under files together with the timeline of the scenario.

Source code in bikeped/sim/overlay.py
def render_demo(out_dir: Any, scenario_id: str = "Bike Approaches Ped", fps: int = 30,
                size: Tuple[int, int] = (1920, 1080)) -> Dict[str, Any]:
    """Render the HUD and the plan view for one scenario without a simulator.

    A pole-height pinhole camera at the testbench camera position looks at the
    crosswalk over an analytic street backdrop, with agents placed from the
    scenario paths. Writes ``demo_view.png`` (the camera view with the HUD),
    ``demo_bev.png``, ``demo_still.png`` (split layout), ``demo_fisheye.png``
    when the fisheye module is available, and ``demo.mp4``. Returns the paths
    under ``files`` together with the timeline of the scenario.
    """
    from .groundtruth import scenario_to_world, synthetic_frame

    out = Path(out_dir)
    out.mkdir(parents=True, exist_ok=True)
    scenario = _find_scenario(scenario_id)
    timeline = build_timeline(scenario, fps)
    width, height = int(size[0]), int(size[1])
    ref = _testbench().FisheyeCamera()
    position = scenario_to_world(ref.cam_x, ref.cam_y, ref.height)
    target = scenario_to_world(_CROSSWALK_X_M + 3.0, 0.0, 0.6)
    pole = _look_at("pole", position, target, width, height, fov_deg=90.0)
    cameras = {"pole": pole}
    run = RunInfo(scenario_name=scenario["name"],
                  scenario_id=str(scenario.get("id", scenario["name"].lower().replace(" ", "-"))),
                  fps=fps, duration_s=timeline["duration"], engine="synthetic",
                  weather="clear_noon", views=["pole"],
                  site={"camera_xy": (ref.cam_x, ref.cam_y), "ground_z": 0.0})
    backdrop = _synthetic_backdrop(pole)
    style = DEFAULT_STYLE
    bev_side = min(height, 1080)

    from .compose import compose, fit_size

    out_size = fit_size("split", (width, height), height)

    def render(index: int, frames: List[FrameRecord]) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
        frame = frames[index]
        base = backdrop.copy()
        _paint_stand_ins(base, frame, pole)
        history = frames[max(0, index - int(style.trail_seconds * fps)):index]
        view = draw_overlay(base, frame, "pole", run=run, scenario=scenario, style=style,
                            history=history, timeline=timeline)
        bev = draw_bev(frame, run=run, scenario=scenario, size=(bev_side, bev_side), style=style)
        composite = compose("split", [view], bev, size=out_size)
        return view, bev, composite

    frames = [synthetic_frame(scenario, i, fps, cameras=cameras, states=timeline["states"])
              for i in range(timeline["n"])]
    alerts = timeline["alert"]
    if alerts:
        start, end = max(alerts, key=lambda iv: iv[1] - iv[0])
        still_index = int(round((start + 0.55 * (end - start)) * fps))
    else:
        still_index = timeline["n"] // 2
    still_index = min(max(still_index, 0), timeline["n"] - 1)
    view, bev, composite = render(still_index, frames)
    files = {"view": out / "demo_view.png", "bev": out / "demo_bev.png",
             "still": out / "demo_still.png", "clip": out / "demo.mp4"}
    cv2.imwrite(str(files["view"]), view)
    cv2.imwrite(str(files["bev"]), bev)
    cv2.imwrite(str(files["still"]), composite)

    try:
        from .fisheye import FisheyeRemap, deployed_fisheye_model

        model = deployed_fisheye_model(position, -90.0, scale=0.5)
        remap = FisheyeRemap(model, face_size=768)
        frame = replace(frames[still_index], cameras={"fisheye": model})
        faces = {}
        for face in remap.faces:
            face_cam = remap.face_camera(face)
            faces[face] = _synthetic_backdrop(face_cam, supersample=1)
            _paint_stand_ins(faces[face], frame, face_cam)
        fisheye = remap.remap(faces)
        files["fisheye"] = out / "demo_fisheye.png"
        cv2.imwrite(str(files["fisheye"]), draw_overlay(
            fisheye, frame, "fisheye", run=run, scenario=scenario, style=style,
            history=frames[max(0, still_index - int(style.trail_seconds * fps)):still_index],
            timeline=timeline))
    except ImportError:
        _LOG.info("fisheye module not available; skipping the fisheye still")

    from .recorder import VideoSink

    writer = VideoSink(files["clip"], fps)
    try:
        for index in range(timeline["n"]):
            writer.write(render(index, frames)[2])
    finally:
        writer.close()
    return {"files": {k: str(v) for k, v in files.items()}, "timeline": timeline,
            "still_index": still_index}

main

main(argv: Optional[Sequence[str]] = None) -> int

Command line entry point: python -m bikeped.sim.overlay --demo OUT_DIR.

Source code in bikeped/sim/overlay.py
def main(argv: Optional[Sequence[str]] = None) -> int:
    """Command line entry point: ``python -m bikeped.sim.overlay --demo OUT_DIR``."""
    parser = argparse.ArgumentParser(
        prog="python -m bikeped.sim.overlay",
        description="Render the HUD and plan view over a synthetic backdrop, without a simulator.")
    parser.add_argument("--demo", metavar="OUT_DIR", required=True,
                        help="directory that receives the stills and the clip")
    parser.add_argument("--scenario", default="Bike Approaches Ped",
                        help="scenario name or id (default: %(default)s)")
    parser.add_argument("--fps", type=int, default=30)
    parser.add_argument("--size", default="1920x1080", help="output size as WIDTHxHEIGHT")
    args = parser.parse_args(argv)
    width, height = (int(v) for v in args.size.lower().split("x"))
    result = render_demo(args.demo, args.scenario, fps=args.fps, size=(width, height))
    for path in result["files"].values():
        print(path)
    return 0

compose

Layouts that arrange camera views and the top-down panel into one frame.

All images are BGR uint8 arrays and sizes are (width, height) in pixels. Every layout returns a frame of exactly the requested size, so the composite video keeps one resolution whatever the views look like. Views are never stretched: each one is scaled to fit its cell and centred on the page background.

letterbox

letterbox(
    image: ndarray, size: Size, background: Tuple[int, int, int] = BACKGROUND_BGR
) -> np.ndarray

Fit image inside a frame of size without changing its aspect ratio.

The image is scaled to the largest size that fits (area averaging when it shrinks, bicubic when it grows), centred, and the rest of the frame is filled with background (BGR). Grayscale and BGRA inputs are converted to BGR.

Source code in bikeped/sim/compose.py
def letterbox(
    image: np.ndarray,
    size: Size,
    background: Tuple[int, int, int] = BACKGROUND_BGR,
) -> np.ndarray:
    """Fit ``image`` inside a frame of ``size`` without changing its aspect ratio.

    The image is scaled to the largest size that fits (area averaging when it
    shrinks, bicubic when it grows), centred, and the rest of the frame is
    filled with ``background`` (BGR). Grayscale and BGRA inputs are converted
    to BGR.
    """
    width, height = _checked_size(size)
    src = _as_bgr(image)
    src_h, src_w = src.shape[:2]
    scale = min(width / src_w, height / src_h)
    new_w = max(1, min(width, int(round(src_w * scale))))
    new_h = max(1, min(height, int(round(src_h * scale))))
    if (new_w, new_h) != (src_w, src_h):
        interpolation = cv2.INTER_AREA if scale < 1.0 else cv2.INTER_CUBIC
        src = cv2.resize(src, (new_w, new_h), interpolation=interpolation)

    canvas = np.empty((height, width, 3), dtype=np.uint8)
    canvas[:] = background
    x0 = (width - new_w) // 2
    y0 = (height - new_h) // 2
    canvas[y0:y0 + new_h, x0:x0 + new_w] = src
    return canvas

fit_size

fit_size(layout: str, view_size: Size, height: int = 1080) -> Size

Frame size at which a view fills its cell of layout without bars.

view_size is the (width, height) of the camera view, of which only the aspect ratio matters, and height the wanted frame height in pixels. The returned (width, height) is even in both dimensions, as H.264 with 4:2:0 chroma requires.

Source code in bikeped/sim/compose.py
def fit_size(layout: str, view_size: Size, height: int = 1080) -> Size:
    """Frame size at which a view fills its cell of ``layout`` without bars.

    ``view_size`` is the ``(width, height)`` of the camera view, of which only
    the aspect ratio matters, and ``height`` the wanted frame height in
    pixels. The returned ``(width, height)`` is even in both dimensions, as
    H.264 with 4:2:0 chroma requires.
    """
    if layout not in LAYOUTS:
        raise ValueError(f"unknown layout {layout!r}; expected one of {', '.join(LAYOUTS)}")
    view_w, view_h = _checked_size(view_size)
    height = _even(int(height))
    aspect = view_w / view_h
    gutter = _gutter(height)
    if layout == "single":
        width = int(round(height * aspect))
    elif layout == "split":
        inner_h = height - 2 * gutter
        width = 3 * gutter + int(round(inner_h * aspect)) + inner_h
    else:
        cell_h = (height - 3 * gutter) // 2
        width = 3 * gutter + 2 * int(round(cell_h * aspect))
    return _even(width), height

compose

compose(
    layout: str,
    images: Sequence[ndarray],
    bev: Optional[ndarray] = None,
    *,
    size: Size = (1920, 1080),
    labels: Optional[Sequence[str]] = None,
) -> np.ndarray

Arrange camera views and the top-down panel into one frame.

Parameters

layout: "single" shows the first view alone. "split" puts the first view on the left and bev in a square panel on the right that spans the inner height of the frame; without bev it is the same as "single". "quad" is a 2x2 grid with up to three views and bev in the last cell; without bev a fourth view takes that cell, and cells with nothing to show stay background. images: Camera views, primary view first. A single array counts as one view. bev: Top-down panel, usually from overlay.draw_bev. size: (width, height) of the returned frame in pixels. labels: One short caption per view, drawn in the top-left corner of its cell.

Returns

numpy.ndarray BGR uint8 frame of shape (height, width, 3). Margins and gutters are 1.5 percent of the frame height and use the page background colour.

Source code in bikeped/sim/compose.py
def compose(
    layout: str,
    images: Sequence[np.ndarray],
    bev: Optional[np.ndarray] = None,
    *,
    size: Size = (1920, 1080),
    labels: Optional[Sequence[str]] = None,
) -> np.ndarray:
    """Arrange camera views and the top-down panel into one frame.

    Parameters
    ----------
    layout:
        ``"single"`` shows the first view alone. ``"split"`` puts the first
        view on the left and ``bev`` in a square panel on the right that spans
        the inner height of the frame; without ``bev`` it is the same as
        ``"single"``. ``"quad"`` is a 2x2 grid with up to three views and
        ``bev`` in the last cell; without ``bev`` a fourth view takes that
        cell, and cells with nothing to show stay background.
    images:
        Camera views, primary view first. A single array counts as one view.
    bev:
        Top-down panel, usually from ``overlay.draw_bev``.
    size:
        ``(width, height)`` of the returned frame in pixels.
    labels:
        One short caption per view, drawn in the top-left corner of its cell.

    Returns
    -------
    numpy.ndarray
        BGR ``uint8`` frame of shape ``(height, width, 3)``. Margins and
        gutters are 1.5 percent of the frame height and use the page
        background colour.
    """
    if layout not in LAYOUTS:
        raise ValueError(f"unknown layout {layout!r}; expected one of {', '.join(LAYOUTS)}")
    if isinstance(images, np.ndarray) and images.ndim < 4:
        images = [images]
    if images is None or len(images) == 0:
        raise ValueError("compose needs at least one image")
    width, height = _checked_size(size)
    captions = list(labels) if labels is not None else []

    if layout == "single" or (layout == "split" and bev is None):
        frame = letterbox(images[0], (width, height))
        if captions:
            _draw_label(frame, captions[0], (0, 0, width, height))
        return frame

    frame = np.empty((height, width, 3), dtype=np.uint8)
    frame[:] = BACKGROUND_BGR

    if layout == "split":
        view_cell, panel_cell = _split_cells((width, height))
        _place(frame, images[0], view_cell)
        _place(frame, bev, panel_cell)
        if captions:
            _draw_label(frame, captions[0], view_cell)
        return frame

    cells = _quad_cells((width, height))
    view_cells = cells[:3] if bev is not None else cells
    for i, (image, cell) in enumerate(zip(images, view_cells)):
        _place(frame, image, cell)
        if i < len(captions):
            _draw_label(frame, captions[i], cell)
    if bev is not None:
        _place(frame, bev, cells[3])
    return frame

recorder

Writes one rendered run to disk: videos, ground truth, stills and the manifest.

Layout of a run directory (run_id = <engine>_<scenario-id>_<weather>)::

<out_dir>/<run_id>/
  manifest.json             run metadata, camera models, per-frame records, timeline
  video/<view>.mp4          clean view, H.264
  video/<view>_overlay.mp4  view with the HUD (when overlay is on)
  video/composite.mp4       the chosen layout
  frames/<view>/000000.jpg  only when frames are saved
  gt/<view>.jsonl           one line per frame: 2D boxes, 3D corners, visibility
  stills/                   first frame, first alert, closest bike-pedestrian approach

Videos are encoded with ffmpeg (libx264, yuv420p) through a pipe; when no ffmpeg executable is found, cv2.VideoWriter with the mp4v codec is used instead and a warning is logged.

VideoSink

VideoSink(path: PathLike, fps: float, crf: int = 18, ffmpeg: Optional[str] = None)

An H.264 mp4 writer fed with BGR uint8 frames of a fixed size.

The size is the first frame's size rounded up to even numbers, as yuv420p requires; later frames are padded or resized to it.

Source code in bikeped/sim/recorder.py
def __init__(self, path: PathLike, fps: float, crf: int = 18,
             ffmpeg: Optional[str] = None) -> None:
    self.path = Path(path)
    self.fps = float(fps)
    self.crf = int(crf)
    self.ffmpeg = ffmpeg if ffmpeg is not None else find_ffmpeg()
    self.size: Optional[Tuple[int, int]] = None
    self.frames = 0
    self._proc: Optional[subprocess.Popen] = None
    self._stderr: Any = None
    self._cv: Optional[cv2.VideoWriter] = None
    self._closed = False

close

close() -> None

Finish the file. Safe to call more than once.

Source code in bikeped/sim/recorder.py
def close(self) -> None:
    """Finish the file. Safe to call more than once."""
    if self._closed:
        return
    self._closed = True
    if self._proc is not None:
        try:
            self._proc.stdin.close()
        except OSError:
            pass
        code = self._proc.wait()
        message = self._ffmpeg_error()
        self._stderr.close()
        if code != 0:
            _LOG.error("ffmpeg exited with %d writing %s: %s", code, self.path, message)
    if self._cv is not None:
        self._cv.release()

Recorder

Recorder(
    out_dir: PathLike,
    run: RunInfo,
    *,
    views: Sequence[str],
    overlay: bool = True,
    layout: str = "single",
    save_frames: str = "none",
    crf: int = 18,
    scenario: Optional[Dict[str, Any]] = None,
    style: Any = None,
    composite_height: Optional[int] = None,
    run_id: Optional[str] = None,
)

Collects :class:FrameRecord objects of one run and writes the run directory.

views are the view names to record from frame.images; the first is the primary view used by the composite. overlay adds a HUD video per view, layout is one of single, split (primary view plus a plan view) or quad (up to three views plus a plan view). save_frames is none, jpg or png. crf is the x264 quality (lower is better). scenario is the scenario dict; it feeds the HUD timeline, the plan view and the manifest. composite_height sets the composite's height in pixels (default: the primary view's height).

Use as a context manager, or call :meth:close; the manifest is written on close even when rendering stopped with an exception.

Source code in bikeped/sim/recorder.py
def __init__(self, out_dir: PathLike, run: RunInfo, *, views: Sequence[str],
             overlay: bool = True, layout: str = "single", save_frames: str = "none",
             crf: int = 18, scenario: Optional[Dict[str, Any]] = None,
             style: Any = None, composite_height: Optional[int] = None,
             run_id: Optional[str] = None) -> None:
    if layout not in LAYOUTS:
        raise ValueError(f"unknown layout {layout!r}; expected one of {LAYOUTS}")
    if save_frames not in SAVE_FRAME_FORMATS:
        raise ValueError(f"save_frames must be one of {SAVE_FRAME_FORMATS}")
    if not views:
        raise ValueError("Recorder needs at least one view")
    self.run = run
    self.views = list(views)
    self.overlay = bool(overlay)
    self.layout = layout
    self.save_frames = save_frames
    self.crf = int(crf)
    self.scenario = scenario
    self.style = style
    self.composite_height = composite_height
    self.run_id = run_id or f"{run.engine}_{run.scenario_id}_{run.weather}"
    self.run_dir = Path(out_dir) / self.run_id
    self.run_dir.mkdir(parents=True, exist_ok=True)
    self.fps = float(run.fps or 30)
    self._ffmpeg = find_ffmpeg()
    self._sinks: Dict[str, VideoSink] = {}
    self._gt_files: Dict[str, Any] = {}
    self._cameras: Dict[str, Dict[str, Any]] = {}
    self._records: List[Dict[str, Any]] = []
    self._history: Deque[FrameRecord] = deque(maxlen=max(2, int(round(1.5 * self.fps)) + 1))
    self._stills: Dict[str, np.ndarray] = {}
    self._closest: Optional[float] = None
    self._timeline: Optional[Dict[str, Any]] = None
    self._missing: set = set()
    self._closed = False
    self._result: Optional[Dict[str, Any]] = None
    self._error: Optional[str] = None

write

write(frame: FrameRecord) -> None

Record one frame: videos, ground truth, saved frames, still candidates.

Source code in bikeped/sim/recorder.py
def write(self, frame: FrameRecord) -> None:
    """Record one frame: videos, ground truth, saved frames, still candidates."""
    if self._closed:
        raise RuntimeError("Recorder is closed")
    shown: Dict[str, np.ndarray] = {}
    for view in self.views:
        image = frame.images.get(view)
        if image is None:
            if view not in self._missing:
                _LOG.warning("frame %d has no image for view %r", frame.index, view)
                self._missing.add(view)
            continue
        cam = frame.cameras.get(view)
        if cam is not None and view not in self._cameras:
            self._cameras[view] = cam.to_dict()
        self._sink(view).write(image)
        h, w = image.shape[:2]
        line = {"index": frame.index, "t": round(float(frame.t), 4), "width": w, "height": h,
                "objects": frame_ground_truth(frame, view)}
        self._gt(view).write(json.dumps(_json_safe(line)) + "\n")
        if self.save_frames != "none":
            folder = self.run_dir / "frames" / view
            folder.mkdir(parents=True, exist_ok=True)
            params = [cv2.IMWRITE_JPEG_QUALITY, 95] if self.save_frames == "jpg" else []
            cv2.imwrite(str(folder / f"{frame.index:06d}.{self.save_frames}"), image, params)
        if self.overlay:
            annotated = self._draw(image, frame, view)
            self._sink(f"{view}_overlay").write(annotated)
            shown[view] = annotated
        else:
            shown[view] = image
    if shown:
        composite = self._composite(frame, shown)
        self._sink("composite").write(composite)
        self._note_stills(frame, composite)
    self._records.append(frame.to_dict())
    self._history.append(replace(frame, images={}, aux={}))

timeline_summary

timeline_summary() -> Dict[str, Any]

Frame counts per state, alert and danger intervals, first alert and lead time.

lead_time_s is the onset of the first ground-truth danger interval minus the first alert time (positive when the alert came first); it is None unless both exist. Times are seconds of scenario time.

Source code in bikeped/sim/recorder.py
def timeline_summary(self) -> Dict[str, Any]:
    """Frame counts per state, alert and danger intervals, first alert and lead time.

    ``lead_time_s`` is the onset of the first ground-truth danger interval
    minus the first alert time (positive when the alert came first); it is
    ``None`` unless both exist. Times are seconds of scenario time.
    """
    from .types import STATES

    times = [r["t"] for r in self._records]
    states = [r["state"] for r in self._records]
    danger = [bool((r.get("danger") or {}).get("is_danger")) for r in self._records]
    actionable = [bool((r.get("danger") or {}).get("actionable")) for r in self._records]
    dt = 1.0 / self.fps
    alert = _intervals(times, [s == "alert" for s in states], dt)
    danger_iv = _intervals(times, danger, dt)
    first_alert = alert[0][0] if alert else None
    first_danger = danger_iv[0][0] if danger_iv else None
    closest = None
    best = None
    for r in self._records:
        people = [a for a in r["agents"] if a["kind"] == "person" and a["role"] == "scripted"]
        bikes = [a for a in r["agents"] if a["kind"] == "bike" and a["role"] == "scripted"]
        for b in bikes:
            for p in people:
                d = math.hypot(b["sx"] - p["sx"], b["sy"] - p["sy"])
                if best is None or d < best:
                    best = d
                    closest = {"index": r["index"], "t": r["t"], "bike_id": b["id"],
                               "person_id": p["id"], "distance_m": round(d, 3)}
    return {
        "frames": len(self._records),
        "state_counts": {s: states.count(s) for s in STATES},
        "alert_intervals": alert,
        "warning_intervals": _intervals(times, [s == "warning" for s in states], dt),
        "danger_intervals": danger_iv,
        "actionable_intervals": _intervals(times, actionable, dt),
        "first_alert_t": first_alert,
        "first_danger_t": first_danger,
        "lead_time_s": (round(first_danger - first_alert, 4)
                        if first_alert is not None and first_danger is not None else None),
        "closest_approach": closest,
    }

close

close() -> Dict[str, Any]

Finish all files and write the manifest; returns paths and counts.

Returns {'run_dir', 'manifest', 'videos', 'stills', 'frames'}. Calling it again returns the same result.

Source code in bikeped/sim/recorder.py
def close(self) -> Dict[str, Any]:
    """Finish all files and write the manifest; returns paths and counts.

    Returns ``{'run_dir', 'manifest', 'videos', 'stills', 'frames'}``.
    Calling it again returns the same result.
    """
    if self._result is not None:
        return self._result
    self._closed = True
    for sink in self._sinks.values():
        try:
            sink.close()
        except Exception as exc:
            _LOG.error("closing %s failed: %s", sink.path, exc)
    for fh in self._gt_files.values():
        fh.close()
    stills = self._write_stills()
    videos = {k: str(s.path.relative_to(self.run_dir)) for k, s in self._sinks.items()
              if s.frames > 0}
    frames_dir = {}
    if self.save_frames != "none":
        frames_dir = {v: f"frames/{v}" for v in self.views
                      if (self.run_dir / "frames" / v).is_dir()}
    manifest = {
        "run_id": self.run_id,
        "run": self.run.to_dict(),
        "scenario": self.scenario,
        "layout": self.layout,
        "overlay": self.overlay,
        "views": self.views,
        "cameras": self._cameras,
        "videos": videos,
        "frames_dir": frames_dir,
        "frame_ext": None if self.save_frames == "none" else self.save_frames,
        "gt": {v: f"gt/{v}.jsonl" for v in self._gt_files},
        "stills": [str(Path(p).relative_to(self.run_dir)) for p in stills],
        "timeline": self.timeline_summary(),
        "complete": self._error is None,
        "error": self._error,
        "frame_count": len(self._records),
        "frames": self._records,
    }
    path = self.run_dir / "manifest.json"
    with open(path, "w") as fh:
        json.dump(_json_safe(manifest), fh, indent=1)
    self._result = {
        "run_dir": str(self.run_dir),
        "manifest": str(path),
        "videos": [str(self.run_dir / v) for v in videos.values()],
        "stills": stills,
        "frames": len(self._records),
    }
    return self._result

find_ffmpeg

find_ffmpeg() -> Optional[str]

Path of an ffmpeg executable: the one on PATH, else the imageio-ffmpeg binary.

Source code in bikeped/sim/recorder.py
def find_ffmpeg() -> Optional[str]:
    """Path of an ffmpeg executable: the one on ``PATH``, else the ``imageio-ffmpeg`` binary."""
    exe = shutil.which("ffmpeg")
    if exe:
        return exe
    try:
        import imageio_ffmpeg

        return imageio_ffmpeg.get_ffmpeg_exe()
    except Exception:
        return None

export

Detection datasets from a rendered run.

A run directory written by bikeped.sim.recorder.Recorder holds the clean video of every view, optionally the individual frames, and one ground-truth file per view. The exporters here turn one view of a run into a COCO annotation file or a YOLO dataset folder with the classes person and bicycle.

Boxes are axis-aligned and given in pixels of the exported image, with the origin at the top-left corner. Saved frames are copied as they are; when a run has no saved frames the images are decoded from the clean video of the view, whose frame k belongs to line k of the ground-truth file.

export_coco

export_coco(
    run_dir: PathLike,
    out_path: PathLike,
    view: str = "pole",
    *,
    min_visible: float = 0.0,
    min_area: float = 0.0,
    every: int = 1,
    images_dir: Optional[PathLike] = None,
) -> Dict[str, Any]

Write one view of a run as a COCO detection dataset.

Parameters

run_dir: Run directory with manifest.json and gt/<view>.jsonl. out_path: Annotation file to write. bbox is [x, y, width, height] in pixels; the categories are person (id 1) and bicycle (id 2). view: Name of the camera view to export. min_visible: Drop boxes whose visible_fraction is known and below this value. Boxes without a visibility estimate are kept. min_area: Drop boxes smaller than this area in square pixels. every: Export every every-th frame. images_dir: Folder for the images; images next to out_path by default. file_name in the annotation file is relative to it.

Returns

dict {"images": n, "annotations": m, "path": out_path}.

Source code in bikeped/sim/export.py
def export_coco(
    run_dir: PathLike,
    out_path: PathLike,
    view: str = "pole",
    *,
    min_visible: float = 0.0,
    min_area: float = 0.0,
    every: int = 1,
    images_dir: Optional[PathLike] = None,
) -> Dict[str, Any]:
    """Write one view of a run as a COCO detection dataset.

    Parameters
    ----------
    run_dir:
        Run directory with ``manifest.json`` and ``gt/<view>.jsonl``.
    out_path:
        Annotation file to write. ``bbox`` is ``[x, y, width, height]`` in
        pixels; the categories are ``person`` (id 1) and ``bicycle`` (id 2).
    view:
        Name of the camera view to export.
    min_visible:
        Drop boxes whose ``visible_fraction`` is known and below this value.
        Boxes without a visibility estimate are kept.
    min_area:
        Drop boxes smaller than this area in square pixels.
    every:
        Export every ``every``-th frame.
    images_dir:
        Folder for the images; ``images`` next to ``out_path`` by default.
        ``file_name`` in the annotation file is relative to it.

    Returns
    -------
    dict
        ``{"images": n, "annotations": m, "path": out_path}``.
    """
    run = _load_run(run_dir, view)
    out_file = Path(out_path)
    images_root = Path(images_dir) if images_dir is not None else out_file.parent / "images"
    out_file.parent.mkdir(parents=True, exist_ok=True)
    images_root.mkdir(parents=True, exist_ok=True)

    images: List[Dict[str, Any]] = []
    annotations: List[Dict[str, Any]] = []
    for sample in _iter_samples(run, every, min_visible, min_area):
        suffix = sample.source.suffix.lower() if sample.source is not None else ".jpg"
        name = f"{run.stem}_{sample.index:06d}{suffix}"
        _store_image(sample, images_root / name)
        image_id = len(images) + 1
        images.append(
            {
                "id": image_id,
                "file_name": name,
                "width": sample.width,
                "height": sample.height,
                "frame_index": sample.index,
                "time_s": round(sample.t, 4),
            }
        )
        for box in sample.boxes:
            annotations.append(
                {
                    "id": len(annotations) + 1,
                    "image_id": image_id,
                    "category_id": box.class_index + 1,
                    "bbox": [round(v, 2) for v in (box.x, box.y, box.w, box.h)],
                    "area": round(box.w * box.h, 2),
                    "iscrowd": 0,
                    "agent_id": box.agent_id,
                    "truncation": box.truncation,
                    "visible_fraction": box.visible_fraction,
                    "distance_m": box.distance_m,
                }
            )

    info = run.manifest.get("run") or {}
    dataset = {
        "info": {
            "description": f"bikeped render {run.run_id}, view {view}",
            "version": "1.0",
            "run_id": run.run_id,
            "view": view,
            "scenario": info.get("scenario_name", ""),
            "engine": info.get("engine", ""),
            "map": info.get("map", ""),
            "weather": info.get("weather", ""),
        },
        "licenses": [],
        "images": images,
        "annotations": annotations,
        "categories": [
            {"id": i + 1, "name": name, "supercategory": _SUPERCATEGORIES[i]}
            for i, name in enumerate(CLASS_NAMES)
        ],
    }
    with open(out_file, "w", encoding="utf-8") as fh:
        json.dump(dataset, fh)
        fh.write("\n")
    return {"images": len(images), "annotations": len(annotations), "path": str(out_file)}

export_yolo

export_yolo(
    run_dir: PathLike,
    out_dir: PathLike,
    view: str = "pole",
    *,
    min_visible: float = 0.0,
    min_area: float = 0.0,
    every: int = 1,
) -> Dict[str, Any]

Write one view of a run as a YOLO detection dataset.

The output folder gets images/<run_id>_<view>_<frame>.jpg, a label file with the same stem under labels/ and a data.yaml. Each label line is class cx cy w h with the box centre and size divided by the image width and height; class 0 is person and class 1 is bicycle. Frames without boxes get an empty label file, so they count as negatives.

min_visible, min_area and every filter as in :func:export_coco. Exports of several runs into the same folder add up, because the file names carry the run id.

Returns

dict {"images": n, "labels": m, "path": out_dir}, where m counts the boxes written.

Source code in bikeped/sim/export.py
def export_yolo(
    run_dir: PathLike,
    out_dir: PathLike,
    view: str = "pole",
    *,
    min_visible: float = 0.0,
    min_area: float = 0.0,
    every: int = 1,
) -> Dict[str, Any]:
    """Write one view of a run as a YOLO detection dataset.

    The output folder gets ``images/<run_id>_<view>_<frame>.jpg``, a label
    file with the same stem under ``labels/`` and a ``data.yaml``. Each label
    line is ``class cx cy w h`` with the box centre and size divided by the
    image width and height; class 0 is ``person`` and class 1 is ``bicycle``.
    Frames without boxes get an empty label file, so they count as negatives.

    ``min_visible``, ``min_area`` and ``every`` filter as in
    :func:`export_coco`. Exports of several runs into the same folder add up,
    because the file names carry the run id.

    Returns
    -------
    dict
        ``{"images": n, "labels": m, "path": out_dir}``, where ``m`` counts
        the boxes written.
    """
    run = _load_run(run_dir, view)
    root = Path(out_dir)
    images_root = root / "images"
    labels_root = root / "labels"
    images_root.mkdir(parents=True, exist_ok=True)
    labels_root.mkdir(parents=True, exist_ok=True)

    n_images = 0
    n_labels = 0
    for sample in _iter_samples(run, every, min_visible, min_area):
        stem = f"{run.stem}_{sample.index:06d}"
        _store_image(sample, images_root / f"{stem}.jpg")
        lines = [
            "{:d} {:.6f} {:.6f} {:.6f} {:.6f}".format(
                box.class_index,
                (box.x + box.w / 2.0) / sample.width,
                (box.y + box.h / 2.0) / sample.height,
                box.w / sample.width,
                box.h / sample.height,
            )
            for box in sample.boxes
        ]
        with open(labels_root / f"{stem}.txt", "w", encoding="utf-8") as fh:
            fh.write("".join(line + "\n" for line in lines))
        n_images += 1
        n_labels += len(lines)

    data = {
        "path": str(root.resolve()),
        "train": "images",
        "val": "images",
        "names": dict(enumerate(CLASS_NAMES)),
    }
    with open(root / "data.yaml", "w", encoding="utf-8") as fh:
        yaml.safe_dump(data, fh, sort_keys=False)
    return {"images": n_images, "labels": n_labels, "path": str(root)}

cli

Command line interface: python -m bikeped.sim {probe,server,presets,render,export}.