Skip to content

Extended suites

The conformance suite is fixed by the paper. Scenarios written after it live in separate suites of bikeped.scenarios, so the published numbers stay reproducible while the library grows. This page describes the extended and stress suites and lists their scenarios.

Working with a suite

python -m bikeped.scenarios list --suite extended
python -m bikeped.scenarios evaluate --suite extended
from bikeped.scenarios import get_suite

for scenario in get_suite('extended'):
    print(scenario['id'], scenario['family'], scenario['expected_dominant'])

Scenarios in these suites use the conformance format, so they run in the testbench, in the browser simulator and in the CARLA renderer without conversion. Per-agent model hints (child, wheelchair, e-scooter, cargo bike and so on) change only how the renderer draws an agent.

extended

Forty-one hand-designed urban conflicts and deliberate negatives in twelve families. Speeds follow published ranges: walking 1.2 to 1.5 m/s, older adults 0.8 to 1.0, wheelchairs 1.0 to 1.3, commuter cyclists 4 to 7, fast road cyclists 8 to 11, e-scooters 5 to 7, cargo bikes 3.5 to 5.5.

Family Scenarios Covers
turning 4 Turning conflicts
occlusion 3 Occlusion and late emergence
yielding 4 Yielding and braking cyclists
counter-flow 2 Wrong-way and counter-flow riders
shared-space 2 Shared space and sidewalk riding
groups 5 Groups and platoons
vulnerable 4 Vulnerable road users
micromobility 4 Micromobility
pedestrian-behaviour 5 Pedestrian behaviour
near-miss 4 Near-miss geometry
signal-platoon 1 Signal-phase platoons
negative 3 True negatives

The dominant state of a run is the most severe state (alert > warning > safe > idle) that the pipeline holds for at least 0.3 s in total over the run. expected_dominant of an extended or stress scenario is the same rule applied to the per-frame labels of compute_ground_truth_danger rather than a judgement, and a test in tests/scenarios/ keeps the two in agreement. Scenarios that the default pipeline gets wrong keep their label and carry the tag known-hard: either the dominant state differs from the expected one, or the first alert comes more than 0.5 s after a danger interval begins. Fourteen scenarios carry it across the library: seven false alarms, five late alerts and two in the stress suite. They are the useful ones when you change the pipeline.

The ground truth is conservative. It labels a comfortable yield, a 2 m tangential pass and the whole ±3 s time-gap sweep of the generators as danger, so a yes in the table below means the pipeline agrees with a cautious reference rather than with a human judgement of the scene.

$ python -m bikeped.scenarios evaluate --suite extended
scenario                             dur  ag danger  alert  false lead s (per interval)  dominant expected  ok
--------------------------------------------------------------------------------------------------------------
right-hook-across-crosswalk         12.0   2   2.50   3.67   1.27 +1.3                      alert    alert  yes
left-hook-across-crosswalk          10.6   2   2.63   3.47   0.83 +0.7                      alert    alert  yes
turn-in-from-cross-street            9.4   2   3.83   3.33   0.00 -0.0                      alert    alert  yes
wide-turn-clears-pedestrian          8.5   2   0.00   3.13   3.13 -                         alert  warning  NO
...
--------------------------------------------------------------------------------------------------------------
41 scenarios, dominant state matches expected in 34; 23 of 33 danger intervals alerted no later than one frame after onset; false-alert time 33.93 s

evaluate runs decision_testbench.simulate_scenario with ideal perception. The columns are seconds of ground-truth danger, of alert and of false alert, the lead of the first alert over each danger interval (negative means late), and the dominant state against the expected one.

Scenario Family Expected Duration Agents
Right Hook Across Crosswalk right-hook-across-crosswalk
A cyclist at 5 m/s turns right off the main road through the far end of the crosswalk, where a pedestrian has just stepped off the kerb.
turning alert 12.0 s 1 adult, 1 bicycle
Left Hook Across Crosswalk left-hook-across-crosswalk
A cyclist turns left across the oncoming lane and sweeps through the near half of the crosswalk while a pedestrian is walking out from that kerb.
turning alert 10.6 s 1 adult, 1 bicycle
Turn In From Cross Street turn-in-from-cross-street
A cyclist comes up the cross street, turns right onto the main road and rides through the middle of the crosswalk as a pedestrian reaches it.
turning alert 9.4 s 1 adult, 1 bicycle
Wide Turn Clears Pedestrian wide-turn-clears-pedestrian
A cyclist swings a wide left turn that stays more than 6 m from a pedestrian walking on the far half of the crosswalk.
turning warning 8.5 s 1 adult, 1 bicycle
Steps Out From Behind Van steps-out-from-behind-van
A pedestrian hidden by a parked van is first detected stepping into the road 3.5 m from the kerb line, 1.5 s before a 6 m/s cyclist reaches the crossing point.
occlusion alert 10.6 s 1 adult, 1 bicycle
Bus Stop Crowd Emergence bus-stop-crowd-emergence
Two people wait at a bus stop; a third steps out from behind them at a brisk 1.8 m/s straight into the path of a cyclist.
occlusion alert 9.9 s 3 adult, 1 bicycle
Cyclist Emerges From Behind Bus cyclist-emerges-from-behind-bus
A cyclist overtaking a stopped bus is first seen 8 m from the crosswalk at 6 m/s while a pedestrian is crossing in front of the bus.
occlusion alert 9.8 s 1 adult, 1 bicycle
Cyclist Stops Before Crosswalk cyclist-stops-before-crosswalk
A cyclist at 5.5 m/s brakes comfortably at 2 m/s^2 to a stop 4 m short of the crosswalk, waits for the pedestrian to clear, then rides on.
yielding alert 16.2 s 1 adult, 1 bicycle
Rolling Yield rolling-yield
A cyclist slows from 6 m/s to 2 m/s and rolls through the crosswalk after the pedestrian has passed the bike lane.
yielding warning 12.8 s 1 adult, 1 bicycle
Track Stand Then Go track-stand-then-go
A cyclist balances in place 5 m before the crosswalk while a pedestrian crosses, then sets off once the pedestrian is on the far side.
yielding warning 14.7 s 1 adult, 1 bicycle
Late Hard Braking late-hard-braking
A cyclist at 6.5 m/s notices the pedestrian late and brakes hard at 5 m/s^2. The bike stops about a metre from them.
yielding alert 9.8 s 1 adult, 1 bicycle
Wrong-Way Rider wrong-way-rider
A cyclist rides against traffic towards +x at 5 m/s; the pedestrian checks the usual direction and steps out into the rider.
counter-flow alert 9.8 s 1 adult, 1 bicycle
Contraflow Lane Pass contraflow-lane-pass
A wrong-way cyclist stays in the far contraflow lane and crosses the crosswalk well ahead of the pedestrian, who is still at the opposite kerb.
counter-flow warning 13.9 s 1 adult, 1 bicycle
Sidewalk Rider Head-On sidewalk-rider-head-on
A cyclist rides along the sidewalk at 4.5 m/s towards a pedestrian walking the other way on the same line.
shared-space alert 17.6 s 1 adult, 1 bicycle
Shared Plaza Slalom shared-plaza-slalom
In a shared plaza a cyclist at 4 m/s weaves between two walkers and cuts close in front of a third.
shared-space alert 9.4 s 3 adult, 1 bicycle
School Group Crossing school-group-crossing
Two adults walk a line of six children across at 1.0 m/s; a cyclist at 5 m/s reaches the crosswalk while the group is strung across the lane.
groups alert 16.0 s 2 adult, 6 child, 1 bicycle
Cyclist Platoon cyclist-platoon
Four commuters ride in single file 1.2 s apart at 5.5 m/s into a pedestrian who is halfway across.
groups alert 11.0 s 1 adult, 4 bicycle
Two Abreast two-abreast
Two cyclists ride side by side at 6 m/s and fill the lane as a pedestrian crosses.
groups alert 9.8 s 1 adult, 2 bicycle
Pedestrian Couple pedestrian-couple
A couple walks across side by side at 1.3 m/s and a cyclist at 6 m/s rides through their line.
groups alert 10.6 s 2 adult, 1 bicycle
Crowd With Straggler crowd-with-straggler
Five people cross together and clear the lane; a straggler follows 3.5 s later and meets the cyclist the rest of the group avoided.
groups alert 14.3 s 6 adult, 1 bicycle
Older Adult Slow Start older-adult-slow-start
An older pedestrian waits 2 s at the kerb, then crosses at 0.85 m/s and is in the lane when a 5.5 m/s cyclist arrives.
vulnerable alert 17.8 s 1 adult, 1 bicycle
Wheelchair Mid-Crossing Pause wheelchair-mid-crossing-pause
A wheelchair user crossing at 1.1 m/s stops for 3 s in the middle of the road; a cyclist rides through the stopping point.
vulnerable alert 15.3 s 1 wheelchair, 1 bicycle
Child Darts Back child-darts-back
A child halfway across turns and runs back at 2.5 m/s for a dropped toy, into the lane the cyclist had counted on being empty.
vulnerable alert 19.0 s 1 child, 1 bicycle
Guide Dog Pace guide-dog-pace
A visually impaired pedestrian with a guide dog crosses at a steady 1.0 m/s and cannot see the approaching 6 m/s cyclist.
vulnerable alert 13.5 s 1 adult, 1 bicycle
E-Scooter Weaving e-scooter-weaving
An e-scooter rider at 6 m/s weaves a metre either side of the lane line and reaches the crosswalk while a pedestrian is in it.
micromobility alert 9.8 s 1 adult, 1 escooter
E-Bike In The Door Zone e-bike-in-the-door-zone
A fast e-bike at 10 m/s rides close to parked cars; a pedestrian steps out from between two of them a car length ahead.
micromobility alert 10.3 s 1 adult, 1 ebike
Cargo Bike Long Stop cargo-bike-long-stop
A loaded cargo bike at 5 m/s brakes at only 1.5 m/s^2 and needs most of the approach to stop. It comes to rest on the crosswalk next to the pedestrian.
micromobility alert 10.5 s 1 adult, 1 cargo_bike
Delivery Rider Cuts Corner delivery-rider-cuts-corner
A delivery rider on an e-bike at 7 m/s cuts the corner from the cross street, diagonally through the crosswalk.
micromobility alert 10.3 s 1 adult, 1 ebike
Hesitation At The Kerb hesitation-at-the-kerb
A pedestrian steps off the kerb, stops for 1.5 s unsure of the cyclist, then goes anyway as the cyclist arrives.
pedestrian-behaviour alert 11.1 s 1 adult, 1 bicycle
Turns Back Mid-Crossing turns-back-mid-crossing
A pedestrian reaches the centre line, changes their mind and walks back towards the kerb they came from, into the cyclist's lane.
pedestrian-behaviour alert 11.1 s 1 adult, 1 bicycle
Diagonal Runner diagonal-runner
A runner crosses diagonally at 3.5 m/s, outside the crosswalk lines, and meets a 6 m/s cyclist in the lane.
pedestrian-behaviour alert 6.6 s 1 runner, 1 bicycle
Phone-Distracted Diagonal phone-distracted-diagonal
Looking at a phone, a pedestrian drifts diagonally across at 0.9 m/s and spends a long time in the lane.
pedestrian-behaviour alert 18.4 s 1 adult, 1 bicycle
Mid-Block Crossing mid-block-crossing
A pedestrian crosses 10 m upstream of the crosswalk, between gaps in traffic, and the cyclist meets them before reaching the marked crossing.
pedestrian-behaviour alert 9.2 s 1 adult, 1 bicycle
Passes In Front passes-in-front
The cyclist crosses the pedestrian's line about a second before the pedestrian reaches it, a margin of a few metres.
near-miss alert 9.8 s 1 adult, 1 bicycle
Passes Behind passes-behind
The cyclist rides through the crosswalk 2.5 m behind a pedestrian who has already crossed the lane.
near-miss warning 10.7 s 1 adult, 1 bicycle
Tangential Pass tangential-pass
A cyclist passes 2 m to the side of a pedestrian standing at the edge of the lane, without changing course.
near-miss alert 7.5 s 1 adult, 1 bicycle
Parallel Never Converges parallel-never-converges
A cyclist rides along the road while a pedestrian walks the same way on the sidewalk 7 m to the side; the paths never meet.
near-miss warning 13.4 s 1 adult, 1 bicycle
Green Wave Burst green-wave-burst
Five cyclists waiting at a light set off together when it turns green and meet a pedestrian who started crossing late.
signal-platoon alert 13.2 s 1 adult, 5 bicycle
Far Lane Moving Away far-lane-moving-away
A cyclist just past the crosswalk rides away along the far lane while a pedestrian starts crossing behind them.
negative warning 10.8 s 1 adult, 1 bicycle
Fast Pass After Clearing fast-pass-after-clearing
A road cyclist at 9 m/s goes through the crosswalk a second after the pedestrian has stepped onto the far kerb.
negative warning 11.9 s 1 adult, 1 road_bike
Waits At The Kerb waits-at-the-kerb
A pedestrian stands on the kerb the whole time and never steps out while a cyclist passes at 7 m/s.
negative warning 8.5 s 1 adult, 1 bicycle

stress

Eight dense scenes with 12 to 24 agents in view at once, for throughput and tracker-load testing. Positions and start times are jittered with fixed seeds, so the suite is the same on every run. Two of them carry known-hard.

$ python -m bikeped.scenarios evaluate --suite stress
...
sidewalk-tour-group                 33.5  16   0.00   5.63   5.63 -                         alert  warning  NO
crosswalk-without-cyclists          21.8  24   0.00   0.00   0.00 -                          safe     safe  yes
...
8 scenarios, dominant state matches expected in 7; 4 of 7 danger intervals alerted no later than one frame after onset; false-alert time 8.37 s
Scenario Family Expected Duration Agents
Rush Hour Crosswalk rush-hour-crosswalk
Twelve pedestrians cross in both directions over 8 s while six cyclists ride through from both ends.
dense alert 19.5 s 12 adult, 6 bicycle
Stadium Exit stadium-exit
Twenty people leave an event and flood the crosswalk in one direction as three cyclists arrive.
dense alert 17.5 s 20 adult, 3 bicycle
Bike Commute Peak bike-commute-peak
Sixteen cyclists in a steady stream at commuter speeds and four pedestrians picking their way across.
dense alert 19.6 s 4 adult, 16 bicycle
Wet Night Mixed Traffic wet-night-mixed-traffic
Ten pedestrians and eight riders on a wet night; some riders go as fast as e-bikes.
dense alert 19.0 s 10 adult, 8 bicycle
Sidewalk Tour Group sidewalk-tour-group
A tour group of ten walks along the sidewalk while six cyclists pass on the road; nobody crosses.
dense warning 33.5 s 10 adult, 6 bicycle
Crosswalk Without Cyclists crosswalk-without-cyclists
Twenty-four pedestrians cross in both directions and no cyclist appears.
dense safe 21.8 s 24 adult
Group Ride Through Crowd group-ride-through-crowd
A group ride of ten passes in two lines through a crosswalk with eight pedestrians on it.
dense alert 15.5 s 8 adult, 10 bicycle
Market Day Shared Street market-day-shared-street
A shared street with fourteen pedestrians wandering across at random angles and six slow cyclists.
dense alert 25.9 s 14 adult, 6 bicycle