Fisheye calibration¶
Ground positions come from one pixel per detection, so the lens model has to match the camera. This page covers capturing checkerboard frames, fitting the equidistant model with calibrate_fisheye.py and comparing projection models. It ends with the files and scripts that read the result.
The three scripts live in calibration/. The commands below are run from that folder, and output paths are relative to the current directory.
The shipped calibration¶
camera_calibration.json at the repository root is the calibration of the reference camera.
| Quantity | Value |
|---|---|
| Model | equidistant, r = f · θ |
| Optical centre | (1752.69, 1804.54) px in the 3500 × 3500 frame |
| Focal length | 1013.31 px |
| Field of view | 197.9° |
| Image circle radius | 1750 px |
k1, k2 |
0, 0 |
| RMS reprojection error | 4.6334 px |
| Data | 42 frames, 1470 corners |
Configuration reference describes every key of the file.
Why a custom solver¶
OpenCV's fisheye calibration fits the Kannala-Brandt model with polynomial distortion coefficients. The module docstring of calibrate_fisheye.py notes that this does not fit a 200° spherical lens on a 16:9 sensor, so the script fits the equidistant projection directly with its own bundle adjustment. The parameters are the optical centre cx, cy, the focal length f and, on request, the correction terms k1 and k2 in
1. Capture frames¶
python calibration/capture_calibration.py
python calibration/capture_calibration.py --camera 0 --outdir my_cal_frames
The script opens the camera at 3840 × 2160 (V4L2 with MJPG on Linux) and shows a preview. Press R to start recording one frame per second and R again to stop. Q or Esc quits. Frames are written as frame_0000.png, frame_0001.png and so on, and numbering continues after frames that are already in the folder.
| Flag | Default | Meaning |
|---|---|---|
--camera |
0 |
Camera index. |
--outdir |
calibration_frames |
Output directory. |
--display-width |
1200 |
Width of the preview window in pixels. |
Move the checkerboard through the whole image circle while recording, including the edge. The lens model matters most there. Calibration needs at least 3 frames with a detected board.
2. Calibrate¶
python calibration/calibrate_fisheye.py --images "calibration_frames/*.png"
python calibration/calibrate_fisheye.py --video calibration.avi --interval 30
python calibration/calibrate_fisheye.py --images "calibration_frames/*.png" --fit-distortion
python calibration/calibrate_fisheye.py --video calibration.avi --interactive
One of --images or --video is required.
| Flag | Default | Meaning |
|---|---|---|
--images |
Image paths or glob patterns showing the checkerboard. | |
--video |
Video file with checkerboard frames. | |
--interactive |
off | Step through the video and pick frames by hand. Requires --video. |
--board-width |
9 |
Inner corners along the board width. |
--board-height |
6 |
Inner corners along the board height. |
--square-size |
0.025 |
Square size in metres. |
--pad-size |
3840 |
Preprocessing pad size in pixels. |
--crop-size |
3500 |
Preprocessing crop size in pixels. |
--interval |
30 |
Frame sampling interval in video mode. |
--max-frames |
50 |
Maximum number of frames taken from a video. |
--initial-fov |
200 |
Initial field-of-view guess in degrees. |
--fit-distortion |
off | Also fit k1 and k2. |
--output |
camera_calibration.json |
Output JSON. A .yaml with the same stem is written next to it. |
--visualize |
off | Show the reprojection of each frame. Q stops the review. |
--save-viz |
Directory for reprojection images. |
In interactive mode the keys are S to keep the current frame (only when a board is detected), Space for the next frame, D and A to jump 10 frames forward or back, and Q to finish.
What the script does¶
- Preprocessing. Each frame is centred on a
pad-sizeblack square and the centrecrop-sizesquare is cut out, the same pad and crop the live system applies. With the defaults a 3840 × 2160 frame becomes 3500 × 3500, and every intrinsic refers to that cropped frame. - Corner detection.
findChessboardCornersneeds straight lines between corners, which a fisheye image does not have. The script remaps the fisheye frame to a 1500 px perspective view with a 160° field of view using the initial focal-length guess, detects the board there, maps the corners back to fisheye pixels and refines them withcornerSubPixon the original image. The remap only has to be good enough to straighten the lines. - Initial poses. Each frame's board pose is initialised with PnP and refined on its own.
- Bundle adjustment. All board poses and the shared intrinsics are optimised together in two stages: Levenberg-Marquardt with a linear loss, then a bounded trust-region solve with a
soft_l1loss to limit the influence of outlier corners. - Output. The field of view is derived from the fitted focal length as
2 · radius · 180 / (f · π)withradius = crop-size / 2.
The report lists the optical centre and its shift from the frame centre, the focal length, the derived field of view, the RMS reprojection error and a per-frame RMS. Frames are tagged OK below 2 px, HIGH below 5 px and BAD above. Drop the BAD frames and run again if a few of them dominate the error.
Output files¶
| File | Content |
|---|---|
camera_calibration.json |
Flat intrinsics plus frame_width, frame_height, pad_size, crop_size. Read by the testbench, the visualizer and the figure script. |
camera_calibration.yaml |
The same calibration in a calibration block, the preprocessing block, and config_yaml_updates with the camera.fov and camera.radius values for config.yaml. |
The script ends by printing the lines to copy into config.yaml. It adds camera.cx and camera.cy when the optical centre is more than 5 px away from the frame centre.
3. Compare projection models¶
calibrate_fisheye.py assumes an equidistant lens. To check that assumption for another lens, fit all four supported models on the same frames:
| Flag | Default | Meaning |
|---|---|---|
--images |
required | Image paths or glob patterns. |
--board-width |
7 |
Inner corners along the board width. |
--board-height |
5 |
Inner corners along the board height. |
--square-size |
0.025 |
Square size in metres. |
--pad-size |
3840 |
Preprocessing pad size in pixels. |
--crop-size |
3500 |
Preprocessing crop size in pixels. |
--initial-fov |
200 |
Initial field-of-view guess in degrees. |
--output |
model_comparison.json |
Output JSON with one entry per model. |
The board defaults differ from calibrate_fisheye.py (7 × 5 here, 9 × 6 there), so pass --board-width and --board-height explicitly when you use both scripts on one set of frames.
The script prints one row per model with the RMS error, optical centre, focal length and field of view, then names the model with the lowest RMS error. The models are equidistant, equisolid, stereographic and orthographic. Projection names maps them to the names config.yaml uses.
4. Use the result¶
| Consumer | What to update |
|---|---|
| Testbench, visualizer, figures | Nothing. They load camera_calibration.json from the repository root. Copy your output file there. |
config.yaml |
camera.fov, camera.radius, camera.cx, camera.cy, and camera.k1, camera.k2 after --fit-distortion. Also set camera.height and camera.pitch_deg for your mount. The testbench reads those two from this file. |
| Starter kit | The CAMERA_* constants in the USER CONFIG block of bridge_starter/bridge.py. |
After replacing the calibration, print the localisation error of the new geometry:
The first line of the output confirms which intrinsics were loaded:
Loaded calibration: FOV=197.90 cx=1752.7 cy=1804.5 f=1013.3 R=1750 frame=3500x3500 pad=3840 pitch=0 k1=0 k2=0