Skip to main content

ChArUco Camera Calibration

Why Does This Matter?​

A fully calibrated camera will provide more accurate 2D and 3D measurements. It's useful for any task that is not based on "servoing" to the crosshair.

With that being said, we recommend using the "servoing" method as often as you can. It's fast, precise, and suitable for tasks such as object acquisition, aiming, and aligning to targets. Don't worry about calibration unless you determine that you need greater accuracy. Every LL unit comes with a default calibration that will provide fairly good results for most use cases

"servoing" is the process of turning your robot until your target is perfectly aligned with a configurable "crosshair". It is the easiest, and in many cases, the most reliable way to appraoch vision problems in robotics.

What is Camera Calibration?​

Intrinsics calibration is the process of determining the two following components:

1. The Camera Matrix​

The camera matrix, often denoted as K, is a 3x3 matrix that describes how the lens and sensor turn the 3D world into a 2D picture.

[fx  0   cx]
[0   fy  cy]
[0   0   1]

cx and cy are the coordinates of the principal point / principal pixel. The principal point / principal pixel is the pixel that is directly aligned with the optical axis / center of the lens. During manufacturing, the camera lens is almost never perfectly aligned with the center of the image sensor. It is crucial that we understand which pixel is the true "center" pixel

fx and fy are computed focal lengths in pixels. fx and fy help us understand the field of view / zoom level of the camera.

2. Distortion Coefficients​

We compute 5 distortion coefficients:

[k1, k2, p1, p2, k3]

k1, k2, k3 are radial distortion coefficients which cause straight lines to appear curved.

p1, p2 are tangential distortion coefficients which occur due to the lens not being parallel to the image plane

Performing ChArUco Calibration​

ChArUco calibration with Limelight was designed to be as seamless and bullet-proof as possible. Read the following steps, and then watch the video below to learn how to calibrate your camera for increased accuracy.

info

You only need to calibrate at one resolution (we recommend 1280x960 for LL3 and 1280x800 for LL3G). Intrinsics are auto-scaled to match your pipeline's chosen resolution, and distortion coefficients are resolution-invariant given a fixed aspect ratio and FOV. Hardware zoom pipelines and 5-megapixel pipelines are the only pipelines that will not make use of custom calibrations.

Preparing the Board​

You can use any laptop screen to calibrate your camera. For maximum accuracy, however, we recommend purchasing this coarse 800mmx600mm board from calib.io

ChArUco Board Example

  1. Print a ChArUco Calibration Board or use an image of a board on a large laptop or computer screen. Use our board from the downloads page, or generate your own.
  2. Find the "width" of the grid by adding the total number of black squares and total number of ArUco markers in one row (11 for the default Limelight calibration board).
  3. Find the "height" of the grid by adding the total number of black squares and total number of ArUco markers in one column (8 for the default Limelight calibration board).
  4. Confirm the "square size" and "marker size" measurements by measuring the side lengths of a square and a marker in millimeters.
    • The square size and marker size measurements are critical, so use calipers.
  5. Note the dictionary type. The default board from the downloads page uses the 5x5_100 dictionary.

Capturing Calibration Images​

The 2026.0 update introduces a streamlined calibration workflow with a live video preview. You can now see your ChArUco detections in real-time before capturing images.

  1. Ensure your board is as flat as possible.
    • You should not perform calibration if your board is not flat. You may purchase special boards from calib.io, or fix your printed board to a clipboard.
  2. Turn on your robot and access the Limelight web UI.
  3. Create a "Charuco Calibration Preview" pipeline.
  4. Configure the charuco board settings (width, height, square size, marker size, dictionary) to match your board.
  5. You will now see a live preview of detected corners overlaid on the video stream. This lets you verify your board configuration is correct before capturing any images.
  6. Delete all saved snapshots by changing the "input" source type from "Camera" to "Snapshot" and clicking "Delete all snapshots".
  7. Change the "input" source type back to "Camera".
  8. Capture at least 25 snapshots of the ChArUco board to begin. Once you're familiar with the process, you'll want to go back to capture a total of at least 50 images.
  9. The main advantage of ChArUco calibration over standard checkerboard calibration is that it works even if only part of the board is visible to the camera. It is important to utilize this advantage.
  10. Your images should have a good mix of the following qualities:
    • Your board spans large regions of the image.
    • Your board extends past at least one edge or corner of the image (This will help compute distortion around the edges of the images).
    • Your board is often positioned such that it has perspective warp (The board should not be parallel to the image plane).
    • Aim for broad diversity in board positions, perspectives, and coverage.
    • 50% of your images should illustrate significant foreshortening. This is the effect of closer markers appearing larger than far away markers. The first and last images in the following screenshot demonstrate foreshortening

Calibration Examples

Calibrating​

  1. Navigate to the "Calibration" tab. It's the third tab in the vertical sidebar.
  2. Click the "Calibrate with Snapshots" button. The process may take a minute or two to complete.
  3. Upon success, the "Latest Calibration Result" card will show the latest calibration result.
  4. Check the "Latest Calibration Result" card for a reasonable result with a low reprojection error (ideally less than 1).
  5. Download the latest calibration result and upload it back to your Limelight.
info

You no longer need to select a calibration. If a custom calibration has been uploaded, it is automatically used for all targeting pipelines.

  • If a custom calibration is active, you will see a bright green header at the top of the calibration tab.
  • If the default calibration is being used, you will see a yellow warning header.

This eliminates the common mistake of uploading a calibration but forgetting to select it.

Inspecting Your Calibration​

You can view a 3D point cloud and 2D mosaic of all detected charuco board corners right in the web interface.

  • The 2D mosaic is the primary tool for dataset improvement. Look for large voids in the image coverage and capture more images to fill them.
  • The 3D point cloud helps advanced users understand the geometric diversity of their calibration image sets and catch degenerate configurations where boards are clustered at similar depths or angles.

You can also inspect individual images by entering screenshot mode within your ChArUco calibration preview pipeline.