Skip to main content

Software Change Log & Feedback

Submit issues and feature requests over email or to the Limelight Feedback Repo

Limelight OS 2024.10.2 (10/28/24)​

Python Snapscript Fix (CREDIT - FTC TEAM 23251 TRIPLE FAULT)​

  • Python Outputs will no longer freeze while using the Control Hub with Limelight3A
  • Python Outputs will no longer freeze while switching between python pipelines

MT2 Edge case​

https://github.com/LimelightVision/limelight-feedback/issues/23

  • While testing MT2 without a robot, it is easy to generate a geometric impossibility.
  • This edge case wouldn't zero-out the mt2 botpose - it would zero-out the camera pose.
  • The robot is now placed at (0,0,0) whenever this edge case is detected

Static IP Addressing​

https://github.com/LimelightVision/limelight-feedback/issues/25

  • Add a warning to the IP Address setter if the address ends in .1-.10 or .20-.255 as these addresses may interfere with the FMS or other devices
  • In 2024.10.1, the UI would sometimes suggest that a static IP was configured when the addressing scheme was set to "automatic"

STDDevs NetworkTables​

https://github.com/LimelightVision/limelight-feedback/issues/24

  • "stddevs" in networktables contains all stddevs for mt1 and mt2 (double array, 12 elements)

Model Upload Edge case​

  • If a classifier model is uploaded to a detector pipeline, "check model" will be displayed on the image
  • If a detector model is uploaded to a classifier pipeline, "check model" will be displayed on the image

Limelight OS 2024.10.1 (9/15/24)​

Limelight 3A Updates​

  • Update default color balance values
  • Update default AprilTag size configuration to 101.6 mm
  • Reduce default video stream framerate to 30FPS
  • Slightly increase video stream compression to reduce stream bitrate

AprilTag Size Warning​

  • Add a warning to the field space visualizer if the configured AprilTag size does not match the sizes in the uploaded field map.
Example banner

CameraPose_RobotSpace Bugfix​

  • Overriding the UI's configued Camera Pose from NetworkTables/LimelightLib/REST works properly again.

Limelight OS 2024.10 (9/4/24)​

Limelight 3A for FTC Support​

  • Limelight3A is fully supported
  • 3A is usable in both FTC and FRC

AprilTag Map Updates​

  • The map editor now supports standard and diamond FTC map generation
  • The LLOS web interface will automatically display the correct field type based on the uploaded map
  • All field visualizers across tools and interfaces show alliance zones and ftc tile grids.
  • 3D visualizer performance has been improved.

USB Limelight Support on ControlHub and RoboRio.​

  • FTC teams can use a single USB-capable Limelight on the Control Hub
  • FRC teams can use up to 16 USB-capable Limelights on the RoboRio. USB limelights auto-populate networktables just like ethernet limelights.
    • To use multiple USB Limelights, give each LL a unique USB index and a unique hostname.

USB Connectivity Upgrades​

  • MacOS no longer uses the USB Ethernet Limelight interface for internet access
  • Windows no longer attempts to use the USB Ethernet Limelight interface for internet access

REST API Updates​

  • Fix update-robotorientation POST request.
  • MT2 is now fully accessible without NetworkTables.
  • Once the update-robotorientation route is used, NetworkTables orientation updates are disabled until reboot.

Limelight OS 2024.9.1 (7/7/24)​

  • The Map Builder Tool now accepts/converts WPILib .json apriltag layouts
  • Add AprilTag3 to Python Snapscripts (from apriltag import apriltag)
  • See example in examples github repo
  • Fix USB connectivity gateway issue on Windows.

mosaic screenshot:

Limelight OS 2024.9 (7/5/24)​

MegaTag Upgrades​

  • Limelight OS has transitioned to NetworkTables 4.0

  • MegaTag2 now uses NT4's getAtomic() to retrieve timestamped IMU updates from the roboRIO.

  • Our timestamped image frames are matched to the two most relevant IMU samples before interpolation is performed.

  • NT4 flush() has been added to LimelightLib. Adding Flush() to older versions of Limelight OS will get you quite close to 2024.9 performance, but NT4 ensures accuracy is always high.

  • The MT2 visualizer robot now has green bumpers, and MT1's visualizer robot uses yellow bumpers.

  • Metrics are now collapsible, and the virtual robots can be hidden.

  • The following video demonstrates how 2024.9's MegaTag 2 (green robot) with robot-side flush() is more robust than 2024.5's MegaTag2 without Flush() (red robot)

USB ID and New USB IP Addresses​

  • Set the "USB ID" in the settings page to use multiple USB Limelights on any system.
  • The USB-Ethernet interface that appears on your system will utilize an IP address that is determined by the USB ID
  • Linux/Android/Mac Systems will now utilize the 172.29.0.0/24 subnet by default
  • Windows systems will now utilize the 172.28.0.0/24 subnet by default.
  • If the USBID is set, the subnet changes to 172.29.(USBID).0/24 for Linux/Android/Mac and 172.28.(USBID).0/24 for Windows.
  • You can now, for example, attach four Limelight devices to a single USB Hub by adjusting their hostnames and USB IDs

CPU Neural classifiers​

  • Upload a CPU .tflite classifier to enable neural classification without Google Coral. You can expect 15-18 FPS on LL3 variants.
  • 2024.9 comes with a default CPU classifier.
  • Set the classifier runtime to "CPU" to enable this feature

CPU Neural detectors​

  • Upload a CPU .tflite detector to enable neural detecyion without Google Coral. You can expect 10 FPS on LL3 variants.
  • 2024.9 comes with a default CPU detector.
  • Set the detector runtime to "CPU" to enable this feature

Limelight OS 2024.8 (7/3/24)​

  • Add python output (PythonOut), tx, ty, txnc, tync, ta to json results object
  • Further improved MT2 latency compensation

Limelight OS 2024.7 (5/21/24)​

  • Upgrade to Linux 6.6

Bugfixes​

  • Fix vision pipeline conversion
  • Fix calibration uploads, snapshot uploads, and nn uploads

Limelight OS 2024.6 (5/8/24)​

LimelightLib Python​

  • pip install limelightlib-python
  • Our Python library allows you to interact with USB and Ethernet Limelights on any platform.
  • It allows for complete Limelight configuration without web UI interaction.
  • Upload pipelines, neural networks, field maps, etc
  • Make realtime changes to any pipeline parameter with an optional "flush to disk" option
  • Post custom python input data, set robot orientation, etc.

MegaTag2 Upgrades​

  • MegaTag2 Gyro latency compensation been improved. Look out for more improvements soon!
  • Add "Gyro latency adjustment" slider to the UI. To manually tune MegaTag 2 latency compensation, you can spin your robot and adjust the slider until localization results are perfect while rotating.

Standard Deviation Metrics​

  • The 3D Field visualizer now includes MegaTag1 and Megatag2 standard deviations for x, y, and yaw.

New "Focus" Pipeline Type​

  • While in "focus" mode, you will have access to a stream quality slider and a crop box slider
  • Spin the lens to maximize the "focus" score.
  • If your camera is in a fixed location, this takes less than one minute. We recommend focusing with a fixed / mounted Limelight.

New "Barcodes" Pipeline Type​

  • 50-60FPS Multi QR Code Detection and Decoding at 1280x800
  • 50-60FPS Multi DataMatrix Detection and Decoding at 1280x800
  • 30FPS Multi UPC, EAN, Code128, and PDF417 at 1280x800
  • Barcode data strings are posted to the "rawbarcodes" nt array.
  • The Barcodes pipeline will populate all 2D metrics such as tx, ty, ta, tcornxy, etc.

All-New REST API​

  • https://docs.limelightvision.io/docs/docs-limelight/apis/rest-http-api
  • Our REST / HTTP API has been rebuilt from the ground up.
  • The REST API allows for complete Limelight configuration without web UI interaction.
  • Upload pipelines, neural networks, field maps, etc
  • Make realtime changes to any pipeline parameter with an optional "flush to disk" option
  • Post python input data, set robot orientation, etc.

Remove Camera Orientation Setting From UI (BREAKING CHANGE)​

  • This has been replaced by the "stream orientation" option. Calibration and targeting are never affecting by this option.
  • The new option only affects the stream. Upside-down, 90 Degree Clockwise, 90 Degree Counter-Clockwise, Horizontal Mirror, and Vertical Mirror
  • Teams will now need to manually invert tx and ty as required while using rotated cameras.

Remove GRIP Support (BREAKING CHANGE)​

Remove "Driver" zero-processing mode (BREAKING CHANGE)​

  • This has been replaced by the "Viewfinder" pipeline type

Add "Viewfinder" Pipeline type​

  • The viewfinder pipeline disables all processing for minimal latency
  • This allows teams to design their own "Driver" pipelines for view-only modes

Pipeline Files now Use JSON format (BREAKING CHANGE)​

  • Pipelines still use the .vpr file extension
  • (Broken in some cases in 2024.6) The UI will auto-convert pipelines to JSON when you use the "upload" button.
  • (Fully functional) You may also https://tools.limelightvision.io/pipeline-upgrade to upgrade your pipelines

Calibration UX Improvement​

  • Calibration settings are now cached. You no longer need to enter your calibration settings every time you want to calibrate.
  • The default calibration dictionary has been updated to work with the recommended 800x600mm coarse board from Calib.io.

Calibration Mosaic​

  • Previously, it was difficult to determine the quality of calibration images
  • The calibration tab now has a "Download Calibration Mosaic" button. The mosaic will show you exactly what each image is contributing to your calibration.

mosaic screenshot:

"Centroid" targeting region​

  • Centroid targeting mode has been added to the "Output" tab to improve object tracking with color pipelines

Dynamic 3D Offset (NT: fiducial_offset_set)​

  • It is now possible to adjust the 3D Offset without changing pipelines. This is useful for situations in which your "aim point" needs to change based on distance or other properties.

Add Modbus Support​

  • Limelight OS now has an always-on modbus server for inspection, logistics, and industrial applications
  • See the modbus register spec here: https://docs.limelightvision.io/docs/docs-limelight/apis/modbus
  • The default modbus server port may be changed in the UI's settings tab
  • Through modbus and snapscript python pipelines, completely custom vision applications with bi-directional communication are now supported.

Custom NT server​

  • The settings tab now contains an entry for a custom NT server.
  • This enables a new workflow which includes a glass NT server running on a PC, and Limelight 3G communicating over USB.

Rawfiducial changes​

  • The "area" value of raw fiducials is now a calibrated, normalized value ranging from ~0-1

All NetworkTables and JSON Changes​

  • Add NT getpipetype - Get the current pipeline type string (eg pipe_color, pipe_fiducial)

  • Add NT tcclass - Classifier pipeline detected class name

  • Add NT tdclass - Detector pipeline detected class name

  • Add NT t2d for guaranteed atomic 2d targeting - [valid,targetcount, targetlatency, capturelatency, tx, ty, txnc, tync, ta, targetid, classifierID, detectorID, tlong, tshort, thor, tvert, ts(skew)]

  • Remove NT tlong, tshort, thor, tvert, and ts

  • Add NT 'crosshairs' array [cx0,cy0,cx1,cy1]

  • Remove NT cx0, cy0, cx1, and cy1

  • Add NT rawbarcodes - NT String Array of barcode data. Up to 32 entries.

  • All "raw" arrays allow for up to 32 targets (up from 8)

  • Add fiducial_offset_set dynamic 3d Offset setter

  • Add "pType" to json top-level result

  • Add "stdev_mt1" and "stdev_mt2" to json top-level result (x,y,z,roll,pitch,yaw) (meters, degrees)

Changes to Other File Formats and JSON Dumps​

  • The calibration file format has been simplified. Old calibrations are auto-converted to the new format upon upload
  • One layer of nesting has been removed from Results and Status JSON dumps

Bugfixes​

  • Previously, if a Google Coral was unplugged while a Neural pipeline was active, the pipeline would permanently revert to "color/retro" mode
    • Now, "CHECK CORAL" or "CHECK MODEL" will be printed to the image. The pipeline type will never change
  • Previously, tags that successfully passed through the fiducial ID filter were sometimes drawn with a red outline instead of a green outline. This visualization problem has been fixed.
  • Apriltag pipelines populate the tcornxy NT array
  • Apriltag pipelines now fully respect the min-max area slider. Previously, AprilTag pipelines would filter 2D results based on Tag Area, but not 3D / Localization Results.

Limelight OS 2024.5.0 (4/9/24)​

  • Upgrade to Linux 6.1

Camera Stack Update​

  • The entire camera stack has been updated to fix a camera peripheral lock-up on Limelight3G.
    • Symptoms include
  • Be sure to retune exposure and gain settings after applying this update.

Dynamic Downscaling​

  • Teams may now set "fiducial_downscale_set" to override the current pipeline's downscale setting
  • 0:UI control, 1:1x, 2:1.5x, 3:2x, 4:3x, 5:4x
  • Use the new Helpers method with 0.0 (UI Control), 1.0, 1.5, 2.0, 3.0, 4.0
  • This is a zero-overhead operation.
  • By combining dynamic downscale and dynamic crop, teams can maximize FPS without managing multiple pipelines

MegaTag2 Improvements​

  • MT2 now works no matter the Limelight orientation, including "portrait" modes with 90 degree and -90 degree rolls

"rawdetections" nt array​

  • [classID, txnc, tync, ta, corner0x, corner0y, corner1x, corner2y, corner3x, corner3y, corner4x, corner4y]
  • corners are in pixel-space without calibration applied

Erode/Dilate Update​

  • Color pipelines now support up to 10 steps of dilation and 10 steps of erosion
  • Color pipelines now have a "reverse morpho" option to reverse the order of the dilation and erosion steps

LimelightLib 1.6 (4/9/24)​

  • Add void SetFiducialDownscalingOverride(float downscale)

Set to 0 for pipeline control, or one of the following to override your pipeline's downscale setting: 1, 1.5, 2, 3, 4

  • Add RawFiducial[] GetRawFiducials()
  • Add RawDetection[] GetRawDetections()

Limelight OS 2024.4.0 (4/3/24)​

Thanks to all of the teams who contributed ideas for this update.

Megatag 2​

Megatag 2 is an ambiguity-free localizer. It has higher accuracy and higher precision than Megatag1, and it was built with the following requirements:

  • Eliminate the pose ambiguity problem and increase robustness against image/corner noise.
  • Provide excellent pose estimates given one or more tags, no matter the perspective.
  • Increase robustness against physical AprilTag placement inaccuracies
  • Reduce the amount of robot-side filtering necessary for good pose estimation results

Notice the difference between MegaTag2 (red robot) and Megatag (blue robot) in this highly ambiguous single-tag case:

Megatag2 requires you to set your robot's heading with a new method call. Here's a complete example:

      LimelightHelpers.SetRobotOrientation("limelight", m_poseEstimator.getEstimatedPosition().getRotation().getDegrees(), 0, 0, 0, 0, 0);
      LimelightHelpers.PoseEstimate mt2 = LimelightHelpers.getBotPoseEstimate_wpiBlue_MegaTag2("limelight");
      if(Math.abs(m_gyro.getRate()) > 720) // if our angular velocity is greater than 720 degrees per second, ignore vision updates
      {
        doRejectUpdate = true;
      }
      if(mt2.tagCount == 0)
      {
        doRejectUpdate = true;
      }
      if(!doRejectUpdate)
      {
        m_poseEstimator.setVisionMeasurementStdDevs(VecBuilder.fill(.6,.6,9999999));
        m_poseEstimator.addVisionMeasurement(
            mt2.pose,
            mt2.timestampSeconds);
      }

Megatag2 provides excellent, ambiguity-free results at any distance given a single tag. This means it is perfectly viable to focus only on tags that are relevant and within your desired placement tolerance. If a tag is not in the correct location or irrelevant, filter it out with the new dynamic filter feature.

Dynamic Apriltag Filtering​

  • Because MegaTag2 is not desperate to accumulate as many AprilTags as possible, you can safely filter for well-placed and relevant tags:
int[] validIDs = {3,4};
LimelightHelpers.SetFiducialIDFiltersOverride("limelight", validIDs);

Transitioning to MegaTag2​

Megatag2 requires your robot heading to work properly. A heading of 0 degrees, 360 degrees, 720 degrees, etc means your robot is facing the red alliance wall. This is the same convention used in PathPlanner, Chorero, Botpose, and Botpose_wpiblue.

Once you have added SetRobotOrientation() to your code, check the built-in 3D visualizer. At close range, Megatag2 and Megatag1 should match closely if not exactly. At long range, Megatag 2 (red robot) should be more accurate and more stable than Megatag1 (blue robot).

Once the built-in visualizer is showing good results, you can safely use Megatag2 to guide your robot during the autonomous period.

The only filter we recommend adding is a "max angular velocity" filter. You may find that at high angular velocities, your pose estimates become slightly less trustworthy.

The examples repo has a Megatag2 example with this filter.

    if(Math.abs(m_gyro.getRate()) > 720) // if our angular velocity is greater than 720 degrees per second, ignore vision updates
    {
      doRejectUpdate = true;
    }
    if(mt2.tagCount == 0)
    {
      doRejectUpdate = true;
    }

LimelightLib 1.5 (4/3/24)​

Add

getBotPoseEstimate_wpiRed_MegaTag2()
getBotPoseEstimate_wpiBlue_MegaTag2()
SetRobotOrientation()

Limelight OS 2024.3.4 (3/20/24)​

Thanks to all of the teams who contributed ideas for this update.

Higher-Precision Single Tag Solver​

MegaTag's single tag 3D solver has been improved. It is far more stable than before at long range.

JSON Disabled by Default (Breaking Change)​

  • JSON has been disabled by default to reduce bandwidth usage and across the board for teams using auto-subscribing dashboards such as Shuffleboard.
  • This should also reduce RoboRIO NT load and CPU usage.
  • Re-enable json per-pipeline in the output tab.
  • This updates includes changes that should allow even more teams to transition away from JSON for pose estimation.

Undistorted Area (Breaking Change)​

Corners are undistorted before computing the area of any target.

Include Per-Fiducial Metrics in botpose, botpose_wpiblue, and botpose_wpired​

[tx, ty, tz, roll, pitch, yaw, tagCount, tagSpan (meters), averageDistance (meters), averageArea (percentage of image), (tags) ]

For every tag used by megatag localization, the above arrays now include (tagID, txnc, tync, ta, distanceToCamera, distanceToRobot, ambiguity)

Ambiguity is a new metric ranging from 0-1 that indicates the ambiguity of the current perspective of the tag. Single-tag-updates with tag ambiguities > .9 should probably be rejected.

"rawtargets" and "rawfiducials" nt arrays (Breaking Change)​

  • rawtargets - (txnc,tync,ta) per target
  • rawfiducials - (tagID, txnc, tync, ta, distanceToCamera, distanceToRobot, ambiguity) per target
  • The previous rawtargets NT entries (tx0,ty0, etc) have been removed.

Bugfixes​

  • Zero-out all single-tag 3D information if the priorityID has not been found. Previously, only Tx, Ta, Ty, and Tv were zeroed-out when the priorityTag was not found
  • Zero-out botpose if the only visible tag has been filtered by the UI's "ID Filters" features. Previously, botposes would reset to the center of the field rather than (0,0,0) if the only visible tag was a filtered tag;
  • 2024.2 would post NANs to certain networktables entries in some rare instances. This will no longer happen.

LimelightLib 1.4 (3/21/24)​

  • Add support for 2024.3.4 Raw Fiducials. PoseEstimates now include an array of rawFiducials which contain id, txnc, tync, ta, distanceToCamera, distanceToRobot, and ambiguity

Limelight Hardware Manager 1.4 (3/18/24)​

Bugfix​

Disovered USB Limelights are properly displayed as a single entry rather than two partial entries.

Limelight OS 2024.2.2 (3/17/24)​

Bugfix​

TX and TY properly respect the crosshair in NT entries.

Limelight OS 2024.2 (3/8/24)​

Zero-Crosshair targeting with Json (tx_nocross, ty_nocross) and NT (txnc, tync)​

If you are using tx/ty targeting with custom intrinsics calibration, you are likely still seeing camera-to-camera variation because the Limelight crosshair is not aligned with the principal pixel of the camera. Teams that require greater tx/ty accuracy can either configure the crosshair to match the principal pixel, or use these new metrics.

Potentially breaking change in tx/ty​

A bug was introduced earlier this season that broke custom calibration specifically for tx, ty, and tx + ty in json. Limelight OS was reverting to default calibrations in several cases.

Calibration Upgrades​

Calibration is now nearly instantaneous, now matter how many images have been captured. We've also fixed a crash caused by having more than around 30 images under certain circumstances.

We're consistently getting a reprojection error of around 1 pixel with 15-20 images of paper targets, and an error of .3 pixels with our high-quality calib.io targets.

Fiducial Filters UI Fix​

Fiducial filter textbox now accepts any number of filters.

Misc​

Apriltag Generator defaults to "no border" to prevent scaling with 165.1 mm tags.

Limelight OS 2024.1.1 (2/24/24)​

  • Fix priorityID

Limelight OS 2024.1 (2/24/24)​

HW Metrics (hw key in networktables, /status GET request)​

Calibration Improvement​

  • Fix crash that could occur if a calibration image contained exactly one valid detection. Improve web ui feedback.

Robot Localization Improvement (tag count and more)​

  • All networktables botpose arrays (botpose, botpose_wpiblue, and botpose_wpired) now include Tag Count, Tag Span (meters), Average Distance (meters), and Average Area (percentage of image)

  • These metrics are computed with tags that are included in the uploaded field map. Custom and/or mobile AprilTags will not affect these metrics.

  • With device calibration and this botpose array upgrade, we do not believe JSON is necessary for the vast majority of use-cases this year.

  • JSON dump now includes botpose_avgarea, botpose_avgdist, botpose_span, and botpose_tagcount for convenience.

[tx,ty,tz,rx,ry,rz,latency,tagcount,tagspan,avgdist,avgarea]

New Feature: Priority ID (NT Key priorityid)​

  • If your robot uses both odometry-based features and tx/ty-based features, you've probably encountered the following UX problem:

  • Before this update, there was no way to easily switch the preferred tag ID for tx/ty targeting.

  • While there is an ID filter in the UI, it

    • is not dynamic
    • removes tags from megaTag localization.

  • This meant teams were creating several pipelines: one for 3D localization, and one per tx/ty tag (one pipeline for blue-side shooting with tag 7, one for blue-side amping with tag 6, etc.).

  • The new priority ID feature (NT Key priorityid) allows you to tell your Limelight "After all tag detection, filtering, and sorting is complete, focus on the tag that matches the priority ID."

  • This does not affect localization in any way, and it only slightly changes the order of tags in JSON results.

  • If your priority id is not -1, tx/ty/ta will return 0 unless the chosen tag is visible.

Misc​

  • Fix "x" across the screen while using dual-target mode in a 3D apriltag pipeline
  • REST API expanded with neural network label uploads (/uploadlabels)
  • Include device nickname in /status json

LimelightLib 1.3​

  • LimelightLib (Java and CPP) have been updated to make localization easier than ever.
  LimelightHelpers.PoseEstimate limelightMeasurement = LimelightHelpers.getBotPoseEstimate_wpiBlue("limelight");
    if(limelightMeasurement.tagCount >= 2)
    {
      m_poseEstimator.setVisionMeasurementStdDevs(VecBuilder.fill(.7,.7,9999999));
      m_poseEstimator.addVisionMeasurement(
          limelightMeasurement.pose,
          limelightMeasurement.timestampSeconds);
    }

New resources for Teams​

Limelight Feedback and Issue Tracker: https://github.com/LimelightVision/limelight-feedback/issues​

Examples Repo: https://github.com/LimelightVision/limelight-examples​

Aiming and Ranging with Swerve Example: https://docs.limelightvision.io/docs/docs-limelight/tutorials/tutorial-swerve-aiming-and-ranging​

MegaTag Localization Example: https://docs.limelightvision.io/docs/docs-limelight/tutorials/tutorial-swerve-pose-estimation​

Thanks to recent contributors jasondaming, Gold876, JosephTLockwood, Andrew Gasser, and virtuald

Limelight 2024 Updates (2/6/24)

Limelight Documentation Upgrade​

  • The documentation has been rewritten to streamline the setup process

Limelight AprilTag Generator​

  • https://tools.limelightvision.io/ now features the first-ever online AprilTag generator.
  • Select your paper size, marker size, and tag IDs to generate a printable PDF.
  • Safari may not properly display tags at the moment.

Limelight Map Builder​

New Hardware Manager​

  • The Finder Tool is now the Limelight Hardware Manager
  • It has been rewritten from scratch. It now reliably detects Limelights, provides more useful diagnostic information, and does not require restarts to work properly.
  • Get it now from the downloads page notebook screenshot:

Train your own Neural Networks​

notebook screenshot:

2024 AprilTag Map and Note Detector​

  • The map and detector model have been added to the downloads page and the latest Limelight OS image.

Limelight OS 2024.0 (2/6/24)​

ChArUco Calibration Fixes​

  • Our ChArUco detector's subpixel accuracy has been increased. A reprojection error of 1-2 pixels is now achievable with clipboard targets and 20 images.
  • Using the same camera and the same target, 2023.6 achieved an RPE of 20 pixels, and 2024.0 achieved an RPE of 1.14 pixels.
  • Input fields no longer accept letters and special characters. This eliminates the potential for a crash.

Out-Of-The-Box Megatag Accuracy Improvement​

  • Before this update, Limelight's internal Megatag map generator referenced the UI's tag size slider instead of the tag sizes supplied by the .fmap file.
  • Megatag now respects the tag sizes configured in fmap files and ignores the size slider.
  • If your size slider has not been set to 165.1 mm, you will notice an immediate improvement in localization accuracy

Performance Upgrades and Bugfixes​

  • Higher FPS AprilTag pipelines
  • The performance of the Field-Space Visualizer has been significantly improved.

Bugfixes​

  • Apriltags in 3D visualizers were sometimes drawn with incorrect or corrupted tag images. Tags are now always displayed correctly.
  • "v" / tv / "valid" will now only return "1" if there are valid detections. Previously, tv was always "1"

2023.6 (4/18/23)​

Easy ChArUco Calibration & Calibration Visualizers​

  • ChArUco calibration is considered to be better than checkerboard calibration because it handles occlusions, bad corner detections, and does not require the entire board to be visible. This makes it much easier to capture calibration board corners close to the edges and corners of your images. This is crucial for distortion coefficient estimation.
  • Limelight’s calibration process provides feedback at every step, and will ensure you do all that is necessary for good calibration results. A ton of effort has gone into making this process as bulletproof as possible.
  • Most importantly, you can visualize your calibration results right next to the default calibration. At a glance, you can understand whether your calibration result is reasonable or not.
  • You can also use the calibration dashboard as a learning tool. You can modify downloaded calibration results files and reupload them to learn how the intrinsics matrix and distortion coefficients affect targeting results, FOV, etc.
  • Take a look at this video:

2023.5.1 & 2023.5.2 (3/22/23)​

  • Fixed regression introduced in 2023.5.0 - While 2023.5 fixed megatag for all non-planar layouts, it reduced the performance of single-tag pose estimates. This has been fixed. Single-tag pose estimates use the exact same solver used in 2023.4.

  • Snappier snapshot interface. Snapshot grid now loads low-res 128p thumbnails.

  • Limeilght Yaw is now properly presented in the 3d visualizers. It is ccw-positive in the visualizer and internally

  • Indicate which targets are currently being tracked in the field space visualizer

2023.5.0 (3/21/23)​

Breaking Changes​

  • Fixed regression - Limelight Robot-Space "Yaw" was inverted in previous releases. Limelight yaw in the web ui is now CCW-Positive internally.

Region Selection Update​

  • Region selection now works as expected in neural detector pipelines.
  • Add 5 new region options to select the center, top, left, right, top, or bottom of the unrotated target rectangle.

"hwreport" REST API​

  • :5807/hwreport will return a JSON response detailing camera intrinsics and distortion information

MegaTag Fix​

  • Certain non-coplanar apriltag layouts were broken in MegaTag. This has been fixed, and pose estimation is now stable with all field tags. This enables stable pose estimation at even greater distances than before.

Greater tx and ty accuracy​

  • TX and TY are more accurate than ever. Targets are fully undistorted, and FOV is determined wholly by camera intrinsics.

2023.4.0 (2/18/23)​

Neural Detector Class Filter​

Specify the classes you want to track for easy filtering of unwanted detections.

Neural Detector expanded support​

Support any input resolution, support additional output shapes to support other object detection architectures. EfficientDet0-based models are now supported.

2023.3.1 (2/14/23)​

AprilTag Accuracy Improvements​

Improved intrinsics matrix and, most importantly, improved distortion coefficients for all models. Noticeable single AprilTag Localization improvements.

Detector Upload​

Detector upload fixed.

2023.3 (2/13/23)​

Capture Latency (NT Key: "cl", JSON Results: "cl")​

The new capture latency entry represents the time between the end of the exposure of the middle row of Limelight's image sensor and the beginning of the processing pipeline.

New Quality Threshold for AprilTags​

Spurious AprilTags are now more easily filtered out with the new Quality Threshold slider. The default value set in 2023.3 should remove most spurious detections.

Camera Pose in Robot Space Override (NT Keys: "camerapose_robotspace_set", "camerapose_robotspace")​

Your Limelight's position in robot space may now be adjusted on-the-fly. If the key is set to an array of zeros, the pose set in the web interface is used.

Here's an example of a Limelight on an elevator: Limelight on Elevator

Increased Max Exposure​

The maximum exposure time is now 33ms (up from 12.5 ms). High-fps capture modes are still limited to (1/fps) seconds. 90hz pipelines, for example, will not have brighter images past 11ms exposure time.

Botpose updates​

All three botpose arrays in networktables have a seventh entry representing total latency (capture latency + targeting latency).

Bugfixes​

  • Fix LL3 MJPEG streams in shuffleboard
  • Fix camMode - driver mode now produces bright, usable images.
  • Exposure label has been corrected - each "tick" represents 0.01ms and not 0.1 ms
  • Fix neural net detector upload

2023.2 (1/28/23)​

Making 3D easier than ever.

WPILib-compatible Botposes​

Botpose is now even easier to use out-of-the-box.

These match the WPILib Coordinate systems. megatag botpose example:

All botposes are printed directly in the field-space visualizer in the web interface, making it easy to confirm at a glance that everything is working properly.

Easier access to 3D Data (Breaking Changes)​

RobotPose in TargetSpace is arguably the most useful data coming out of Limelight with repsect to AprilTags. Using this alone, you can perfectly align a drivetrain with an AprilTag on the field.

  • NetworkTables Key “campose” is now “camerapose_targetspace”
  • NetworkTables Key “targetpose” is now “targetpose_cameraspace”
  • New NetworkTables Key - “targetpose_robotspace”
  • New NetworkTables Key - “botpose_targetspace”

Neural Net Upload​

Upload teachable machine models to the Limelight Classifier Pipeline. Make sure they are Tensorflow Lite EdgeTPU compatible models. Upload .tflite and .txt label files separately.

2023.1 (1/19/23)​

MegaTag and Performance Boosts

Correcting A Mistake​

The default marker size parameter in the UI has been corrected to 152.4mm (down from 203.2mm). This was the root of most accuracy issues.

Increased Tracking Stability​

There are several ways to tune AprilTag detection and decoding. We’ve improved stability across the board, especially in low light / low exposure environments.

Ultra Fast Grayscaling​

Grayscaling is 3x-6x faster than before. Teams will always see a gray video stream while tracking AprilTags.

Cropping For Performance​

AprilTag pipelines now have crop sliders. Cropping your image will result in improved framerates at any resolution.

Easier Filtering​

There is now a single “ID filter” field in AprilTag pipelines which filters JSON output, botpose-enabled tags, and tx/ty-enabled tags. The dual-filter setup was buggy and confusing.

Breaking Change​

The NT Key “camtran” is now “campose”

JSON update​

"botpose" is now a part of the json results dump

Field Space Visualizer Update​

The Field-space visualizer now shows the 2023 FRC field. It should now be easier to judge botpose accuracy at a glance.

Limelight MegaTag (new botpose)​

My #1 priority has been rewriting botpose for greater accuracy, reduced noise, and ambiguity resilience. Limelight’s new botpose implementation is called MegaTag. Instead of computing botpose with a dumb average of multiple individual field-space poses, MegaTag essentially combines all tags into one giant 3D tag with several keypoints. This has enormous benefits.

The following GIF shows a situation designed to induce tag flipping: Green Cylinder: Individual per-tag bot pose Blue Cylinder: 2023.0.1 BotPose White Cylinder: New MegaTag Botpose

Notice how the new botpose (white cylinder) is extremely stable compared to the old botpose (blue cylinder). You can watch the tx and ty values as well.

Here’s the full screen, showing the tag ambiguity:

Here are the advantages:

Botpose is now resilient to ambiguities (tag flipping) if more than one tag is in view (unless they are close and coplanar. Ideally the keypoints are not coplanar). Botpose is now more resilient to noise in tag corners if more than one tag is in view. The farther away the tags are from each other, the better. This is not restricted to planar tags. It scales to any number of tags in full 3D and in any orientation. Floor tags and ceiling tags would work perfectly.

Here’s a diagram demonstrating one aspect of how this works with a simple planar case. The results are actually better than what is depicted, as the MegaTag depicted has a significant error applied to three points instead of one point. As the 3D combined MegaTag increases in size and in keypoint count, its stability increases.

Nerual Net upload is being pushed to 2023.2!

2023.0.0 and 2023.0.1 (1/11/23)​

Introducing AprilTags, Robot localization, Deep Neural Networks, a rewritten screenshot interface, and more.

Features, Changes, and Bugfixes​

  • New sensor capture pipeline and Gain control
    • Our new capture pipeline allows for exposure times 100x shorter than what they were in 2022. The new pipeline also enables Gain Control. This is extremely important for AprilTags tracking, and will serve to make retroreflective targeting more reliable than ever. Before Limelight OS 2023, Limelight's sensor gain was non-deterministic (we implemented some tricks to make it work anyways).
    • With the new "Sensor Gain" slider, teams can make images darker or brighter than ever before without touching the exposure slider. Increasing gain will increase noise in the image.
    • Combining lower gain with the new lower exposure times, it is now possible to produce nearly completely black images with full-brightness LEDs and retroreflective targets. This will help mitigate LED and sunlight reflections while tracking retroreflective targets.
    • By increasing Sensor Gain and reducing exposure, teams will be able to minimize the effects of motion blur due to high exposure times while tracking AprilTags.
    • We have managed to develop this new pipeline while retaining all features - 90fps, hardware zoom, etc.
  • More Resolution Options
    • There two new capture resolutsions for LL1, LL2, and LL2+: 640x480x90fps, and 1280x960x22fps
  • Optimized Web Interface
    • The web gui will now load and initialize up to 3x faster on robot networks.
  • Rewritten Snapshots Interface
    • The snapshots feature has been completely rewritten to allow for image uploads, image downloads, and image deletion. There are also new APIs for capturing snapshots detailed in the documentation.
  • SolvePnP Improvements
    • Our solvePnP-based camera localization feature had a nasty bug that was seriously limiting its accuracy every four frames. This has been addressed, and a brand new full 3D canvas has been built for Retroreflective/Color SolvePNP visualizations.
  • Web Interface Bugfix
    • There was an extremely rare issue 2022 that caused the web interface to permanently break during the first boot after flashing, which would force the user to re-flash. The root cause was found and fixed for good.
  • New APIs
    • Limelight now include REST and Websocket APIs. REST, Websocket, and NetworkTables APIs all support the new JSON dump feature, which lists all data for all targets in a human readable, simple-to-parse format for FRC and all other applications.

Zero-Code Learning-Based Vision & Google Coral Support​

  • Google Coral is now supported by all Limelight models. Google Coral is a 4TOPs (Trillions-of-Operations / second) USB hardware accelerator that is purpose built for inference on 8-bit neural networks.
  • Just like retroreflective tracking a few years ago, the barrier to entry for learning-based vision on FRC robots has been too high for the average team to even make an attempt. We have developed all of the infrastructure required to make learning-based vision as easy as retroreflective targets with Limelight.
  • We have a cloud GPU cluster, training scripts, a dataset aggregation tool, and a human labelling team ready to go. We are excited to bring deep neural networks to the FRC community for the first time.
  • We currently support two types of models: Object Detection models, and Image classification models.
    • Object detection models will provide "class IDs" and bounding boxes (just like our retroreflective targets) for all detected objects. This is perfect for real-time game piece tracking.
      • Please contribute to the first-ever FRC object detection model by submitting images here: https://datasets.limelightvision.io/frc2023
      • Use tx, ty, ta, and tclass networktables keys or the JSON dump to use detection networks
    • Image classification models will ingest an image, and produce a single class label.
      • To learn more and to start training your own models for Limelight, check out Teachable Machine by google.
      • https://www.youtube.com/watch?v=T2qQGqZxkD0
      • Teachable machine models are directly compatible with Limelight.
      • Image classifiers can be used to classify internal robot state, the state of field features, and so much more.
      • Use the tclass networktables key to use these models.
  • Limelight OS 2023.0 does not provide the ability to upload custom models. This will be enabled shortly in 2023.1

Zero-Code AprilTag Support​

  • AprilTags are as easy as retroreflective targets with Limelight. Because they have a natural hard filter in the form of an ID, there is even less of a reason to have your roboRIO do any vision-related filtering.
  • To start, use tx, ty, and ta as normal. Zero code changes are required. Sort by any target characteristic, utilize target groups, etc.
  • Because AprilTags both always square and always uniquely identifiable, they provide the perfect platform for full 3D pose calculations.
  • The feedback we've received for this feature in our support channels has been extremely positive. We've made AprilTags as easy as possible, from 2D tracking to a full 3D robot localization on the field
  • Check out the Field Map Specification and Coordinate System Doc for more detailed information.
  • There are four ways to use AprilTags with Limelight:
  • AprilTags in 2D
    • Use tx, ty, and ta. Configure your pipelines to seek out a specific tag ID.
    • <gif>
  • Point-of-Interest 3D AprilTags
    • Use tx and ty, ta, and tid networktables keys. The point of interest offset is all most teams will need to track targets do not directly have AprilTags attached to them.
    • <gif>
  • Full 3D
    • Track your LL, your robot, or tags in full 3D. Use campose or json to pull relevant data into your roboRio.
    • <gif>
  • Field-Space Robot Localization
    • Tell your Limelight how it's mounted, upload a field map, and your LL will provide the field pose of your robot for use with the WPILib Pose Estimator.
    • Our field coordinate system places (0,0) at the center of the field instead of a corner.
    • Use the botpose networktables key for this feature.
    • <gif>

2022.3.0 (4/13/22)​

Bugfixes and heartbeat.

Bugfixes​

  • Fix performance, stream stability, and stream lag issues related to USB Camera streams and multiple stream instances.

Features and Changes​

  • "hb" Heartbeat NetworkTable key
    • The "hb" value increments once per processing frame, and resets to zero at 2000000000.

2022.2.3 (3/16/22)​

Bugfixes and robot-code crop filtering.

Bugfixes​

  • Fix "stream" networktables key and Picture-In-Picture Modes
  • Fix "snapshot" networktables key. Users must set the "snapshot" key to "0" before setting it to "1" to take a screenshot.
  • Remove superfluous python-related alerts from web interface

Features and Changes​

  • Manual Crop Filtering
    • Using the "crop" networktables array, teams can now control crop rectangles from robot code.
    • For the "crop" key to work, the current pipeline must utilize the default, wide-open crop rectangle (-1 for minX and minY, +1 for maxX and +1 maxY).
    • In addition, the "crop" networktable array must have exactly 4 values, and at least one of those values must be non-zero.

2022.2.2 (2/23/22)​

Mandatory upgrade for all teams based on Week 0 and FMS reliability testing.

Bugfixes​

  • Fix hang / loss of connection / loss of targeting related to open web interfaces, FMS, FMS-like setups, Multiple viewer devices etc.

Features and Changes​

  • Crop Filtering

    • Ignore all pixels outside of a specified crop rectangle
    • If your flywheel has any sweet spots on the field, you can make use of the crop filter to ignore the vast majority of pixels in specific pipelines. This feature should help teams reduce the probability of tracking non-targets.
    • If you are tracking cargo, use this feature to look for cargo only within a specific part of the image. Consider ignoring your team's bumpers, far-away targets, etc.

  • Corners feature now compatible with smart target grouping

    • This one is for the teams that want to do more advanced custom vision on the RIO
    • "tcornxy" corner limit increased to 64 corners
    • Contour simplification and force convex features now work properly with smart target grouping and corner sending

  • IQR Filter max increased to 3.0

  • Web interface live target update rate reduced from 30fps to 15fps to reduce bandwidth and cpu load while the web interface is open

2022.1 (1/25/22)​

Bugfixes​

  • We acquired information from one of our suppliers about an issue (and a fix!) that affects roughly 1/75 of the CPUs specifically used in Limelight 2 (it may be related to a specific batch). It makes sense, and it was one of the only remaining boot differences between the 2022 image and the 2020 image.
  • Fix the upload buttons for GRIP inputs and SolvePNP Models

Features​

  • Hue Rainbow

    • The new hue rainbow makes it easier to configure the hue threshold.

  • Hue Inversion

    • The new hue inversion feature is a critical feature if you want to track red objects, as red is at both the beginning and the end of the hue range:

  • New Python Libraries

    • Added scipy, scikit-image, pywavelets, pillow, and pyserial to our python sandbox.

2022.0 and 2022.0.3 (1/15/22)​

This is a big one. Here are the four primary changes:

Features​

  • Smart Target Grouping

    • Automatically group targets that pass all individual target filters.
    • Will dynamically group any number of targets between -group size slider minimum- and -group size slider maximum-

  • Outlier Rejection

    • While this goal is more challenging than other goals, it gives us more opportunities for filtering. Conceptually, this goal is more than a “green blob.” Since we know that the goal is comprised of multiple targets that are close to each other, we can actually reject outlier targets that stand on their own.
    • You should rely almost entirely on good target filtering for this year’s goal, and only use outlier rejection if you see or expect spurious outliers in your camera stream. If you have poor standard target filtering, outlier detection could begin to work against you!

  • Limelight 2022 Image Upgrades We have removed hundreds of moving parts from our software. These are the results:

    • Compressed Image Size: 1.3 GB in 2020 → 76MB for 2022 (Reduced by a factor of 17!)
    • Download time: 10s of minutes in 2020 → seconds for 2022
    • Flash time: 5+ minutes in 2020 → seconds for 2022
    • Boot time: 35+ seconds in 2020 → 14 seconds for 2022 (10 seconds to LEDS on)

  • Full Python Scripting

    • Limelight has successfully exposed a large number of students to some of the capabilities of computer vision in robotics. With python scripting, teams can now take another step forward by writing their own image processing pipelines.

  • This update is compatible with all Limelight Hardware, including Limelight 1.

  • Known issues: Using hardware zoom with python will produce unexpected results.

  • 2022.0.3 restores the 5802 GRIP stream, and addresses boot issues on some LL2 units by reverting some of the boot time optimizations. Boot time is increased to 16 seconds.

2020.4 (3/11/20)​

2020.4 is another critical update that eliminates the intermittent 2-4 second crash that could occur during an abrupt mjpeg stream disconnection. This often happened at the very end of the match, and in some cases could happen during matches.

2020.3 (2/25/20)​

2020.3 is a critical update that addresses intermittent networking-related software restarts, and addresses a crash that would occur upon USB camera disconnection.

2020.2 (1/30/20)​

2020.2 pushes Limelight's hardware even further by adding the ability to pan and zoom to any point. It also adds a brand new video mode and important bug fixes. Let us know what features you would like to see in future updates!

Features​

  • 3x Hardware Zoom at 60 FPS

    • Our last update added 2x Hardware Zoom for all Limelight models at no cost whatsoever. We’ve managed to push our hardware zoom even further with 3x Hardware Zoom. All Limelight models are now capable of 3x Hardware Zoom at 60fps. This makes full-court tracking even easier for every team.

  • Hardware Panning

    • Robots capable of shooting from both long and short distances in 2020 would have found it difficult to use Hardware Zoom on their Limelights without the use of an active, adjustable Limelight mount. We have incorporated Hardware Panning to solve this problem.

Bugfixes​

  • Address driver issues that were breaking the color balance sliders
  • Revert all networking-related drivers to 2019 season variants to address a small number of network performance reports from teams

2020.1 (1/16/20)​

2020.1 adds one of Limelight's coolest features yet: 2x Hardware Zoom. This is not digital zoom, and it is now enabled for all Limelight versions.

Features​

  • 2x Hardware Zoom

    • Not Digital zoom
    • Enable 2x Hardware zoom to achieve vastly improved tracking at long range, and stable tracking at full-court distances.
    • This feature comes with absolutely no latency or framerate cost - tracking continues to execute at 90fps with no additional overhead.

  • Manual Crosshairs

    • Manually adjust single and dual crosshair locations with sliders in the web interface

  • New SolvePNP / 3D features (Still experimental)

    • We have added the "Force Convex" option to use only the outermost points of a target - this is necessary in 2020 due to the thin tape used for the hexagonal target.
    • The "bind target" option has been added. This feature binds "tx" and "ty" to the 3D target. This is required to guide robots to score using 3D targets.
    • Finally, add the "Goal Z-Offset" option to automatically offset your desired target location in 3D space on the Z-axis.
    • In Infinite Recharge, the "Goal Z-Offset" would be used to track the center of the small, circular goal behind the hexagonal goal.

  • Color sensing with "tc"

    • Read the new "tc" array in Network Tables to obtain Red, Green, and Blue values at the crosshair's location. Adjust "Blue Balance" and "Red Balance" on the input tab to achieve perfect color sensing.

Bugfixes​

  • USB Camera functionality broken in 2020.0 is now fixed in 2020.1
  • SolvePnP functionality broken in 2020.0 is now fixed in 2020.1
  • SolvePnP properly uses the model center as the axis visualization center

2019.7 (4/5/19)​

2019.7 adds two new features.

Features​

  • 160x120 30fps streaming
    • Smoother, lower-bandwidth streaming for teams that use wide-angle USB cameras. Our 180-degree camera stream bandwidth dropped from ~3.7mbps to ~1.8 mbps.
    • Change the stream resolution in the settings tab.
    • Changing the stream resolution on a Limelight with a wide-angle USB camera attached. No Picture-in-Picture, and normal stream rate.
  • Smart Speckle Rejection
    • Teams that have mounted their cameras in-line with the target this year have had to deal with unwanted LED reflections.
    • The area slider does not always solve this problem, as teams want to track small vision targets at large distances.
    • This new feature will automatically reject relatively small contours that have passed through all other filters.
    • As a robot moves away from a vision target (decreasing its size), SSR will automatically adjust to only reject relatively small contours.

Changes​

  • Tooltips
    • Tooltips are now available on some Limelight controls

2019.6.1 Hotfix (3/14/19)​

2019.6.1 fixes Grip uploads.

2019.6 (3/12/19)​

2019.6 is all about reliability.

Bugfixes​

  • USB Cameras
    • Address issue that could cause some USB cameras to fail on boot.
  • FMS
    • Make Limelight more (if not completely) resistant to FMS restarts and field / laptop networking changes.
    • Limelight will no longer hang after a sudden client networking change.
  • Raw Contour Sorting (BREAKING CHANGE)
    • Intersection filters no longer affect raw contour sorting.
  • Smartdashboard auto-posting
    • LL auto-posts certain pieces of information to SmartDashboard (IP Address, Interface url, etc.). The names of these values now contain the hostname.

Features & Changes​

  • Significantly increase precision and stability of the compute3d feature. Translation and rotation measurements are stable at larger distances.
  • Max Black Level Offset increased to 40 (from 25) for even darker images.
  • New "Closest" sort options in the "Output" tab while "Dual Crosshair" mode is enabled.
    • Standard - Current "closest" sorting implementation with "Dual Crosshair" mode.
    • Standard V2 - Experimental, smart "closest" sorting implementation with "Dual Crosshair" mode.
    • Average of Crosshairs - "Closest" sort origin with "Dual Crosshair" mode is the average of the two crosshairs.
    • Crosshair A - "Closest" sort origin with "Dual Crosshair" mode is crosshair A.
    • Crosshair B - "Closest" sort origin with "Dual Crosshair" mode is crosshair B.
  • New "LED Mode" pipeline options: "Left Half", "Right Half"
  • Floating-point raw corner values while compute3D is enabled.
  • Hide image decorations while using magic wands
  • Larger stream in web interface

2019.5 (2/9/19)​

With 2019.5 we are introducing the brand new compute3D camera localization feature. Only a handful of teams have ever attempted to add this feature to their vision systems, and now it is available to all Limelight 1 and Limelight 2 users.

This is not a silver bullet for this year's game. We highly recommend thinking of creative ways to use the standard high-speed 90 fps tracking unless this feature is absolutely necessary.

Features​

  • High-Precision Mode and PnP
    • In the following gif, a Limelight 2 was placed 37 inches behind and 14.5 inches to the right of the target.
    • The Limelight was later turned by hand. Notice how the distances remain mostly unchanged.
    • With 2019.4, we introduced corner sending. This allowed advanced teams to write their own algorithms using OpenCV's solvePNP(). With 2019.5, this is all done on-board.
    • Upload a plain-text csv file with a model of your target. We have pre-built models of 2019 targets hosted on our website. All models must have a centered origin, and use counter-clockwise point ordering with inch scaling.
    • Enable the new high-res 960x720 mode, and then enable "Solve 3D" to acquire the position and rotation of your Limelight relative to your target.
    • Corner numbers are now displayed on the image for easier model creation and threshold tuning.
    • Read all 6 dimensions of your camera's transform (x,y,z,pitch,yaw,roll) by reading the "camtran" networktable number array.
  • Black Level
    • With the new black level slider, thresholding is even easier. Increase the black level offset to further darken your images.

Breaking Changes​

  • The reported vertical FOV for LL2 has been fixed to match the listed value of 49.7 degrees. This will change your "ty" values

Bug Fixes​

  • Fix stream-only crash that could occur when fisheye USB cameras were attached.
  • Fix rare hang caused by networking-related driver.
  • Corner approximation is now always active.

2019.4 (1/26/19)​

We believe all critical bug reports are covered with this release.

Features​

  • Corners
    • Send the corners of your target as two arrays (tcornx, tcorny) *NOW tcornxy by enabling "send corners" in the "Output" tab. This will further enable teams that are interested in advanced pipelines with methods like solvePNP().
    • Adjust corner approximation with the "corner approximation" slider in the "Output" tab.

Bug Fixes​

  • Fix hang + rare crash that would occur when two targets had exactly the same area, x coordinate, or y coordinate.
  • Fix area calculation in dual- and tri-target modes.
  • Optimize contour sorting for better performance.

2019.3 (1/19/19)​

2019.3 addresses a number of bugs and feature requests.

Features​

  • Stream Rate (bandwidth reduction)
    • Set the stream rate to "low" in the settings page for a lower-bandwidth 15fps video stream.
  • Raw Contours disabled in NetworkTables (bandwidth reduction)
    • Raw contours are now disabled by default. This will further reduce Limelight's overall bandwidth.
    • There are now ~180 fewer networktables entries submitted per second by default.
    • See the new "Raw Contours" pipeline option in the "Output" tab to re-enable raw contours.
  • Active Pipeline Index
    • Read the networktable key "getpipe" to get the true active pipeline index of the camera. This is updated at 90hz.
    • The active pipeline index is now written to the videostream underneath the FPS display.
  • Left and Right Intersection Filters
    • Teams may now choose between "left" and "right" dual-contour intersection filters. The "above" and "below" intersection filters were added in 2019.2.

Bug Fixes​

  • Fix LabView Dashboard streaming bug introduced in 2019.2
  • The webpage no longer requests icon fonts from a the internet. All fonts are stored locally. This should speed up interface loading.
  • Reduce "driver mode" exposure.
  • Fix "Distance Transform" GRIP implementation
  • Fix 20-second communication delays caused by changing the hostname.

2019.2 (1/7/19)​

2019.2 adds new features to better equip teams for the 2019 season.

  • Intersection Filter
  • Direction Filter

Additional Changes and Fixes​

  • IP Address is auto-posted to SmartDashboard/Shuffleboard for easier event troubleshooting. We have had almost no reports of this being an issue, but this serves as another contingency feature.
  • Pipeline name is auto-posted to SmartDashboard/Shuffleboard
  • Access the width and height of the rough bounding box in pixels with new networktables keys.
  • Access the longest side and shortest side of the perfect bounding box in pixels with new networktables keys.
  • "Sort Mode" now applies to dual-contours
  • "Sort Mode" is now fixed
  • 5802 stream is less taxing on the camera

2019.1 (12/18/18)​

2019.1 fixes all major bugs reported during the beta

  • Performance is back to a steady 90fps. A driver issue was the root cause of our performance problems during beta
  • IP and hostname settings actually apply and "stick"
  • Magic Wands have been re-ordered to match those of Limelight 2018 software
  • We now support Grip inputs like masks through the web interface
  • NetworkTables freeze fixed

2019.0 (12/10/18)​

With 2019, we are introducing GRIP support, a brand-new interface, and a cleaner flashing procedure.

  • Grip Support

    • Build pipelines with GRIP For Limelight, and export "LL Script" files to upload to your camera.
    • Masks and NetworkTables support will be added in a future update
    • Expose a bare video stream at http://limelighturl:5802 for testing in grip

  • All-new web interface

    • Smaller controls
    • More tooltips
    • Add the ability to turn off the LEDs from the interface
    • Move display combobox underneath stream for easier display switching
    • Faster communication to Limelight. New web tech allowed us to simplify other parts of our code.

  • Flashing

    • We have migrated to "Balena Etcher"
    • Etcher is twice as fast and works on all platforms
    • Flash popups are fixed with the migration.

  • Other

    • The LED Mode has been modified to allow for pipeline-specific LED Modes
    • Faster boot times in automatic IP assignment mode.
      • Optimizations and minor bug fixes

2018.5 (3/28/18)​

2018.5 fixes a critical issue that would prevent users from tuning pipelines on snapshots.

2018.4 (3/19/18)​

2018.4 adds new contour sorting options. These are fairly important for cube tracking this year, as teams don't necessarily want to track the largest cube in view. In many cases, teams want to track the cube that is closest to their intakes. Many users have had to use the raw contours feature to implement their own sorting, so we want to make this as easy as possible.

Features​

  • Contour Sort Mode
    • Select between "largest", "smallest", "highest", "lowest", "leftmost", "rightmost", and "closest" sort options.
    • We feel that many teams will make use of the "closest" option for cube tracking.

2018.3 (2/28/18)​

2018.3 fixes a major networktables reconnection bug which would cause NetworkTables settings changes to not propagate to Limelight. Thanks to Peter Johnson and the WPILib team for pinpointing and fixing the underlying NT bug. This was (as far as we know) the last high-priority bug facing Limelight.

Settings changes such as ledMode, pipeline, and camMode should always apply to Limelight. You should no longer need workarounds to change Limelight settings while debugging, after restarting robot code, and after rebooting the roborio.

Changes​

  • Fix major NT syncing issue which broke settings changes (ledMode, pipeline, and camMode) during LabView debugging, and after a reset/reboot of the roborio.
  • Eye-dropper wand:
    • The eye dropper wand uses the same 10 unit window for Hue, but now uses a 30 unit window for saturation and value. This means that thresholding is more often a one-click operation, rather than a multi-step process.
  • Snapshots
    • Setting the snapshot value to "1" will only take a single snapshot and reset the value to 0. Snapshotting is throttled to 2 snapshots per second.
    • Snapshot limit increased to 100 images.
    • Snapshot selector area is now scrollable to support 100 images.

2018.2 (2/10/18)​

2018.2 fixes all known streaming bugs with various FRC dashboards. It also makes Limelight easier to tune and more versatile during events.

Features​

  • Thresholding wands
    • Setup HSV threshold parameters in a matter of clicks
    • The "Set" wand centers HSV parameters around the selected pixel
    • The "Add" wand adjusts HSV parameters to include the selected pixel
    • The "Subtract" wand adjusts HSV parameters to ignore the selected pixel
  • Snapshots
    • Snapshots allow users to save what Limelight is seeing during matches or event calibration, and tune pipelines while away from the field.
    • Save a snapshot with the web interface, or by posting a "1" to the "snapshot" NetworkTables key
    • To view snapshots, change the "Image Source" combo box on the input tab. This will allow you to test your pipelines on snapshots rather than Limelight's camera feed
    • Limelight will store up to 32 snapshots. It will automatically delete old snapshots if you exceed this limit.
  • New Streaming options
    • We've introduced the "stream" NetworkTables key to control Limelight's streaming mode. We've received requests for PiP (Picture-in-Picture) modes to better accommodate certain dashboards.
    • 0 - Standard - Side-by-side streams if a webcam is attached to Limelight
    • 1 - PiP Main - The secondary camera stream is placed in the lower-right corner of the primary camera stream.
    • 2 - PiP Secondary - The primary camera stream is placed in the lower-right corner of the secondary camera stream.
  • Increase streaming framerate to 22fps
  • Erosion and Dilation
    • Enable up to one iteration of both erosion and dilation.
    • Erosion will slightly erode the result of an HSV threshold. This is useful if many objects are passing through a tuned HSV threshold.
    • Dilation will slightly inflate the result of an HSV threshold. Use this to patch holes in thresholding results.
  • Restart Button
    • Restart Limelight's vision tracking from the web interface. This is only useful for teams that experience intermittent issues while debugging LabView code.

Optimizations​

  • Drop steady-state pipeline execution time to 3.5-4ms.

Bug Fixes​

  • Fix Shuffleboard streaming issues
  • Fix LabView dashboard streaming issues

2018.1 (1/8/18)​

  • Red-Balance slider
  • Blue-Balance slider
  • Better default color balance settings
  • Increased max exposure setting

2018.0 (1/3/18)​

On top of a ton of new case studies, more detailed documentation, and a full example program for an autonomous STEAMWORKS shooter, the software has received a major upgrade.

Features​

  • New Vision Pipeline interface
    • Add up to 10 unique vision pipelines, each with custom crosshairs, thresholding options, exposure, filtering options, etc.
    • Name each vision pipeline.
    • Mark any pipeline as the "default" pipeline.
    • Instantly switch between pipelines during a match with the new "pipeline" NetworkTables value. This is useful for games that have multiple vision targets (eg. the gear peg and boiler from 2017). This is also useful for teams that need to use slightly different crosshair options per robot, field, alliance, etc.
    • Download vision pipelines from Limelight to backup or share with other teams.
    • Upload vision pipelines to any "slot" to use downloaded pipelines.
  • Target "Grouping" option
    • Instantly prefer targets that consist of two shapes with the "dual" grouping mode". "Single" and "Tri" options are also available
  • New Crosshair Calibration interface
    • "Single" and "Dual" crosshair modes.
    • "Single" mode is what Limelight utilized prior to this update. Teams align their robots manually, and "calibrate" to re-zero targeting values about the crosshair.
    • "Dual" mode is an advanced feature for robots that need a dynamic crosshair that automatically adjusts as a target's area / distance to target changes. We've used this feature on some of our shooting robots, as some of them shot with a slight curve. This feature will also be useful for robots with uncentered andor misaligned Limelight mounts.
    • Separate X and Y calibration.
  • Add Valid Target "tv" key to Network Tables.
  • Add Targeting Latency "tl" key to Network Tables. "tl" measures the vision pipeline execution time. Add at least 11 ms for capture time.
  • Draw additional rectangle to help explain aspect ratio calculation.
  • Remove throttling feature, and lock Limelight to 90fps.
  • Disable focusing on most web interface buttons. Fixes workflow problem reported by teams who would calibrate their crosshairs, then press "enter" to enable their robots.
  • Post three "raw" contours and both crosshairs to Network Tables.
    • Access a raw contour with tx0, tx1, ta0, ta1, etc.
    • Access both raw crosshairs with cx0, cy0, cx1, cy1.
    • All x/y values are in normalized screen space (-1.0 to 1.0)
  • Add "suffix" option to web interface. Allows users to add a suffix to their Limelights' hostnames and NetworkTables (e.g. limelight-boiler). This feature should only be utilized if teams intend to use multiple Limelights on a single robot.
  • Display image version on web interface

Optimizations​

  • Decrease networking-related latency to ~0.2 ms from ~10ms (Thanks Thad House)
  • Move stream encoding and jpg compression to third core, eliminating 10ms hitch (25 - 30ms hitch with two cameras) seen every six frames.
  • Drop steady-state pipeline execution time to 5ms with SIMD optimizations.

Bugfixes​

  • Fix minor area value inaccuracy which prevented value from reaching 100% (maxed ~99%).
  • Fix half-pixel offset in all targeting calculations
  • Fix camera stream info not populating for NT servers started after Limelight's boot sequence. Regularly refresh camera stream info.
  • Fix bug which caused aspect ratio to "flip" occasionally.
  • Force standard stream output (rather than thresholded output) in driver mode.
  • Fix bug which prevented LEDs from blinking after resetting Networking information

2017.7 (11/21/17)​

  • Improved contour sorting. Was favoring small contours over larger contours.
  • New Coordinate system: Center is (0,0). ty increases as the target moves "up" the y-axis, and tx increases as the target moves "right" along the x-axis.
  • More accurate angle calculations (Pinhole camera model).
  • Display targeting info (tx, ty, ta, and ts) on webpage
  • Default targeting values are zeros. This means zeros are returned if no target is in view.
  • New side-by-side webpage layout. Still collapses to single column on small devices.
  • Continuous slider updates don't hurt config panel performance.
  • Area sent to NT as a percentage (0-100)
  • Image size down to 700MB from 2.1GB

2017.6 (11/13/17)​

  • New Imaging tool. Tested on Win7, Win8 and Win10.
  • Post camera stream to cameraserver streams. Works with smart dashboard camera streams, but shuffleboard has known bugs here
  • Quartic scaling on area sliders, quadratic scaling on aspect ratio sliders. This makes tuning much easier
  • Organize controls into “input”, “threshold”, “filter”, and “output” tabs
  • Continuous updates while dragging sliders
  • Area sent to NT as a percentage (0-100)
  • Display image size down to 700MB from 2.1GB

2017.5 (11/9/17)​

  • Image size down to 2.1GB from 3.9GB
  • Add driver mode and led mode APIs
  • Set ledMode to 0, 1, or 2 in the limelight table.
  • Set camMode to 0 or 1 in the limelight table.
  • Add ability to toggle between threshold image and raw image via web interface (will clean up in later release)
  • Post camera stream to network tables under CameraPublishing/limelight/streams (will need a hotfix)
  • Add skew to targeting information (“ts” in limelight table)
  • Add base “CommInterface” in anticipation of more protocols

2017.4 (10/30/17)​

  • Lots of boot and shutdown bullet-proofing

2017.3 (10/25/17)​

  • Hue range is 0-179 from 0-255
  • Decrease max log size, clear logs, clear apt cache

2017.2 (10/23/17)​

  • Manual ISO sensitivity
  • Minimum exposure increased to 2

2017.1 (10/21/17)​

  • Optimizations
  • “Convexity” changed to “Fullness”
  • Exposure range set to 0-128 ms from 0-255 ms
  • Support two cameras
  • Fully support single-point calibration