Building A Pipeline¶
To configure a limelight vision pipeline, first access its web interface at http://10.te.am.11:5801. If you have opted to use a dynamically assigned ip-address, access the interface at http://limelight.local:5801.
The “Tracking” page is comprised of five tuning tabs:
The Input Tab hosts controls to change the raw camera image before it is passed through the processing pipeline.
Controls the desired pipeline type. Switch to “GRIP” if you want to use a GRIP pipeline.
Controls the source of the image that is passed through the pipeline. Switch to “Snapshot” to test your vision pipeline on a stored Snapshot.
This control auto-resets to “Camera” when the GUI is closed.
Controls the resolution of the camera and vision pipeline. We recommend using the 320x240 pipeline unless you are utilizing 3D functionality.
320x240 pipelines execute at 90fps, while 960x720 pipelines execute at 22 fps.
Controls the default LED mode for this pipeline. This may be overidden during a match with the “LED” network table option.
Controls the orientation of incoming frames. Set it to “inverted” if your camera is mounted upside-down.
Controls the camera’s exposure setting in .1 millisecond intervals. Think of a camera as a grid of light-collecting buckets - exposure time controls how long your camera’s “buckets” are open per frame. Lowering the exposure time will effectively darken your image. Low and fixed exposure times are crucial in FRC, as they black-out the bulk of incoming image data. Well-lit retroreflective tape will stand out in a mostly black image, making vision processing a straightforward process.
Black Level Offset¶
Increasing the black level offset can significantly darken your camera stream. This should be increased to further remove arena lights and bright spots from your image.
Red Balance, Blue Balance¶
Controls the intensity of Red and Blue color components in your image. These collecively control your Limelight’s white balance. We recommend leaving these at their default values of
Thresholding is a critical component of most FRC vision tracking algorithms. It is the act of taking an image, and throwing away any pixels that aren’t in a specific color range. The result of thresholding is generally a one-dimensional image in which a pixel is either “on” or “off.
Video Feed (Located beneath stream)¶
Controls which image is streamed from the mjpeg server. You should switch to the “threshold” image if you need to tune your HSV thresholding.
- Wands enable users to click on Limelights’s video stream to perform automatic HSV thresholding.
- The “Eyedropper” wand centers HSV parameters around the selected pixel
- The “Add” wand adjusts HSV parameters to include the selected pixel
- The “Subtract” wand adjust HSV paramters to ignore the selected pixel
Describes a “pure” color. A Hue of “0” describes pure red, and a hue of 1/3 (59 on the slider) describes pure green. Hue is useful because it doesn’t change as a pixel “brightens” or “darkens”. This is the most important parameter to tune. If you make your hue range as small as possible, you will have little if any trouble transitioning to an actual FRC field.
Describes the extent to which a color is “pure”. Another way to think of this is how washed-out a color appears, that is, how much “white” is in a color. Low saturation means a color is almost white, and high saturation means a color is almost “pure”.
Describes the darkness of a color, or how much “black” is in a color. A low value corresponds to a near-black color. You should absolutely increase the minimum value from zero, so that black pixels are not passed through the processing pipeline.
Erosion and Dilation¶
Erosion slightly erodes the result of an HSV threshold. This is useful if many objects are passing through a tuned HSV threshold. Dilation slightly inflates the result of an HSV threshold. Use this to patch holes in thresholding results.
After thresholding, Limelight generates a list of contours. After that, each contour is wrapped in a bounding rectangle an unrotated rectangle, and a “convex hull”. These are passed through a series of filters to determine the “best” contour. If multiple contours pass through all filters, Limelight chooses the best contour using the “Sort Mode” Control.
Controls how contours are sorted after they are passed through all other filters.
Controls the range of acceptable bounding-rectangle areas, as percentages of the screen. You can increase the minimum area to help filter-out stadium lights, and decrease the maximum value to help filter-out things like large displays near the field.
The area slider is not linearly scaled, but quarticly scaled. This is done to provide extra precision near the lower-end of area values, where many FRC targets lie. The area of a square scales quadratically with its side length, but x^4 scaling provides even greater precision where it is needed.
Fullness is the percentage of “on” pixels in the chosen contour’s bounding rectangle. A solid rectangle target will have a near-1.0 fullness, while a U-shaped target will have a low fullness.
Target Aspect Ratio¶
Aspect ratio is defined by the width of the bounding rectangle of the chosen contour divided by its height. A low aspect ratio describes a “tall” rectangle, while a high aspect ratio describes a “wide” rectangle.
The aspect ratio slider is also quadratically scaled.
Rejects contours on the basis of their orientation.
Smart Speckle Rejection¶
Rejects relatively small contours that have passed through all other filters. This is essential if a target must remain trackable from short-range and long-range. This feature was introduced in the 2019 season to reject Limelight’s LED reflections when robots were very close to targets.
Contour Simplification %¶
Simplifies contours to reduce the number of corners in each contour. This is useful when using the raw corner or 3D features.
Controls target “grouping”. Set to dual mode to look for “targets” that consist of two shapes, or tri mode to look for targets that consist of three shapes.
Intersection Filter (Dual Targets Only)¶
Rejects groups of contours based on how they would intersect if extended to infinity.
This tab controls what happens during the last stage of the vision pipeline
Controls the point of interest of the chosen contour’s bounding rectangle. By default, the tracking parameters tx and ty represent the offsets from your crosshair to the center of the chosen rectangle. You can use another option if a target changes in size, or is comprised of two targets that sometimes blend together.
Send Raw Corners?¶
Set this control to “yes” to submit raw corners over network tables. Tune the number of corners submitted by adjusting the “Contour Simplification” value in the “Contour Filtering” page.
Send Raw Contours?¶
Set this control to “yes” to submit raw contours over network tables. The top 3 passing contours will be submitted.
Controls the “origin” of your targeting values. Let’s say a shooter on your robot needs to be calibrated such that it always points a bit left-of-center. You can line up your robot, click “calibrate,” and all of your targeting values will be sent relative to your new crosshair. See the calibration page for more details!