Vehicle Annotation Toolbox V2:

A comprehensive annotation toolbox that uses YOLO for vehicle detection and BoTSORT for tracking, with an integrated GUI for single file processing, batch jobs, manual annotations, road mask annotation (closed polygon road segmentation), and tagging vehicle behaviors. It now supports complete workflows for both vehicle detection/tracking and road/area annotation.

Requirements (Ideally):

• Windows 11
• Python 3.11
• CUDA 12.1

Installation:

To install the required dependencies, run the following command:

pip install -r requirements.txt

Note: This project uses newFastReID for ReID feature extraction. Ensure it is installed according to the repository's instructions.

Model Setup:

Download the required models and place them in the models/ directory:

1. YOLO Model: Place yolo11m-obb.pt/yolo11m.pt or your custom pretrained model in the models/ directory. This is the default model path used by the application.
2. ReID Model: Download veriwild_bot_R50-ibn.pth from the newFastReID MODEL_ZOO or your custom ReID model and place it in the models/ directory. This model is required for the custom BoTSORT tracker with ReID enabled.

Folder Structure:

project/
├── main.py              #Main GUI application entry point
├── batch_process.py     #Batch processing functions
├── config.py            #Configuration settings
├── utils.py             #Utility functions
├── setup.py             #Package setup
├── requirements.txt     #Dependencies
├── README.md            #This file
├── botsort.yaml         #BoTSORT tracker configuration
├── modules/             #Core modules
│   ├── __init__.py
│   ├── detector.py      #YOLO based detection logic
│   ├── tracker.py       #BoTSORT tracking logic
│   ├── botsort.py       #Custom BoTSORT implementation with ReID
│   ├── annotator.py     #Manual annotation GUI
│   ├── road_mask.py     #Road mask annotation GUI (closed polygons)
│   ├── behavior.py      #Behavior tagging GUI
│   └── exporter.py      #Data merging logic
├── models/              #YOLO and ReID models
│   ├── yolo11m-obb.pt
│   └── veriwild_bot_R50-ibn.pth
└── output/              #Default output directory
    ├── detections/
    ├── tracking/
    ├── manual/
    ├── road_masks/
    └── behaviors/

Usage:

Run the main application from the command line, which will open the Vehicle Annotation Toolbox GUI.

python main.py

The application is organized into several tabs for different functionalities.

Detect Tab:

This tab is used for running vehicle detection on a single image file.

• Input Image: Select the image file to process.
• YOLO Model: Specify the path to the .pt model file.
• Output Directory: Choose where the annotated image and detections.json file will be saved.
• Confidence Threshold: Set the confidence threshold for detection.

Track Tab:

This tab is used for running vehicle tracking on a single video file.

• Input Video: Select the video file to process.
• YOLO Model: Specify the path to the .pt model file.
• Output Directory: Choose where the annotated video and _tracks.json file will be saved.
• Confidence Threshold: Set the confidence threshold for tracking.

Manual Annotate Tab:

Launches a dedicated window for manual annotation of objects in images or videos.

• Input Video/Image: Select the media file to annotate.
• Output Annotation File: Specify the .json file to save annotations to.
• Load Existing Annotations: Optionally load a previously saved annotation file to continue editing.
• Controls:
  • Mouse: Draw bounding boxes on the canvas.
  • Delete Key: Deletes the currently selected annotation box in the current frame.
  • Ctrl+Z: Undo the last action.
  • Ctrl+Shift+Z: Redo the last undone action.
  • Buttons: Use the "Previous" and "Next" buttons for frame navigation, and "Save" to save progress.

Road Mask Tab:

This tab provides a dedicated tool for annotating road areas in images or videos by drawing closed polygons. It is designed for traffic analysis, autonomous vehicle training data, and road segmentation tasks, with special support for aerial drone footage where the scene remains static throughout the video.

Key Features:

• Closed polygon drawing workflow (similar to Roboflow's polygon tool).
• Additive polygons to mark road areas (displayed with green outlines).
• Exclusion polygons to mark islands, medians, or holes within roads (displayed with red outlines).
• Support for complex road geometries like roundabouts with center islands.
• Vertex level undo/redo functionality (Ctrl+Z / Ctrl+Y).
• Support for both images and videos.
• For videos: annotation applies to the entire video .(assumes static aerial scene).
• Outputs: binary mask PNG, visualization PNG, and JSON metadata with polygon coordinates.

Usage:

1. Launch the application and navigate to the Road Mask tab.
2. Click Load Image/Video and select your source file (supports .jpg, .png, .mp4, .avi, .mov, .mkv)
3. Choose the polygon type using the radio buttons:
   • Add to Road Mask for road areas (polygons will be outlined in green).
   • Exclude (Island) for non-road areas within roads such as medians or islands (polygons will be outlined in red).
4. Draw polygons by left clicking on the canvas to add vertices.
5. Close the polygon using one of three methods:
   • Click near the starting vertex (auto-snap activates within approximately 12 pixels).
   • Right click anywhere on the canvas.
   • Click the Close Polygon button in the control panel.
6. Repeat steps 3–5 to add multiple polygons (like draw an outer road boundary as additive, then inner island exclusions).
7. Use Ctrl+Z to undo the last added vertex (or remove the last closed polygon if the current polygon is empty); use Ctrl+Y to redo.
8. Adjust the Mask Opacity % slider (10–100%) to change the overlay transparency for better visibility of the underlying image.
9. Click Save Annotation to export the three output files.

Output Files:

The tool saves three files to the same directory as the source file:

• {filename}_road_mask.png: Binary mask image (white pixels = road areas, black pixels = non-road areas).
• {filename}_road_viz.png: Visualization overlay showing the mask composited on the original frame for visual verification.
• {filename}_road_annotation.json: JSON metadata file containing polygon vertices, polygon types (additive/exclusion), source file information, and timestamp.

For videos, only one set of output files is generated (no per-frame outputs), as the annotation is assumed to apply to the entire video. The visualization uses the currently displayed frame at the time of saving.

Example Use Case:

Annotating a roundabout: First, draw a large outer polygon around the entire roundabout area and mark it as Add to Road Mask (green outline). Then, draw an inner circular polygon around the center island and mark it as Exclude (Island) (red outline). The resulting binary mask will show a ring-shaped road area with the center island properly excluded.

Keyboard Shortcuts:

• Ctrl+Z: Undo last vertex or polygon.
• Ctrl+Y: Redo last undone action.
• Left Click: Add vertex to current polygon.
• Right Click: Close current polygon (requires at least 3 vertices).

Tag Behavior Tab:

Launches a dedicated window for tagging behaviors of tracked vehicles in a video. This requires a video file and its corresponding tracks JSON file generated from the "Track" tab.

• Input Video File: Select the video file for behavior tagging.
• Output Behaviors File: Specify the .json file where behavior tags will be saved.
• Load Existing Behaviors: Optionally load a previously saved behaviors file.
• Controls:
  • Click: Select a tracked vehicle in the video frame to tag its behavior.
  • Keyboard Shortcuts: Use 'A' and 'S' to set the start and end frames for a behavior, and 'Left'/'Right' arrow keys for frame navigation.

Merge Tab:

This tab is used to merge annotations from manual edits, automatic tracking, and behavior tagging into a single, comprehensive JSON file.

• Manual Annotations (Optional): Select the JSON file containing manual annotations.
• Tracks File (Optional): Select the _tracks.json file generated from the "Track" tab.
• Behaviors File (Optional): Select the JSON file containing behavior tags.
• Merged Output File: Specify the output .json file to save the merged data.

Batch Process Tab:

This tab allows for running detection or tracking on a directory of images or videos, respectively.

• Input Directory: Select the folder containing the media files.
• YOLO Model: Specify the path to the .pt model file.
• Output Directory: Choose a parent directory for the output results. The tool will create subdirectories for each processed file.

BoTSORT Configuration:

The tracker uses a custom implementation of BoTSORT with ReID features for improved tracking accuracy. The configuration is located in botsort.yaml.

• Custom BoTSORT: Implementation located in modules/botsort.py with Kalman filtering, Hungarian matching, and a combined IoU + ReID similarity cost
• ReID Model: Uses a VeRi-Wild pretrained model for vehicle re-identification.
• Track Buffer: Set to 100 frames for better handling of long-term occlusions.
• Appearance Matching: Combines motion and appearance (ReID) features for robust tracking.
• Global Motion Compensation: Uses sparse optical flow to compensate for camera movement.

Output Format:

All detections, tracks, and behaviors are saved in a structured JSON format for consistency and ease of use in downstream tasks.

Download the pretrained models:

1. YOLOv11: https://docs.ultralytics.com/models/yolo11/#performance-metrics
2. Veri-Wild ReID: https://github.com/JDAI-CV/fast-reid/blob/master/MODEL_ZOO.md

Troubleshooting:

1. ReID Model Not Found: Ensure the veriwild_bot_R50-ibn.pth file is correctly placed in the models/ directory.
2. CUDA Issues: Verify that your PyTorch installation is compatible with your installed CUDA 12.1 toolkit.
3. Ideal Structure (to avoid issues): Make sure to create a 'models' and an (optional) 'output' folder for creation of the datatset.