feat: initial HSAP platform
Huaxu Sentinel Active Safety Platform with embedded algorithm code, Docker Compose setup, and vendored dataset scaffolds for clone-and-run. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,137 @@
|
||||
# Ultralytics YOLO Interactive Object Tracking UI 🚀
|
||||
|
||||
A real-time [object detection](https://docs.ultralytics.com/tasks/detect/) and [tracking](https://docs.ultralytics.com/modes/track/) UI built with [Ultralytics YOLO11](https://github.com/ultralytics/ultralytics) and [OpenCV](https://opencv.org/), designed for interactive demos and seamless integration of tracking overlays. Whether you're just getting started with object tracking or looking to enhance it with additional features, this project provides a solid foundation.
|
||||
|
||||
https://github.com/user-attachments/assets/723e919e-555b-4cca-8e60-18e711d4f3b2
|
||||
|
||||
## ✨ Features
|
||||
|
||||
- Real-time object detection and visual tracking
|
||||
- Click-to-track any detected object
|
||||
- Scope lines and bold [bounding boxes](https://docs.ultralytics.com/usage/simple-utilities/#bounding-boxes) for active tracking
|
||||
- Dashed boxes for passive (non-tracked) objects
|
||||
- [Live terminal output](https://docs.ultralytics.com/guides/view-results-in-terminal/): object ID, label, [confidence](https://www.ultralytics.com/glossary/confidence), and center coordinates
|
||||
- Adjustable object tracking algorithms ([ByteTrack](https://docs.ultralytics.com/reference/trackers/byte_tracker/), [BoT-SORT](https://docs.ultralytics.com/reference/trackers/bot_sort/))
|
||||
- Supports:
|
||||
- [PyTorch](https://pytorch.org/) `.pt` models (for GPU devices like [NVIDIA Jetson](https://docs.ultralytics.com/guides/nvidia-jetson/) or [CUDA](https://developer.nvidia.com/cuda)-enabled desktops)
|
||||
- [NCNN](https://docs.ultralytics.com/integrations/ncnn/) `.param + .bin` models (for CPU-only devices like [Raspberry Pi](https://www.raspberrypi.org/) or ARM boards)
|
||||
|
||||
## 🏗️ Project Structure
|
||||
|
||||
```
|
||||
YOLO-Interactive-Tracking-UI/
|
||||
├── interactive_tracker.py # Main Python tracking UI script
|
||||
└── README.md # You're here!
|
||||
```
|
||||
|
||||
## 💻 Hardware & Model Compatibility
|
||||
|
||||
| Platform | Model Format | Example Model | GPU Acceleration | Notes |
|
||||
| ---------------- | ------------------ | -------------------- | ---------------- | ------------------------------- |
|
||||
| Raspberry Pi 4/5 | NCNN (.param/.bin) | `yolov8n_ncnn_model` | ❌ CPU only | Recommended format for Pi/ARM |
|
||||
| Jetson Nano | PyTorch (.pt) | `yolov8n.pt` | ✅ CUDA | Real-time performance possible |
|
||||
| Desktop w/ GPU | PyTorch (.pt) | `yolov8s.pt` | ✅ CUDA | Best performance |
|
||||
| CPU-only laptops | NCNN (.param/.bin) | `yolov8n_ncnn_model` | ❌ | Decent performance (~10–15 FPS) |
|
||||
|
||||
_Note: Performance may vary based on the specific hardware, model complexity, and input resolution._
|
||||
|
||||
## 🛠️ Installation
|
||||
|
||||
### Basic Dependencies
|
||||
|
||||
Install the core `ultralytics` package:
|
||||
|
||||
```bash
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
> **Tip:** Use a virtual environment like `venv` or [`conda`](https://docs.ultralytics.com/guides/conda-quickstart/) (recommended) to manage dependencies.
|
||||
|
||||
> **GPU Support:** Install PyTorch based on your system and CUDA version by following the official guide: [https://pytorch.org/get-started/locally/](https://pytorch.org/get-started/locally/)
|
||||
|
||||
## 🚀 Quickstart
|
||||
|
||||
### Step 1: Download, Convert, or Specify Model
|
||||
|
||||
- For pre-trained Ultralytics YOLO [models](https://docs.ultralytics.com/models/) (e.g., `yolo11s.pt` or `yolov8s.pt`), simply specify the model name in the script parameters (`model_file`). These models will be automatically downloaded and cached. You can also manually download them from [Ultralytics Assets Releases](https://github.com/ultralytics/assets/releases) and place them in the project folder.
|
||||
- If you're using a custom-trained YOLO model, ensure the model file is in the project folder or provide its relative path.
|
||||
- For CPU-only devices, export your chosen model (e.g., `yolov8n.pt`) to the [NCNN format](https://docs.ultralytics.com/integrations/ncnn/) using the Ultralytics `export` mode.
|
||||
|
||||
- **Supported Formats:**
|
||||
- `yolo11s.pt` (for GPU with PyTorch)
|
||||
- `yolov8n_ncnn_model` (directory containing `.param` and `.bin` files for CPU with NCNN)
|
||||
|
||||
### Step 2: Configure the Script
|
||||
|
||||
Edit the global parameters at the top of `interactive_tracker.py`:
|
||||
|
||||
```python
|
||||
# --- Configuration ---
|
||||
enable_gpu = False # Set True if running with CUDA and PyTorch model
|
||||
model_file = "yolo11s.pt" # Path to model file (.pt for GPU, _ncnn_model dir for CPU)
|
||||
show_fps = True # Display current FPS in the top-left corner
|
||||
show_conf = False # Display confidence score for each detection
|
||||
save_video = False # Set True to save the output video stream
|
||||
video_output_path = "interactive_tracker_output.avi" # Output video file name
|
||||
|
||||
# --- Detection & Tracking Parameters ---
|
||||
conf = 0.3 # Minimum confidence threshold for object detection
|
||||
iou = 0.3 # IoU threshold for Non-Maximum Suppression (NMS)
|
||||
max_det = 20 # Maximum number of objects to detect per frame
|
||||
|
||||
tracker = "bytetrack.yaml" # Tracker configuration: 'bytetrack.yaml' or 'botsort.yaml'
|
||||
track_args = {
|
||||
"persist": True, # Keep track history across frames
|
||||
"verbose": False, # Suppress detailed tracker debug output
|
||||
}
|
||||
|
||||
window_name = "Ultralytics YOLO Interactive Tracking" # Name for the OpenCV display window
|
||||
# --- End Configuration ---
|
||||
```
|
||||
|
||||
- **`enable_gpu`**: Set to `True` if you have a CUDA-compatible GPU and are using a `.pt` model. Keep `False` for NCNN models or CPU-only execution.
|
||||
- **`model_file`**: Ensure this points to the correct model file or directory based on `enable_gpu`.
|
||||
- **`conf`**: Adjust the [confidence](https://www.ultralytics.com/glossary/confidence) threshold. Lower values detect more objects but may increase false positives.
|
||||
- **`iou`**: Set the [Intersection over Union (IoU)](https://www.ultralytics.com/glossary/intersection-over-union-iou) threshold for [Non-Maximum Suppression (NMS)](https://www.ultralytics.com/glossary/non-maximum-suppression-nms). Higher values allow more overlapping boxes.
|
||||
- **`tracker`**: Choose between available tracker configuration files ([ByteTrack](https://docs.ultralytics.com/reference/trackers/byte_tracker/), [BoT-SORT](https://docs.ultralytics.com/reference/trackers/bot_sort/)).
|
||||
|
||||
### Step 3: Run the Object Tracking
|
||||
|
||||
Execute the script from your terminal:
|
||||
|
||||
```bash
|
||||
python interactive_tracker.py
|
||||
```
|
||||
|
||||
### Controls
|
||||
|
||||
- 🖱️ **Left-click** on a detected object's bounding box to start tracking it.
|
||||
- 🔄 Press the **`c`** key to cancel the current tracking and select a new object.
|
||||
- ❌ Press the **`q`** key to quit the application.
|
||||
|
||||
### Saving Output Video (Optional)
|
||||
|
||||
If you want to record the tracking session, enable the `save_video` option in the configuration:
|
||||
|
||||
```python
|
||||
save_video = True # Enables video recording
|
||||
video_output_path = "output.avi" # Customize your output file name (e.g., .mp4, .avi)
|
||||
```
|
||||
|
||||
The video file will be saved in the project's working directory when you quit the application by pressing `q`.
|
||||
|
||||
## 👤 Author
|
||||
|
||||
- **Alireza**
|
||||
- [Connect on LinkedIn](https://www.linkedin.com/in/alireza787b)
|
||||
- Published: 2025-04-01
|
||||
|
||||
## 📜 License & Disclaimer
|
||||
|
||||
This project is released under the [AGPL-3.0 license](https://www.ultralytics.com/legal/agpl-3-0-software-license). For full licensing details, please refer to the [Ultralytics Licensing page](https://www.ultralytics.com/license).
|
||||
|
||||
This software is provided "as is" for educational and demonstration purposes. Use it responsibly and at your own risk. The author assumes no liability for misuse or unintended consequences.
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
Contributions, feedback, and bug reports are welcome! Feel free to open an issue or submit a pull request on the original repository if you have improvements or suggestions.
|
||||
@@ -0,0 +1,241 @@
|
||||
# Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
import cv2
|
||||
|
||||
from ultralytics import YOLO
|
||||
from ultralytics.utils import LOGGER
|
||||
from ultralytics.utils.plotting import Annotator, colors
|
||||
|
||||
enable_gpu = False # Set True if running with CUDA
|
||||
model_file = "yolo11s.pt" # Path to model file
|
||||
show_fps = True # If True, shows current FPS in top-left corner
|
||||
show_conf = False # Display or hide the confidence score
|
||||
save_video = False # Set True to save output video
|
||||
video_output_path = "interactive_tracker_output.avi" # Output video file name
|
||||
|
||||
|
||||
conf = 0.3 # Min confidence for object detection (lower = more detections, possibly more false positives)
|
||||
iou = 0.3 # IoU threshold for NMS (higher = less overlap allowed)
|
||||
max_det = 20 # Maximum objects per image (increase for crowded scenes)
|
||||
|
||||
tracker = "bytetrack.yaml" # Tracker config: 'bytetrack.yaml', 'botsort.yaml', etc.
|
||||
track_args = {
|
||||
"persist": True, # Keep frames history as a stream for continuous tracking
|
||||
"verbose": False, # Print debug info from tracker
|
||||
}
|
||||
|
||||
window_name = "Ultralytics YOLO Interactive Tracking" # Output window name
|
||||
|
||||
LOGGER.info("🚀 Initializing model...")
|
||||
if enable_gpu:
|
||||
LOGGER.info("Using GPU...")
|
||||
model = YOLO(model_file)
|
||||
model.to("cuda")
|
||||
else:
|
||||
LOGGER.info("Using CPU...")
|
||||
model = YOLO(model_file, task="detect")
|
||||
|
||||
classes = model.names # Store model class names
|
||||
|
||||
cap = cv2.VideoCapture(0) # Replace with video path if needed
|
||||
if not cap.isOpened():
|
||||
raise SystemError("Failed to open video source.")
|
||||
|
||||
vw = None # Initialized lazily after the first frame is read
|
||||
|
||||
selected_object_id = None
|
||||
selected_bbox = None
|
||||
selected_center = None
|
||||
latest_detections: list[list[float]] = []
|
||||
|
||||
|
||||
def get_center(x1: int, y1: int, x2: int, y2: int) -> tuple[int, int]:
|
||||
"""Calculate the center point of a bounding box.
|
||||
|
||||
Args:
|
||||
x1 (int): Top-left X coordinate.
|
||||
y1 (int): Top-left Y coordinate.
|
||||
x2 (int): Bottom-right X coordinate.
|
||||
y2 (int): Bottom-right Y coordinate.
|
||||
|
||||
Returns:
|
||||
center_x (int): X-coordinate of the center point.
|
||||
center_y (int): Y-coordinate of the center point.
|
||||
"""
|
||||
return (x1 + x2) // 2, (y1 + y2) // 2
|
||||
|
||||
|
||||
def extend_line_from_edge(mid_x: int, mid_y: int, direction: str, img_shape: tuple[int, int, int]) -> tuple[int, int]:
|
||||
"""Calculate the endpoint to extend a line from the center toward an image edge.
|
||||
|
||||
Args:
|
||||
mid_x (int): X-coordinate of the midpoint.
|
||||
mid_y (int): Y-coordinate of the midpoint.
|
||||
direction (str): Direction to extend ('left', 'right', 'up', 'down').
|
||||
img_shape (tuple[int, int, int]): Image shape in (height, width, channels).
|
||||
|
||||
Returns:
|
||||
end_x (int): X-coordinate of the endpoint.
|
||||
end_y (int): Y-coordinate of the endpoint.
|
||||
"""
|
||||
h, w = img_shape[:2]
|
||||
if direction == "down":
|
||||
return mid_x, h - 1
|
||||
elif direction == "left":
|
||||
return 0, mid_y
|
||||
elif direction == "right":
|
||||
return w - 1, mid_y
|
||||
elif direction == "up":
|
||||
return mid_x, 0
|
||||
else:
|
||||
return mid_x, mid_y
|
||||
|
||||
|
||||
def draw_tracking_scope(im, bbox: tuple, color: tuple) -> None:
|
||||
"""Draw tracking scope lines extending from the bounding box to image edges.
|
||||
|
||||
Args:
|
||||
im (np.ndarray): Image array to draw on.
|
||||
bbox (tuple): Bounding box coordinates (x1, y1, x2, y2).
|
||||
color (tuple): Color in BGR format for drawing.
|
||||
"""
|
||||
x1, y1, x2, y2 = bbox
|
||||
mid_top = ((x1 + x2) // 2, y1)
|
||||
mid_bottom = ((x1 + x2) // 2, y2)
|
||||
mid_left = (x1, (y1 + y2) // 2)
|
||||
mid_right = (x2, (y1 + y2) // 2)
|
||||
cv2.line(im, mid_top, extend_line_from_edge(*mid_top, "up", im.shape), color, 2)
|
||||
cv2.line(im, mid_bottom, extend_line_from_edge(*mid_bottom, "down", im.shape), color, 2)
|
||||
cv2.line(im, mid_left, extend_line_from_edge(*mid_left, "left", im.shape), color, 2)
|
||||
cv2.line(im, mid_right, extend_line_from_edge(*mid_right, "right", im.shape), color, 2)
|
||||
|
||||
|
||||
def click_event(event: int, x: int, y: int, flags: int, param) -> None:
|
||||
"""Handle mouse click events to select an object for focused tracking.
|
||||
|
||||
Args:
|
||||
event (int): OpenCV mouse event type.
|
||||
x (int): X-coordinate of the mouse event.
|
||||
y (int): Y-coordinate of the mouse event.
|
||||
flags (int): Any relevant flags passed by OpenCV.
|
||||
param (Any): Additional parameters (not used).
|
||||
"""
|
||||
global selected_object_id, latest_detections
|
||||
if event == cv2.EVENT_LBUTTONDOWN:
|
||||
if not latest_detections:
|
||||
return
|
||||
min_area = float("inf")
|
||||
best_match = None
|
||||
for track in latest_detections:
|
||||
if len(track) < 6:
|
||||
continue
|
||||
x1, y1, x2, y2 = map(int, track[:4])
|
||||
if x1 <= x <= x2 and y1 <= y <= y2:
|
||||
area = max(0, x2 - x1) * max(0, y2 - y1)
|
||||
if area < min_area:
|
||||
track_id = int(track[4]) if len(track) >= 7 else -1
|
||||
class_id = int(track[6]) if len(track) >= 7 else int(track[5])
|
||||
min_area = area
|
||||
best_match = (track_id, classes.get(class_id, str(class_id)))
|
||||
if best_match:
|
||||
selected_object_id, label = best_match
|
||||
LOGGER.info(f"Tracking started: {label} (ID {selected_object_id})")
|
||||
|
||||
|
||||
cv2.namedWindow(window_name)
|
||||
cv2.setMouseCallback(window_name, click_event)
|
||||
|
||||
fps_counter, fps_timer, fps_display = 0, time.time(), 0
|
||||
|
||||
while cap.isOpened():
|
||||
success, im = cap.read()
|
||||
if not success:
|
||||
break
|
||||
|
||||
results = model.track(im, conf=conf, iou=iou, max_det=max_det, tracker=tracker, **track_args)
|
||||
annotator = Annotator(im)
|
||||
detections = results[0].boxes.data if results[0].boxes is not None else []
|
||||
latest_detections = detections.cpu().tolist() if hasattr(detections, "cpu") else list(detections) # type: ignore[arg-type]
|
||||
detected_objects: list[str] = []
|
||||
for track in detections:
|
||||
track = track.tolist()
|
||||
if len(track) < 6:
|
||||
continue
|
||||
x1, y1, x2, y2 = map(int, track[:4])
|
||||
class_id = int(track[6]) if len(track) >= 7 else int(track[5])
|
||||
track_id = int(track[4]) if len(track) == 7 else -1
|
||||
color = colors(track_id, True)
|
||||
txt_color = annotator.get_txt_color(color)
|
||||
conf_score = float(track[5]) if len(track) >= 7 else 0.0
|
||||
class_name = classes.get(class_id, str(class_id))
|
||||
label = f"{class_name} ID {track_id}" + (f" ({conf_score:.2f})" if show_conf else "")
|
||||
center = get_center(x1, y1, x2, y2)
|
||||
detected_objects.append(f"{class_name}#{track_id}@{center[0]},{center[1]}")
|
||||
if track_id == selected_object_id:
|
||||
draw_tracking_scope(im, (x1, y1, x2, y2), color)
|
||||
cv2.circle(im, center, 6, color, -1)
|
||||
|
||||
# Pulsing circle for attention
|
||||
pulse_radius = 8 + int(4 * abs(time.time() % 1 - 0.5))
|
||||
cv2.circle(im, center, pulse_radius, color, 2)
|
||||
|
||||
annotator.box_label([x1, y1, x2, y2], label=f"ACTIVE: TRACK {track_id}", color=color)
|
||||
else:
|
||||
# Draw dashed box for other objects
|
||||
for i in range(x1, x2, 10):
|
||||
cv2.line(im, (i, y1), (i + 5, y1), color, 3)
|
||||
cv2.line(im, (i, y2), (i + 5, y2), color, 3)
|
||||
for i in range(y1, y2, 10):
|
||||
cv2.line(im, (x1, i), (x1, i + 5), color, 3)
|
||||
cv2.line(im, (x2, i), (x2, i + 5), color, 3)
|
||||
# Draw label text with background
|
||||
(tw, th), bl = cv2.getTextSize(label, 0, 0.7, 2)
|
||||
cv2.rectangle(im, (x1 + 5 - 5, y1 + 20 - th - 5), (x1 + 5 + tw + 5, y1 + 20 + bl), color, -1)
|
||||
cv2.putText(im, label, (x1 + 5, y1 + 20), 0, 0.7, txt_color, 1, cv2.LINE_AA)
|
||||
|
||||
if show_fps:
|
||||
fps_counter += 1
|
||||
if time.time() - fps_timer >= 1.0:
|
||||
fps_display = fps_counter
|
||||
fps_counter = 0
|
||||
fps_timer = time.time()
|
||||
|
||||
# Draw FPS text with background
|
||||
fps_text = f"FPS: {fps_display}"
|
||||
(tw, th), bl = cv2.getTextSize(fps_text, 0, 0.7, 2)
|
||||
cv2.rectangle(im, (10 - 5, 25 - th - 5), (10 + tw + 5, 25 + bl), (255, 255, 255), -1)
|
||||
cv2.putText(im, fps_text, (10, 25), 0, 0.7, (104, 31, 17), 1, cv2.LINE_AA)
|
||||
|
||||
if save_video and vw is None:
|
||||
h, w = im.shape[:2]
|
||||
fps = cap.get(cv2.CAP_PROP_FPS) or 0
|
||||
fps = float(fps) if fps and fps > 0 else 30.0
|
||||
ext = video_output_path.lower()
|
||||
fourcc = cv2.VideoWriter_fourcc(*("MJPG" if ext.endswith(".avi") else "mp4v"))
|
||||
vw = cv2.VideoWriter(video_output_path, fourcc, fps, (w, h))
|
||||
|
||||
cv2.imshow(window_name, im)
|
||||
if save_video and vw is not None:
|
||||
vw.write(im)
|
||||
# Terminal logging
|
||||
LOGGER.info(
|
||||
f"Detected {len(detections)} object(s): {' | '.join(detected_objects)}"
|
||||
if detected_objects
|
||||
else f"Detected {len(detections)} object(s)."
|
||||
)
|
||||
|
||||
key = cv2.waitKey(1) & 0xFF
|
||||
if key == ord("q"):
|
||||
break
|
||||
elif key == ord("c"):
|
||||
LOGGER.info("Tracking reset.")
|
||||
selected_object_id = None
|
||||
|
||||
cap.release()
|
||||
if save_video and vw is not None:
|
||||
vw.release()
|
||||
cv2.destroyAllWindows()
|
||||
Reference in New Issue
Block a user