feat: initial HSAP platform

Huaxu Sentinel Active Safety Platform with embedded algorithm code, Docker Compose setup, and vendored dataset scaffolds for clone-and-run. Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 16:59:59 +08:00
commit 7c43b44c57
1619 changed files with 373355 additions and 0 deletions
--- a/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/CMakeLists.txt
+++ b/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/CMakeLists.txt
@@ -0,0 +1,21 @@
+cmake_minimum_required(VERSION 3.12)
+project(yolov8_openvino_example)
+
+set(CMAKE_CXX_STANDARD 14)
+
+find_package(OpenCV REQUIRED)
+
+include_directories(
+	${OpenCV_INCLUDE_DIRS}
+	/path/to/intel/openvino/runtime/include
+)
+
+add_executable(detect
+	main.cc
+	inference.cc
+)
+
+target_link_libraries(detect
+	${OpenCV_LIBS}
+	/path/to/intel/openvino/runtime/lib/intel64/libopenvino.so
+)
--- a/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/README.md
+++ b/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/README.md
@@ -0,0 +1,81 @@
+# YOLOv8 OpenVINO Inference in C++
+
+Welcome to the [Ultralytics YOLOv8](https://docs.ultralytics.com/models/yolov8/) OpenVINO Inference example in C++! This guide will help you get started with leveraging the powerful YOLOv8 models using the [Intel OpenVINO™ toolkit](https://docs.openvino.ai/) and [OpenCV API](https://docs.opencv.org/) in your C++ projects. Whether you're looking to enhance performance on Intel hardware or add flexibility to your applications, this example provides a solid foundation. Learn more about optimizing models on the [Ultralytics blog](https://www.ultralytics.com/blog).
+
+## 🌟 Features
+
+- 🚀 **Model Format Support**: Compatible with [ONNX](https://onnx.ai/) and [OpenVINO Intermediate Representation (IR)](https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_IR_and_opsets.html) formats. Check the [Ultralytics ONNX integration](https://docs.ultralytics.com/integrations/onnx/) for more details.
+- ⚡ **Precision Options**: Run models in **FP32**, **FP16** ([half-precision](https://www.ultralytics.com/glossary/half-precision)), and **INT8** ([quantization](https://www.ultralytics.com/glossary/model-quantization)) precisions for optimized performance.
+- 🔄 **Dynamic Shape Loading**: Easily handle models with dynamic input shapes, common in many [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) tasks.
+
+## 📋 Dependencies
+
+To ensure smooth execution, please make sure you have the following dependencies installed:
+
+| Dependency                                            | Version  |
+| ----------------------------------------------------- | -------- |
+| [OpenVINO](https://docs.openvino.ai/latest/home.html) | >=2023.3 |
+| [OpenCV](https://opencv.org/)                         | >=4.5.0  |
+| [C++](https://en.cppreference.com/w/)                 | >=14     |
+| [CMake](https://cmake.org/documentation/)             | >=3.12.0 |
+
+## ⚙️ Build Instructions
+
+Follow these steps to build the project:
+
+1.  Clone the Ultralytics repository:
+
+    ```bash
+    git clone https://github.com/ultralytics/ultralytics.git
+    cd ultralytics/examples/YOLOv8-OpenVINO-CPP-Inference
+    ```
+
+2.  Create a build directory and compile the project using CMake:
+    ```bash
+    mkdir build
+    cd build
+    cmake ..
+    make
+    ```
+
+## 🛠️ Usage
+
+Once built, you can run [inference](https://www.ultralytics.com/glossary/real-time-inference) on an image using the compiled executable. Provide the path to your model file (either `.xml` for OpenVINO IR or `.onnx`) and the path to your image:
+
+```bash
+# Example using an OpenVINO IR model
+./detect path/to/your/model.xml path/to/your/image.jpg
+
+# Example using an ONNX model
+./detect path/to/your/model.onnx path/to/your/image.jpg
+```
+
+This command will process the image using the specified YOLOv8 model and display the [object detection](https://www.ultralytics.com/glossary/object-detection) results. Explore various [Ultralytics Solutions](https://docs.ultralytics.com/solutions/) for real-world applications.
+
+## 🔄 Exporting YOLOv8 Models
+
+To use your Ultralytics YOLOv8 model with this C++ example, you first need to export it to the OpenVINO IR or ONNX format. Use the `yolo export` command available in the Ultralytics Python package. Find detailed instructions in the [Export mode documentation](https://docs.ultralytics.com/modes/export/).
+
+```bash
+# Export to OpenVINO format (generates .xml and .bin files)
+yolo export model=yolov8s.pt imgsz=640 format=openvino
+
+# Export to ONNX format
+yolo export model=yolov8s.pt imgsz=640 format=onnx
+```
+
+For more details on exporting and optimizing models for OpenVINO, refer to the [Ultralytics OpenVINO integration guide](https://docs.ultralytics.com/integrations/openvino/).
+
+## 📸 Screenshots
+
+### Running Using OpenVINO Model
+
+![Running OpenVINO Model](https://github.com/ultralytics/ultralytics/assets/76827698/2d7cf201-3def-4357-824c-12446ccf85a9)
+
+### Running Using ONNX Model
+
+![Running ONNX Model](https://github.com/ultralytics/ultralytics/assets/76827698/9b90031c-cc81-4cfb-8b34-c619e09035a7)
+
+## ❤️ Contributions
+
+We hope this example helps you integrate YOLOv8 with OpenVINO and OpenCV into your C++ projects effortlessly. Contributions to improve this example or add new features are welcome! Please see the [Ultralytics contribution guidelines](https://docs.ultralytics.com/help/contributing/) for more information. Visit the main [Ultralytics documentation](https://docs.ultralytics.com/) for further guides and resources. Happy coding! 🚀
--- a/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/inference.cc
+++ b/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/inference.cc
@@ -0,0 +1,178 @@
+#include "inference.h"
+
+#include <memory>
+#include <opencv2/dnn.hpp>
+#include <random>
+
+namespace yolo {
+
+// Constructor to initialize the model with default input shape
+Inference::Inference(const std::string &model_path, const float &model_confidence_threshold, const float &model_NMS_threshold) {
+	model_input_shape_ = cv::Size(640, 640); // Set the default size for models with dynamic shapes to prevent errors.
+	model_confidence_threshold_ = model_confidence_threshold;
+	model_NMS_threshold_ = model_NMS_threshold;
+	InitializeModel(model_path);
+}
+
+// Constructor to initialize the model with specified input shape
+Inference::Inference(const std::string &model_path, const cv::Size model_input_shape, const float &model_confidence_threshold, const float &model_NMS_threshold) {
+	model_input_shape_ = model_input_shape;
+	model_confidence_threshold_ = model_confidence_threshold;
+	model_NMS_threshold_ = model_NMS_threshold;
+	InitializeModel(model_path);
+}
+
+void Inference::InitializeModel(const std::string &model_path) {
+	ov::Core core; // OpenVINO core object
+	std::shared_ptr<ov::Model> model = core.read_model(model_path); // Read the model from file
+
+	// If the model has dynamic shapes, reshape it to the specified input shape
+	if (model->is_dynamic()) {
+		model->reshape({1, 3, static_cast<long int>(model_input_shape_.height), static_cast<long int>(model_input_shape_.width)});
+	}
+
+	// Preprocessing setup for the model
+	ov::preprocess::PrePostProcessor ppp = ov::preprocess::PrePostProcessor(model);
+	ppp.input().tensor().set_element_type(ov::element::u8).set_layout("NHWC").set_color_format(ov::preprocess::ColorFormat::BGR);
+	ppp.input().preprocess().convert_element_type(ov::element::f32).convert_color(ov::preprocess::ColorFormat::RGB).scale({255, 255, 255});
+	ppp.input().model().set_layout("NCHW");
+	ppp.output().tensor().set_element_type(ov::element::f32);
+	model = ppp.build(); // Build the preprocessed model
+
+	// Compile the model for inference
+	compiled_model_ = core.compile_model(model, "AUTO");
+	inference_request_ = compiled_model_.create_infer_request(); // Create inference request
+
+	short width, height;
+
+	// Get input shape from the model
+	const std::vector<ov::Output<ov::Node>> inputs = model->inputs();
+	const ov::Shape input_shape = inputs[0].get_shape();
+	height = input_shape[1];
+	width = input_shape[2];
+	model_input_shape_ = cv::Size2f(width, height);
+
+	// Get output shape from the model
+	const std::vector<ov::Output<ov::Node>> outputs = model->outputs();
+	const ov::Shape output_shape = outputs[0].get_shape();
+	height = output_shape[1];
+	width = output_shape[2];
+	model_output_shape_ = cv::Size(width, height);
+}
+
+// Method to run inference on an input frame
+void Inference::RunInference(cv::Mat &frame) {
+	Preprocessing(frame); // Preprocess the input frame
+	inference_request_.infer(); // Run inference
+	PostProcessing(frame); // Postprocess the inference results
+}
+
+// Method to preprocess the input frame
+void Inference::Preprocessing(const cv::Mat &frame) {
+	cv::Mat resized_frame;
+	cv::resize(frame, resized_frame, model_input_shape_, 0, 0, cv::INTER_AREA); // Resize the frame to match the model input shape
+
+	// Calculate scaling factor
+	scale_factor_.x = static_cast<float>(frame.cols / model_input_shape_.width);
+	scale_factor_.y = static_cast<float>(frame.rows / model_input_shape_.height);
+
+    ov::Tensor input_tensor = inference_request_.get_input_tensor();
+    uint8_t* input_data = input_tensor.data<uint8_t>();
+    size_t bytes_to_copy = resized_frame.total() * resized_frame.elemSize();
+    memcpy(input_data, resized_frame.data, bytes_to_copy);
+
+	inference_request_.set_input_tensor(input_tensor); // Set input tensor for inference
+}
+
+// Method to postprocess the inference results
+void Inference::PostProcessing(cv::Mat &frame) {
+	std::vector<int> class_list;
+	std::vector<float> confidence_list;
+	std::vector<cv::Rect> box_list;
+
+	// Get the output tensor from the inference request
+	const float *detections = inference_request_.get_output_tensor().data<const float>();
+	const cv::Mat detection_outputs(model_output_shape_, CV_32F, (float *)detections); // Create OpenCV matrix from output tensor
+
+	// Iterate over detections and collect class IDs, confidence scores, and bounding boxes
+	for (int i = 0; i < detection_outputs.cols; ++i) {
+		const cv::Mat classes_scores = detection_outputs.col(i).rowRange(4, detection_outputs.rows);
+
+		cv::Point class_id;
+		double score;
+		cv::minMaxLoc(classes_scores, nullptr, &score, nullptr, &class_id); // Find the class with the highest score
+
+		// Check if the detection meets the confidence threshold
+		if (score > model_confidence_threshold_) {
+			class_list.push_back(class_id.y);
+			confidence_list.push_back(score);
+
+			const float cx = detection_outputs.at<float>(0, i);
+			const float cy = detection_outputs.at<float>(1, i);
+			const float w = detection_outputs.at<float>(2, i);
+			const float h = detection_outputs.at<float>(3, i);
+
+			cv::Rect box;
+			box.x = static_cast<int>((cx - w / 2));
+			box.y = static_cast<int>((cy - h / 2));
+			box.width = static_cast<int>(w);
+			box.height = static_cast<int>(h);
+			box_list.push_back(box);
+		}
+	}
+
+	// Apply Non-Maximum Suppression (NMS) to filter overlapping bounding boxes
+	std::vector<int> NMS_result;
+	cv::dnn::NMSBoxes(box_list, confidence_list, model_confidence_threshold_, model_NMS_threshold_, NMS_result);
+
+	// Collect final detections after NMS
+	for (int i = 0; i < NMS_result.size(); ++i) {
+		Detection result;
+		const unsigned short id = NMS_result[i];
+
+		result.class_id = class_list[id];
+		result.confidence = confidence_list[id];
+		result.box = GetBoundingBox(box_list[id]);
+
+		DrawDetectedObject(frame, result);
+	}
+}
+
+// Method to get the bounding box in the correct scale
+cv::Rect Inference::GetBoundingBox(const cv::Rect &src) const {
+	cv::Rect box = src;
+	box.x = static_cast<int>(box.x * scale_factor_.x);
+	box.y = static_cast<int>(box.y * scale_factor_.y);
+	box.width = static_cast<int>(box.width * scale_factor_.x);
+	box.height = static_cast<int>(box.height * scale_factor_.y);
+	return box;
+}
+
+void Inference::DrawDetectedObject(cv::Mat &frame, const Detection &detection) const {
+	const cv::Rect &box = detection.box;
+	const float &confidence = detection.confidence;
+	const int &class_id = detection.class_id;
+
+	// Generate a random color for the bounding box
+	std::random_device rd;
+	std::mt19937 gen(rd());
+	std::uniform_int_distribution<int> dis(120, 255);
+	const cv::Scalar &color = cv::Scalar(dis(gen), dis(gen), dis(gen));
+
+	// Draw the bounding box around the detected object
+	cv::rectangle(frame, cv::Point(box.x, box.y), cv::Point(box.x + box.width, box.y + box.height), color, 3);
+
+	// Prepare the class label and confidence text
+	std::string classString = classes_[class_id] + std::to_string(confidence).substr(0, 4);
+
+	// Get the size of the text box
+	cv::Size textSize = cv::getTextSize(classString, cv::FONT_HERSHEY_DUPLEX, 0.75, 2, 0);
+	cv::Rect textBox(box.x, box.y - 40, textSize.width + 10, textSize.height + 20);
+
+	// Draw the text box
+	cv::rectangle(frame, textBox, color, cv::FILLED);
+
+	// Put the class label and confidence text above the bounding box
+	cv::putText(frame, classString, cv::Point(box.x + 5, box.y - 10), cv::FONT_HERSHEY_DUPLEX, 0.75, cv::Scalar(0, 0, 0), 2, 0);
+}
+} // namespace yolo
--- a/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/inference.h
+++ b/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/inference.h
@@ -0,0 +1,61 @@
+// Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license
+
+#ifndef YOLO_INFERENCE_H_
+#define YOLO_INFERENCE_H_
+
+#include <string>
+#include <vector>
+#include <opencv2/imgproc.hpp>
+#include <openvino/openvino.hpp>
+
+namespace yolo {
+
+struct Detection {
+	short class_id;
+	float confidence;
+	cv::Rect box;
+};
+
+class Inference {
+ public:
+	Inference() {}
+	// Constructor to initialize the model with default input shape
+	Inference(const std::string &model_path, const float &model_confidence_threshold, const float &model_NMS_threshold);
+	// Constructor to initialize the model with specified input shape
+	Inference(const std::string &model_path, const cv::Size model_input_shape, const float &model_confidence_threshold, const float &model_NMS_threshold);
+
+	void RunInference(cv::Mat &frame);
+
+ private:
+	void InitializeModel(const std::string &model_path);
+	void Preprocessing(const cv::Mat &frame);
+	void PostProcessing(cv::Mat &frame);
+	cv::Rect GetBoundingBox(const cv::Rect &src) const;
+	void DrawDetectedObject(cv::Mat &frame, const Detection &detections) const;
+
+	cv::Point2f scale_factor_;			// Scaling factor for the input frame
+	cv::Size2f model_input_shape_;	// Input shape of the model
+	cv::Size model_output_shape_;		// Output shape of the model
+
+	ov::InferRequest inference_request_;  // OpenVINO inference request
+	ov::CompiledModel compiled_model_;    // OpenVINO compiled model
+
+	float model_confidence_threshold_;  // Confidence threshold for detections
+	float model_NMS_threshold_;         // Non-Maximum Suppression threshold
+
+	std::vector<std::string> classes_ {
+		"person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light",
+		"fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow",
+		"elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee",
+		"skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard",
+		"tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple",
+		"sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch",
+		"potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard",
+		"cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase",
+		"scissors", "teddy bear", "hair drier", "toothbrush"
+	};
+};
+
+} // namespace yolo
+
+#endif // YOLO_INFERENCE_H_
--- a/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/main.cc
+++ b/algorithms/dms_yolo/code/examples/YOLOv8-OpenVINO-CPP-Inference/main.cc
@@ -0,0 +1,42 @@
+#include "inference.h"
+
+#include <iostream>
+#include <opencv2/highgui.hpp>
+
+int main(int argc, char **argv) {
+	// Check if the correct number of arguments is provided
+	if (argc != 3) {
+		std::cerr << "usage: " << argv[0] << " <model_path> <image_path>" << std::endl;
+		return 1;
+	}
+
+	// Get the model and image paths from the command-line arguments
+	const std::string model_path = argv[1];
+	const std::string image_path = argv[2];
+
+	// Read the input image
+	cv::Mat image = cv::imread(image_path);
+
+	// Check if the image was successfully loaded
+	if (image.empty()) {
+		std::cerr << "ERROR: image is empty" << std::endl;
+		return 1;
+	}
+
+	// Define the confidence and NMS thresholds
+	const float confidence_threshold = 0.5;
+	const float NMS_threshold = 0.5;
+
+	// Initialize the YOLO inference with the specified model and parameters
+	yolo::Inference inference(model_path, cv::Size(640, 640), confidence_threshold, NMS_threshold);
+
+	// Run inference on the input image
+	inference.RunInference(image);
+
+	// Display the image with the detections
+	cv::imshow("image", image);
+	cv::waitKey(0);
+	cv::destroyAllWindows();
+
+	return 0;
+}