feat: HSAP platform v2 — modular navigation, quality review, audit log, world model simulation
Major changes: - New frontend (platform/web/): Vite + React 18 + TypeScript + Tailwind - 4-module navigation: 数据送标 / 模型管理 / 车队管理 / 系统管理 - Data catalog with charts (DMS/ADAS/Lane 3-tab view) - Quality review workflow (标注质检): Good/Fine/Bad scoring with auto-advance - Audit enhancements: batch operations, rejection categories, Feishu notifications - Operation audit log (操作日志) - World model simulation studio (仿真工坊) - Dataset version management with snapshots and diff - ADAS 7-class dataset integration (138K images organized + compressed) - User management with Feishu integration and pagination - CRUD/search/filter on all pages, card layout redesign - PIL-optimized image overlay rendering - Auto-snapshot on build, in_review workflow stage - Removed embedded algorithm code (now in workspace)
This commit is contained in:
@@ -0,0 +1,28 @@
|
||||
cmake_minimum_required(VERSION 3.5)
|
||||
|
||||
project(Yolov8CPPInference VERSION 0.1)
|
||||
|
||||
set(CMAKE_INCLUDE_CURRENT_DIR ON)
|
||||
|
||||
# CUDA
|
||||
set(CUDA_TOOLKIT_ROOT_DIR "/usr/local/cuda")
|
||||
find_package(CUDA 11 REQUIRED)
|
||||
|
||||
set(CMAKE_CUDA_STANDARD 11)
|
||||
set(CMAKE_CUDA_STANDARD_REQUIRED ON)
|
||||
# !CUDA
|
||||
|
||||
# OpenCV
|
||||
find_package(OpenCV REQUIRED)
|
||||
include_directories(${OpenCV_INCLUDE_DIRS})
|
||||
# !OpenCV
|
||||
|
||||
set(PROJECT_SOURCES
|
||||
main.cpp
|
||||
|
||||
inference.h
|
||||
inference.cpp
|
||||
)
|
||||
|
||||
add_executable(Yolov8CPPInference ${PROJECT_SOURCES})
|
||||
target_link_libraries(Yolov8CPPInference ${OpenCV_LIBS})
|
||||
@@ -0,0 +1,85 @@
|
||||
# YOLOv8/YOLOv5 C++ Inference with OpenCV DNN
|
||||
|
||||
This example demonstrates how to perform inference using Ultralytics YOLOv8 and YOLOv5 models in C++ leveraging the [OpenCV DNN module](https://docs.opencv.org/4.x/d6/d0f/group__dnn.html).
|
||||
|
||||
## 🛠️ Usage
|
||||
|
||||
Follow these steps to set up and run the C++ inference example:
|
||||
|
||||
```bash
|
||||
# 1. Clone the Ultralytics repository
|
||||
git clone https://github.com/ultralytics/ultralytics
|
||||
cd ultralytics
|
||||
|
||||
# 2. Install Ultralytics Python package (needed for exporting models)
|
||||
pip install .
|
||||
|
||||
# 3. Navigate to the C++ example directory
|
||||
cd examples/YOLOv8-CPP-Inference
|
||||
|
||||
# 4. Export Models: Add yolov8*.onnx and/or yolov5*.onnx models (see export instructions below)
|
||||
# Place the exported ONNX models in the current directory (YOLOv8-CPP-Inference).
|
||||
|
||||
# 5. Update Source Code: Edit main.cpp and set the 'projectBasePath' variable
|
||||
# to the absolute path of the 'YOLOv8-CPP-Inference' directory on your system.
|
||||
# Example: std::string projectBasePath = "/path/to/your/ultralytics/examples/YOLOv8-CPP-Inference";
|
||||
|
||||
# 6. Configure OpenCV DNN Backend (Optional - CUDA):
|
||||
# - The default CMakeLists.txt attempts to use CUDA for GPU acceleration with OpenCV DNN.
|
||||
# - If your OpenCV build doesn't support CUDA/cuDNN, or you want CPU inference,
|
||||
# remove the CUDA-related lines from CMakeLists.txt.
|
||||
|
||||
# 7. Build the project
|
||||
mkdir build
|
||||
cd build
|
||||
cmake ..
|
||||
make
|
||||
|
||||
# 8. Run the inference executable
|
||||
./Yolov8CPPInference
|
||||
```
|
||||
|
||||
## ✨ Exporting YOLOv8 and YOLOv5 Models
|
||||
|
||||
You need to export your trained PyTorch models to the [ONNX](https://onnx.ai/) format to use them with OpenCV DNN.
|
||||
|
||||
**Exporting Ultralytics YOLOv8 Models:**
|
||||
|
||||
Use the Ultralytics CLI to export. Ensure you specify the desired `imgsz` and `opset`. For compatibility with this example, `opset=12` is recommended.
|
||||
|
||||
```bash
|
||||
yolo export model=yolov8s.pt imgsz=640,480 format=onnx opset=12 # Example: 640x480 resolution
|
||||
```
|
||||
|
||||
**Exporting YOLOv5 Models:**
|
||||
|
||||
Use the `export.py` script from the YOLOv5 repository structure (included within the cloned `ultralytics` repo).
|
||||
|
||||
```bash
|
||||
# Assuming you are in the 'ultralytics' base directory after cloning
|
||||
python export.py --weights yolov5s.pt --imgsz 640 480 --include onnx --opset 12 # Example: 640x480 resolution
|
||||
```
|
||||
|
||||
Place the generated `.onnx` files (e.g., `yolov8s.onnx`, `yolov5s.onnx`) into the `ultralytics/examples/YOLOv8-CPP-Inference/` directory.
|
||||
|
||||
**Example Output:**
|
||||
|
||||
_yolov8s.onnx:_
|
||||
|
||||
|
||||
|
||||
_yolov5s.onnx:_
|
||||
|
||||
|
||||
|
||||
## 📝 Notes
|
||||
|
||||
- This repository utilizes the [OpenCV DNN API](https://docs.opencv.org/4.x/d6/d0f/group__dnn.html) to run [ONNX](https://onnx.ai/) exported models of YOLOv5 and Ultralytics YOLOv8.
|
||||
- While not explicitly tested, it might theoretically work for other YOLO architectures like YOLOv6 and YOLOv7 if their ONNX export formats are compatible.
|
||||
- The example models are exported with a rectangular resolution (640x480), but the code should handle models exported with different resolutions. Consider using techniques like [letterboxing](https://docs.ultralytics.com/modes/predict/#letterbox) if your input images have different aspect ratios than the model's training resolution, especially for square `imgsz` exports.
|
||||
- The `main` branch version includes a simple GUI wrapper using [Qt](https://www.qt.io/). However, the core logic resides in the `Inference` class (`inference.h`, `inference.cpp`).
|
||||
- A key part of the `Inference` class demonstrates how to handle the output differences between YOLOv5 and YOLOv8 models, effectively transposing YOLOv8's output format to match the structure expected from YOLOv5 for consistent post-processing.
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
Contributions are welcome! If you find any issues or have suggestions for improvement, please feel free to open an issue or submit a pull request. See our [Contributing Guide](https://docs.ultralytics.com/help/contributing/) for more details.
|
||||
@@ -0,0 +1,197 @@
|
||||
// Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license
|
||||
|
||||
#include "inference.h"
|
||||
|
||||
Inference::Inference(const std::string &onnxModelPath, const cv::Size &modelInputShape, const std::string &classesTxtFile, const bool &runWithCuda)
|
||||
{
|
||||
modelPath = onnxModelPath;
|
||||
modelShape = modelInputShape;
|
||||
classesPath = classesTxtFile;
|
||||
cudaEnabled = runWithCuda;
|
||||
|
||||
loadOnnxNetwork();
|
||||
// loadClassesFromFile(); The classes are hard-coded for this example
|
||||
}
|
||||
|
||||
std::vector<Detection> Inference::runInference(const cv::Mat &input)
|
||||
{
|
||||
cv::Mat modelInput = input;
|
||||
int pad_x, pad_y;
|
||||
float scale;
|
||||
if (letterBoxForSquare && modelShape.width == modelShape.height)
|
||||
modelInput = formatToSquare(modelInput, &pad_x, &pad_y, &scale);
|
||||
|
||||
cv::Mat blob;
|
||||
cv::dnn::blobFromImage(modelInput, blob, 1.0/255.0, modelShape, cv::Scalar(), true, false);
|
||||
net.setInput(blob);
|
||||
|
||||
std::vector<cv::Mat> outputs;
|
||||
net.forward(outputs, net.getUnconnectedOutLayersNames());
|
||||
|
||||
int rows = outputs[0].size[1];
|
||||
int dimensions = outputs[0].size[2];
|
||||
|
||||
bool yolov8 = false;
|
||||
// yolov5 has an output of shape (batchSize, 25200, 85) (Num classes + box[x,y,w,h] + confidence[c])
|
||||
// yolov8 has an output of shape (batchSize, 84, 8400) (Num classes + box[x,y,w,h])
|
||||
if (dimensions > rows) // Check if the shape[2] is more than shape[1] (yolov8)
|
||||
{
|
||||
yolov8 = true;
|
||||
rows = outputs[0].size[2];
|
||||
dimensions = outputs[0].size[1];
|
||||
|
||||
outputs[0] = outputs[0].reshape(1, dimensions);
|
||||
cv::transpose(outputs[0], outputs[0]);
|
||||
}
|
||||
float *data = (float *)outputs[0].data;
|
||||
|
||||
std::vector<int> class_ids;
|
||||
std::vector<float> confidences;
|
||||
std::vector<cv::Rect> boxes;
|
||||
|
||||
for (int i = 0; i < rows; ++i)
|
||||
{
|
||||
if (yolov8)
|
||||
{
|
||||
float *classes_scores = data+4;
|
||||
|
||||
cv::Mat scores(1, classes.size(), CV_32FC1, classes_scores);
|
||||
cv::Point class_id;
|
||||
double maxClassScore;
|
||||
|
||||
minMaxLoc(scores, 0, &maxClassScore, 0, &class_id);
|
||||
|
||||
if (maxClassScore > modelScoreThreshold)
|
||||
{
|
||||
confidences.push_back(maxClassScore);
|
||||
class_ids.push_back(class_id.x);
|
||||
|
||||
float x = data[0];
|
||||
float y = data[1];
|
||||
float w = data[2];
|
||||
float h = data[3];
|
||||
|
||||
int left = int((x - 0.5 * w - pad_x) / scale);
|
||||
int top = int((y - 0.5 * h - pad_y) / scale);
|
||||
|
||||
int width = int(w / scale);
|
||||
int height = int(h / scale);
|
||||
|
||||
boxes.push_back(cv::Rect(left, top, width, height));
|
||||
}
|
||||
}
|
||||
else // yolov5
|
||||
{
|
||||
float confidence = data[4];
|
||||
|
||||
if (confidence >= modelConfidenceThreshold)
|
||||
{
|
||||
float *classes_scores = data+5;
|
||||
|
||||
cv::Mat scores(1, classes.size(), CV_32FC1, classes_scores);
|
||||
cv::Point class_id;
|
||||
double max_class_score;
|
||||
|
||||
minMaxLoc(scores, 0, &max_class_score, 0, &class_id);
|
||||
|
||||
if (max_class_score > modelScoreThreshold)
|
||||
{
|
||||
confidences.push_back(confidence);
|
||||
class_ids.push_back(class_id.x);
|
||||
|
||||
float x = data[0];
|
||||
float y = data[1];
|
||||
float w = data[2];
|
||||
float h = data[3];
|
||||
|
||||
int left = int((x - 0.5 * w - pad_x) / scale);
|
||||
int top = int((y - 0.5 * h - pad_y) / scale);
|
||||
|
||||
int width = int(w / scale);
|
||||
int height = int(h / scale);
|
||||
|
||||
boxes.push_back(cv::Rect(left, top, width, height));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
data += dimensions;
|
||||
}
|
||||
|
||||
std::vector<int> nms_result;
|
||||
cv::dnn::NMSBoxes(boxes, confidences, modelScoreThreshold, modelNMSThreshold, nms_result);
|
||||
|
||||
std::vector<Detection> detections{};
|
||||
for (unsigned long i = 0; i < nms_result.size(); ++i)
|
||||
{
|
||||
int idx = nms_result[i];
|
||||
|
||||
Detection result;
|
||||
result.class_id = class_ids[idx];
|
||||
result.confidence = confidences[idx];
|
||||
|
||||
std::random_device rd;
|
||||
std::mt19937 gen(rd());
|
||||
std::uniform_int_distribution<int> dis(100, 255);
|
||||
result.color = cv::Scalar(dis(gen),
|
||||
dis(gen),
|
||||
dis(gen));
|
||||
|
||||
result.className = classes[result.class_id];
|
||||
result.box = boxes[idx];
|
||||
|
||||
detections.push_back(result);
|
||||
}
|
||||
|
||||
return detections;
|
||||
}
|
||||
|
||||
void Inference::loadClassesFromFile()
|
||||
{
|
||||
std::ifstream inputFile(classesPath);
|
||||
if (inputFile.is_open())
|
||||
{
|
||||
std::string classLine;
|
||||
while (std::getline(inputFile, classLine))
|
||||
classes.push_back(classLine);
|
||||
inputFile.close();
|
||||
}
|
||||
}
|
||||
|
||||
void Inference::loadOnnxNetwork()
|
||||
{
|
||||
net = cv::dnn::readNetFromONNX(modelPath);
|
||||
if (cudaEnabled)
|
||||
{
|
||||
std::cout << "\nRunning on CUDA" << std::endl;
|
||||
net.setPreferableBackend(cv::dnn::DNN_BACKEND_CUDA);
|
||||
net.setPreferableTarget(cv::dnn::DNN_TARGET_CUDA);
|
||||
}
|
||||
else
|
||||
{
|
||||
std::cout << "\nRunning on CPU" << std::endl;
|
||||
net.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV);
|
||||
net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);
|
||||
}
|
||||
}
|
||||
|
||||
cv::Mat Inference::formatToSquare(const cv::Mat &source, int *pad_x, int *pad_y, float *scale)
|
||||
{
|
||||
int col = source.cols;
|
||||
int row = source.rows;
|
||||
int m_inputWidth = modelShape.width;
|
||||
int m_inputHeight = modelShape.height;
|
||||
|
||||
*scale = std::min(m_inputWidth / (float)col, m_inputHeight / (float)row);
|
||||
int resized_w = col * *scale;
|
||||
int resized_h = row * *scale;
|
||||
*pad_x = (m_inputWidth - resized_w) / 2;
|
||||
*pad_y = (m_inputHeight - resized_h) / 2;
|
||||
|
||||
cv::Mat resized;
|
||||
cv::resize(source, resized, cv::Size(resized_w, resized_h));
|
||||
cv::Mat result = cv::Mat::zeros(m_inputHeight, m_inputWidth, source.type());
|
||||
resized.copyTo(result(cv::Rect(*pad_x, *pad_y, resized_w, resized_h)));
|
||||
resized.release();
|
||||
return result;
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
// Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license
|
||||
|
||||
#ifndef INFERENCE_H
|
||||
#define INFERENCE_H
|
||||
|
||||
// Cpp native
|
||||
#include <fstream>
|
||||
#include <vector>
|
||||
#include <string>
|
||||
#include <random>
|
||||
|
||||
// OpenCV / DNN / Inference
|
||||
#include <opencv2/imgproc.hpp>
|
||||
#include <opencv2/opencv.hpp>
|
||||
#include <opencv2/dnn.hpp>
|
||||
|
||||
struct Detection
|
||||
{
|
||||
int class_id{0};
|
||||
std::string className{};
|
||||
float confidence{0.0};
|
||||
cv::Scalar color{};
|
||||
cv::Rect box{};
|
||||
};
|
||||
|
||||
class Inference
|
||||
{
|
||||
public:
|
||||
Inference(const std::string &onnxModelPath, const cv::Size &modelInputShape = {640, 640}, const std::string &classesTxtFile = "", const bool &runWithCuda = true);
|
||||
std::vector<Detection> runInference(const cv::Mat &input);
|
||||
|
||||
private:
|
||||
void loadClassesFromFile();
|
||||
void loadOnnxNetwork();
|
||||
cv::Mat formatToSquare(const cv::Mat &source, int *pad_x, int *pad_y, float *scale);
|
||||
|
||||
std::string modelPath{};
|
||||
std::string classesPath{};
|
||||
bool cudaEnabled{};
|
||||
|
||||
std::vector<std::string> classes{"person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"};
|
||||
|
||||
cv::Size2f modelShape{};
|
||||
|
||||
float modelConfidenceThreshold {0.25};
|
||||
float modelScoreThreshold {0.45};
|
||||
float modelNMSThreshold {0.50};
|
||||
|
||||
bool letterBoxForSquare = true;
|
||||
|
||||
cv::dnn::Net net;
|
||||
};
|
||||
|
||||
#endif // INFERENCE_H
|
||||
@@ -0,0 +1,72 @@
|
||||
// Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license
|
||||
|
||||
#include <iostream>
|
||||
#include <vector>
|
||||
#include <getopt.h>
|
||||
|
||||
#include <opencv2/opencv.hpp>
|
||||
|
||||
#include "inference.h"
|
||||
|
||||
using namespace std;
|
||||
using namespace cv;
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
std::string projectBasePath = "/home/user/ultralytics"; // Set your ultralytics base path
|
||||
|
||||
bool runOnGPU = true;
|
||||
|
||||
//
|
||||
// Pass in either:
|
||||
//
|
||||
// "yolov8s.onnx" or "yolov5s.onnx"
|
||||
//
|
||||
// To run Inference with yolov8/yolov5 (ONNX)
|
||||
//
|
||||
|
||||
// Note that in this example the classes are hard-coded and 'classes.txt' is a place holder.
|
||||
Inference inf(projectBasePath + "/yolov8s.onnx", cv::Size(640, 640), "classes.txt", runOnGPU);
|
||||
|
||||
std::vector<std::string> imageNames;
|
||||
imageNames.push_back(projectBasePath + "/ultralytics/assets/bus.jpg");
|
||||
imageNames.push_back(projectBasePath + "/ultralytics/assets/zidane.jpg");
|
||||
|
||||
for (int i = 0; i < imageNames.size(); ++i)
|
||||
{
|
||||
cv::Mat frame = cv::imread(imageNames[i]);
|
||||
|
||||
// Inference starts here...
|
||||
std::vector<Detection> output = inf.runInference(frame);
|
||||
|
||||
int detections = output.size();
|
||||
std::cout << "Number of detections:" << detections << std::endl;
|
||||
|
||||
for (int i = 0; i < detections; ++i)
|
||||
{
|
||||
Detection detection = output[i];
|
||||
|
||||
cv::Rect box = detection.box;
|
||||
cv::Scalar color = detection.color;
|
||||
|
||||
// Detection box
|
||||
cv::rectangle(frame, box, color, 2);
|
||||
|
||||
// Detection box text
|
||||
std::string classString = detection.className + ' ' + std::to_string(detection.confidence).substr(0, 4);
|
||||
cv::Size textSize = cv::getTextSize(classString, cv::FONT_HERSHEY_DUPLEX, 1, 2, 0);
|
||||
cv::Rect textBox(box.x, box.y - 40, textSize.width + 10, textSize.height + 20);
|
||||
|
||||
cv::rectangle(frame, textBox, color, cv::FILLED);
|
||||
cv::putText(frame, classString, cv::Point(box.x + 5, box.y - 10), cv::FONT_HERSHEY_DUPLEX, 1, cv::Scalar(0, 0, 0), 2, 0);
|
||||
}
|
||||
// Inference ends here...
|
||||
|
||||
// This is only for preview purposes
|
||||
float scale = 0.8;
|
||||
cv::resize(frame, frame, cv::Size(frame.cols*scale, frame.rows*scale));
|
||||
cv::imshow("Inference", frame);
|
||||
|
||||
cv::waitKey(-1);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user