ADAS/HSAP
2
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: initial HSAP platform

Browse Source 
Huaxu Sentinel Active Safety Platform with embedded algorithm code,
Docker Compose setup, and vendored dataset scaffolds for clone-and-run.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Chengfang Lu
2026-05-25 16:59:59 +08:00
commit 7c43b44c57
1619 changed files with 373355 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

93
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/ADVANCED_TUTORIAL.md Normal file
View File

@@ -0,0 +1,93 @@
# Advanced Tutorial
Here we give same advanced use case and coding guide from easy to hard.
## Understand and Write config files
A config file in PytorchAutoDrive (look at [this example](../configs/semantic_segmentation/erfnet/cityscapes_512x1024.py) while reading) is defined by 9 mandatory dicts including:
**Data pipeline:**
1. dataset: the Dataset class
2. train_augmentation: transforms for training
3. test_augmentation: transforms for testing
**Optimization pipeline:**
1. loss: the loss function (e.g. CrossEntropy)
2. optimizer: the optimizer (e.g. SGD)
3. lr_scheduler: the learning rate scheduler (e.g. step lr)
**Model specific options:**
1. train: options for training (e.g. epochs, batch size, DDP options)
2. test: options for evaluation, profiling and visualization (e.g. checkpoint path, image size)
3. model: options to define your model
Also there are 3 optional dicts (beta):
1. vis: used to replace `test` if specified
2. test_dataset: used to replace `dataset` in testing mode if specified
3. vis_dataset: used to replace `dataset` in visualization if specified
Other than `train` and `test`, each dict defines the `__init__` function of a Class, or the input args for a function. Some args are dynamic (e.g. there is a arg for the network instance in optimizers) and will be replaced on the run, which means you don't have to set them in configs. To write a config file is exactly the same as writing Python dicts, you can pitch in maths and imports as well. Do remember to import from `configs.xxx` and most of the first 6 dicts can be directly imported from `configs.xxx.common`. For more details on PytorchAutoDrive's config mechanism, refer to [configs/README.md](../configs/README.md).
## Hyper-parameter tuning and Shortcuts
*Configs are all good and clear, but what if I want to quickly try a set of hyper-parameters for my model? Do I have to write a bunch of config files?*
**The answer is No, you don't have to.**
PytorchAutoDrive provides `--cfg-options` for you to conveniently replace most of the config options in commandline, for instance to replace learning rate to `0.1` and batch size to `32` for ERFNet, you can simply do this:
```
python main_semseg.py --train --config=configs/semantic_segmentation/erfnet/cityscapes_512x1024.py --cfg-options="optimizer.lr=0.1 train.batch_size=32"
```
In practice, you can turn a hyper-parameter search to a shell `search.sh` like this:
```
#!/bin/bash
for lr in 1 2 3; do
exp_name=erfnet_hyperparameters_${lr}
python main_semseg.py --train --config=configs/semantic_segmentation/erfnet/cityscapes_512x1024.py --cfg-options="optimizer.lr=0.${lr} train.exp_name=${exp_name}"
python main_semseg.py --val --config=configs/semantic_segmentation/erfnet/cityscapes_512x1024.py --cfg-options="test.exp_name=${exp_name} test.checkpoint=checkpoints/${exp_name}/model.pt"
done
```
Then you'll see results stored at `checkpoints/` in separate directories named by `exp_name` and tensorboard logs at `checkpoints/tb_logs/`.
The format for `--cfg-options` is `x1=y1 x2=y2`. Argument parsing is based on Python3 `eval()` and supports common `string`, `number`, `list` or `tuple` (even `dict` if you do it right). Don't forget the double quotation marks!
*--cfg-options is too difficult to write, can I get normal argparse args to use?*
**Yes you can.** You can define **shortcuts** in [statics](../configs/statics.py), we have some preset args in place already, for instance, the equivalent for `--cfg-options="optimizer.lr=0.1 train.batch_size=32"` is simply: `--lr=0.1 --batch-size=32`. Most of legacy args before the great refactoring are included here.
## The PytorchAutoDrive code trip
PytorchAutoDrive is built by registration, and executed by Runners.
**Algorithms:**
In `utils/` you can find directories containing registered classes or functions for models, datasets, optimizers, etc., and files for utility use. Most algorithm-related codes you need to read can be found here. For instance, you have a dict in config file as:
```
dict(
name='ERFNet',
...
)
```
You can search for the Class/function named `ERFNet` here for its implementation.
**Executions:**
`utils/runners/` implements Runners. A Runner parses config file and constructs the execution process for training/testing/visualization. It defines the behavior of the entire logical process, do read them carefully (best start from `base.py`) if you want to get some deeper customizations from coding.
## Examples
1. [Code a dataset to visualize lane lines and compare with dataset GT](./advanced/VISUALIZE_LANE_DATASETS.md)
2. Code a model: Checkout `utils/models/`, more details is coming...
3. Code a dataset: Checkout `utils/datasets/`, more details is coming...
4. Code a runner for advanced visualization: Checkout `utils/runners/*visualizer.py`, more details is coming...

120
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/BENCHMARK.md Normal file
View File

@@ -0,0 +1,120 @@
# Welcome to PytorchAutoDrive benchmark
*The current benchmark's FLOPs & Param count is entirely based on [thop](https://github.com/Lyken17/pytorch-OpCounter) to identify underlying basic ops, which might be inaccurate. But FLOPs count is an [estimate](https://discuss.pytorch.org/t/correct-way-to-calculate-flops-in-model/67198/6) to begin with. What we are doing here, is simply providing a relatively fair benchmark for comparing different methods.*
## Lane detection performance
| method | backbone | resolution | FPS | FLOPS(G) | Params(M) |
| :---: | :---: | :---: | :---: | :---: | :---: |
| Baseline | VGG16 | 360 x 640 | 56.36 | 214.50 | 20.37 |
| Baseline | ResNet18 | 360 x 640 | 148.59 | 85.24 | 12.04 |
| Baseline | ResNet34 | 360 x 640 | 79.97 | 159.60 | 22.15 |
| Baseline | ResNet50 | 360 x 640 | 50.58 | 177.62 | 24.57 |
| Baseline | ResNet101 | 360 x 640 | 27.41 | 314.36 | 43.56 |
| Baseline | ERFNet | 360 x 640 | 85.87 | 26.32 | 2.67 |
| Baseline | ENet | 360 x 640 | 56.63 | 4.26 | 0.95 |
| Baseline | MobileNetV2 | 360 x 640 | 126.54 | 4.49 | 2.06 |
| Baseline | MobileNetV3-Large | 360 x 640 | 104.34 | 3.63 | 3.30 |
| SCNN | VGG16 | 360 x 640 | 21.18 | 218.64 | 20.96 |
| SCNN | ResNet18 | 360 x 640 | 21.12 | 89.38 | 12.63 |
| SCNN | ResNet34 | 360 x 640 | 20.77 | 163.74 | 22.74 |
| SCNN | ResNet50 | 360 x 640 | 19.59 | 181.76 | 25.16 |
| SCNN | ResNet101 | 360 x 640 | 13.50 | 318.50 | 44.15 |
| SCNN | ERFNet | 360 x 640 | 18.40 | 30.46 | 3.26 |
| LSTR | ResNet18s | 360 x 640 | 98.13 | 1.15 | 0.77 |
| LSTR | ResNet18s-2x | 360 x 640 | 97.27 | 4.05 | 3.05 |
| LSTR | ResNet18s | 1080 x 1920 | 91.23 | 10.20 | 0.77 |
| LSTR | ResNet18s | 2160 x 4320 | 23.60 | 40.75 | 0.77 |
| LSTR | ResNet34 | 360 x 640 | 63.52 | 34.54 | 22.34 |
| RESA | ResNet18 | 360 x 640 | 67.66 | 61.35 | 6.61 |
| RESA | ResNet34 | 360 x 640 | 54.49 | 101.74 | 11.99 |
| RESA | ResNet50 | 360 x 640 | 44.80 | 105.71 | 12.46 |
| RESA | ResNet101 | 360 x 640 | 25.14 | 242.45 | 31.46 |
| RESA | MobileNetV2 | 360 x 640 | 60.53 | 12.80 | 4.63 |
| RESA | MobileNetV3-Large | 360 x 640 | 54.39 | 11.95 | 5.88 |
| LaneATT | ResNet18 | 360 x 640 | 198.29 | 18.67 | 12.02 |
| LaneATT | ResNet34 | 360 x 640 | 133.84 | 36.01 | 22.12 |
| BézierLaneNet | ResNet18 | 360 x 640 | 212.83 | 14.77 | 4.10 |
| BézierLaneNet | ResNet34 | 360 x 640 | 149.52 | 29.85 | 9.49 |
| Baseline | VGG16 | 288 x 800 | 55.31 | 214.50 | 20.15 |
| Baseline | ResNet18 | 288 x 800 | 136.28 | 85.22 | 11.82 |
| Baseline | ResNet34 | 288 x 800 | 72.42 | 159.60 | 21.93 |
| Baseline | ResNet50 | 288 x 800 | 49.41 | 177.60 | 24.35 |
| Baseline | ResNet101 | 288 x 800 | 27.19 | 314.34 | 43.34 |
| Baseline | ERFNet | 288 x 800 | 88.76 | 26.26 | 2.68 |
| Baseline | ENet | 288 x 800 | 57.99 | 4.12 | 0.96 |
| Baseline | MobileNetV2 | 288 x 800 | 129.24 | 4.41 | 2.00 |
| Baseline | MobileNetV3-Large | 288 x 800 | 107.83 | 3.56 | 3.25 |
| Baseline | RepVGG-A0 | 288 x 800 | 162.61 | 207.81 | 9.06 |
| Baseline | RepVGG-A1 | 288 x 800 | 117.30 | 339.83 | 13.54 |
| Baseline | RepVGG-B0 | 288 x 800 | 103.68 | 390.83 | 15.09 |
| Baseline | RepVGG-B1g2 | 288 x 800 | 36.91 | 1166.76 | 42.20 |
| Baseline | RepVGG-B2 | 288 x 800 | 18.98 | 2310.13 | 81.23 |
| Baseline | Swin-Tiny | 288 x 800 | 51.90 | 44.24 | 27.72 |
| SCNN | VGG16 | 288 x 800 | 21.40 | 218.62 | 20.74 |
| SCNN | ResNet18 | 288 x 800 | 20.80 | 89.34 | 12.42 |
| SCNN | ResNet34 | 288 x 800 | 19.77 | 163.72 | 22.52 |
| SCNN | ResNet50 | 288 x 800 | 18.88 | 181.72 | 24.94 |
| SCNN | ResNet101 | 288 x 800 | 13.42 | 318.46 | 43.94 |
| SCNN | ERFNet | 288 x 800 | 18.80 | 30.40 | 3.27 |
| SCNN | RepVGG-A1 | 288 x 800 | 20.53 | 343.96 | 14.13 |
| RESA | ResNet18 | 288 x 800 | 69.58 | 61.33 | 6.62 |
| RESA | ResNet34 | 288 x 800 | 55.61 | 101.72 | 12.01 |
| RESA | ResNet50 | 288 x 800 | 46.75 | 105.70 | 12.48 |
| RESA | ResNet101 | 288 x 800 | 26.08 | 242.44 | 31.47 |
| RESA | MobileNetV2 | 288 x 800 | 59.49 | 12.55 | 4.63 |
| RESA | MobileNetV3-Large | 288 x 800 | 53.85 | 11.70 | 5.88 |
| LSTR | ResNet34 | 288 x 800 | 65.39 | 33.86 | 22.34 |
| BézierLaneNet | ResNet18 | 288 x 800 | 210.79 | 14.66 | 4.10 |
| BézierLaneNet | ResNet34 | 288 x 800 | 144.65 | 29.54 | 9.49 |
## Segmentation performance:
| method | resolution | FPS | FLOPS(G) | Params(M) |
| :---: | :---: | :---: | :---: | :---: |
| FCN | 256 x 512 | 43.32 | 216.42 | 51.95 |
| FCN | 512 x 1024 | 12.06 | 865.69 | 51.95 |
| FCN | 1024 x 2048 | 3.06 | 3462.77 | 51.95 |
| ERFNet | 256 x 512 | 91.20 | 15.03 | 2.07 |
| ERFNet | 512 x 1024 | 85.51 | 60.11 | 2.07 |
| ERFNet | 1024 x 2048 | 21.53 | 240.44 | 2.07 |
| ENet | 256 x 512 | 59.31 | 2.72 | 0.35 |
| ENet | 512 x 1024 | 55.69 | 10.88 | 0.35 |
| ENet | 1024 x 2048 | 30.88 | 43.53 | 0.35 |
| DeeplabV2 | 256 x 512 | 44.87 | 180.59 | 43.90 |
| DeeplabV2 | 512 x 1024 | 12.93 | 722.37 | 43.90 |
| DeeplabV2 | 1024 x 2048 | 3.23 | 2889.49 | 43.90 |
| DeeplabV3 | 256 x 512 | 35.26 | 241.65 | 58.63 |
| DeeplabV3 | 512 x 1024 | 10.26 | 966.61 | 58.63 |
| DeeplabV3 | 1024 x 2048 | 2.56 | 3866.45| 58.63 |
*All results are the maximum value of 3 times on a RTX 2080Ti.*
*Lane detection post-processing are not counted.*
*LaneATT NMS is not counted yet.*
## Profiling Models Yourself
In the setting of `mode=simple`, we employ a random tensor to replace the real image.
Therefore, we can avoid using the DataLoader to obtain the best performance of models.
**This is also the setting for the above benchmark.**
```
python tools/profiling.py --mode=simple \
--config=<config file path> \
--times=3 \
--height=<image height in pixels> \
--width=<image width in pixels>
```
Same config mechanism and commandline overwrite by `--cfg-options` as in training/testing.
In the setting of `mode=real`, so as to simulate that the real camera transmit frames to models, we set 'batch_size=1' and 'num_workers=0' in the DataLoader. Just use `--mode=real` and probably provide an actual model by `--checkpoint`.
For detailed instructions and commandline shortcuts available, run:
```
python tools/profiling.py --help
```

39
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,39 @@
# Welcome to PytorchAutoDrive Contributor Guide
We welcome **Pull Requests** to fix bugs, update docs or implement new features etc. We also welcome **Issues** to report problems and needs, or ask questions (since your question might be more common and helpful to the community than you presume). The maintainers will respond to all PR or Issues within 24 hours.
## Where Should I Pick Up Works?
Search for issues with [help wanted](https://github.com/voldemortX/pytorch-auto-drive/issues?q=is%3Aissue+label%3A%22help+wanted%22+) or [good first issue](https://github.com/voldemortX/pytorch-auto-drive/issues?q=is%3Aissue+label%3A%22good+first+issue%22) marks, or improve the code/doc based on your user experience. Contact Zhengyang Feng (zyfeng97@outlook.com) for in-depth collaborations.
## Micro Plans
See our **RoadMap** #4 for quarterly iteration plans.
## Macro Plans
This repository implements (or plan to implement) the following interesting papers in a unified PyTorch framework:
[Fully Convolutional Networks for Semantic Segmentation](https://arxiv.org/abs/1605.06211) CVPR 2015
[DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://arxiv.org/abs/1606.00915) TPAMI 2017
[Rethinking Atrous Convolution for Semantic Image Segmentation](https://arxiv.org/abs/1706.05587) ArXiv preprint 2017
[ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://arxiv.org/abs/1606.02147) ArXiv preprint 2016
[ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation](https://ieeexplore.ieee.org/abstract/document/8063438/) ITS 2017
[Spatial As Deep: Spatial CNN for Traffic Scene Understanding](https://arxiv.org/abs/1712.06080) AAAI 2018
[RESA: Recurrent Feature-Shift Aggregator for Lane Detection](https://arxiv.org/abs/2008.13719) AAAI 2021
[Polynomial Regression Network for Variable-Number Lane Detection](http://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123630698.pdf) ECCV 2020
[End-to-end Lane Shape Prediction with Transformers](https://arxiv.org/abs/2011.04233) WACV 2021
[Keep Your Eyes on The Lane: Real-Time Attention-Guided Lane Detection](https://arxiv.org/abs/2010.12035) CVPR 2021
[Rethinking Efficient Lane Detection via Curve Modeling](https://arxiv.org/abs/2203.02431) CVPR 2022
You are also welcomed to make additions on this paper list, or open-source your related works here.

108
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/CURVE.md Normal file
View File

@@ -0,0 +1,108 @@
## Generate Bézier labels
Generation script:
```
python ./tools/curve_fitting_tools/gen_bezier_annotations.py
--dataset=<dataset name>
--image-set=<image set name: train\test\val>
--order=<the order of Bézier curves>
```
**Notes:**
1. We generate Bézier control points from original key points without normalization.
The normalized control points can be obtained by using `--norm` in the generated script.
2. The test set of LLAMAS dataset is unavailable, thus we cannot obtain the LLAMAS test set 's Bézier labels.
3. In CurveLanes dataset, some lanes were marked by sparse key points (for instance, 2 and 3 points) , therefore, before obtain Bézier labels we interpolate lanes.
## Bézier label format
All labels are saved in a json file, named `<image-set>_<order>.json`.
```
{"raw_file":filename1, "Bezier_control_points": [[...],[...], ..., [...]}
{"raw_file":filename2, "Bezier_control_points": [[...],[...], ..., [...]}
...
{"raw_file":filenamen, "Bezier_control_points": [[...],[...], ..., [...]}
```
## Upper-bound test script
This script is used to obtain prediction results from fitted curves.
```
python ./tools/curve_fitting_tools/upperbound.py
--dataset=<dataset name>
--state=<1: test set/2: val test>
--fit-function=<bezier/poly>
--num-points=<the number of generating key points>
--order=<the order of generating curves>
```
We still need to run `autotest_<culane\llamas\tusimple>.sh` to get F1/Accuracy.
LPD metric script:
```
python ./tools/curve_fitting_tools/lpd_mertic.py
--pred=<.json with the predictions>
--gt=<.json with the gt>
--gt-type='tusimple'
```
The `lpd_metric.py` is used to get lpd metric, which was employed in [PolyLaneNet](https://arxiv.org/abs/2004.10924).
We copy this test script from this [repo](https://github.com/lucastabelini/PolyLaneNet), you can find more information in this [issue](https://github.com/lucastabelini/PolyLaneNet/issues/50).
**Notes:**
1. The upper-bound test on TuSimple dataset does not require `--num-points`.
2. The lpd metric only supports TuSimple dataset.
3. Bézier curves are simply fitted with least-squares, which is not optimal.
## Upper-bounds on test set (except LLAMAS uses val)
**100 sample points for the CULane eval.**
### CULane F1
| Order | Bézier | polynomial |
| :---: | :---: | :---: |
| 1st | 99.6024 | 99.6177 |
| 2nd | 99.9733 | 99.9685 |
| 3rd | 99.9962 | 99.9971 |
| 4th | 99.9962 | 99.9990 |
| 5th | 99.9847 | 99.9990 |
### TuSimple Accuracy
| Order | Bézier | polynomial |
| :---: | :---: | :---: |
| 1st | 96.4738 | 97.9629 |
| 2nd | 98.4588 | 99.0760 |
| 3rd | 99.5239 | 99.7463 |
| 4th | 99.8120 | 99.9498 |
| 5th | 99.9106 | 99.9883 |
### LLAMAS F1
| Order | Bézier | polynomial |
| :---: | :---: | :---: |
| 1st | 98.8978 | 99.1409 |
| 2nd | 99.4408 | 99.5178 |
| 3rd | 99.7191 | 99.6259 |
| 4th | 99.7987 | 99.6961 |
| 5th | 99.8501 | 99.7501 |
### TuSimple LPD
| Order | Bézier | polynomial |
| :---: | :---: | :---: |
| 1st | 0.956382 | 1.415590 |
| 2nd | 0.652477 | 0.944469 |
| 3rd | 0.471154 | 0.557482 |
| 4th | 0.314884 | 0.329481 |
| 5th | 0.238662 | 0.208554 |
*LPD metric: lower is better.*

26
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/DATASET.md Normal file
View File

@@ -0,0 +1,26 @@
# Datasets
## Basic information of lane detection datasets
| Name | #Train | #Validation | #Test | Resolution (H x W) | #Lanes | |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| CULane | 88880 | 9675 | 34680 | 590 x 1640 | <=4 | [instruction](./datasets/CULANE.md)|
| TuSimple | 3268 | 358 | 2782 | 720 x 1280 | <=5 | [instruction](./datasets/TUSIMPLE.md) |
| LLAMAS* | 58269 | 20844 | 20929 | 717 x 1276 | <= 4 | [instruction](./datasets/LLAMAS.md) |
*\* The number of lanes in llamas dataset is more than 4, but most methods & evaluation metrics only use 4 lanes.*
## Basic information of semantic segmentation datasets
| Name | #Train | #Validation | #Test | Resolution (H x W) | #Classes | |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| PASCAL VOC 2012* | 10582 | 1449 | - | various | 21 | [instruction](./datasets/PASCALVOC.md) |
| Cityscapes | 2975 | 500 | - | 1024 x 2048 | 19 | [instruction](./datasets/CITYSCAPES.md) |
| GTAV | 24966 | - | - | mostly 1052 x 1914 | 19 | [instruction](./datasets/GTAV.md) |
| SYNTHIA** | 9400 | - | - | 760 x 1280 | 23 | [instruction](./datasets/SYNTHIA.md) |
*\* Extended by SBD.*
*\*\* SYNTHIA-RAND-CITYSCAPES.*
*- Not used or label not available.*

98
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/DEPLOY.md Normal file
View File

@@ -0,0 +1,98 @@
# Deployment Guide
- [x] PyTorch -> ONNX
- [x] ONNX -> TensorRT
- [ ] ONNX inference
- [ ] TensorRT inference
- [ ] ONNX visualization
- [ ] TensorRT visualization
## Installation
**A separate Python virtual environment is recommended here to avoid effects to your training & testing environment.**
Install all deployment packages (our tested conda version) by:
```
conda install cudatoolkit=10.2 -c pytorch
conda install cudnn==8.0.4 -c nvidia
pip install onnx==1.10.2 onnxruntime-gpu==1.6.0
python3 -m pip install --upgrade nvidia-tensorrt==8.2.1.8 // you may need to add --extra-index-url https://pypi.ngc.nvidia.com
```
In this version, TensorRT may use CUDA runtime >= 11, you might avoid using conda if you have sudo access on your device.
Or you can incrementally install **Extra Dependencies** through the tutorial.
## Important Note
Note that we only convert the model `forward()` function,
post-processing (i.e., `inference()`) is not included. Typical post-processing includes: segmentation map interpolation,
line NMS for anchor-based lane detection, sigmoid/softmax activations, etc.
## PyTorch -> ONNX:
**PyTorch version >= 1.6.0 is recommended for this feature.**
### Extra Dependencies:
```
pip install onnx==1.10.2 onnxruntime-gpu==<version>
```
`<version>` depends on your CUDA/CuDNN version, see [here](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), if you met version issues,
try install exact cudatoolkit and cudnn from conda. Or you can just install the CPU onnxruntime for this functionality.
### Conversion:
The conversion is based on Torch JIT's tracing on random input. To convert a checkpoint (*e.g.,* ckpt.pt) to ONNX, simply run this command:
```
python tools/to_onnx.py --config=<config file path> --height=<input height> --width=<input width> --checkpoint=ckpt.pt
```
You'll then see the saved `ckpt.onnx` file and a report on the conversion quality.
Same config mechanism and commandline overwrite by `--cfg-options` as in training/testing.
For detailed instructions and commandline shortcuts available, run:
```
python tools/to_onnx.py --help
```
### Currently Unsupported Models:
- ENet (segmentation)
- ENet backbone (lane detection)
- DCNv2 in BézierLaneNet (lane detection)
- Swin backbone (supported if pytorch >= 1.10.0)
- LaneATT (supported if pytorch >= 1.8.0)
## ONNX -> TensorRT:
### Extra Dependencies:
```
python3 -m pip install --upgrade nvidia-tensorrt==<version>
```
TensorRT `<version>` is recommended to be at least 7.2, you can also install it via other means than pip.
To work better with onnxruntime (for checking of conversion quality), you best checkout the [compatibility](https://onnxruntime.ai/docs/execution-providers/TensorRT-ExecutionProvider.html#requirements).
### Conversion:
The conversion is mainly a building of TensorRT engine. To convert a checkpoint (*e.g.,* ckpt.onnx) to TensorRT, simply run this command:
```
python tools/to_tensorrt.py --height=<input height> --width=<input width> --onnx-path=<ckpt.onnx>
```
You'll then see the saved `ckpt.engine` file and a report on the conversion quality.
### Currently Unsupported Models:
- ENet (segmentation)
- ENet backbone (lane detection)
- SCNN (lane detection)
- Swin backbone (lane detection)
- DCNv2 in BézierLaneNet (lane detection, could support if built custom op from mmcv and directly convert from PyTorch to TensorRT)
- LaneATT (supported if TensorRT >= 8.4.1.5)

21
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/IMAGENET_MODELS.md Normal file
View File

@@ -0,0 +1,21 @@
# ImageNet Pre-trained weights download
**Notification:** We use only the ImageNet-1K pre-trained weights, from TorchVision whenever possible (for the update on pre-trained weights in TorchVision, we always prefer the v0.7.0 weights). Most weights will be automatically downloaded, except:
*Need to provide download url in config.
**Need to provide pt/pth file path in config.
**All Weights:**
| Model | Filename | Link |
| :---: | :---: | :---: |
| ERFNet's Encoder** | erfnet_encoder_pretrained.pth.tar | [Official Repo Weight](https://github.com/Eromera/erfnet_pytorch/tree/master/trained_models) |
| MobileNetV2* | mobilenet_v2-b0353104.pth | [TorchVision Weight](https://download.pytorch.org/models/mobilenet_v2-b0353104.pth) |
| MobileNetV3-Large** | mobilenet_v3_large-8738ca79.pth | [TorchVision Weight](https://download.pytorch.org/models/mobilenet_v3_large-8738ca79.pth) |
| VGG-16 | vgg16-397923af.pth | [TorchVision Weight](https://download.pytorch.org/models/vgg16-397923af.pth) |
| ResNet-18 | resnet18-5c106cde.pth | [TorchVision Weight](https://download.pytorch.org/models/resnet18-5c106cde.pth) |
| ResNet-34 | resnet34-333f7ec4.pth | [TorchVision Weight](https://download.pytorch.org/models/resnet34-333f7ec4.pth) |
| ResNet-50 | resnet50-19c8e357.pth | [TorchVision Weight](https://download.pytorch.org/models/resnet50-19c8e357.pth) |
| ResNet-101 | resnet101-5d3b4d8f.pth | [TorchVision Weight](https://download.pytorch.org/models/resnet101-5d3b4d8f.pth) |
| Swin-Tiny** | swin_tiny_patch4_window7_224.pth | [Official Repo Weight](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_tiny_patch4_window7_224.pth) |

52
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/INSTALL.md Normal file
View File

@@ -0,0 +1,52 @@
# Installation
## Download the code:
```
git clone https://github.com/voldemortX/pytorch-auto-drive.git
cd pytorch-auto-drive
```
## Requirements
- Linux (recommended) or Windows (not fully tested, could have problems)
- Python >= 3.6
- CUDA >= 9.2 (for CUDA version < 9.2, the code is tested only with PyTorch 1.3 & CUDA 9.0 & CuDNN 7.6.0)
- PyTorch >= 1.6 (2.x are not tested)
- TorchVision >= 0.7.0
- [mmcv-full](https://github.com/open-mmlab/mmcv) >= 1.3.5 (according to PyTorch/CUDA version)
- Other pip dependencies: `pip install -r requirements.txt`
The default Conda env (step-by-step):
```
conda create -n pad python=3.6
conda activate pad
conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.2 -c pytorch
pip install mmcv-full==1.3.5 -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.6.0/index.html
pip install -r requirements.txt
```
## Prepare the code:
```
chmod 777 *.sh tools/shells/*.sh
mkdir output
```
## Improve training speed with [Pillow-SIMD](https://github.com/uploadcare/pillow-simd) (optional, advanced):
```
pip uninstall pillow
CC="cc -mavx2" pip install -U --force-reinstall pillow-simd
```
Note that you need to use ToTensor transform as late as possible for this speedup.
## Enable tensorboard (optional):
```
tensorboard --logdir=<path to tb_logs>
```
`<path to tb_logs>` is usually `./checkpoints/tb_logs` if you did not customized `save_dir` in config file.

80
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/LANEDETECTION.md Normal file
View File

@@ -0,0 +1,80 @@
# Lane detection
**Before diving into this, please make sure you followed the instructions to prepare datasets in [DATASET.md](./DATASET.md)**
**Execution is based on [config files](../configs/README.md)**
## Training:
Some models' ImageNet pre-trained weights need to be manually downloaded, refer to [this table](./IMAGENET_MODELS.md).
```
python main_landet.py --train \
--config=<config file path> \
--mixed-precision # Optional, enable mixed precision \
--cfg-options=<overwrite cfg dict> # Optional
```
Your `<overwrite cfg dict>` is used to manually override config file options in commandline so you don't have to modify config file each time. It should look like this (**the quotation marks are necessary!**): `"train.batch_size=8 train.workers=4 model.lane_classifier_cfg.dropout=0.1"`
Some options can be used by shortcuts, such as `--batch-size` will set both `train.batch_size` and `test.batch_size`, for more info:
```
python main_landet.py --help
```
Example shells are provided in [tools/shells](../tools/shells/).
## Distributed Training
We support multi-GPU training with Distributed Data Parallel (DDP):
```
python -m torch.distributed.launch --nproc_per_node=<number of GPU per-node> --use_env main_landet.py <your normal args>
```
With DDP, batch size and number of workers are **per-GPU**. Do not forget to set device args like `world_size` in your config.
## Testing
### Evaluation:
**Important Notice: Do not simoutanously run multiple evaluation on CULane, since the eval use the same pytorch-auto-drive/output cache directory, the results could be overwritten! Same goes for LLAMAS!**
1. Predict lane lines:
```
python main_landet.py --test \ # Or --val for validation
--config=<config file path> \
--mixed-precision # Optional, enable mixed precision \
--cfg-options=<overwrite cfg dict> # Optional
```
To test a downloaded pt file, try add `--checkpoint=<pt file path>`.
Note that LLAMAS doesn't have test set labels.
2. Test with official scripts on `<my_dataset>`:
```
./autotest_<my_dataset>.sh <exp_name> <mode> <save_dir>
```
`<mode>` includes `test` and `val`.
`<save_dir>` and `<exp_name>` are recommended to set the same as in config file, so detail evaluation results will be saved to `<save_dir>/<exp_name>/`
Overall result will be saved to `log.txt`.
### Fast evaluation in mIoU [Not Recommended]:
Training contains online fast validations by using `val_num_steps` and the best model is saved, but we find that the best checkpoint is usually the last, so probably no need for validations. For log details you can checkout tensorboard.
To validate a trained model on mean IoU, you can use either mixed-precision or fp32 for any model trained with/without mixed-precision:
```
python main_landet.py --valfast \
--config=<config file path> \
--mixed-precision # Optional, enable mixed precision \
--cfg-options=<overwrite cfg dict> # Optional
```

213
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/MODEL_ZOO.md Normal file
View File

@@ -0,0 +1,213 @@
# Welcome to PytorchAutoDrive model zoo
## Lane detection performance
**Data Augmentation levels:**
- **level 0**: only small rotation and resize
- **level 1a**: the LSTR augmentations
- **level 1b**: the BézierLaneNet augmentations
- **level 1c**: the LaneATT augmentations
| method | backbone | data<br>augmentation | resolution | mixed precision? | dataset | metric | average | best | training time <br> *(2080 Ti)* |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Baseline | VGG16 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 93.79% | 93.94% | 1.5h |
| Baseline | ResNet18 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 94.18% | 94.25% | 0.7h |
| Baseline | ResNet34 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 95.23% | 95.31% | 1.1h |
| Baseline | ResNet34 | level 1a | 360 x 640 | *no* | TuSimple | Accuracy | 92.14% | 92.68% | 1.2h* |
| Baseline | ResNet50 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 95.07% | 95.12% | 1.5h |
| Baseline | ResNet101 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 95.15% | 95.19% | 2.6h |
| Baseline | ERFNet | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 96.02% | 96.04% | 0.8h |
| Baseline | ERFNet | level 1a | 360 x 640 | *no* | TuSimple | Accuracy | 94.21% | 94.37% | 0.9h* |
| Baseline | ENet<sup>#</sup> | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 95.55% | 95.61% | 1h<sup>+</sup> |
| Baseline | MobileNetV2 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 93.98% | 94.07% | 0.5h |
| Baseline | MobileNetV3-Large | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 92.09% | 92.18% | 0.5h |
| SCNN | VGG16 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 95.01% | 95.17% | 2h |
| SCNN | ResNet18 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 94.69% | 94.77% | 1.2h |
| SCNN | ResNet34 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 95.19% | 95.25% | 1.6h |
| SCNN | ResNet34 | level 1a | 360 x 640 | *no* | TuSimple | Accuracy | 92.62% | 93.42% | 1.7h* |
| SCNN | ResNet50 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 95.43% | 95.56% | 2.4h |
| SCNN | ResNet101 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 95.56% | 95.69% | 3.5h |
| SCNN | ERFNet | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 96.18% | 96.29% | 1.6h |
| SCNN | ERFNet | level 1a | 360 x 640 | *no* | TuSimple | Accuracy | 95.00% | 95.26% | 1.7h* |
| RESA | ResNet18 | level 0 | 360 x 640 | *no* | TuSimple | Accuracy | 94.64% | 95.24% | 1.2h* |
| RESA | ResNet34 | level 0 | 360 x 640 | *no* | TuSimple | Accuracy | 94.84% | 95.15% | 1.6h* |
| RESA | ResNet50 | level 0 | 360 x 640 | *no* | TuSimple | Accuracy | 95.34% | 95.50% | 2.4h* |
| RESA | ResNet101 | level 0 | 360 x 640 | *no* | TuSimple | Accuracy | 95.24% | 95.56% | 3.5h* |
| RESA | ERFNet | level 0 | 360 x 640 | *no* | TuSimple | Accuracy | 95.73% | 95.76% | 1.7h |
| RESA | MobileNetV2 | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 94.61% | 95.21% | 0.7h |
| RESA | MobileNetV3-Large | level 0 | 360 x 640 | *yes* | TuSimple | Accuracy | 94.56% | 94.99% | 0.7h |
| LSTR | ResNet18s<sup>#</sup> | level 0 | 360 x 640 | *no* | TuSimple | Accuracy | 91.91% | 92.40% | 14.2h |
| LSTR | ResNet18s<sup>#</sup> | level 1a | 360 x 640 | *no* | TuSimple | Accuracy | 94.91% | 95.06% | 15.5h |
| BézierLaneNet | ResNet18 | level 1b | 360 x 640 | *no* | TuSimple | Accuracy | 95.01% | 95.41% | 5.5h |
| BézierLaneNet | ResNet34 | level 1b | 360 x 640 | *no* | TuSimple | Accuracy | 95.17% | 95.65% | 6.5h |
| Baseline | VGG16 | level 0 | 288 x 800 | *yes* | CULane | F1 | 65.93 | 66.09 | 9.3h |
| Baseline | ResNet18 | level 0 | 288 x 800 | *yes* | CULane | F1 | 65.19 | 65.30 | 5.3h |
| Baseline | ResNet34 | level 0 | 288 x 800 | *yes* | CULane | F1 | 69.82 | 69.92 | 7.3h |
| Baseline | ResNet50 | level 0 | 288 x 800 | *yes* | CULane | F1 | 68.31 | 68.48 | 12.4h |
| Baseline | ResNet101 | level 0 | 288 x 800 | *yes* | CULane | F1 | 71.29 | 71.37 | 20.0h |
| Baseline | ERFNet | level 0 | 288 x 800 | *yes* | CULane | F1 | 73.40 | 73.49 | 6h |
| Baseline | ENet<sup>#</sup> | level 0 | 288 x 800 | *yes* | CULane | F1 | 69.39 | 69.90 | 6.4h<sup>+</sup> |
| Baseline | MobileNetV2 | level 0 | 288 x 800 | *yes* | CULane | F1 | 67.34 | 67.41 | 3.0h |
| Baseline | MobileNetV3-Large | level 0 | 288 x 800 | *yes* | CULane | F1 | 68.27 | 68.42 | 3.0h |
| Baseline | RepVGG-A0| level 0 | 288 x 800 | *yes* | CULane | F1 | 70.22 | 70.56 | 3.3h** |
| Baseline | RepVGG-A1 | level 0 | 288 x 800 | *yes* | CULane | F1 | 70.73 | 70.85 | 4.1h** |
| Baseline | RepVGG-B0 | level 0 | 288 x 800 | *yes* | CULane | F1 | 71.77 | 71.81 | 6.2h** |
| Baseline | RepVGG-B1g2 | level 0 | 288 x 800 | *yes* | CULane | F1 | 72.08 | 72.20 | 10.0h** |
| Baseline | RepVGG-B2 | level 0 | 288 x 800 | *yes* | CULane | F1 | 72.24 | 72.33 | 13.2h** |
| Baseline | Swin-Tiny | level 0 | 288 x 800 | *yes* | CULane | F1 | 69.75 | 69.90 | 12.1h** |
| SCNN | VGG16 | level 0 | 288 x 800 | *yes* | CULane | F1 | 74.02 | 74.29 | 12.8h |
| SCNN | ResNet18 | level 0 | 288 x 800 | *yes* | CULane | F1 | 71.94 | 72.19 | 8.0h |
| SCNN | ResNet34 | level 0 | 288 x 800 | *yes* | CULane | F1 | 72.44 | 72.70 | 10.7h |
| SCNN | ResNet50 | level 0 | 288 x 800 | *yes* | CULane | F1 | 72.95 | 73.03 | 17.9h |
| SCNN | ResNet101 | level 0 | 288 x 800 | *yes* | CULane | F1 | 73.29 | 73.58 | 25.7h |
| SCNN | ERFNet | level 0 | 288 x 800 | *yes* | CULane | F1 | 73.85 | 74.03 | 11.3h |
| SCNN | RepVGG-A1 | level 0 | 288 x 800 | *yes* | CULane | F1 | 72.88 | 72.89 | 5.7h** |
| RESA | ResNet18 | level 0 | 288 x 800 | *no* | CULane | F1 | 72.76 | 72.90 | 8.0h* |
| RESA | ResNet34 | level 0 | 288 x 800 | *no* | CULane | F1 | 73.29 | 73.66 | 10.7h* |
| RESA | ResNet50 | level 0 | 288 x 800 | *no* | CULane | F1 | 73.99 | 74.19 | 17.9h* |
| RESA | ResNet101 | level 0 | 288 x 800 | *no* | CULane | F1 | 73.96 | 74.04 | 25.7h* |
| RESA | ERFNet | level 0 | 288 x 800 | *no* | CULane | F1 | 73.28 | 73.32 | 9.1h |
| RESA | MobileNetV2 | level 0 | 288 x 800 | *yes* | CULane | F1 | 72.28 | 72.36 | 4.6h |
| RESA | MobileNetV3-Large | level 0 | 288 x 800 | *yes* | CULane | F1 | 70.23 | 70.61 | 4.6h |
| LSTR | ResNet18s-2X<sup>#</sup> | level 0 | 288 x 800 | *no* | CULane | F1 | 36.27 | 39.77 | 28.5h* |
| LSTR | ResNet18s-2X<sup>#</sup> | level 1a | 288 x 800 | *no* | CULane | F1 | 68.35 | 68.72 | 31.5h* |
| LSTR | ResNet34 | level 1a | 288 x 800 | *no* | CULane | F1 | 72.17 | 72.48 | 45.0h* |
| LaneATT | ResNet18 | level 1c | 360 x 640 | *no* | CULane | F1 | 74.71 | 74.87 | 3.6h** |
| LaneATT | ResNet34 | level 1c | 360 x 640 | *no* | CULane | F1 | 75.76 | 75.82 | 4.0h** |
| BézierLaneNet | ResNet18 | level 1b | 288 x 800 | *yes* | CULane | F1 | 73.36 | 73.67 | 9.9h |
| BézierLaneNet | ResNet34 | level 1b | 288 x 800 | *yes* | CULane | F1 | 75.30 | 75.57 | 11.0h |
| Baseline | ERFNet | level 0 | 360 x 640 | *yes* | LLAMAS | F1 | 95.94 | 96.13 | 10.9h<sup>+</sup> |
| Baseline | VGG16 | level 0 | 360 x 640 | *yes* | LLAMAS | F1 | 95.05 | 95.11 | 9.3h |
| Baseline | ResNet34 | level 0 | 360 x 640 | *yes* | LLAMAS | F1 | 95.90 | 95.91 | 7.0h |
| SCNN | ERFNet | level 0 | 360 x 640 | *yes* | LLAMAS | F1 | 95.89 | 95.94 | 14.2h<sup>+</sup> |
| SCNN | VGG16 | level 0 | 360 x 640 | *yes* | LLAMAS | F1 | 96.39 | 96.42 | 12.5h |
| SCNN | ResNet34 | level 0 | 360 x 640 | *yes* | LLAMAS | F1 | 96.17 | 96.19 | 10.1h |
| BézierLaneNet | ResNet18 | level 1b | 360 x 640 | *yes* | LLAMAS | F1 | 95.42 | 95.52 | 5.5h |
| BézierLaneNet | ResNet34 | level 1b | 360 x 640 | *yes* | LLAMAS | F1 | 96.04 | 96.11 | 6.1h |
*All performance is measured with ImageNet pre-training and reported as 3 times average/best on test set.*
*The test set annotations of LLAMAS are not public, so we provide validation set result in this table.*
*<sup>+</sup> Measured on a single GTX 1080Ti.*
*<sup>#</sup> No pre-training.*
*\* Trained on a 1080 Ti cluster, with CUDA 9.0 PyTorch 1.3, training time is estimated as: single 2080 Ti, mixed precision.*
*\*\* Trained on two 2080ti.*
### TuSimple detailed performance (best):
| method | backbone | data<br>augmentation | accuracy | FP | FN | |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Baseline | VGG16 | level 0 | 93.94% | 0.0998 | 0.1021 | [model](https://drive.google.com/file/d/1ChK0hApqLU0xUiEm4Wul-gNYQDQka151/view?usp=sharing) \| [shell](../tools/shells/vgg16_baseline_tusimple.sh) |
| Baseline | ResNet18 | level 0 | 94.25% | 0.0881 | 0.0894 | [model](https://drive.google.com/file/d/17VKnwsN4WMbpnD4DgaaerppjXybqn-LG/view?usp=sharing) \| [shell](../tools/shells/resnet18_baseline_tusimple.sh) |
| Baseline | ResNet34 | level 0 | 95.31% | 0.0640 | 0.0622 | [model](https://drive.google.com/file/d/1NAck0aQZK_wAHer4xB8xzegxDWk9EFtG/view?usp=sharing) \| [shell](../tools/shells/resnet34_baseline_tusimple.sh) |
| Baseline | ResNet34 | level 1a | 92.68% | 0.1073 | 0.1221 | [model](https://drive.google.com/file/d/1OhN2tWIep9ncKFf-_2RqUEaSJvPK60cn/view?usp=sharing) \| [shell](../tools/shells/resnet34_baseline-aug_tusimple.sh) |
| Baseline | ResNet50 | level 0 | 95.12% | 0.0649 | 0.0653 | [model](https://drive.google.com/file/d/10KBMVGc63kPvqL_2deaLfTfC3fSAtnju/view?usp=sharing) \| [shell](../tools/shells/resnet50_baseline_tusimple.sh) |
| Baseline | ResNet101 | level 0 | 95.19% | 0.0619 | 0.0620 | [model](https://drive.google.com/file/d/1mELtKB3e8ntOmPovhnMphXWKf_bv83ef/view?usp=sharing) \| [shell](../tools/shells/resnet101_baseline_tusimple.sh) |
| Baseline | ERFNet | level 0 | 96.04% | 0.0591 | 0.0365 | [model](https://drive.google.com/file/d/1rLWDP_dkIQ7sBsCEzJi8T7ET1EPghhJJ/view?usp=sharing) \| [shell](../tools/shells/erfnet_baseline_tusimple.sh) |
| Baseline | ERFNet | level 1a | 94.37% | 0.0846 | 0.0770 | [model](https://drive.google.com/file/d/1LPmxT8rnyZL2M08lSLrlvrM0H_hMrFvq/view?usp=sharing) \| [shell](../tools/shells/erfnet_baseline-aug_tusimple.sh) |
| Baseline | ENet | level 0 | 95.61% | 0.0655 | 0.0503 | [model](https://drive.google.com/file/d/1CNSox62ghs0ArDVJb9mTZ1NVvqSkUNYC/view?usp=sharing) \| [shell](../tools/shells/enet_baseline_tusimple.sh) |
| Baseline | MobileNetV2 | level 0 | 94.07% | 0.0792 | 0.0866 | [model](https://drive.google.com/file/d/1SUqt3BDXSMhAv68F9VIKncY0lDUg9My8/view?usp=sharing) \| [shell](../tools/shells/mobilenetv2_baseline_tusimple.sh) |
| Baseline | MobileNetV3-Large | level 0 | 92.18% | 0.1149 | 0.1322 | [model](https://drive.google.com/file/d/1I5SPlkmC8TnNeANoQGxzP3P1_iVxms3u/view?usp=sharing) \| [shell](../tools/shells/mobilenetv3-large_baseline_tusimple.sh) |
| SCNN | VGG16 | level 0 | 95.17% | 0.0637 | 0.0622 | [model](https://drive.google.com/file/d/1Fd46-f_8q-fGcJEI_PhPyh7aBY1uqbIw/view?usp=sharing) \| [shell](../tools/shells/vgg16_scnn_tusimple.sh) |
| SCNN | ResNet18 | level 0 | 94.77% | 0.0753 | 0.0737 | [model](https://drive.google.com/file/d/1cHp9gG2NgtC1iSp2LZMPF_UKiCb-fVkn/view?usp=sharing) \| [shell](../tools/shells/resnet18_scnn_tusimple.sh) |
| SCNN | ResNet34 | level 0 | 95.25% | 0.0627 | 0.0634 | [model](https://drive.google.com/file/d/1M0ROpEHV8DGJT4xWq2eURcbqMzpea1q7/view?usp=sharing) \| [shell](../tools/shells/resnet34_scnn_tusimple.sh) |
| SCNN | ResNet34 | level 1a | 93.42% | 0.0868 | 0.0998 | [model](https://drive.google.com/file/d/1t-cmUjBbLzSjODMvcpwoRPJKnUxLfWKk/view?usp=sharing) \| [shell](../tools/shells/resnet34_scnn-aug_tusimple.sh) |
| SCNN | ResNet50 | level 0 | 95.56% | 0.0561 | 0.0556 | [model](https://drive.google.com/file/d/1YK-PzdE9q8zn48isiBxwaZEdRsFw_oHe/view?usp=sharing) \| [shell](../tools/shells/resnet50_scnn_tusimple.sh) |
| SCNN | ResNet101 | level 0 | 95.69% | 0.0519 | 0.0504 | [model](https://drive.google.com/file/d/13qk5rIHqhDlwylZP9S-8fN53DexPTBQy/view?usp=sharing) \| [shell](../tools/shells/resnet101_scnn_tusimple.sh) |
| SCNN | ERFNet | level 0 | 96.29% | 0.0470 | 0.0318 | [model](https://drive.google.com/file/d/1rzE2fZ5mQswMIm6ICK1lWH-rsQyjRbxL/view?usp=sharing) \| [shell](../tools/shells/erfnet_scnn_tusimple.sh) |
| SCNN | ERFNet | level 1a | 95.26% | 0.0625 | 0.0512 | [model](https://drive.google.com/file/d/14XJ-W_wIOndjkhtPiUghwy0cLXrnl0PS/view?usp=sharing) \| [shell](../tools/shells/erfnet_scnn-aug_tusimple.sh) |
| RESA | ResNet18 | level 0 | 95.24% | 0.0685 | 0.0571 | [model](https://drive.google.com/file/d/1I3XpUB_I0SEkvE4sejIFyInOPZ5XsWyG/view?usp=sharing) \| [shell](../tools/shells/resnet18_resa_tusimple.sh) |
| RESA | ResNet34 | level 0 | 95.15% | 0.0690 | 0.0592 | [model](https://drive.google.com/file/d/1Spa1bCXoFyjCgOO-ordSPP5a3GMW6E0N/view?usp=sharing) \| [shell](../tools/shells/resnet34_resa_tusimple.sh) |
| RESA | ResNet50 | level 0 | 95.50% | 0.0550 | 0.0507 | [model](https://drive.google.com/file/d/1Mmb_4AFzSpZBcB7UxKr3Vlhwi27exqp0/view?usp=sharing) \| [shell](../tools/shells/resnet50_resa_tusimple.sh) |
| RESA | ResNet101 | level 0 | 95.56% | 0.0580 | 0.0513 | [model](https://drive.google.com/file/d/1i--g7uzt3dTeMlNnAXbBCSJ8RXVjIljR/view?usp=sharing) \| [shell](../tools/shells/resnet101_resa_tusimple.sh) |
| RESA | ERFNet | level 0 | 95.76% | 0.0648 | 0.0439 | [model](https://drive.google.com/file/d/1bbq28A7Gt9cWdrhAp_V4ENzDkde2sNXh/view?usp=sharing) \| [shell](../tools/shells/erfnet_resa_tusimple.sh) |
| RESA | MobileNetV2 | level 0 | 95.21% | 0.0642 | 0.0552 | [model](https://drive.google.com/file/d/1XuQ-jak8qViNK9NXeeVmya2EQ-Z3XxZW/view?usp=sharing) \| [shell](../tools/shells/mobilenetv2_resa_tusimple.sh) |
| RESA | MobileNetV3-Large | level 0 | 94.99% | 0.0841 | 0.0597 | [model](https://drive.google.com/file/d/1ax7YTH6r8o9PIKSLT4fh7GVdaKgdujMO/view?usp=sharing) \| [shell](../tools/shells/mobilenetv3-large_resa_tusimple.sh) |
| LSTR | ResNet18s | level 1a | 95.06% | 0.0486 | 0.0418 | [model](https://drive.google.com/file/d/1z1ikrcgboyLFO3ysJUIf8qlBv7zEUvjK/view?usp=sharing) \| [shell](../tools/shells/resnet18s_lstr-aug_tusimple.sh) |
| LSTR | ResNet18s | level 0 | 92.40% | 0.1289 | 0.1127 | [model](https://drive.google.com/file/d/1iHArGHnOlSbS01RPFlLYI1mPJSX7o4sR/view?usp=sharing) \| [shell](../tools/shells/resnet18s_lstr_tusimple.sh) |
| BézierLaneNet | ResNet18 | level 1b | 95.41% | 0.0531 | 0.0458 | [model](https://drive.google.com/file/d/10qMdvPBnZP4P88EQXYZxsXZgj7sz6LvS/view?usp=sharing) \| [shell](../tools/shells/resnet18_bezierlanenet_tusimple-aug1b.sh) |
| BézierLaneNet | ResNet34 | level 1b | 95.65% | 0.0513 | 0.0386 | [model](https://drive.google.com/file/d/1FFn8j2BoUsyj8UbBcfeGWKvCQj9Qg-44/view?usp=sharing) \| [shell](../tools/shells/resnet34_bezierlanenet_tusimple-aug1b.sh) |
### CULane detailed performance (best):
| method | backbone | data<br>augmentation | normal | crowded | night | no line | shadow | arrow | dazzle<br>light | curve | crossroad | total | |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Baseline | VGG16 | level 0 | 85.51 | 64.05 | 61.14 | 35.96 | 59.76 | 78.43 | 53.25 | 62.16 | 2224 | 66.09 | [model](https://drive.google.com/file/d/1wVz1a7S1e5Dgy7ERk7E8dqQ8gyK-dWLG/view?usp=sharing) \| [shell](../tools/shells/vgg16_baseline_culane.sh) |
| Baseline | ResNet18 | level 0 | 85.45 | 62.63 | 61.04 | 33.88 | 51.72 | 78.15 | 53.05 | 59.70 | 1915 | 65.30 | [model](https://drive.google.com/file/d/1wkaTp8v1ceXrd6AjRccqpNxxxkd_qg1U/view?usp=sharing) \| [shell](../tools/shells/resnet18_baseline_culane.sh) |
| Baseline | ResNet34 | level 0 | 89.46 | 66.66 | 65.38 | 40.43 | 62.17 | 83.18 | 58.51 | 63.00 | 1713 | 69.92 | [model](https://drive.google.com/file/d/16VIJcd3wDOjFjg3UCVekUPcAb_F1K604/view?usp=sharing) \| [shell](../tools/shells/resnet34_baseline_culane.sh) |
| Baseline | ResNet50 | level 0 | 88.15 | 65.73 | 63.74 | 37.96 | 62.59 | 81.68 | 59.47 | 64.01 | 2046 | 68.48 | [model](https://drive.google.com/file/d/1DYVeH9kdSPhEMA4fsJFdEiw8qOwvQBl8/view?usp=sharing) \| [shell](../tools/shells/resnet50_baseline_culane.sh) |
| Baseline | ResNet101 | level 0 | 90.11 | 67.89 | 67.01 | 43.10 | 70.56 | 85.09 | 61.77 | 65.47 | 1883 | 71.37 | [model](https://drive.google.com/file/d/1iubFjWetsKE2VI4BEIWLDd80gB7IQUaP/view?usp=sharing) \| [shell](../tools/shells/resnet101_baseline_culane.sh) |
| Baseline | ERFNet | level 0 | 91.48 | 71.27 | 68.09 | 46.76 | 74.47 | 86.09 | 64.18 | 66.89 | 2102 | 73.49 | [model](https://drive.google.com/file/d/16-Q_jZYc9IIKUEHhClSTwZI4ClMeVvQS/view?usp=sharing) \| [shell](../tools/shells/erfnet_baseline_culane.sh) |
| Baseline | ENet | level 0 | 89.26 | 68.15 | 62.99 | 42.43 | 68.59 | 83.10 | 58.49 | 63.23 | 2464 | 69.90 | [model](https://drive.google.com/file/d/1DNgOpAVq87GIPUeAdMP6fnS4LhUmqyRB/view?usp=sharing) \| [shell](../tools/shells/enet_baseline_culane.sh) |
| Baseline | MobileNetV2 | level 0 | 87.82 | 65.09 | 61.46 | 38.15 | 57.34 | 79.29 | 55.89 | 60.29 | 2114 | 67.41 | [model](https://drive.google.com/file/d/1xTW24b0bW_tzeXQc0znHrMrkBK4BL7_t/view?usp=sharing) \| [shell](../tools/shells/mobilenetv2_baseline_culane.sh) |
| Baseline | MobileNetV3-Large | level 0 | 88.20 | 66.33 | 63.08 | 40.41 | 56.15 | 79.81 | 59.15 | 61.96 | 2304 | 68.42 | [model](https://drive.google.com/file/d/1JJ6gGcH6fAwR3UcGAnmdels5Vm8Bz48Q/view?usp=sharing) \| [shell](../tools/shells/mobilenetv3-large_baseline_culane.sh) |
| Baseline | RepVGG-A0 | level 0 | 89.74 | 67.68 | 65.21 | 42.51 | 67.85 | 83.13 | 60.86 | 63.63 | 2011 | 70.56 | [model](https://drive.google.com/file/d/1IJtM5LT0GTsHHlO0USLpuZLA_KyuLRd_/view?usp=sharing) \| [shell](../tools/shells/repvgg-a0_baseline_culane.sh) |
| Baseline | RepVGG-A1 | level 0 | 89.92 | 68.60 | 65.43 | 41.99 | 66.64 | 84.78 | 61.38 | 64.85 | 2127 | 70.85 | [model](https://drive.google.com/file/d/1cQMaXCww-a3mPssQK9iFzHJh6SinxcOo/view?usp=sharing) \| [shell](../tools/shells/repvgg-a1_baseline_culane.sh) |
| Baseline | RepVGG-B0 | level 0 | 90.86 | 69.32 | 66.68 | 43.53 | 67.83 | 85.43 | 59.80 | 66.47 | 2189 | 71.81 | [model](https://drive.google.com/file/d/1NR4n7N7mK3yKvRAWZUbtRYQ0xHM2vL60/view?usp=sharing) \| [shell](../tools/shells/repvgg-b0_baseline_culane.sh) |
| Baseline | RepVGG-B1g2 | level 0 | 90.85 | 69.31 | 67.94 | 43.81 | 68.45 | 85.85 | 60.64 | 67.69 | 2092 | 72.20 | [model](https://drive.google.com/file/d/1tKo69RroMYMn_v_C51BuHJQDSN0I7R-m/view?usp=sharing) \| [shell](../tools/shells/repvgg-b1g2_baseline_culane.sh) |
| Baseline | RepVGG-B2 | level 0 | 90.82 | 69.84 | 67.65 | 43.02 | 72.08 | 85.76 | 61.75 | 67.67 | 2000 | 72.33 | [model](https://drive.google.com/file/d/1_3sS5U20lTDIsq5jS4cev0kZaUER9NPH/view?usp=sharing) \| [shell](../tools/shells/repvgg-b2_baseline_culane.sh) |
| Baseline | Swin-Tiny | level 0 | 89.55 | 68.36 | 63.56 | 42.53 | 61.96 | 82.64 | 60.81 | 65.21 | 2813 | 69.90 | [model](https://drive.google.com/file/d/1YI_pHGVuuWYyTLeUgdZVDS14WoQjPon4/view?usp=sharing) \| [shell](../tools/shells/swin-tiny_baseline_culane.sh) |
| SCNN | VGG16 | level 0 | 92.02 | 72.31 | 69.13 | 46.01 | 76.37 | 87.71 | 64.68 | 68.96 | 1924 | 74.29 | [model](https://drive.google.com/file/d/1vm8B1SSH0nlAIbz3aEGC1kqWP4YdFb3A/view?usp=sharing) \| [shell](../tools/shells/vgg16_scnn_culane.sh) |
| SCNN | ResNet18 | level 0 | 90.98 | 70.17 | 66.54 | 43.12 | 66.31 | 85.62 | 62.20 | 65.58 | 1808 | 72.19 | [model](https://drive.google.com/file/d/1i08KOS3b0hOTuzn866j4oUWtw3TYcndn/view?usp=sharing) \| [shell](../tools/shells/resnet18_scnn_culane.sh) |
| SCNN | ResNet34 | level 0 | 91.06 | 70.41 | 67.75 | 44.64 | 68.98 | 86.50 | 61.57 | 65.75 | 2017 | 72.70 | [model](https://drive.google.com/file/d/1JyPJQv8gpFZbr1sh7zRRiekUuDR4Aea8/view?usp=sharing) \| [shell](../tools/shells/resnet34_scnn_culane.sh) |
| SCNN | ResNet50 | level 0 | 91.38 | 70.60 | 67.62 | 45.02 | 71.24 | 86.90 | 66.03 | 66.17 | 1958 | 73.03 | [model](https://drive.google.com/file/d/1DxqUONEpT47RvJlCg7kDWIsdkYH0Fv8E/view?usp=sharing) \| [shell](../tools/shells/resnet50_scnn_culane.sh) |
| SCNN | ResNet101 | level 0 | 91.10 | 71.43 | 68.53 | 46.39 | 72.61 | 86.87 | 61.95 | 67.01 | 1720 | 73.58 | [model](https://drive.google.com/file/d/11O4ZDvNqQsKodnl9kJar6Mx1UKnB70L9/view?usp=sharing) \| [shell](../tools/shells/resnet101_scnn_culane.sh) |
| SCNN | ERFNet | level 0 | 91.82 | 72.13 | 69.49 | 46.68 | 70.59 | 87.40 | 64.18 | 68.30 | 2236 | 74.03 | [model](https://drive.google.com/file/d/1YOAuIJqh0M1RsPN5zISY7kTx9xt29IS3/view?usp=sharing) \| [shell](../tools/shells/erfnet_scnn_culane.sh) |
| SCNN | RepVGG-A0 | level 0 | 91.06 | 71.30 | 67.23 | 44.75 | 70.51 | 87.11 | 61.73 | 66.61 | 1963 | 72.89 | [model](https://drive.google.com/file/d/1ayyVr5lVW5HxnF5QZWJl0IKW1nO5Q_KJ/view?usp=sharing) \| [shell](../tools/shells/repvgg-a1_scnn_culane.sh) |
| RESA | ResNet18 | level 0 | 91.23 | 70.57 | 67.16 | 45.24 | 68.01 | 86.56 | 64.32 | 66.19 | 1679 | 72.90 | [model](https://drive.google.com/file/d/1VkjZ1v-uMSy2VW8VdWPOOO1pFCXhfEDK/view?usp=sharing) \| [shell](../tools/shells/resnet18_resa_culane.sh) |
| RESA | ResNet34 | level 0 | 91.31 | 71.80 | 67.54 | 46.57 | 72.74 | 86.94 | 64.46 | 67.31 | 1701 | 73.66 | [model](https://drive.google.com/file/d/1x9JWhW7AIbiADqkzmKgmBUQL-JXexABM/view?usp=sharing) \| [shell](../tools/shells/resnet34_resa_culane.sh) |
| RESA | ResNet50 | level 0 | 91.52 | 72.49 | 68.44 | 47.02 | 72.56 | 87.34 | 63.11 | 68.21 | 1493 | 74.19 | [model](https://drive.google.com/file/d/1tmp5JO2CKWekbKVNX6nVKIj3AoQVCOkG/view?usp=sharing) \| [shell](../tools/shells/resnet50_resa_culane.sh) |
| RESA | ResNet101 | level 0 | 91.45 | 71.51 | 69.01 | 46.54 | 75.52 | 87.75 | 63.90 | 68.24 | 1522 | 74.04 | [model](https://drive.google.com/file/d/1RLLo8MUZDl4wahTXbn49FlxBq5IUsWru/view?usp=sharing) \| [shell](../tools/shells/resnet101_resa_culane.sh) |
| RESA | ERFNet | level 0 | 91.18 | 71.07 | 68.50 | 45.49 | 69.53 | 87.68 | 64.52 | 65.56 | 1777 | 73.32 | [model](https://drive.google.com/file/d/175jDY6gBm1VT_CQvKCtax9fq3YubKDTA/view?usp=sharing) \| [shell](../tools/shells/erfnet_resa_culane.sh) |
| RESA | MobileNetV2 | level 0 | 90.58 | 70.42 | 67.19 | 45.29 | 62.80 | 85.52 | 66.00 | 65.19 | 1945 | 72.36 | [model](https://drive.google.com/file/d/1Dh9Laid33aMGFBQuTqasKgqluWKTU5Si/view?usp=sharing) \| [shell](../tools/shells/mobilenetv2_resa_culane.sh) |
| RESA | MobileNetV3-Large | level 0 | 89.53 | 67.63 | 65.74 | 43.08 | 66.07 | 84.61 | 60.10 | 63.14 | 2218 | 70.61 | [model](https://drive.google.com/file/d/1mgqkDgss9nDQgAOQnNIoHuprxofod1V4/view?usp=sharing) \| [shell](../tools/shells/mobilenetv3-large_resa_culane.sh) |
| LSTR | ResNet18s-2X | level 0 | 56.17 | 39.10 | 22.90 | 25.62 | 25.49 | 52.09 | 40.21 | 30.33 | 1690 | 39.77 | [model](https://drive.google.com/file/d/1vdYwM0xDcQLjMAibjmls8hX-IsUe0xcq/view?usp=sharing) \| [shell](../tools/shells/resnet18s_lstr_culane.sh) |
| LSTR | ResNet18s-2X | level 1a | 86.78 | 67.34 | 59.92 | 40.10 | 59.82 | 78.66 | 56.63 | 56.64 | 1166 | 68.72 | [model](https://drive.google.com/file/d/11Tv_nowlWmQtTYQfhGsziDIzb20kPo8o/view?usp=sharing) \| [shell](../tools/shells/resnet18s_lstr-aug_culane.sh) |
| LSTR | ResNet34 | level 1a | 89.73 | 69.77 | 66.72 | 45.32 | 68.16 | 85.03 | 64.34 | 64.13 | 1247 | 72.48 | [model](https://drive.google.com/file/d/1KfmXubuAtUoE9MO8iViMyB_3XhTxWnwH/view?usp=sharing) \| [shell](../tools/shells/resnet34_lstr-aug_culane.sh) |
| LaneATT | ResNet18 | level 1c | 90.74 | 72.63 | 69.53 | 47.71 | 70.38 | 86.55 | 65.02 | 65.73 | 1036 | 74.87 | [model](https://drive.google.com/file/d/17Ve7RiqxHK4aEXOt7MJzycj6mvvrieZ1/view?usp=sharing) \| [shell](../tools/shells/resnet18_laneatt_culane.sh) |
| LaneATT | ResNet34 | level 1c | 91.36 | 73.72 | 70.71 | 48.40 | 73.69 | 86.86 | 68.95 | 66.00 | 965 | 75.82 | [model](https://drive.google.com/file/d/1AEyNZFAskPg2MKcD3KCaL95JsBeJG_48/view?usp=sharing) \| [shell](../tools/shells/resnet34_laneatt_culane.sh) |
| BézierLaneNet | ResNet18 | level 1b | 90.22 | 71.55 | 68.70 | 45.30 | 70.91 | 84.09 | 62.49 | 58.98 | 996 | 73.67 | [model](https://drive.google.com/file/d/1IpfusHvFeMEGe8wv0fer6KF3pH4X2Tj3/view?usp=sharing) \| [shell](../tools/shells/resnet18_bezierlanenet_culane-aug1b.sh) |
| BézierLaneNet | ResNet34 | level 1b | 91.59 | 73.20 | 69.90 | 48.05 | 76.74 | 87.16 | 69.20 | 62.45 | 888 | 75.57 | [model](https://drive.google.com/file/d/1342FQeDQKRHMo283jW2T1WDgfgsYbR5q/view?usp=sharing) \| [shell](../tools/shells/resnet34_bezierlanenet_culane-aug1b.sh) |
### LLAMAS detailed performance (best):
| method | backbone | data<br>augmentation | F1 | TP | FP | FN | Precision | Recall | val / test | |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Baseline | VGG16 | level 0 | 95.11 | 70263 | 3460 | 3772 | 95.31 | 94.91 | val | [model](https://drive.google.com/file/d/1k2b7iRw3_YMJDMUsKVjtdOXRdZsXNXnO/view?usp=sharing) \| [shell](../tools/shells/vgg16_baseline_llamas.sh) |
| Baseline | ResNet34 | level 0 | 95.91 | 70841 | 2847 | 3194 | 96.14 | 95.69 | val | [model](https://drive.google.com/file/d/1YXNgwhQqwxoMkHDbRqAdnuAXIe1IdLSm/view?usp=sharing) \| [shell](../tools/shells/resnet34_baseline_llamas.sh) |
| Baseline | ERFNet | level 0 | 96.13 | 71136 | 2830 | 2899 | 96.17 | 96.08 | val | [model](https://drive.google.com/file/d/15oNU4iffIPYTuKSCH2j0boRkYmm3-uSh/view?usp=sharing) \| [shell](../tools/shells/erfnet_baseline_llamas.sh) |
| SCNN | VGG16 | level 0 | 96.42 | 71274 | 2526 | 2761 | 96.27 | 96.42 | val | [model](https://drive.google.com/file/d/1qE-euGGMZTxcHED_VDUW6eR-SKPkUDD-/view?usp=sharing) \| [shell](../tools/shells/vgg16_scnn_llamas.sh) |
| SCNN | ERFNet | level 0 | 95.94 | 71036 | 3019 | 2999 | 95.92 | 95.95 | val | [model](https://drive.google.com/file/d/1oTdmP_tsguqa1-6bBIikT4gPThUVdXCr/view?usp=sharing) \| [shell](../tools/shells/erfnet_scnn_llamas.sh) |
| SCNN | ResNet34 | level 0 | 96.19 | 71109 | 2705 | 2926 | 96.34 | 96.05 | val | [model](https://drive.google.com/file/d/1-vribu32iXViBqYumApmKQK4mjQd6BEp/view?usp=sharing) \| [shell](../tools/shells/resnet34_scnn_llamas.sh) |
| BézierLaneNet | ResNet18 | level 1b | 95.52 | 70515 | 3102 | 3520 | 95.79 | 95.25 | val | [model](https://drive.google.com/file/d/1fTQEZnr2wVQ20P3B2AyM3c_dFp5BHKwQ/view?usp=sharing) \| [shell](../tools/shells/resnet18_bezierlanenet_llamas-aug1b.sh) |
| BézierLaneNet | ResNet34 | level 1b | 96.11 | 70959 | 2667 | 3076 | 96.38 | 95.85 | val | [model](https://drive.google.com/file/d/1RhYTJB_VlHL9hFYuwAX_T4Nev9ZIlmHt/view?usp=sharing) \| [shell](../tools/shells/resnet34_bezierlanenet_llamas-aug1b.sh) |
Their test performance can be found at the [LLAMAS leaderboard](https://unsupervised-llamas.com/llamas/benchmark_splines).
## Semantic segmentation performance
| model | resolution | mixed precision? | dataset | average | best | training time<br>*(2080 Ti)* | best model link |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| FCN | 321 x 321 | *yes* | PASCAL VOC 2012 | 70.72 | 70.83 | 3.3h | [model](https://drive.google.com/file/d/1SIIpApBdL0wXanlLeLWBSJJmX3AYLBnf/view?usp=sharing) \| [shell](../tools/shells/fcn_pascalvoc_321x321.sh) |
| FCN | 321 x 321 | *no* | PASCAL VOC 2012 | 70.91 | 71.55 | 6.3h | [model](https://drive.google.com/file/d/1ZunsGFjXxSIwR8Blckyk-Ils6IdhSqV1/view?usp=sharing) \| [shell](../tools/shells/fcn_pascalvoc_321x321_fp32.sh) |
| DeeplabV2 | 321 x 321 | *yes* | PASCAL VOC 2012 | 74.59 | 74.74 | 3.3h | [model](https://drive.google.com/file/d/1UGR4u1qvJcczLfcgmSHoVd0CGqHMfLoU/view?usp=sharing) \| [shell](../tools/shells/deeplabv2_pascalvoc_321x321.sh) |
| DeeplabV3 | 321 x 321 | *yes* | PASCAL VOC 2012 | 78.11 | 78.17 | 7h | [model](https://drive.google.com/file/d/1iYN73iqDD74HPZFGorARb6T2w7KkhbPM/view?usp=sharing) \| [shell](../tools/shells/deeplabv3_pascalvoc_321x321.sh) |
| FCN | 256 x 512 | *yes* | Cityscapes | 68.05 | 68.20 | 2.2h | [model](https://drive.google.com/file/d/1zT-lBElfkD1Sratu4WYiTCRU9PF16lLj/view?usp=sharing) \| [shell](../tools/shells/fcn_cityscapes_256x512.sh) |
| DeeplabV2 | 256 x 512 | *yes* | Cityscapes | 68.65 | 68.90 | 2.2h | [model](https://drive.google.com/file/d/16SR6EEdsuOtU6xyu7BsP-GQ16-y3OfGe/view?usp=sharing) \| [shell](../tools/shells/deeplabv2_cityscapes_256x512.sh) |
| DeeplabV3 | 256 x 512 | *yes* | Cityscapes | 69.87 | 70.37 | 4.5h | [model](https://drive.google.com/file/d/1HUR09zcPpjD5Q3LAm4p5t7e9gl1ZkpqU/view?usp=sharing) \| [shell](../tools/shells/deeplabv3_cityscapes_256x512.sh) |
| DeeplabV2 | 256 x 512 | *no* | Cityscapes | 68.45 | 68.89 | 4h | [model](https://drive.google.com/file/d/1fbxsPGu31plfgyQ0N0eiqk659F9osbRm/view?usp=sharing) \| [shell](../tools/shells/deeplabv2_cityscapes_256x512_fp32.sh) |
| ERFNet | 512 x 1024 | *yes* | Cityscapes | 71.99 | 72.47 | 5h | [model](https://drive.google.com/file/d/1uzBSboKD-Xt0K6VHd2aF561Cy13q9xRe/view?usp=sharing) \| [shell](../tools/shells/erfnet_cityscapes_512x1024.sh) |
| ENet | 512 x 1024 | *yes* | Cityscapes | 65.54 | 65.74 | 10.6h | [model](https://drive.google.com/file/d/1oK2mKCetOtY8KFaKLjs7-jOMkxZjbIQD/view?usp=sharing) \| [shell](../tools/shells/enet_cityscapes_512x1024.sh) |
| DeeplabV2 | 512 x 1024 | *yes* | Cityscapes | 71.78 | 72.12 | 9h | [model](https://drive.google.com/file/d/1MUG3PMMlFOtiX7G-TYCZhG_8D9aLqTPE/view?usp=sharing) \| [shell](../tools/shells/deeplabv2_cityscapes_512x1024.sh) |
| DeeplabV3 | 512 x 1024 | *yes* | Cityscapes | 74.64 | 74.67 | 20.1h | [model](https://drive.google.com/file/d/11xrX9AdNBdupSb8cOdGASiWrjYBXdQ48/view?usp=sharing) \| [shell](../tools/shells/deeplabv3_cityscapes_512x1024.sh) |
| DeeplabV2 | 512 x 1024 | *yes* | GTAV | 32.90 | 33.88 | 13.8h | [model](https://drive.google.com/file/d/1udHozZzwka9ktMxaV0tynL1HToy0H6sI/view?usp=sharing) \| [shell](../tools/shells/deeplabv2_gtav_512x1024.sh) |
| DeeplabV2 | 512 x 1024 | *yes* | SYNTHIA* | 33.89 | 34.86 | 10.4h | [model](https://drive.google.com/file/d/1M-CO46zjXbVo8pguISUEw3M6NoKHIN0l/view?usp=sharing) \| [shell](../tools/shells/deeplabv2_synthia_512x1024.sh) |
*All performance is measured with ImageNet pre-training and reported as 3 times average/best mIoU (%) on val set.*
*\* mIoU-16.*

18
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/MODEL_ZOO_J.md Normal file
View File

@@ -0,0 +1,18 @@
The whole [MODEL_ZOO](./MODEL_ZOO.md) with specifications.
## Lane Detection
- [Baseline](/configs/lane_detection/baseline)
- [SCNN](./MODEL_ZOO.md)
- [RESA](./MODEL_ZOO.md)
- [LSTR](./MODEL_ZOO.md)
- [LaneATT](./MODEL_ZOO.md)
- [BézierLaneNet](/configs/lane_detection/bezierlanenet)
## Semantic Segmentation
- [FCN](/configs/semantic_segmentation/fcn)
- [DeeplabV2](./MODEL_ZOO.md)
- [DeeplabV3](./MODEL_ZOO.md)
- [ENet](./MODEL_ZOO.md)
- [ERFNet](/configs/semantic_segmentation/erfnet)

63
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/SEGMENTATION.md Normal file
View File

@@ -0,0 +1,63 @@
# Semantic segmentation
**Before diving into this, please make sure you followed the instructions to prepare datasets in [DATASET.md](./DATASET.md)**
**Execution is based on [config files](../configs/README.md)**
## Training:
Some models' ImageNet pre-trained weights need to be manually downloaded, refer to [this table](./IMAGENET_MODELS.md).
```
python main_semseg.py --train \
--config=<config file path> \
--mixed-precision # Optional, enable mixed precision \
--cfg-options=<overwrite cfg dict> # Optional
```
Your `<overwrite cfg dict>` is used to manually override config file options in commandline so you don't have to modify config file each time. It should look like this (**the quotation marks are necessary!**): `"train.batch_size=8 train.workers=4 model.classifier_cfg.num_classes=21"`
Some options can be used by shortcuts, such as `--batch-size` will set both `train.batch_size` and `test.batch_size`, for more info:
```
python main_semseg.py --help
```
Example shells are provided in [tools/shells](../tools/shells/).
## Distributed Training
We support multi-GPU training with Distributed Data Parallel (DDP):
```
python -m torch.distributed.launch --nproc_per_node=<number of GPU per-node> --use_env main_semseg.py <your normal args>
```
With DDP, batch size and number of workers are **per-GPU**. Do not forget to set device args like `world_size` in your config.
## Testing:
Training contains online evaluations and the best model is saved.
To evaluate a trained model:
```
python main_semseg.py --val \ # No test set labels available
--config=<config file path> \
--mixed-precision # Optional, enable mixed precision \
--cfg-options=<overwrite cfg dict> # Optional
```
To test a downloaded pt file, try add `--checkpoint=<pt file path>`.
Detail results will be saved to `<save_dir>/<exp_name>/`.
Overall result will be saved to `log.txt`.
Recommend `workers=0 batch_size=1` for high precision inference.
## Notes:
1. Cityscapes dataset is down-sampled by 2 when training at 256 x 512, to specify different sizes, modify them in config files if needed.
2. All segmentation results reported are from single model without CRF and without multi-scale testing.

129
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/VISUALIZATION.md Normal file
View File

@@ -0,0 +1,129 @@
# Welcome to PytorchAutoDrive visualization tutorial
Trained models used for inference can be found at [MODEL_ZOO.md](../docs/MODEL_ZOO.md).
To quickly get some results, download [PAD_test_images](https://drive.google.com/file/d/1XQvBS1uoHeIgUv7oDQ4Vp1tWYi0oAGhU/view?usp=sharing) (129MB). It includes sample images and videos from TuSimple, CULane, Cityscapes and PASCAL VOC 2012. [\[Alternate Baidu Yun download link\]](https://pan.baidu.com/s/1KjHJJdBzx0pa50jsJfHtaw?pwd=uoeg)
After download, unzip it as:
```
unzip PAD_test_images.zip -d PAD_test_images
```
## Segmentation mask (Image Folder)
Use [tools/vis/seg_img_dir.py](../tools/vis/seg_img_dir.py) to visualize segmentation results, by providing the image folder with `--image-path` and mask (**not the colored ones**) with `--target-path`. For detailed instructions, run:
```
python tools/vis/seg_img_dir.py --help
```
For example, visualize a PASCAL VOC image:
```
python tools/vis/seg_img_dir.py --image-path=PAD_test_images/seg_test_images/voc --image-suffix=_image.jpg --target-path=PAD_test_images/seg_test_images/voc --target-suffix=_mask.png --save-path=PAD_test_images/seg_test_images/voc_res --config=<any PASCAL VOC config>
```
You should be able to see the result like this stored at `--save-path` with same image names as in `--image-path`:
<div align="center">
<img src="assets/vis_voc1.jpg"/>
</div>
You can also do inference by `--pred`, in this case you'll need the correct `--config` and `--checkpoint`, for example:
```
python tools/vis/seg_img_dir.py --image-path=PAD_test_images/seg_test_images/voc --image-suffix=_image.jpg --save-path=PAD_test_images/seg_test_images/voc_pred --pred --config=configs/semantic_segmentation/deeplabv2/resnet101_pascalvoc_321x321.py --checkpoint=deeplabv2_pascalvoc_321x321_20201108.pt
```
Another example for visualizing complex filename and structure such as Cityscapes (Remember to use `--map-id` for Cityscapes style annotation files):
```
python tools/vis/seg_img_dir.py --image-path=PAD_test_images/seg_test_images/munster --image-suffix=_leftImg8bit.png --target-path=PAD_test_images/seg_test_images/labels/munster --target-suffix=_gtFine_labelIds.png --save-path=PAD_test_images/seg_test_images/city_res --config=configs/semantic_segmentation/erfnet/cityscapes_512x1024.py --map-id
```
## Segmentation mask (Video)
Since there are no video labels available from all supported datasets, inference must be conducted, for example:
```
python tools/vis/seg_video.py --video-path=PAD_test_images/seg_test_images/stuttgart_00.avi --save-path=PAD_test_images/seg_test_images/stuttgart_pred.avi --config=configs/semantic_segmentation/erfnet/cityscapes_512x1024.py --checkpoint=erfnet_cityscapes_512x1024_20200918.pt
```
## Lane points (Image Folder)
Use [tools/vis/lane_img_dir.py](../tools/vis/lane_img_dir.py) to visualize arbitrary lane detection results, by providing the image folder with `--image-path`, keypoint file folder (**CULane format**) with `--keypoint-path` and optional segmentation mask (**not the colored ones**) with `--mask-path`. For detailed instructions, run:
```
python tools/vis/lane_img_dir.py --help
```
Try `--style` for different lane line styles (currently supports `point`, `line` & `bezier`):
- `--style=point`: Sample points will be drawn directly
- `--style=line`: Sample points will be connected with semi-transparent lines (you can get a neat visual on curve-based methods, but possible unpleasant zigzag lines for segmentation-based methods)
- `--style=bezier`: Same as `--style=line`, additionally adds Bézier control points for Bézier curve-based methods
For example, visualize on CULane:
```
python tools/vis/lane_img_dir.py --image-path=PAD_test_images/lane_test_images/05171008_0748.MP4 --keypoint-path=PAD_test_images/lane_test_images/05171008_0748.MP4 --mask-path=PAD_test_images/lane_test_images/laneseg_label_w16/05171008_0748.MP4 --image-suffix=.jpg --keypoint-suffix=.lines.txt --mask-suffix=.png --save-path=PAD_test_images/lane_test_images/culane_res --config=<any CULane config>
```
You should be able to see the result like this stored at `--save-path` with same image names as in `--image-path`:
<div align="center">
<img src="assets/vis_culane1.jpg"/>
</div>
You can also do inference by `--pred`, in this case you'll need the correct `--config` and `--checkpoint`, for example:
```
python tools/vis/lane_img_dir.py --image-path=PAD_test_images/lane_test_images/05171008_0748.MP4 --image-suffix=.jpg --save-path=PAD_test_images/lane_test_images/culane_pred --pred --config=configs/lane_detection/baseline/erfnet_culane.py --checkpoint=erfnet_baseline_culane_20210204.pt
```
## Lane points (Video)
Since there are no video labels available from all supported datasets, inference must be conducted, for example:
```
python tools/vis/lane_video.py --video-path=PAD_test_images/lane_test_images/tusimple_val_1min.avi --save-path=PAD_test_images/lane_test_images/tusimple_val_1min_pred.avi --config=configs/lane_detection/baseline/erfnet_tusimple.py --checkpoint=erfnet_baseline_tusimple_20210424.pt
```
## Lane points (compare with GT)
In **Lane points (Image Folder)**, you literally visualized one set of text files on one set of images. While with two sets of text files, you can additionally compare with Ground Truth (typical use case when analyzing or worse: writing a paper). Provide GT keypoints by `--gt-keypoint-path` and set the evaluation metric by `--metric`, currently only `culane` and `tusimple` are supported (note that the `culane` metric here is the Python version, which slightly differs from the official C++ implementation). You may also use `--pred` to predict from a model instead of `--keypoint-path` to provide predictions.
For example, compare against GT with BézierLaneNet:
```
python tools/vis/lane_img_dir.py --image-path=PAD_test_images/lane_test_images/05171008_0748.MP4 --gt-keypoint-path=PAD_test_images/lane_test_images/05171008_0748.MP4 --image-suffix=.jpg --gt-keypoint-suffix=.lines.txt --save-path=PAD_test_images/lane_test_images/culane_gt_compare --config=configs/lane_detection/bezierlanenet/resnet34_culane-aug1b.py --style=bezier --checkpoint=resnet34_bezierlanenet_culane_aug1b_20211109.pt --mixed-precision --pred
```
You can get red **False Positive**, green **True Positive** and blue **Ground Truth**, like this:
<div align="center">
<img src="assets/vis_culane_gt.jpg"/>
</div>
The extra takeaway here is: you can visualize prediction results from implementations other than PytorchAutoDrive, which makes things easier when comparing a wide range of methods. Refer to [ADVANCED_TUTORIAL.md](./ADVANCED_TUTORIAL.md) for visualizing on specific dataset file structures (e.g., TuSimple selectively use images from different sub folders, and use one json file for all annotations).
## Advanced Visualization
You can set `--image-path` or `--target-path` to your dataset paths in order to visualize a customized dataset, but keep in mind about `--save-path`: **don't accidentally overwrite your dataset!**
Use large `batch_size`, more `workers` or `--mixed-precision` to accelerate visualization same as training/testing.
To generate videos from Cityscapes like our sample video, you can download the demo files from Cityscapes website, modify and run `tools/generate_cityscapes_demo.py`
To generate videos from TuSimple like our sample video, modify and run `tools/tusimple_to_video.py`
Colors can be customized for each dataset in `utils/datasets/`, although we recommend not doing that and keep the official colors for each dataset. If you are using custom data/custom model with more predicted classes but never did modify the code, the class colors will be used iteratively.
[vis_utils.py](../utils/vis_utils.py) contains batch-wise visualization functions to modify for your own use case.
You can set a `vis_dataset` dict in your config file to init the `ImageFolder` or `VideoLoader` classes from config file, otherwise it is automatically initiated by commandline args.
You can set a `vis` dict in your config file, this way the visualizer will use this dict's options instead of `test`.
For advanced visualization that includes actual coding, refer to [ADVANCED_TUTORIAL.md](./ADVANCED_TUTORIAL.md).

74
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/advanced/VISUALIZE_LANE_DATASETS.md Normal file
View File

@@ -0,0 +1,74 @@
# Visualize & Compare with GT on lane detection datasets
Here we make visualization functions work the same for specific dataset file structures by writing customized Dataset class and the optional `vis_dataset` dict in configs, trough examples.
## Expected Usage
*Note that TuSimple, CULane and LLAMAS are already supported in this manner.*
Write a config file that imports all options from the config file you want to visualize with, and provide a `vis_dataset` dict, e.g., [configs/lane_detection/bezierlanenet/vis_resnet34_tusimple_aug1b.py](../../configs/lane_detection/bezierlanenet/vis_resnet34_tusimple_aug1b.py):
```
from configs.lane_detection.bezierlanenet.resnet34_tusimple_aug1b import *
from configs.lane_detection.common.datasets._utils import TUSIMPLE_ROOT
vis_dataset = dict(
name='TuSimpleVis',
root_dataset=TUSIMPLE_ROOT,
root_output='./test_tusimple_vis',
keypoint_json="./output/resnet34_bezierlanenet-aug2_tusimple.json",
image_set='test'
)
```
Then simply run:
```
python tools/vis/lane_img_dir.py --config=configs/lane_detection/bezierlanenet/vis_resnet34_tusimple_aug1b.py --metric tusimple --style line
```
Beware `--style=bezier` can only be used with `--pred`. An example for CULane: [configs/lane_detection/bezierlanenet/vis_resnet34_culane_aug1b.py](../../configs/lane_detection/bezierlanenet/vis_resnet34_culane_aug1b.py)
## Implementation (TuSimple)
The `vis_dataset` can be used to replace `dataset` if specified, as shown in [get_loader()](https://github.com/voldemortX/pytorch-auto-drive/blob/cf314875ff0108f863b9ea8ac8d15141116b8f19/utils/runners/lane_det_visualizer.py#L92) of the default image folder runner. So instead of writing a new runner class, we can simply write a hybrid of [TuSimple](https://github.com/voldemortX/pytorch-auto-drive/blob/cf314875ff0108f863b9ea8ac8d15141116b8f19/utils/datasets/tusimple.py#L15) and [ImageFolderLaneDataset](https://github.com/voldemortX/pytorch-auto-drive/blob/cf314875ff0108f863b9ea8ac8d15141116b8f19/utils/datasets/image_folder.py#L56). It should be able to (functional behavior):
- extract images from the dataset path
- optionally load the dataset GT
- optionally load user-specified predictions
There are two major problems you'll need to address.
1. The training dataset do not load annotations for `val` and `test`, so you'll need to load them same as `train`, and you don't need to append `-2` for training, it would be something like this:
```
def preload_tusimple_labels(json_contents):
# Load a TuSimple label json's content
print('Loading json annotation/prediction...')
targets = []
for i in tqdm(range(len(json_contents))):
lines = json_contents[i]['lanes']
h_samples = json_contents[i]['h_samples']
temp = []
for j in range(len(lines)):
temp.append(np.array([[float(x), float(y)] for x, y in zip(lines[j], h_samples)]))
targets.append(temp)
return targets
```
2. The default pipeline uses a `keypoint_process_fn` that is passed dynamically in the runner (not specified in config dict), and processes only the CULane format keypoints in a per-file manner. For TuSimple, where labels are in a single file, it will do best to preload all of them (as above) and replace the `keypoint_process_fn` with a dummy function:
```
def dummy_keypoint_process_fn(label):
return label
# Then your class `__init__()` will start like this:
class TuSimpleVis(ImageFolderLaneBase):
def __init__(self, root_dataset, root_output, keypoint_json, image_set, transforms=None,
keypoint_process_fn=None, use_gt=True):
super().__init__(root_dataset, root_output, transforms, dummy_keypoint_process_fn)
```
With help of existing base classes, the full code is provided as reference in [utils/datasets/tusimple_vis.py](../../utils/datasets/tusimple_vis.py).

BIN
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/assets/vis_culane1.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 171 KiB

BIN
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/assets/vis_culane_gt.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 149 KiB

BIN
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/assets/vis_voc1.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

28
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/datasets/CITYSCAPES.md Normal file
View File

@@ -0,0 +1,28 @@
# Cityscapes
## Prepare the dataset
1. The dataset can be downloaded in their [official website](https://www.cityscapes-dataset.com/).
2. Change the `CITYSCAPES_ROOT` in [configs/semantic_segmentation/common/datasets/_utils.py](../../configs/semantic_segmentation/common/datasets/_utils.py) to your dataset's location.
3. Pre-processing:
```
python tools/cityscapes_data_list.py
```
## Description
### Directory Structure
```
├── <CITYSCAPES.BASE_DIR>
├── data_lists
├── gtFine
├── leftImage8bit
├── all_demoVideo
└── ...
```
*More details are coming soon.*

99
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/datasets/CULANE.md Normal file
View File

@@ -0,0 +1,99 @@
# CULane
## Prepare the dataset
1. The CULane dataset can be downloaded in their [official website](https://xingangpan.github.io/projects/CULane.html).
2. Change the `CULANE_ROOT` in [configs/lane_detection/common/datasets/_utils.py](../../configs/lane_detection/common/datasets/_utils.py) to your dataset's location..
3. Pre-processing:
```
cd <CULANE.BASE_DIR>
mkdir lists
cp -r ./list/* ./lists/
cd -
python tools/culane_list_convertor.py
```
4. Prepare official evaluation scripts (**require CP++ OpenCV**):
*comment Line 21 in Makefile if using opencv2.*
```
cd tools/culane_evaluation
make
mkdir output
chmod 777 eval*
cd -
```
Check [this issue](https://github.com/voldemortX/pytorch-auto-drive/issues/80) if you have OpenCV >= 4.0.
5. Prepare python evaluation scripts (**If you have difficulty compiling in 4.**):
```
cd tools/culane_evaluation_py
mkdir output
chmod 777 *.sh
# change $backend from cpp to python in autotest_culane.sh
```
Then change `data_dir` to your CULane base directory in [eval.sh](../../tools/culane_evaluation/eval.sh) and [eval_validation.sh](../../tools/culane_evaluation/eval_validation.sh). *Mind that you need extra ../../ if relative path is used.*
If you use Bézier curve methods, download Bézier curve GT from [here](https://drive.google.com/file/d/1s7N45IjUWxZEUIuyDUCIqtDfhzxRjmfL/view?usp=sharing) and unzip them in `CULANE_ROOT/bezier_labels/`. More info on curves are in [CURVE.md](../CURVE.md).
## Description
### Directory Structure
```
<CULANE.BASE_DIR>
├─ driver_100_30frame
├─ driver_161_90frame
├─ driver_182_30frame
├─ driver_193_90frame
├─ driver_23_30frame
├─ driver_37_30frame
├─ laneseg_label_w16
├─ bezier_labels
│ ├─ train_3.json
│ └─ ...
├─ list
└─ lists
```
### Label Data Format
```
x1, y, x2, y-10, x3, y-20, ... , xn, y-10(n-1)
```
For each image, there would be a .txt annotation file, in which each line gives the x,y coordinates for key points of a lane marking. The CULane dataset, focus attention on the detection of four lane markings, which are paid most attention to in real applications.
For example,
```
-20.4835 580 19.3893 570 58.1682 560 98.1783 550 137.929 540 177.709 530 216.495 520 256.512 510 296.276 500 336.008 490 375.78 480 415.941 470 456.696 460 496.456 450 537.226 440 577.47 430 618.252 420 659.177 410
532.893 590 542.567 580 553.704 570 564.84 560 575.977 550 587.139 540 598.302 530 609.465 520 620.628 510 631.944 500 643.107 490 654.27 480 665.432 470 676.595 460 687.912 450 699.075 440 710.237 430 721.4 420 732.563 410
1170.27 590 1151.37 580 1130.68 570 1110.6 560 1089.91 550 1068.95 540 1048.26 530 1027.56 520 1007.77 510 986.81 500 966.115 490 945.788 480 925.092 470 904.506 460 883.81 450 863.13 440 842.434 430 821.739 420 801.059 410
1679.87 560 1626.23 550 1574.15 540 1520.62 530 1467.55 520 1414.57 510 1361.5 500 1307.8 490 1255.24 480 1202.18 470 1149.46 460 1096.4 450 1042.63 440 990.059 430 936.993 420 884.22 410
```
*Each row in xxx.txt represents a lane mark.*
Training/validation/testing list:
For train_gt.txt, which is used for training.
```
input image per-pixel label four 0/1 numbers which indicate the existance of four lane markings from left to right
```
For example,
```
/driver_23_30frame/05151649_0422.MP4/00000.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00000.png 1 1 1 1
/driver_23_30frame/05151649_0422.MP4/00300.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00300.png 1 1 1 1
...
/driver_23_30frame/05151649_0422.MP4/00330.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00330.png 1 1 1 1
```

26
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/datasets/GTAV.md Normal file
View File

@@ -0,0 +1,26 @@
# GTAV
## Prepare the dataset
1. The dataset can be downloaded in their [official website](https://download.visinf.tu-darmstadt.de/data/from_games/).
2. Change the `GTAV_ROOT` in [configs/semantic_segmentation/common/datasets/_utils.py](../../configs/semantic_segmentation/common/datasets/_utils.py) to your dataset's location.
3. Pre-processing:
```
python tools/gtav_data_list.py
```
## Description
### Directory Structure
```
├── <GTAV.BASE_DIR>
├── data_lists
├── images
└── labels
```
*More details are coming soon.*

88
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/datasets/LLAMAS.md Normal file
View File

@@ -0,0 +1,88 @@
# LLAMAS
## Prepare the dataset
1. The LLAMAS dataset can be downloaded in their [official website](https://unsupervised-llamas.com/llamas/).
2. Change the `LLAMAS_ROOT` in [configs/lane_detection/common/datasets/_utils.py](../../configs/lane_detection/common/datasets/_utils.py) to your dataset's location..
3. Pre-processing:
```
python tools/llamas_list_convertor.py
```
LLAMAS dataset provides both color and gray images. We use color images in our framework.
4. Prepare official evaluation scripts:
```
cd tools/llamas_evaluation
mkdir output
```
Then change `data_dir` to your LLAMAS base directory in [autotest_llamas.sh](../../autotest_llamas.sh). *Mind that you need extra ../../ if relative path is used.*
5. If you use Bézier curve methods, download Bézier curve GT from [here](https://drive.google.com/file/d/1klugsr5lT9SxW0r5oyYjfo2qplt1TyxU/view?usp=sharing) and unzip them in `LLAMAS_ROOT/bezier_labels/`. More info on curves are in [CURVE.md](../CURVE.md).
## Description
### Directory Structure
```
<LLAMAS.BASE_DIR>
├─ color_images
│ ├─ test
│ ├─ train
│ └─ valid
├─ labels
│ ├─ train
│ └─ valid
├─ laneseg_labels
│ ├─ train
│ └─ valid
├─ bezier_labels
│ ├─ train_3.json
│ └─ ...
└─ lists
```
The test set' s annotations are not public.
### Label Data Format
```
{
"image_name": "...",
"projection_matrix": [[x11, 0, x13], [0, x21, x23], [0, 0, 1]],
"lanes":
[
{
"lane_id": "...",
"markers":
[
{
"lane_marker_id": "...",
"world_start":{"x":"...", "y": "...", "z": "..."}, "pixel_start": {"x": "...", "y": "..."},
"world_end":{"x":"...", "y": "...", "z": "..."}, "pixel_start": {"x": "...", "y": "..."}}
},
...
]
},
...
]
}
```
LLAMAS dataset employs json files to save annotations. Each image corresponds to a json file.
We utilize [the format of culane](CULANE.md) to reformat LLAMAS dataset.
We use the [script](https://github.com/XingangPan/seg_label_generate) provided by Xingang Pan to generate per-pixel labels.
The generated labels can be downloaded from [Google Drive](https://drive.google.com/file/d/1XA4nRLuAzsjJUSUs4HCjz7dksI9dHDNd/view?usp=sharing).

25
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/datasets/PASCALVOC.md Normal file
View File

@@ -0,0 +1,25 @@
# PASCAL VOC 2012
## Prepare the dataset
1. The PASCAL VOC 2012 dataset we use is the commonly used 10582 training set version. If you don't already have that dataset, we refer you to [Google](https://www.google.com) or this [blog](https://www.sun11.me/blog/2018/how-to-use-10582-trainaug-images-on-DeeplabV3-code/).
2. Change the `PASCAL_ROOT` in [configs/semantic_segmentation/common/datasets/_utils.py](../../configs/semantic_segmentation/common/datasets/_utils.py) to your dataset's location.
## Description
### Directory Structure
```
├── <PASCAL_VOC.BASE_DIR>/VOCtrainval_11-May-2012/VOCdevkit/VOC2012
├── Annotations
├── ImageSets
│ ├── Segmentation
│ └── ...
├── JPEGImages
├── SegmentationClass
├── SegmentationClassAug
└── ...
```
*More details are coming soon.*

28
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/datasets/SYNTHIA.md Normal file
View File

@@ -0,0 +1,28 @@
# SYNTHIA
## Prepare the dataset
1. The dataset can be downloaded in their [official website](http://synthia-dataset.net/downloads/).
2. Change the `SYNTHIA_ROOT` in [configs/semantic_segmentation/common/datasets/_utils.py](../../configs/semantic_segmentation/common/datasets/_utils.py) to your dataset's location.
3. Pre-processing:
```
python tools/synthia_label_convertor.py
python tools/synthia_data_list.py
```
## Description
### Directory Structure
```
├── <SYNTHIA.BASE_DIR>
├── data_lists
├── RGB
├── GT
└── ...
```
*More details are coming soon.*

79
algorithms/lane_ufld/code/pytorch-auto-drive-master/docs/datasets/TUSIMPLE.md Normal file
View File

@@ -0,0 +1,79 @@
Do not overwrite folders with the same name, as the content of the training and testing sets is different
# Tusimple
## Prepare the dataset
1. The TuSimple dataset can be downloaded at their [github repo](https://github.com/TuSimple/tusimple-benchmark/issues/3). However, you'll also need [segmentation labels](https://drive.google.com/open?id=1LZDCnr79zuNH73NstZ8oIPDud0INCwb9), [list6_train.txt](https://github.com/cardwing/Codes-for-Lane-Detection/blob/master/ENet-TuSimple-Torch/list6/list6_train.txt), [list6_val.txt](https://github.com/cardwing/Codes-for-Lane-Detection/blob/master/ENet-TuSimple-Torch/list6/list6_val.txt) and [list_test.txt](https://github.com/cardwing/Codes-for-Lane-Detection/blob/master/ENet-TuSimple-Torch/list/list_test.txt) provided by [@cardwing](https://github.com/cardwing), thanks for their efforts.
2. Change the `TUSIMPLE_ROOT` in [configs/lane_detection/common/datasets/_utils.py](../../configs/lane_detection/common/datasets/_utils.py) to your dataset's location.
3. Pre-processing:
First put the data lists you downloaded before in `TUSIMPLE.BASE_DIR/lists`. Then:
```
python tools/tusimple_list_convertor.py
```
4. Prepare official evaluation scripts:
```
cd tools/tusimple_evaluation
mkdir output
```
Then change `data_dir` to your TuSimple base directory in [autotest_tusimple.sh](../../autotest_tusimple.sh). *Mind that you need extra ../../ if relative path is used.*
5. If you use Bézier curve methods, download Bézier curve GT from [here](https://drive.google.com/file/d/1aV1e5MAReIvtgW8RCoOMnvCnAK6uYtwn/view?usp=sharing) and unzip them in `TUSIMPLE_ROOT/bezier_labels/`. More info on curves are in [CURVE.md](../CURVE.md).
## Description
### Directory Structure
Note that the structure of Tusimple dataset downloaded from kaggle is different from the original structure. It contains `train_set` folder, `test_set` folder. You need to move the clips in the `test_set` folder to the `train_set` folder. **Do not overwrite folders with the same name**, as the content of the training and testing sets is different.
```
<TUSIMPLE.BASE_DIR>
├─ clips
├─ lists
├─ segGT6
├─ label_data_0313.json
├─ label_data_0531.json
├─ label_data_0601.json
├─ bezier_labels
│ ├─ train_3.json
│ └─ ...
└─ test_label.json
```
### Label Data Format
```
{
'lanes': list. A list of lanes. For each list of one lane, the elements are width values on image.
'h_samples': list. A list of height values corresponding to the 'lanes', which means len(h_samples) == len(lanes[i])
'raw_file': str. 20th frame file path in a clip.
}
```
There will be at most 5 lane markings in `lanes`. The polylines are organized by the same distance gap (`h_sample` in each label data) from the recording car. It means you can pair each element in one lane and h_samples to get position of lane marking in images.
For example,
```
{
"lanes": [
[-2, -2, -2, -2, 632, 625, 617, 609, 601, 594, 586, 578, 570, 563, 555, 547, 539, 532, 524, 516, 508, 501, 493, 485, 477, 469, 462, 454, 446, 438, 431, 423, 415, 407, 400, 392, 384, 376, 369, 361, 353, 345, 338, 330, 322, 314, 307, 299],
[-2, -2, -2, -2, 719, 734, 748, 762, 777, 791, 805, 820, 834, 848, 863, 877, 891, 906, 920, 934, 949, 963, 978, 992, 1006, 1021, 1035, 1049, 1064, 1078, 1092, 1107, 1121, 1135, 1150, 1164, 1178, 1193, 1207, 1221, 1236, 1250, 1265, -2, -2, -2, -2, -2],
[-2, -2, -2, -2, -2, 532, 503, 474, 445, 416, 387, 358, 329, 300, 271, 241, 212, 183, 154, 125, 96, 67, 38, 9, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2],
[-2, -2, -2, 781, 822, 862, 903, 944, 984, 1025, 1066, 1107, 1147, 1188, 1229, 1269, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2]
],
"h_samples": [240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710],
"raw_file": "path_to_clip"
}
```
*`-2` in `lanes` means on some h_sample, there is no exsiting lane marking. The first existing point in the first lane is (632, 280).*
## Segmentation Label Generation \[Advanced\]
There is no precise corresponding generation script for the provided segmentation labels, although they are in good quality. If you plan to generate segmentation labels yourself, or simply can't download from Google Drive, refer to [#40](https://github.com/voldemortX/pytorch-auto-drive/issues/40) for links to [Baidu Drive](https://github.com/voldemortX/pytorch-auto-drive/issues/40#issuecomment-978728543), [C++ scripts](https://github.com/XingangPan/seg_label_generate) and [Python scripts](https://github.com/ZJULearning/resa/blob/main/tools/generate_seg_tusimple.py).