yolov26_3d/eval_tools/docs/TWO_LEVEL_PATH_SUPPORT.md

# Two-Level Path Support for Model Evaluation

## Overview

The evaluation system now supports both 1-level and 2-level directory structures for detection results and ground truth labels. This allows for more flexible organization of test data.

## Directory Structures

### 1-Level Structure (Default)

```
det_root/
    case1/
        txt_results/
            frame001.txt
            frame002.txt
    case2/
        txt_results/
            frame001.txt

gt_root/
    case1/
        labels/
            frame001.txt
            frame002.txt
    case2/
        labels/
            frame001.txt
```

### 2-Level Structure

```
det_root/
    level1_dir/
        case1/
            txt_results/
                frame001.txt
                frame002.txt
        case2/
            txt_results/
                frame001.txt
    level2_dir/
        case3/
            txt_results/
                frame001.txt

gt_root/
    level1_dir/
        case1/
            labels/
                frame001.txt
                frame002.txt
        case2/
            labels/
                frame001.txt
    level2_dir/
        case3/
            labels/
                frame001.txt
```

## Configuration

### YAML Configuration File

Add the `path_depth` parameter to your config file:

```yaml
dataset:
  det_path: "/path/to/detection/results"
  gt_path: "/path/to/ground/truth"
  path_depth: 2  # Set to 1 for 1-level, 2 for 2-level structure
```

**Example for 1-level structure:**
```yaml
dataset:
  det_path: "/data1/dongying/Mono3d/G1M3/CNCAP_results/mono3d/evalset_roi0"
  gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Testdata"
  path_depth: 1  # Default
```

**Example for 2-level structure:**
```yaml
dataset:
  det_path: "/data1/dongying/Mono3d/G1M3/CNCAP_results/mono3d"
  gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png"
  path_depth: 2
```

### Command-Line Arguments

You can also specify the path depth via command line:

```bash
# 1-level structure (default)
python eval_tools/core/eval.py \
  --config eval_tools/configs/eval_config_mono3d.yaml

# 2-level structure
python eval_tools/core/eval.py \
  --config eval_tools/configs/eval_config_mono3d.yaml \
  --path-depth 2

# Or without config file
python eval_tools/core/eval.py \
  --det-path /path/to/detections \
  --gt-path /path/to/labels \
  --path-depth 2 \
  --output-dir results
```

## How It Works

### 1-Level Mode (`path_depth=1`)

1. Scans `det_root` for all subdirectories (cases)
2. For each case, looks for `txt_results/` subdirectory
3. Matches with corresponding `gt_root/case/labels/` directory

### 2-Level Mode (`path_depth=2`)

1. Scans `det_root` for all subdirectories (level1 directories)
2. For each level1 directory, scans for case subdirectories
3. For each case, looks for `txt_results/` subdirectory
4. Matches with corresponding `gt_root/level1/case/labels/` directory

**Important:** The level1 directory names must match between detection and ground truth paths.

## Model Comparison Script

The comparison script automatically inherits the `path_depth` setting from the config files:

```bash
bash eval_tools/model_comparison/compare_models_with_common_matches.sh
```

The script will:
1. Read `path_depth` from `eval_config_mono3d.yaml` for Model 1
2. Read `path_depth` from `eval_config_yolov5s.yaml` for Model 2
3. Evaluate both models with their respective path structures
4. Compare results using common matches

## Examples

### Example 1: Evaluating with 2-level structure

```bash
# Update your config file
cat > eval_tools/configs/eval_config_2level.yaml << EOF
dataset:
  det_path: "/data/results/all_models"
  gt_path: "/data/ground_truth/all_datasets"
  path_depth: 2

image:
  width: 1920
  height: 1080

# ... other settings ...
EOF

# Run evaluation
python eval_tools/core/eval.py --config eval_tools/configs/eval_config_2level.yaml
```

### Example 2: Comparing models with different path structures

```yaml
# Model 1 config (1-level)
dataset:
  det_path: "/data/model1/results"
  gt_path: "/data/gt"
  path_depth: 1

# Model 2 config (2-level)
dataset:
  det_path: "/data/model2/results"
  gt_path: "/data/gt_organized"
  path_depth: 2
```

Both models can be compared even with different directory structures.

## Backward Compatibility

- If `path_depth` is not specified, it defaults to `1` (1-level structure)
- All existing config files and scripts continue to work without modification
- The system automatically detects and handles both structures

## Troubleshooting

### Issue: "GT case directory not found"

**Cause:** Level1 directory names don't match between detection and ground truth paths.

**Solution:** Ensure that the intermediate directory names are identical:
```
det_root/dataset_A/case1/  ← "dataset_A" must match
gt_root/dataset_A/case1/   ← "dataset_A" must match
```

### Issue: "No image pairs found"

**Cause:** Incorrect `path_depth` setting.

**Solution:**
- Check your actual directory structure
- Set `path_depth: 1` for `root/case/txt_results`
- Set `path_depth: 2` for `root/level1/case/txt_results`

### Issue: Cases are being skipped

**Cause:** Missing `txt_results/` or `labels/` subdirectories.

**Solution:** Verify that each case directory contains:
- Detection: `case/txt_results/*.txt`
- Ground truth: `case/labels/*.txt`

## Implementation Details

The changes are implemented in:
- `eval_tools/evaluator/evaluator.py`: Modified `load_data_from_paths()` method
- `eval_tools/core/eval.py`: Added `--path-depth` argument and config support
- `eval_tools/configs/*.yaml`: Added `path_depth` parameter

The implementation maintains full backward compatibility while adding support for 2-level structures.