单目3D初始代码

eval_tools/analysis/README_2D_FP_FN.md

@@ -0,0 +1,129 @@
+# 2D FP/FN 分析工具
+
+本目录用于分析单模型 2D 检测中的误检（FP）和漏检（FN），帮助把总指标拆成更可操作的问题类型。
+
+## 文件
+
+- `analyze_2d_fp_fn.py`
+  读取检测结果和 GT，复用现有 `Evaluator` / `parser` / `matcher`，输出 2D FP/FN 分类分析报告。
+- `export_2d_fp_fn_badcases.py`
+  从 `analysis_report.json` 中筛选指定类别、错误类型、距离或置信度范围，导出 badcase 清单。
+
+## FP 分类
+
+- `duplicate`
+  同类 GT 其实已经被更高分框匹配走了，当前框属于重复检出。
+- `class_confusion`
+  框和其他类别 GT 有较高 IoU，说明更像分类混淆。
+- `localization`
+  和同类 GT 靠得比较近，但 IoU 没过主阈值，更像框偏了。
+- `background`
+  附近没有足够重叠的 GT，更像背景误检或漏标。
+
+## FN 分类
+
+- `class_confusion`
+  其他类别的激活框和该 GT 重叠较高。
+- `low_score`
+  同类框位置够准，但分数低于工作点阈值。
+- `localization`
+  同类高分框在附近，但 IoU 没过主阈值。
+- `low_score_localization`
+  同类低分框在附近，但又低分又偏框。
+- `missing`
+  附近没有合理同类框。
+
+## 运行分析
+
+推荐直接复用现有评测配置：
+
+```bash
+source /deeplearning_team/ydong/dongying/miniconda/etc/profile.d/conda.sh
+conda activate yolov5
+
+python eval_tools/analysis/analyze_2d_fp_fn.py \
+  --config eval_tools/configs/eval_config_mono3d-roi0.yaml \
+  --classes vehicle pedestrian bicycle rider \
+  --near-iou-threshold 0.1 \
+  --output-dir eval_tools/analysis/results/mono3d_fp_fn
+```
+
+也可以直接传路径：
+
+```bash
+python eval_tools/analysis/analyze_2d_fp_fn.py \
+  --det-path /path/to/dets \
+  --gt-path /path/to/gts \
+  --det-format json \
+  --gt-format json \
+  --img-width 1920 \
+  --img-height 1080 \
+  --iou-threshold 0.5 \
+  --conf-threshold 0.4
+```
+
+## 输出
+
+运行后会生成：
+
+- `analysis_report.json`
+  完整结构化结果，包含 summary、top frames、FP/FN example 明细。
+- `analysis_report.txt`
+  适合快速查看的文本摘要。
+
+JSON 中最常用的字段：
+
+- `summary.fp_by_type`
+- `summary.fn_by_type`
+- `summary.per_class`
+- `top_frames`
+- `false_positive_examples`
+- `false_negative_examples`
+
+## 导出 badcase 清单
+
+例如导出 `vehicle` 的高置信背景误检：
+
+```bash
+python eval_tools/analysis/export_2d_fp_fn_badcases.py \
+  --input eval_tools/analysis/results/mono3d_fp_fn/analysis_report.json \
+  --mode fp \
+  --classes vehicle \
+  --error-types background \
+  --min-confidence 0.5 \
+  --top-k 200 \
+  --dedup-frame
+```
+
+例如导出 `pedestrian` 的低分漏检：
+
+```bash
+python eval_tools/analysis/export_2d_fp_fn_badcases.py \
+  --input eval_tools/analysis/results/mono3d_fp_fn/analysis_report.json \
+  --mode fn \
+  --classes pedestrian \
+  --error-types low_score low_score_localization \
+  --top-k 200
+```
+
+导出结果包括：
+
+- `*_badcases.json`
+  过滤后的明细
+- `*_badcases.txt`
+  文本摘要
+- `*_badcases_case_frame_list.txt`
+  每行一个 `case<TAB>frame`，方便接可视化或人工排查
+
+## 推荐排查流程
+
+1. 先看 `summary.fp_by_type` / `summary.fn_by_type`，确认主要矛盾是误检、漏检、分类混淆还是定位偏差。
+2. 再看 `summary.per_class`，确认问题集中在哪些类别。
+3. 用 `export_2d_fp_fn_badcases.py` 筛出某个错误桶的 top case。
+4. 把导出的 `case/frame` 清单接到你们现有可视化脚本里做人工复核。
+
+## 注意
+
+- 当前分析严格复用了评测时的 ROI GT 过滤和 `Matcher2D` 规则，因此口径会尽量和正式评测保持一致。
+- `background` 不能完全等价于“真的背景误检”，也可能包含漏标样本，最好配合人工复查。
+- `duplicate` 往往与 NMS、同物体多框、训练标签分布有关。

eval_tools/analysis/analyze_2d_fp_fn.sh

@@ -0,0 +1,8 @@
+NUM_WORKERS=${NUM_WORKERS:-32}
+
+python eval_tools/analysis/analyze_2d_fp_fn.py \
+  --config eval_tools/configs/eval_config_yolov26s-roi0.yaml \
+  --classes car \
+  --near-iou-threshold 0.1 \
+  --num-workers "${NUM_WORKERS}" \
+  --output-dir /data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_2d/mono3d_fp_fn-roi0

eval_tools/analysis/analyze_3d.md

@@ -0,0 +1,5 @@
+evaluation_results/eval_results_yolo26s_768_20260407_DL_KPI_SCENE/yolo26s-20260407-conf0.27/20260410_162018_roi0/evaluation_report.json
+
+上述是一组模型评测报告，对于报告中的2d问题的进一步分析，目前有eval_tools/analysis/analyze_2d_fp_fn.py和eval_tools/analysis/visualize_2d_fn_cases.py脚本进行分析。
+
+现在期望有对3d指标的进一步分析，比如报告和可视化结果等。

eval_tools/analysis/analyze_3d_badcases.py

@@ -0,0 +1,799 @@
+#!/usr/bin/env python3
+"""
+Analyze 3D bad cases from saved detailed matches or by rebuilding matches.
+
+Preferred input is ``detailed_3d_matches.json`` produced by the evaluator.
+If that file is unavailable, this tool can rebuild the detailed matches from
+the evaluation config and the underlying det/gt directories.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import math
+import sys
+from collections import Counter, defaultdict
+from datetime import datetime
+from pathlib import Path
+from statistics import mean, median
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from eval_tools.analysis.analyze_2d_fp_fn import (
+    build_config,
+    build_case_key,
+    class_name,
+    parse_class_ids,
+    round_float,
+)
+from eval_tools.evaluator.evaluator import Evaluator
+
+
+DEFAULT_DISTANCE_RANGES = [
+    [0, 10],
+    [10, 20],
+    [20, 30],
+    [30, 40],
+    [40, 50],
+    [50, 60],
+    [60, 70],
+    [70, 80],
+    [80, 90],
+    [90, 100],
+    [100, 999],
+]
+DEFAULT_LATERAL_DISTANCE_RANGES = [
+    [-50, -40],
+    [-40, -30],
+    [-30, -20],
+    [-20, -10],
+    [-10, 0],
+    [0, 10],
+    [10, 20],
+    [20, 30],
+    [30, 40],
+    [40, 50],
+]
+METRIC_KEYS = (
+    "lateral_error",
+    "longitudinal_error",
+    "longitudinal_relative_error",
+    "heading_error",
+    "heading_error_relaxed",
+    "reversal",
+)
+CSV_FIELDNAMES = [
+    "case_name",
+    "frame_name",
+    "class_id",
+    "class_name",
+    "gt_id",
+    "confidence",
+    "iou",
+    "distance_longitudinal_m",
+    "distance_lateral_m",
+    "distance_bin",
+    "lateral_bin",
+    "metric_name",
+    "metric_value",
+    "metric_value_display",
+    "is_reversal",
+    "lateral_error_m",
+    "longitudinal_error_m",
+    "longitudinal_relative_error",
+    "heading_error_rad",
+    "heading_error_deg",
+    "heading_error_relaxed_rad",
+    "heading_error_relaxed_deg",
+    "gt_bbox",
+    "det_bbox",
+    "gt_center_3d",
+    "det_center_3d",
+    "gt_rotation_rad",
+    "det_rotation_rad",
+]
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Analyze 3D bad cases from detailed_3d_matches.json or rebuild them from config."
+    )
+    parser.add_argument(
+        "--detailed-matches",
+        type=str,
+        default=None,
+        help="Path to detailed_3d_matches.json generated by evaluator.",
+    )
+    parser.add_argument(
+        "--evaluation-report",
+        type=str,
+        default=None,
+        help="Path to evaluation_report.json. Used to infer sibling detailed_3d_matches.json.",
+    )
+    parser.add_argument("--config", type=str, default=None, help="Path to YAML evaluation config.")
+    parser.add_argument("--det-path", type=str, help="Detection results root directory")
+    parser.add_argument("--gt-path", type=str, help="Ground-truth labels root directory")
+    parser.add_argument("--path-depth", type=int, choices=[1, 2], help="Directory depth")
+    parser.add_argument(
+        "--det-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Detection file format",
+    )
+    parser.add_argument(
+        "--gt-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Ground-truth file format",
+    )
+    parser.add_argument("--img-width", type=int, help="Image width")
+    parser.add_argument("--img-height", type=int, help="Image height")
+    parser.add_argument(
+        "--coord-system",
+        type=str,
+        choices=["camera", "ego"],
+        help="Coordinate system used by the parser/evaluator",
+    )
+    parser.add_argument(
+        "--iou-threshold",
+        type=float,
+        help="IoU threshold used for evaluator loading",
+    )
+    parser.add_argument(
+        "--conf-threshold",
+        type=float,
+        help="Confidence threshold for rebuilding detailed matches",
+    )
+    parser.add_argument(
+        "--classes",
+        nargs="+",
+        default=None,
+        help="Optional class filter, e.g. car suv pedestrian or numeric IDs",
+    )
+    parser.add_argument(
+        "--metrics",
+        nargs="+",
+        default=list(METRIC_KEYS),
+        choices=METRIC_KEYS,
+        help="Metrics to rank and export.",
+    )
+    parser.add_argument(
+        "--top-k",
+        type=int,
+        default=200,
+        help="Top bad cases to keep per metric overall.",
+    )
+    parser.add_argument(
+        "--top-k-per-class",
+        type=int,
+        default=100,
+        help="Top bad cases to keep per metric and class.",
+    )
+    parser.add_argument(
+        "--top-k-frames",
+        type=int,
+        default=50,
+        help="Number of worst frames to keep in the summary.",
+    )
+    parser.add_argument(
+        "--min-confidence",
+        type=float,
+        default=None,
+        help="Optional minimum confidence filter on matched detections.",
+    )
+    parser.add_argument(
+        "--max-confidence",
+        type=float,
+        default=None,
+        help="Optional maximum confidence filter on matched detections.",
+    )
+    parser.add_argument(
+        "--min-iou",
+        type=float,
+        default=None,
+        help="Optional minimum 2D IoU filter on matched samples.",
+    )
+    parser.add_argument(
+        "--max-iou",
+        type=float,
+        default=None,
+        help="Optional maximum 2D IoU filter on matched samples.",
+    )
+    parser.add_argument(
+        "--bad-lateral-threshold",
+        type=float,
+        default=1.0,
+        help="Threshold in meters for counting bad lateral errors.",
+    )
+    parser.add_argument(
+        "--bad-longitudinal-threshold",
+        type=float,
+        default=3.0,
+        help="Threshold in meters for counting bad longitudinal errors.",
+    )
+    parser.add_argument(
+        "--bad-longitudinal-relative-threshold",
+        type=float,
+        default=0.2,
+        help="Threshold for counting bad longitudinal relative errors.",
+    )
+    parser.add_argument(
+        "--bad-heading-threshold-deg",
+        type=float,
+        default=15.0,
+        help="Threshold in degrees for counting bad heading errors.",
+    )
+    parser.add_argument(
+        "--num-workers",
+        type=int,
+        default=None,
+        help="Worker count for rebuilding detailed matches (default: evaluator auto-detect).",
+    )
+    parser.add_argument(
+        "--save-rebuilt-matches",
+        action="store_true",
+        help="When rebuilding detailed matches, also save them into the output directory.",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="Output directory. Defaults to eval_tools/analysis/results_3d/<timestamp>.",
+    )
+    return parser.parse_args()
+
+
+def metric_display_value(metric_name, sample):
+    if metric_name == "lateral_error":
+        return float(sample["lateral_error_m"])
+    if metric_name == "longitudinal_error":
+        return float(sample["longitudinal_error_m"])
+    if metric_name == "longitudinal_relative_error":
+        return float(sample["longitudinal_relative_error"])
+    if metric_name == "heading_error":
+        return float(sample["heading_error_deg"])
+    if metric_name == "heading_error_relaxed":
+        return float(sample["heading_error_relaxed_deg"])
+    if metric_name == "reversal":
+        return 1.0 if sample["is_reversal"] else 0.0
+    raise KeyError(f"Unsupported metric: {metric_name}")
+
+
+def metric_raw_value(metric_name, sample):
+    if metric_name == "lateral_error":
+        return float(sample["lateral_error_m"])
+    if metric_name == "longitudinal_error":
+        return float(sample["longitudinal_error_m"])
+    if metric_name == "longitudinal_relative_error":
+        return float(sample["longitudinal_relative_error"])
+    if metric_name == "heading_error":
+        return float(sample["heading_error_rad"])
+    if metric_name == "heading_error_relaxed":
+        return float(sample["heading_error_relaxed_rad"])
+    if metric_name == "reversal":
+        return 1.0 if sample["is_reversal"] else 0.0
+    raise KeyError(f"Unsupported metric: {metric_name}")
+
+
+def metric_unit(metric_name):
+    if metric_name in ("lateral_error", "longitudinal_error"):
+        return "m"
+    if metric_name in ("heading_error", "heading_error_relaxed"):
+        return "deg"
+    return ""
+
+
+def threshold_hit(metric_name, sample, thresholds):
+    if metric_name == "lateral_error":
+        return sample["lateral_error_m"] >= thresholds["bad_lateral_threshold"]
+    if metric_name == "longitudinal_error":
+        return sample["longitudinal_error_m"] >= thresholds["bad_longitudinal_threshold"]
+    if metric_name == "longitudinal_relative_error":
+        return sample["longitudinal_relative_error"] >= thresholds["bad_longitudinal_relative_threshold"]
+    if metric_name in ("heading_error", "heading_error_relaxed"):
+        limit = thresholds["bad_heading_threshold_deg"]
+        key = "heading_error_deg" if metric_name == "heading_error" else "heading_error_relaxed_deg"
+        return sample[key] >= limit
+    if metric_name == "reversal":
+        return bool(sample["is_reversal"])
+    raise KeyError(f"Unsupported metric: {metric_name}")
+
+
+def make_stats(values):
+    if not values:
+        return {"mean": 0.0, "median": 0.0, "std": 0.0, "percentile_90": 0.0}
+
+    values = [float(v) for v in values]
+    avg = mean(values)
+    med = median(values)
+    variance = sum((v - avg) ** 2 for v in values) / len(values)
+    values_sorted = sorted(values)
+    p90_index = min(len(values_sorted) - 1, max(0, math.ceil(0.9 * len(values_sorted)) - 1))
+    return {
+        "mean": round_float(avg),
+        "median": round_float(med),
+        "std": round_float(math.sqrt(variance)),
+        "percentile_90": round_float(values_sorted[p90_index]),
+    }
+
+
+def bucket_label(prefix, range_pair):
+    return f"{prefix}_{range_pair[0]}-{range_pair[1]}m"
+
+
+def build_distance_ranges(config):
+    metrics_cfg = config.get("metrics_3d", {}) if config else {}
+    return metrics_cfg.get("distance_ranges") or DEFAULT_DISTANCE_RANGES
+
+
+def build_lateral_ranges(config):
+    metrics_cfg = config.get("metrics_3d", {}) if config else {}
+    return metrics_cfg.get("lateral_distance_ranges") or DEFAULT_LATERAL_DISTANCE_RANGES
+
+
+def classify_range(value, prefix, ranges):
+    if value is None:
+        return None
+    for range_pair in ranges:
+        lo, hi = float(range_pair[0]), float(range_pair[1])
+        if lo <= float(value) < hi:
+            return bucket_label(prefix, [int(lo) if lo.is_integer() else lo, int(hi) if hi.is_integer() else hi])
+    return None
+
+
+def infer_detailed_matches_path(args):
+    if args.detailed_matches:
+        return Path(args.detailed_matches)
+    if args.evaluation_report:
+        report_path = Path(args.evaluation_report)
+        sibling = report_path.parent / "detailed_3d_matches.json"
+        if sibling.exists():
+            return sibling
+    return None
+
+
+def load_saved_detailed_matches(path):
+    with open(path, "r") as file:
+        return json.load(file)
+
+
+def rebuild_detailed_matches(args):
+    config = build_config(args)
+    evaluator = Evaluator(
+        config=config,
+        iou_threshold=float(config.get("matching", {}).get("iou_threshold", 0.5)),
+        num_workers=args.num_workers,
+        save_detailed_matches=True,
+    )
+    dataset_cfg = config["dataset"]
+    image_cfg = config["image"]
+    evaluator.load_data_from_paths(
+        det_root=dataset_cfg["det_path"],
+        gt_root=dataset_cfg["gt_path"],
+        img_width=image_cfg.get("width", 1920),
+        img_height=image_cfg.get("height", 1080),
+        path_depth=dataset_cfg.get("path_depth", 1),
+        det_format=dataset_cfg.get("det_format", "auto"),
+        gt_format=dataset_cfg.get("gt_format", "auto"),
+    )
+    evaluator.evaluate_3d()
+    return evaluator.detailed_3d_matches or {}, config
+
+
+def load_detailed_matches(args):
+    detailed_path = infer_detailed_matches_path(args)
+    config = build_config(args) if (
+        args.config
+        or args.det_path
+        or args.gt_path
+        or args.path_depth is not None
+        or args.det_format
+        or args.gt_format
+        or args.img_width is not None
+        or args.img_height is not None
+        or args.coord_system
+        or args.iou_threshold is not None
+        or args.conf_threshold is not None
+    ) else None
+
+    if detailed_path and detailed_path.exists():
+        return load_saved_detailed_matches(detailed_path), detailed_path, config
+
+    if config is None:
+        raise FileNotFoundError(
+            "Failed to locate detailed_3d_matches.json. Please provide --detailed-matches, "
+            "--evaluation-report with a sibling matches file, or --config to rebuild matches."
+        )
+
+    matches, rebuilt_config = rebuild_detailed_matches(args)
+    return matches, None, rebuilt_config
+
+
+def collect_samples(detailed_matches, class_ids, distance_ranges, lateral_ranges, args):
+    selected_class_names = {class_name(class_id) for class_id in class_ids}
+    samples = []
+    for case_name, frames in detailed_matches.items():
+        for frame_name, class_groups in frames.items():
+            for class_name_str, items in class_groups.items():
+                if class_name_str not in selected_class_names:
+                    continue
+                class_id = next((cid for cid in class_ids if class_name(cid) == class_name_str), None)
+                if class_id is None:
+                    continue
+                for item in items:
+                    confidence = float(item.get("confidence", 0.0))
+                    iou = float(item.get("iou", 0.0))
+                    if args.min_confidence is not None and confidence < args.min_confidence:
+                        continue
+                    if args.max_confidence is not None and confidence > args.max_confidence:
+                        continue
+                    if args.min_iou is not None and iou < args.min_iou:
+                        continue
+                    if args.max_iou is not None and iou > args.max_iou:
+                        continue
+
+                    distance = item.get("distance") or {}
+                    errors = item.get("errors") or {}
+                    longitudinal_distance = distance.get("longitudinal")
+                    lateral_distance = distance.get("lateral")
+                    sample = {
+                        "case_name": case_name,
+                        "frame_name": frame_name,
+                        "class_id": class_id,
+                        "class_name": class_name_str,
+                        "gt_id": str(item.get("gt_id")),
+                        "confidence": round_float(confidence),
+                        "iou": round_float(iou),
+                        "distance_longitudinal_m": None if longitudinal_distance is None else round_float(longitudinal_distance),
+                        "distance_lateral_m": None if lateral_distance is None else round_float(lateral_distance),
+                        "distance_bin": classify_range(longitudinal_distance, "long", distance_ranges),
+                        "lateral_bin": classify_range(lateral_distance, "lat", lateral_ranges),
+                        "gt_bbox": [round_float(v) for v in item.get("gt_bbox", [])],
+                        "det_bbox": [round_float(v) for v in item.get("det_bbox", [])],
+                        "gt_center_3d": [round_float(v) for v in item.get("gt_center_3d", [])],
+                        "det_center_3d": [round_float(v) for v in item.get("det_center_3d", [])],
+                        "gt_rotation_rad": round_float(item.get("gt_rotation", 0.0)),
+                        "det_rotation_rad": round_float(item.get("det_rotation", 0.0)),
+                        "lateral_error_m": round_float(errors.get("lateral", 0.0)),
+                        "longitudinal_error_m": round_float(errors.get("longitudinal", 0.0)),
+                        "longitudinal_relative_error": round_float(errors.get("longitudinal_relative", 0.0)),
+                        "heading_error_rad": round_float(errors.get("heading", 0.0)),
+                        "heading_error_deg": round_float(math.degrees(float(errors.get("heading", 0.0)))),
+                        "heading_error_relaxed_rad": round_float(errors.get("heading_relaxed", errors.get("heading", 0.0))),
+                        "heading_error_relaxed_deg": round_float(
+                            math.degrees(float(errors.get("heading_relaxed", errors.get("heading", 0.0))))
+                        ),
+                        "is_reversal": bool(errors.get("is_reversal", False)),
+                    }
+                    samples.append(sample)
+    return samples
+
+
+def summarize_metric(samples, metric_name, thresholds):
+    values = [metric_raw_value(metric_name, sample) for sample in samples]
+    summary = {
+        "stats": make_stats(values),
+        "bad_count": sum(1 for sample in samples if threshold_hit(metric_name, sample, thresholds)),
+        "bad_percentage": round_float(
+            100.0 * sum(1 for sample in samples if threshold_hit(metric_name, sample, thresholds)) / len(samples)
+        ) if samples else 0.0,
+    }
+    if metric_name == "reversal":
+        count = sum(1 for sample in samples if sample["is_reversal"])
+        summary = {
+            "count": count,
+            "percentage": round_float(100.0 * count / len(samples)) if samples else 0.0,
+        }
+    return summary
+
+
+def summarize_sample_group(samples, metrics, thresholds):
+    result = {"num_samples": len(samples)}
+    for metric_name in metrics:
+        result[metric_name] = summarize_metric(samples, metric_name, thresholds)
+    return result
+
+
+def build_top_frames(samples, metrics, thresholds, top_k_frames):
+    grouped = defaultdict(list)
+    for sample in samples:
+        grouped[(sample["case_name"], sample["frame_name"])].append(sample)
+
+    top_frames = []
+    for (case_name, frame_name), frame_samples in grouped.items():
+        bad_by_metric = {
+            metric_name: sum(1 for sample in frame_samples if threshold_hit(metric_name, sample, thresholds))
+            for metric_name in metrics
+        }
+        frame_record = {
+            "case_name": case_name,
+            "frame_name": frame_name,
+            "num_samples": len(frame_samples),
+            "bad_objects": sum(1 for sample in frame_samples if any(threshold_hit(metric_name, sample, thresholds) for metric_name in metrics)),
+            "bad_by_metric": bad_by_metric,
+            "mean_lateral_error_m": round_float(mean(sample["lateral_error_m"] for sample in frame_samples)),
+            "mean_longitudinal_error_m": round_float(mean(sample["longitudinal_error_m"] for sample in frame_samples)),
+            "mean_heading_error_deg": round_float(mean(sample["heading_error_deg"] for sample in frame_samples)),
+        }
+        top_frames.append(frame_record)
+
+    top_frames.sort(
+        key=lambda item: (
+            item["bad_objects"],
+            item["bad_by_metric"].get("reversal", 0),
+            item["mean_longitudinal_error_m"],
+            item["mean_heading_error_deg"],
+        ),
+        reverse=True,
+    )
+    return top_frames[:top_k_frames]
+
+
+def rank_key(metric_name, sample):
+    if metric_name == "reversal":
+        return (
+            1 if sample["is_reversal"] else 0,
+            sample["heading_error_deg"],
+            sample["confidence"],
+            sample["iou"],
+        )
+    return (
+        metric_display_value(metric_name, sample),
+        sample["confidence"],
+        sample["iou"],
+    )
+
+
+def sample_to_example(sample, metric_name):
+    example = dict(sample)
+    example["metric_name"] = metric_name
+    example["metric_value"] = round_float(metric_raw_value(metric_name, sample))
+    example["metric_value_display"] = round_float(metric_display_value(metric_name, sample))
+    example["metric_unit"] = metric_unit(metric_name)
+    return example
+
+
+def build_badcase_examples(samples, metrics, top_k, top_k_per_class):
+    badcase_examples = {}
+    badcase_examples_per_class = defaultdict(dict)
+
+    by_class = defaultdict(list)
+    for sample in samples:
+        by_class[sample["class_name"]].append(sample)
+
+    for metric_name in metrics:
+        ranked = sorted(samples, key=lambda sample: rank_key(metric_name, sample), reverse=True)
+        if metric_name == "reversal":
+            ranked = [sample for sample in ranked if sample["is_reversal"]]
+        badcase_examples[metric_name] = [sample_to_example(sample, metric_name) for sample in ranked[:top_k]]
+
+        for class_name_str, class_samples in by_class.items():
+            class_ranked = sorted(class_samples, key=lambda sample: rank_key(metric_name, sample), reverse=True)
+            if metric_name == "reversal":
+                class_ranked = [sample for sample in class_ranked if sample["is_reversal"]]
+            badcase_examples_per_class[class_name_str][metric_name] = [
+                sample_to_example(sample, metric_name) for sample in class_ranked[:top_k_per_class]
+            ]
+
+    return badcase_examples, dict(sorted(badcase_examples_per_class.items()))
+
+
+def build_bin_summary(samples, key_name, metrics, thresholds):
+    grouped = defaultdict(list)
+    for sample in samples:
+        bucket = sample.get(key_name)
+        if bucket:
+            grouped[bucket].append(sample)
+    return {
+        bucket: summarize_sample_group(bucket_samples, metrics, thresholds)
+        for bucket, bucket_samples in sorted(grouped.items())
+    }
+
+
+def write_csv_exports(output_dir, badcase_examples):
+    csv_dir = output_dir / "csv"
+    csv_dir.mkdir(parents=True, exist_ok=True)
+    for metric_name, examples in badcase_examples.items():
+        csv_path = csv_dir / f"top_{metric_name}.csv"
+        with open(csv_path, "w", newline="") as file:
+            writer = csv.DictWriter(file, fieldnames=CSV_FIELDNAMES)
+            writer.writeheader()
+            for example in examples:
+                row = {field: example.get(field) for field in CSV_FIELDNAMES}
+                writer.writerow(row)
+
+
+def write_markdown_report(report, output_path):
+    metadata = report["metadata"]
+    summary = report["summary"]
+    top_frames = report["top_frames"]
+    badcase_examples = report["badcase_examples"]
+
+    with open(output_path, "w") as file:
+        file.write("# 3D Badcase Analysis Report\n\n")
+
+        file.write("## Configuration\n\n")
+        file.write("| Item | Value |\n")
+        file.write("| --- | --- |\n")
+        for key in (
+            "source",
+            "detailed_matches_path",
+            "config_path",
+            "coord_system",
+            "iou_threshold",
+            "conf_threshold",
+            "classes",
+            "metrics",
+        ):
+            value = metadata.get(key)
+            if isinstance(value, list):
+                value = ", ".join(str(v) for v in value)
+            file.write(f"| {key} | `{value}` |\n")
+        file.write("\n")
+
+        file.write("## Overall Summary\n\n")
+        file.write("| Metric | Samples | Mean | P90 | Bad Count | Bad % |\n")
+        file.write("| --- | ---: | ---: | ---: | ---: | ---: |\n")
+        for metric_name in metadata["metrics"]:
+            item = summary["metrics"][metric_name]
+            if metric_name == "reversal":
+                file.write(
+                    f"| `{metric_name}` | {summary['num_samples']} | - | - | {item['count']} | {item['percentage']:.2f}% |\n"
+                )
+            else:
+                stats = item["stats"]
+                file.write(
+                    f"| `{metric_name}` | {summary['num_samples']} | {stats['mean']:.4f} | {stats['percentile_90']:.4f} "
+                    f"| {item['bad_count']} | {item['bad_percentage']:.2f}% |\n"
+                )
+        file.write("\n")
+
+        file.write("## Per-Class Summary\n\n")
+        file.write("| Class | Samples |")
+        for metric_name in metadata["metrics"]:
+            if metric_name == "reversal":
+                file.write(" Reversal % |")
+            else:
+                file.write(f" {metric_name} Mean |")
+        file.write("\n")
+        file.write("| --- | ---: |")
+        for _metric_name in metadata["metrics"]:
+            file.write(" ---: |")
+        file.write("\n")
+        for class_name_str, class_summary in report["per_class"].items():
+            file.write(f"| `{class_name_str}` | {class_summary['num_samples']} |")
+            for metric_name in metadata["metrics"]:
+                metric_summary = class_summary.get(metric_name, {})
+                if metric_name == "reversal":
+                    file.write(f" {metric_summary.get('percentage', 0.0):.2f}% |")
+                else:
+                    file.write(f" {metric_summary.get('stats', {}).get('mean', 0.0):.4f} |")
+            file.write("\n")
+        file.write("\n")
+
+        file.write("## Top Frames\n\n")
+        file.write("| Case / Frame | Samples | Bad Objects | Longitudinal Mean | Heading Mean(deg) |\n")
+        file.write("| --- | ---: | ---: | ---: | ---: |\n")
+        for item in top_frames:
+            file.write(
+                f"| `{item['case_name']}/{item['frame_name']}` | {item['num_samples']} | {item['bad_objects']} "
+                f"| {item['mean_longitudinal_error_m']:.4f} | {item['mean_heading_error_deg']:.4f} |\n"
+            )
+        file.write("\n")
+
+        file.write("## Top Badcases\n\n")
+        for metric_name, examples in badcase_examples.items():
+            file.write(f"### {metric_name}\n\n")
+            file.write("| Class | Case / Frame | Metric | Conf | IoU |\n")
+            file.write("| --- | --- | ---: | ---: | ---: |\n")
+            for example in examples[:10]:
+                file.write(
+                    f"| `{example['class_name']}` | `{example['case_name']}/{example['frame_name']}` | "
+                    f"{example['metric_value_display']:.4f} | {example['confidence']:.4f} | {example['iou']:.4f} |\n"
+                )
+            file.write("\n")
+
+
+def default_output_dir():
+    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+    return REPO_ROOT / "eval_tools" / "analysis" / "results_3d" / timestamp
+
+
+def main():
+    args = parse_args()
+    output_dir = Path(args.output_dir) if args.output_dir else default_output_dir()
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    detailed_matches, detailed_matches_path, config = load_detailed_matches(args)
+    if not detailed_matches:
+        raise RuntimeError("No detailed 3D matches were available for analysis.")
+
+    class_ids = parse_class_ids(args.classes)
+    thresholds = {
+        "bad_lateral_threshold": float(args.bad_lateral_threshold),
+        "bad_longitudinal_threshold": float(args.bad_longitudinal_threshold),
+        "bad_longitudinal_relative_threshold": float(args.bad_longitudinal_relative_threshold),
+        "bad_heading_threshold_deg": float(args.bad_heading_threshold_deg),
+    }
+    distance_ranges = build_distance_ranges(config or {})
+    lateral_ranges = build_lateral_ranges(config or {})
+    samples = collect_samples(detailed_matches, class_ids, distance_ranges, lateral_ranges, args)
+    if not samples:
+        raise RuntimeError("No 3D samples remained after filtering.")
+
+    metrics = list(args.metrics)
+    summary = {
+        "num_cases": len({sample["case_name"] for sample in samples}),
+        "num_frames": len({(sample["case_name"], sample["frame_name"]) for sample in samples}),
+        "num_samples": len(samples),
+        "metrics": {metric_name: summarize_metric(samples, metric_name, thresholds) for metric_name in metrics},
+    }
+    per_class = {}
+    by_class = defaultdict(list)
+    for sample in samples:
+        by_class[sample["class_name"]].append(sample)
+    for class_name_str, class_samples in sorted(by_class.items()):
+        per_class[class_name_str] = summarize_sample_group(class_samples, metrics, thresholds)
+
+    badcase_examples, badcase_examples_per_class = build_badcase_examples(
+        samples=samples,
+        metrics=metrics,
+        top_k=args.top_k,
+        top_k_per_class=args.top_k_per_class,
+    )
+    top_frames = build_top_frames(samples, metrics, thresholds, args.top_k_frames)
+
+    report = {
+        "metadata": {
+            "created_at": datetime.now().isoformat(timespec="seconds"),
+            "source": "detailed_3d_matches" if detailed_matches_path is not None else "rebuilt_from_config",
+            "detailed_matches_path": str(detailed_matches_path.resolve()) if detailed_matches_path else None,
+            "config_path": args.config,
+            "coord_system": (config or {}).get("metrics_3d", {}).get("coordinate_system", "camera"),
+            "iou_threshold": (config or {}).get("matching", {}).get("iou_threshold"),
+            "conf_threshold": (config or {}).get("metrics_3d", {}).get(
+                "conf_threshold",
+                (config or {}).get("metrics_2d", {}).get("conf_threshold"),
+            ),
+            "classes": [class_name(class_id) for class_id in class_ids],
+            "metrics": metrics,
+            "bad_thresholds": thresholds,
+            "distance_ranges": distance_ranges,
+            "lateral_distance_ranges": lateral_ranges,
+            "vehicle_size_split_3d": (config or {}).get("metrics_3d", {}).get("vehicle_size_split"),
+        },
+        "summary": summary,
+        "per_class": per_class,
+        "per_distance_bin": build_bin_summary(samples, "distance_bin", metrics, thresholds),
+        "per_lateral_bin": build_bin_summary(samples, "lateral_bin", metrics, thresholds),
+        "top_frames": top_frames,
+        "badcase_examples": badcase_examples,
+        "badcase_examples_per_class": badcase_examples_per_class,
+    }
+
+    report_path = output_dir / "analysis_report.json"
+    with open(report_path, "w") as file:
+        json.dump(report, file, indent=2)
+
+    markdown_path = output_dir / "analysis_report.md"
+    write_markdown_report(report, markdown_path)
+    write_csv_exports(output_dir, badcase_examples)
+
+    if args.save_rebuilt_matches and detailed_matches_path is None:
+        rebuilt_path = output_dir / "detailed_3d_matches.json"
+        with open(rebuilt_path, "w") as file:
+            json.dump(detailed_matches, file, indent=2)
+        print(f"Rebuilt detailed matches saved to: {rebuilt_path}")
+
+    print(f"JSON report saved to:      {report_path}")
+    print(f"Markdown report saved to:  {markdown_path}")
+    print(f"CSV exports saved to:      {output_dir / 'csv'}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/analysis/analyze_3d_badcases.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+EVALUATION_REPORT=${EVALUATION_REPORT:-evaluation_results/eval_results_yolo26s_768_20260407_DL_KPI_SCENE/yolo26s-20260407-conf0.27/20260412_155342_roi0/evaluation_report.json}
+DETAILED_MATCHES=${DETAILED_MATCHES:-$(dirname "${EVALUATION_REPORT}")/detailed_3d_matches.json}
+CONFIG_PATH=${CONFIG_PATH:-eval_tools/configs/eval_config_yolov26s-roi0.yaml}
+OUTPUT_DIR=${OUTPUT_DIR:-/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_3d}
+TOP_K=${TOP_K:-200}
+TOP_K_PER_CLASS=${TOP_K_PER_CLASS:-100}
+TOP_K_FRAMES=${TOP_K_FRAMES:-50}
+CLASSES=${CLASSES:-car suv van bus truck pedestrian bicycle}
+
+PYTHON_BIN=${PYTHON_BIN:-/deeplearning_team/ydong/dongying/miniconda/envs/dev/bin/python}
+
+"${PYTHON_BIN}" eval_tools/analysis/analyze_3d_badcases.py \
+  --evaluation-report "${EVALUATION_REPORT}" \
+  --detailed-matches "${DETAILED_MATCHES}" \
+  --config "${CONFIG_PATH}" \
+  --classes ${CLASSES} \
+  --top-k "${TOP_K}" \
+  --top-k-per-class "${TOP_K_PER_CLASS}" \
+  --top-k-frames "${TOP_K_FRAMES}" \
+  --output-dir "${OUTPUT_DIR}"

eval_tools/analysis/export_2d_fp_fn_badcases.py

@@ -0,0 +1,339 @@
+#!/usr/bin/env python3
+"""
+Export filtered badcase lists from analyze_2d_fp_fn.py results.
+
+The script reads ``analysis_report.json`` and produces:
+  - a filtered JSON file with matching examples
+  - a plain-text case/frame list for downstream visualization
+  - a compact text summary
+"""
+
+import argparse
+import json
+from collections import Counter, defaultdict
+from pathlib import Path
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Filter and export 2D FP/FN badcases from analysis_report.json."
+    )
+    parser.add_argument(
+        "--input",
+        type=str,
+        required=True,
+        help="Path to analysis_report.json generated by analyze_2d_fp_fn.py",
+    )
+    parser.add_argument(
+        "--mode",
+        type=str,
+        default="both",
+        choices=["fp", "fn", "both"],
+        help="Which badcase pool to export",
+    )
+    parser.add_argument(
+        "--classes",
+        nargs="+",
+        default=None,
+        help="Optional class-name filter, e.g. vehicle pedestrian rider",
+    )
+    parser.add_argument(
+        "--error-types",
+        nargs="+",
+        default=None,
+        help="Optional error-type filter, e.g. duplicate localization low_score",
+    )
+    parser.add_argument(
+        "--min-confidence",
+        type=float,
+        default=None,
+        help="Minimum detection confidence for FP examples or matched dets in FN examples",
+    )
+    parser.add_argument(
+        "--max-confidence",
+        type=float,
+        default=None,
+        help="Maximum detection confidence for FP examples or matched dets in FN examples",
+    )
+    parser.add_argument(
+        "--min-distance",
+        type=float,
+        default=None,
+        help="Minimum target distance in metres",
+    )
+    parser.add_argument(
+        "--max-distance",
+        type=float,
+        default=None,
+        help="Maximum target distance in metres",
+    )
+    parser.add_argument(
+        "--min-best-iou",
+        type=float,
+        default=None,
+        help="Minimum best IoU field to keep an example",
+    )
+    parser.add_argument(
+        "--top-k",
+        type=int,
+        default=None,
+        help="Only keep the first K filtered examples after sorting",
+    )
+    parser.add_argument(
+        "--dedup-frame",
+        action="store_true",
+        help="Keep at most one example per case/frame/error/class combination",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="Output directory. Defaults to sibling folder next to the input report.",
+    )
+    return parser.parse_args()
+
+
+def normalize_tokens(values):
+    if not values:
+        return None
+    return {str(value).strip().lower() for value in values if str(value).strip()}
+
+
+def get_confidence(item):
+    if "confidence" in item and item["confidence"] is not None:
+        return float(item["confidence"])
+    if "best_det_confidence" in item and item["best_det_confidence"] is not None:
+        return float(item["best_det_confidence"])
+    return None
+
+
+def get_best_iou(item):
+    if "best_det_iou" in item and item["best_det_iou"] is not None:
+        return float(item["best_det_iou"])
+    return max(
+        float(item.get("best_same_class_iou", 0.0)),
+        float(item.get("best_other_class_iou", 0.0)),
+    )
+
+
+def passes_filters(item, class_filter, error_filter, args):
+    class_name = str(item.get("class_name", "")).lower()
+    error_type = str(item.get("error_type", "")).lower()
+
+    if class_filter and class_name not in class_filter:
+        return False
+    if error_filter and error_type not in error_filter:
+        return False
+
+    confidence = get_confidence(item)
+    if args.min_confidence is not None and (confidence is None or confidence < args.min_confidence):
+        return False
+    if args.max_confidence is not None and (confidence is None or confidence > args.max_confidence):
+        return False
+
+    distance = item.get("distance_m")
+    if args.min_distance is not None and (distance is None or float(distance) < args.min_distance):
+        return False
+    if args.max_distance is not None and (distance is None or float(distance) > args.max_distance):
+        return False
+
+    best_iou = get_best_iou(item)
+    if args.min_best_iou is not None and best_iou < args.min_best_iou:
+        return False
+
+    return True
+
+
+def rank_key(item):
+    confidence = get_confidence(item) or 0.0
+    best_iou = get_best_iou(item)
+    distance = item.get("distance_m")
+    distance = float(distance) if distance is not None else -1.0
+    return (confidence, best_iou, distance)
+
+
+def summarize(items):
+    by_error = Counter()
+    by_class = Counter()
+    by_class_error = defaultdict(Counter)
+
+    for item in items:
+        class_name = item.get("class_name", "unknown")
+        error_type = item.get("error_type", "unknown")
+        by_error[error_type] += 1
+        by_class[class_name] += 1
+        by_class_error[class_name][error_type] += 1
+
+    return {
+        "total_examples": len(items),
+        "by_error_type": dict(sorted(by_error.items())),
+        "by_class": dict(sorted(by_class.items())),
+        "by_class_and_error": {
+            class_name: dict(sorted(counter.items()))
+            for class_name, counter in sorted(by_class_error.items())
+        },
+    }
+
+
+def ensure_output_dir(input_path, output_dir):
+    if output_dir:
+        path = Path(output_dir)
+    else:
+        path = Path(input_path).resolve().parent / "exported_badcases"
+    path.mkdir(parents=True, exist_ok=True)
+    return path
+
+
+def write_summary(path, mode, filters, summary, items):
+    with open(path, "w") as file:
+        file.write("=" * 90 + "\n")
+        file.write("2D FP/FN BADCASE EXPORT SUMMARY\n")
+        file.write("=" * 90 + "\n\n")
+        file.write(f"Mode: {mode}\n")
+        file.write(f"Classes: {filters['classes']}\n")
+        file.write(f"Error types: {filters['error_types']}\n")
+        file.write(f"Min confidence: {filters['min_confidence']}\n")
+        file.write(f"Max confidence: {filters['max_confidence']}\n")
+        file.write(f"Min distance: {filters['min_distance']}\n")
+        file.write(f"Max distance: {filters['max_distance']}\n")
+        file.write(f"Min best IoU: {filters['min_best_iou']}\n")
+        file.write(f"Top K: {filters['top_k']}\n")
+        file.write(f"Dedup frame: {filters['dedup_frame']}\n\n")
+
+        file.write("SUMMARY\n")
+        file.write("-" * 90 + "\n")
+        file.write(f"Total examples: {summary['total_examples']}\n\n")
+
+        file.write("BY ERROR TYPE\n")
+        file.write("-" * 90 + "\n")
+        for key, value in summary["by_error_type"].items():
+            file.write(f"{key:<24} {value}\n")
+        file.write("\n")
+
+        file.write("BY CLASS\n")
+        file.write("-" * 90 + "\n")
+        for key, value in summary["by_class"].items():
+            file.write(f"{key:<24} {value}\n")
+        file.write("\n")
+
+        file.write("TOP EXAMPLES\n")
+        file.write("-" * 90 + "\n")
+        for item in items[:50]:
+            confidence = get_confidence(item)
+            best_iou = get_best_iou(item)
+            file.write(
+                f"{item.get('case_name')}/{item.get('frame_name')}  "
+                f"{item.get('class_name')}  {item.get('error_type')}  "
+                f"conf={confidence if confidence is not None else '-'}  "
+                f"best_iou={best_iou:.4f}\n"
+            )
+
+
+def write_case_frame_list(path, items):
+    seen = set()
+    with open(path, "w") as file:
+        for item in items:
+            key = (item.get("case_name"), item.get("frame_name"))
+            if key in seen:
+                continue
+            seen.add(key)
+            file.write(f"{key[0]}\t{key[1]}\n")
+
+
+def main():
+    args = parse_args()
+
+    with open(args.input, "r") as file:
+        data = json.load(file)
+
+    pools = []
+    if args.mode in ("fp", "both"):
+        pools.extend(data.get("false_positive_examples", []))
+    if args.mode in ("fn", "both"):
+        pools.extend(data.get("false_negative_examples", []))
+
+    class_filter = normalize_tokens(args.classes)
+    error_filter = normalize_tokens(args.error_types)
+
+    filtered = [
+        item for item in pools if passes_filters(item, class_filter, error_filter, args)
+    ]
+
+    filtered.sort(key=rank_key, reverse=True)
+
+    if args.dedup_frame:
+        deduped = []
+        seen = set()
+        for item in filtered:
+            key = (
+                item.get("case_name"),
+                item.get("frame_name"),
+                item.get("class_name"),
+                item.get("error_type"),
+            )
+            if key in seen:
+                continue
+            seen.add(key)
+            deduped.append(item)
+        filtered = deduped
+
+    if args.top_k is not None:
+        filtered = filtered[: args.top_k]
+
+    summary = summarize(filtered)
+    output_dir = ensure_output_dir(args.input, args.output_dir)
+
+    base_name = f"{args.mode}_badcases"
+    json_path = output_dir / f"{base_name}.json"
+    txt_path = output_dir / f"{base_name}.txt"
+    list_path = output_dir / f"{base_name}_case_frame_list.txt"
+
+    with open(json_path, "w") as file:
+        json.dump(
+            {
+                "source": str(Path(args.input).resolve()),
+                "mode": args.mode,
+                "filters": {
+                    "classes": sorted(class_filter) if class_filter else None,
+                    "error_types": sorted(error_filter) if error_filter else None,
+                    "min_confidence": args.min_confidence,
+                    "max_confidence": args.max_confidence,
+                    "min_distance": args.min_distance,
+                    "max_distance": args.max_distance,
+                    "min_best_iou": args.min_best_iou,
+                    "top_k": args.top_k,
+                    "dedup_frame": args.dedup_frame,
+                },
+                "summary": summary,
+                "examples": filtered,
+            },
+            file,
+            indent=2,
+        )
+
+    write_summary(
+        txt_path,
+        args.mode,
+        {
+            "classes": sorted(class_filter) if class_filter else None,
+            "error_types": sorted(error_filter) if error_filter else None,
+            "min_confidence": args.min_confidence,
+            "max_confidence": args.max_confidence,
+            "min_distance": args.min_distance,
+            "max_distance": args.max_distance,
+            "min_best_iou": args.min_best_iou,
+            "top_k": args.top_k,
+            "dedup_frame": args.dedup_frame,
+        },
+        summary,
+        filtered,
+    )
+    write_case_frame_list(list_path, filtered)
+
+    print(f"Filtered JSON saved to: {json_path}")
+    print(f"Summary text saved to:  {txt_path}")
+    print(f"Case/frame list saved to: {list_path}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/analysis/visualize_2d_fn_cases.py

@@ -0,0 +1,783 @@
+#!/usr/bin/env python3
+"""
+Visualize 2D FN cases from analyze_2d_fp_fn.py results on source images.
+
+This script is designed for image-based inspection of false negatives,
+especially FN-localization. It reads ``analysis_report.json``, reloads the
+corresponding GT/detections using the same evaluator pipeline, and saves:
+
+  - frame-level overlays (all GT / active detections / highlighted FN targets)
+  - per-example panels (full-frame + local crop)
+  - a summary index JSON
+"""
+
+import argparse
+import json
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+import cv2
+import numpy as np
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from eval_tools.analysis.analyze_2d_fp_fn import build_config, class_name, parse_class_ids
+from eval_tools.evaluator.evaluator import Evaluator
+
+
+BOX_COLORS = {
+    "gt_all": (80, 220, 80),
+    # Keep normal detections visually quiet so highlighted error targets stand out.
+    "det_all": (150, 150, 150),
+    "fn_gt": (40, 40, 255),
+    "fn_det": (0, 215, 255),
+    "fp_det": (255, 0, 220),
+    "fp_ref_gt": (255, 255, 0),
+    "title_bg": (30, 30, 30),
+}
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Visualize FN cases from analysis_report.json on source images."
+    )
+    parser.add_argument(
+        "--analysis-report",
+        type=str,
+        required=True,
+        help="Path to analysis_report.json generated by analyze_2d_fp_fn.py",
+    )
+    parser.add_argument("--config", type=str, help="Path to YAML evaluation config")
+    parser.add_argument("--det-path", type=str, help="Detection results root directory")
+    parser.add_argument("--gt-path", type=str, help="Ground-truth labels root directory")
+    parser.add_argument("--path-depth", type=int, choices=[1, 2], help="Directory depth")
+    parser.add_argument(
+        "--det-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Detection file format",
+    )
+    parser.add_argument(
+        "--gt-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Ground-truth file format",
+    )
+    parser.add_argument("--img-width", type=int, help="Image width")
+    parser.add_argument("--img-height", type=int, help="Image height")
+    parser.add_argument(
+        "--coord-system",
+        type=str,
+        choices=["camera", "ego"],
+        help="Coordinate system used by the parser/evaluator",
+    )
+    parser.add_argument(
+        "--iou-threshold",
+        type=float,
+        help="IoU threshold used for evaluator loading",
+    )
+    parser.add_argument(
+        "--conf-threshold",
+        type=float,
+        help="Confidence threshold for active detections shown on overlays",
+    )
+    parser.add_argument(
+        "--mode",
+        type=str,
+        default="fn",
+        choices=["fn", "fp", "both"],
+        help="Which example pool to visualize",
+    )
+    parser.add_argument(
+        "--error-types",
+        nargs="+",
+        default=["localization"],
+        help="Error types to visualize. Default: localization",
+    )
+    parser.add_argument(
+        "--classes",
+        nargs="+",
+        default=None,
+        help="Optional class filter, e.g. vehicle pedestrian bicycle rider",
+    )
+    parser.add_argument(
+        "--case-names",
+        nargs="+",
+        default=None,
+        help="Optional case-name filter",
+    )
+    parser.add_argument(
+        "--min-confidence",
+        type=float,
+        default=None,
+        help="Minimum confidence for the associated detection (best det for FN, det confidence for FP)",
+    )
+    parser.add_argument(
+        "--max-confidence",
+        type=float,
+        default=None,
+        help="Maximum confidence for the associated detection",
+    )
+    parser.add_argument(
+        "--min-distance",
+        type=float,
+        default=None,
+        help="Minimum target distance in metres",
+    )
+    parser.add_argument(
+        "--max-distance",
+        type=float,
+        default=None,
+        help="Maximum target distance in metres",
+    )
+    parser.add_argument(
+        "--max-best-iou",
+        type=float,
+        default=None,
+        help="Maximum best IoU. Useful for focusing on badly localized examples.",
+    )
+    parser.add_argument(
+        "--top-k",
+        type=int,
+        default=200,
+        help="Maximum number of examples to visualize after filtering",
+    )
+    parser.add_argument(
+        "--dedup-frame",
+        action="store_true",
+        help="Keep at most one example per case/frame/class/error combination",
+    )
+    parser.add_argument(
+        "--line-thickness",
+        type=int,
+        default=2,
+        help="Base line thickness for non-highlight boxes",
+    )
+    parser.add_argument(
+        "--crop-scale",
+        type=float,
+        default=1.8,
+        help="Expand crop window around GT/det union box by this factor",
+    )
+    parser.add_argument(
+        "--jpeg-quality",
+        type=int,
+        default=92,
+        help="JPEG quality for saved visualizations",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="Output directory. Defaults to evaluation_results/fn_vis_<report_name>",
+    )
+    return parser.parse_args()
+
+
+def get_confidence(example):
+    if example.get("confidence") is not None:
+        return float(example["confidence"])
+    if example.get("best_det_confidence") is not None:
+        return float(example["best_det_confidence"])
+    return None
+
+
+def get_best_iou(example):
+    if example.get("best_det_iou") is not None:
+        return float(example["best_det_iou"])
+    return max(
+        float(example.get("best_same_class_iou", 0.0)),
+        float(example.get("best_other_class_iou", 0.0)),
+    )
+
+
+def normalize_token_set(values):
+    if not values:
+        return None
+    return {str(v).strip().lower() for v in values if str(v).strip()}
+
+
+def rank_examples(examples):
+    def key(item):
+        conf = get_confidence(item) or 0.0
+        best_iou = get_best_iou(item)
+        distance = item.get("distance_m")
+        distance = float(distance) if distance is not None else -1.0
+        area = float(item.get("gt_bbox_area", item.get("det_bbox_area", 0.0)) or 0.0)
+        return (conf, -best_iou, area, distance)
+
+    return sorted(examples, key=key, reverse=True)
+
+
+def filter_examples(report, args):
+    pools = []
+    if args.mode in ("fn", "both"):
+        pools.extend(report.get("false_negative_examples", []))
+    if args.mode in ("fp", "both"):
+        pools.extend(report.get("false_positive_examples", []))
+
+    class_filter = normalize_token_set(args.classes)
+    error_filter = normalize_token_set(args.error_types)
+    case_filter = set(args.case_names) if args.case_names else None
+
+    filtered = []
+    for item in pools:
+        class_str = str(item.get("class_name", "")).lower()
+        error_type = str(item.get("error_type", "")).lower()
+        case_name = item.get("case_name")
+        conf = get_confidence(item)
+        distance = item.get("distance_m")
+        best_iou = get_best_iou(item)
+
+        if class_filter and class_str not in class_filter:
+            continue
+        if error_filter and error_type not in error_filter:
+            continue
+        if case_filter and case_name not in case_filter:
+            continue
+        if args.min_confidence is not None and (conf is None or conf < args.min_confidence):
+            continue
+        if args.max_confidence is not None and (conf is None or conf > args.max_confidence):
+            continue
+        if args.min_distance is not None and (distance is None or float(distance) < args.min_distance):
+            continue
+        if args.max_distance is not None and (distance is None or float(distance) > args.max_distance):
+            continue
+        if args.max_best_iou is not None and best_iou > args.max_best_iou:
+            continue
+        filtered.append(item)
+
+    filtered = rank_examples(filtered)
+
+    if args.dedup_frame:
+        deduped = []
+        seen = set()
+        for item in filtered:
+            key = (
+                item.get("case_name"),
+                item.get("frame_name"),
+                item.get("class_name"),
+                item.get("error_type"),
+            )
+            if key in seen:
+                continue
+            seen.add(key)
+            deduped.append(item)
+        filtered = deduped
+
+    if args.top_k is not None:
+        filtered = filtered[: args.top_k]
+
+    return filtered
+
+
+def bbox_to_int(bbox):
+    return [int(round(float(v))) for v in bbox]
+
+
+def get_example_gt_bbox(example):
+    return example.get("gt_bbox")
+
+
+def get_example_det_bbox(example):
+    if example.get("best_det_bbox") is not None:
+        return example.get("best_det_bbox")
+    return example.get("det_bbox")
+
+
+def is_fn_example(example):
+    return example.get("gt_bbox") is not None
+
+
+def parse_generated_gt_index(gt_id):
+    if not gt_id:
+        return None
+    gt_id = str(gt_id)
+    if gt_id.startswith("gt_") and gt_id[3:].isdigit():
+        return int(gt_id[3:])
+    return None
+
+
+def resolve_reference_gt(example, gts):
+    if not gts:
+        return None, None, None
+
+    def find_by_explicit_id(target_id):
+        if target_id is None:
+            return None
+        for gt in gts:
+            if gt.get("id") is not None and str(gt.get("id")) == str(target_id):
+                return gt
+        return None
+
+    best_same_gt_id = example.get("best_same_gt_id")
+    best_other_gt_id = example.get("best_other_gt_id")
+
+    gt = find_by_explicit_id(best_same_gt_id)
+    if gt is not None:
+        return gt.get("bbox_2d"), class_name(gt["label"]), best_same_gt_id
+
+    gt = find_by_explicit_id(best_other_gt_id)
+    if gt is not None:
+        return gt.get("bbox_2d"), class_name(gt["label"]), best_other_gt_id
+
+    same_idx = parse_generated_gt_index(best_same_gt_id)
+    if same_idx is not None:
+        same_class_gts = [gt for gt in gts if gt["label"] == example.get("class_id")]
+        if 0 <= same_idx < len(same_class_gts):
+            gt = same_class_gts[same_idx]
+            return gt.get("bbox_2d"), class_name(gt["label"]), best_same_gt_id
+
+    other_idx = parse_generated_gt_index(best_other_gt_id)
+    if other_idx is not None and 0 <= other_idx < len(gts):
+        gt = gts[other_idx]
+        return gt.get("bbox_2d"), class_name(gt["label"]), best_other_gt_id
+
+    return None, None, None
+
+
+def get_target_box_color(example, kind):
+    if is_fn_example(example):
+        return BOX_COLORS["fn_gt"] if kind == "gt" else BOX_COLORS["fn_det"]
+    if kind == "det":
+        return BOX_COLORS["fp_det"]
+    return BOX_COLORS["fp_ref_gt"]
+
+
+def draw_box(image, bbox, color, label=None, thickness=2):
+    x1, y1, x2, y2 = bbox_to_int(bbox)
+    cv2.rectangle(image, (x1, y1), (x2, y2), color, thickness, cv2.LINE_AA)
+    if label:
+        (tw, th), _ = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.55, 1)
+        y_text = max(0, y1 - th - 8)
+        cv2.rectangle(image, (x1, y_text), (x1 + tw + 8, y_text + th + 8), color, -1)
+        cv2.putText(
+            image,
+            label,
+            (x1 + 4, y_text + th + 2),
+            cv2.FONT_HERSHEY_SIMPLEX,
+            0.55,
+            (255, 255, 255),
+            1,
+            cv2.LINE_AA,
+        )
+
+
+def add_header(image, text):
+    h, w = image.shape[:2]
+    overlay = image.copy()
+    cv2.rectangle(overlay, (0, 0), (w, 42), BOX_COLORS["title_bg"], -1)
+    cv2.addWeighted(overlay, 0.55, image, 0.45, 0, image)
+    cv2.putText(
+        image,
+        text,
+        (10, 28),
+        cv2.FONT_HERSHEY_SIMPLEX,
+        0.7,
+        (255, 255, 255),
+        2,
+        cv2.LINE_AA,
+    )
+
+
+def make_crop(image, boxes, scale=1.8):
+    h, w = image.shape[:2]
+    valid = [bbox for bbox in boxes if bbox is not None]
+    if not valid:
+        return image.copy(), (0, 0)
+
+    x1 = min(float(b[0]) for b in valid)
+    y1 = min(float(b[1]) for b in valid)
+    x2 = max(float(b[2]) for b in valid)
+    y2 = max(float(b[3]) for b in valid)
+
+    cx = 0.5 * (x1 + x2)
+    cy = 0.5 * (y1 + y2)
+    bw = max(32.0, (x2 - x1) * scale)
+    bh = max(32.0, (y2 - y1) * scale)
+
+    crop_x1 = max(0, int(round(cx - bw / 2)))
+    crop_y1 = max(0, int(round(cy - bh / 2)))
+    crop_x2 = min(w, int(round(cx + bw / 2)))
+    crop_y2 = min(h, int(round(cy + bh / 2)))
+
+    return image[crop_y1:crop_y2, crop_x1:crop_x2].copy(), (crop_x1, crop_y1)
+
+
+def draw_crop_panel(image, example, gts, crop_scale):
+    gt_bbox = get_example_gt_bbox(example)
+    det_bbox = get_example_det_bbox(example)
+    ref_gt_bbox, ref_gt_class, ref_gt_id = resolve_reference_gt(example, gts)
+    crop, (off_x, off_y) = make_crop(
+        image, [gt_bbox, det_bbox, ref_gt_bbox], scale=crop_scale
+    )
+
+    def shift_box(box):
+        if box is None:
+            return None
+        return [
+            float(box[0]) - off_x,
+            float(box[1]) - off_y,
+            float(box[2]) - off_x,
+            float(box[3]) - off_y,
+        ]
+
+    gt_local = shift_box(gt_bbox)
+    det_local = shift_box(det_bbox)
+    ref_gt_local = shift_box(ref_gt_bbox)
+
+    if gt_local is not None:
+        draw_box(
+            crop,
+            gt_local,
+            get_target_box_color(example, "gt"),
+            label=f"GT {example['class_name']}",
+            thickness=3,
+        )
+    elif ref_gt_local is not None:
+        draw_box(
+            crop,
+            ref_gt_local,
+            get_target_box_color(example, "gt"),
+            label=f"RefGT {ref_gt_class or '-'}",
+            thickness=3,
+        )
+
+    if det_local is not None:
+        conf = get_confidence(example)
+        iou = get_best_iou(example)
+        if example.get("best_det_bbox") is not None:
+            label = f"BestDet {example.get('best_det_class', '-')}"
+            if conf is not None:
+                label += f" {conf:.2f}"
+            label += f" IoU {iou:.3f}"
+        else:
+            label = f"FP Det {example.get('class_name', '-')}"
+            if conf is not None:
+                label += f" {conf:.2f}"
+            label += f" IoU {iou:.3f}"
+        draw_box(crop, det_local, get_target_box_color(example, "det"), label=label, thickness=3)
+
+    add_header(
+        crop,
+        f"crop | {'FN' if is_fn_example(example) else 'FP'} | {example['class_name']} | {example['error_type']} | dist={example.get('distance_m')}",
+    )
+    return crop
+
+
+def add_sidebar(panel, example):
+    h, _ = panel.shape[:2]
+    sidebar = np.full((h, 360, 3), 28, dtype=np.uint8)
+    lines = [
+        f"case: {example.get('case_name')}",
+        f"frame: {example.get('frame_name')}",
+        f"class: {example.get('class_name')}",
+        f"error: {example.get('error_type')}",
+        f"mode: {'fn' if is_fn_example(example) else 'fp'}",
+        f"gt_id: {example.get('gt_id', '-')}",
+        f"ref_gt_id: {example.get('best_same_gt_id') or example.get('best_other_gt_id') or '-'}",
+        f"best_det_id: {example.get('best_det_id', '-')}",
+        f"best_det_cls: {example.get('best_det_class', '-')}",
+        f"det_id: {example.get('det_id', '-')}",
+        f"conf: {get_confidence(example)}",
+        f"best_iou: {get_best_iou(example):.4f}",
+        f"distance_m: {example.get('distance_m')}",
+        f"lateral_m: {example.get('lateral_m')}",
+        f"gt_area: {example.get('gt_bbox_area')}",
+        f"det_area: {example.get('det_bbox_area')}",
+    ]
+
+    y = 36
+    for line in lines:
+        cv2.putText(
+            sidebar,
+            str(line),
+            (12, y),
+            cv2.FONT_HERSHEY_SIMPLEX,
+            0.56,
+            (235, 235, 235),
+            1,
+            cv2.LINE_AA,
+        )
+        y += 30
+
+    return np.hstack([panel, sidebar])
+
+
+def resize_to_height(image, target_height):
+    h, w = image.shape[:2]
+    if h == target_height:
+        return image
+    scale = target_height / max(h, 1)
+    return cv2.resize(image, (max(1, int(round(w * scale))), target_height))
+
+
+def combine_full_and_crop(full_image, crop_image, example):
+    target_h = max(full_image.shape[0], crop_image.shape[0])
+    full_resized = resize_to_height(full_image, target_h)
+    crop_resized = resize_to_height(crop_image, target_h)
+    panel = np.hstack([full_resized, crop_resized])
+    return add_sidebar(panel, example)
+
+
+def find_pair_map(config):
+    evaluator = Evaluator(
+        config=config,
+        iou_threshold=float(config.get("matching", {}).get("iou_threshold", 0.5)),
+        num_workers=1,
+        save_detailed_matches=False,
+    )
+    dataset_cfg = config["dataset"]
+    image_cfg = config["image"]
+    evaluator.load_data_from_paths(
+        det_root=dataset_cfg["det_path"],
+        gt_root=dataset_cfg["gt_path"],
+        img_width=image_cfg.get("width", 1920),
+        img_height=image_cfg.get("height", 1080),
+        path_depth=dataset_cfg.get("path_depth", 1),
+        det_format=dataset_cfg.get("det_format", "auto"),
+        gt_format=dataset_cfg.get("gt_format", "auto"),
+    )
+
+    pair_map = {}
+    for pair in evaluator.image_pairs:
+        level1_name = pair.get("level1_name")
+        if level1_name:
+            case_key = f"{level1_name}/{pair['case']}"
+        else:
+            case_key = pair["case"]
+        pair_map[(case_key, pair["frame"])] = pair
+    return pair_map, evaluator
+
+
+def find_image_path(pair):
+    gt_file = Path(pair["gt_file"])
+    case_dir = gt_file.parent.parent
+    images_dir = case_dir / "images"
+    stem = gt_file.stem
+    for suffix in (".png", ".jpg", ".jpeg", ".bmp"):
+        candidate = images_dir / f"{stem}{suffix}"
+        if candidate.exists():
+            return candidate
+    matches = list(images_dir.glob(f"{stem}.*"))
+    return matches[0] if matches else None
+
+
+def render_frame_overlay(image, gts, active_dets, frame_examples, class_ids, line_thickness):
+    canvas = image.copy()
+
+    selected_class_ids = set(class_ids)
+    for gt in gts:
+        if gt["label"] not in selected_class_ids:
+            continue
+        draw_box(
+            canvas,
+            gt["bbox_2d"],
+            BOX_COLORS["gt_all"],
+            label=f"GT {class_name(gt['label'])}",
+            thickness=line_thickness,
+        )
+
+    for det in active_dets:
+        if det["label"] not in selected_class_ids:
+            continue
+        conf = float(det.get("confidence", 0.0))
+        draw_box(
+            canvas,
+            det["bbox_2d"],
+            BOX_COLORS["det_all"],
+            label=f"Det {class_name(det['label'])} {conf:.2f}",
+            thickness=line_thickness,
+        )
+
+    for idx, example in enumerate(frame_examples, 1):
+        gt_bbox = get_example_gt_bbox(example)
+        det_bbox = get_example_det_bbox(example)
+        ref_gt_bbox, ref_gt_class, _ref_gt_id = resolve_reference_gt(example, gts)
+        if gt_bbox is not None:
+            draw_box(
+                canvas,
+                gt_bbox,
+                get_target_box_color(example, "gt"),
+                label=f"FN#{idx} GT {example['class_name']}",
+                thickness=max(3, line_thickness + 1),
+            )
+        elif ref_gt_bbox is not None:
+            draw_box(
+                canvas,
+                ref_gt_bbox,
+                get_target_box_color(example, "gt"),
+                label=f"FP#{idx} RefGT {ref_gt_class or '-'}",
+                thickness=max(3, line_thickness + 1),
+            )
+        if det_bbox is not None:
+            conf = get_confidence(example)
+            iou = get_best_iou(example)
+            if example.get("best_det_bbox") is not None:
+                label = f"FN#{idx} BestDet {example.get('best_det_class', '-')}"
+                if conf is not None:
+                    label += f" {conf:.2f}"
+                label += f" IoU {iou:.3f}"
+            else:
+                label = f"FP#{idx} Det {example.get('class_name', '-')}"
+                if conf is not None:
+                    label += f" {conf:.2f}"
+                label += f" IoU {iou:.3f}"
+            draw_box(
+                canvas,
+                det_bbox,
+                get_target_box_color(example, "det"),
+                label=label,
+                thickness=max(3, line_thickness + 1),
+            )
+
+    example_modes = {("FN" if is_fn_example(example) else "FP") for example in frame_examples}
+    if len(example_modes) == 1:
+        mode_label = next(iter(example_modes))
+    else:
+        mode_label = "MIXED"
+
+    headline = (
+        f"2D error visualization | mode={mode_label} | examples={len(frame_examples)} | "
+        f"GT=green Det=orange FN-GT=red FN-det=yellow FP-det=magenta FP-refGT=cyan"
+    )
+    add_header(canvas, headline)
+    return canvas
+
+
+def ensure_dir(path):
+    path.mkdir(parents=True, exist_ok=True)
+    return path
+
+
+def sanitize_token(value):
+    return str(value).replace("/", "__").replace("\\", "__").replace(" ", "_")
+
+
+def default_output_dir(report_path):
+    report_path = Path(report_path)
+    return report_path.parent / f"fn_vis_{report_path.stem}"
+
+
+def main():
+    args = parse_args()
+
+    with open(args.analysis_report, "r") as file:
+        report = json.load(file)
+
+    config = build_config(args)
+    class_ids = parse_class_ids(args.classes) if args.classes else parse_class_ids(report["summary"]["classes"])
+    filtered_examples = filter_examples(report, args)
+
+    if not filtered_examples:
+        print("No examples matched the current filters.")
+        return
+
+    pair_map, evaluator = find_pair_map(config)
+
+    output_dir = Path(args.output_dir) if args.output_dir else default_output_dir(args.analysis_report)
+    frame_dir = ensure_dir(output_dir / "frames")
+    example_dir = ensure_dir(output_dir / "examples")
+
+    by_frame = defaultdict(list)
+    for item in filtered_examples:
+        by_frame[(item["case_name"], item["frame_name"])].append(item)
+
+    index = {
+        "analysis_report": str(Path(args.analysis_report).resolve()),
+        "num_examples": len(filtered_examples),
+        "num_frames": len(by_frame),
+        "mode": args.mode,
+        "error_types": args.error_types,
+        "classes": [class_name(cid) for cid in class_ids],
+        "frames": [],
+    }
+
+    conf_threshold = float(config.get("metrics_2d", {}).get("conf_threshold", 0.5))
+
+    for frame_idx, ((case_name, frame_name), frame_examples) in enumerate(by_frame.items(), 1):
+        pair = pair_map.get((case_name, frame_name))
+        if pair is None:
+            print(f"Warning: failed to locate pair for {case_name}/{frame_name}, skipping")
+            continue
+
+        image_path = find_image_path(pair)
+        if image_path is None or not image_path.exists():
+            print(f"Warning: image not found for {case_name}/{frame_name}, skipping")
+            continue
+
+        image = cv2.imread(str(image_path))
+        if image is None:
+            print(f"Warning: failed to read image: {image_path}")
+            continue
+
+        gts = Evaluator._parse_ground_truths_for_pair(pair, evaluator.coord_system)
+        dets = Evaluator._parse_detections_for_pair(pair, evaluator.coord_system)
+        active_dets = [det for det in dets if float(det.get("confidence", 0.0)) >= conf_threshold]
+
+        frame_overlay = render_frame_overlay(
+            image,
+            gts,
+            active_dets,
+            frame_examples,
+            class_ids,
+            line_thickness=args.line_thickness,
+        )
+
+        frame_rel = Path("frames") / (
+            f"{frame_idx:04d}_{sanitize_token(case_name)}_{sanitize_token(frame_name)}.jpg"
+        )
+        frame_path = output_dir / frame_rel
+        cv2.imwrite(
+            str(frame_path),
+            frame_overlay,
+            [int(cv2.IMWRITE_JPEG_QUALITY), int(args.jpeg_quality)],
+        )
+
+        frame_entry = {
+            "case_name": case_name,
+            "frame_name": frame_name,
+            "image_path": str(image_path),
+            "frame_visualization": str(frame_rel),
+            "num_examples": len(frame_examples),
+            "examples": [],
+        }
+
+        for ex_idx, example in enumerate(frame_examples, 1):
+            crop_image = draw_crop_panel(
+                image.copy(), example, gts, crop_scale=args.crop_scale
+            )
+            panel = combine_full_and_crop(frame_overlay.copy(), crop_image, example)
+            rel = Path("examples") / (
+                f"{frame_idx:04d}_{ex_idx:02d}_"
+                f"{sanitize_token(case_name)}_{sanitize_token(frame_name)}_"
+                f"{sanitize_token(example['class_name'])}_{sanitize_token(example['error_type'])}.jpg"
+            )
+            panel_path = output_dir / rel
+            cv2.imwrite(
+                str(panel_path),
+                panel,
+                [int(cv2.IMWRITE_JPEG_QUALITY), int(args.jpeg_quality)],
+            )
+
+            example_record = dict(example)
+            example_record["visualization"] = str(rel)
+            frame_entry["examples"].append(example_record)
+
+        index["frames"].append(frame_entry)
+
+    index_path = output_dir / "index.json"
+    with open(index_path, "w") as file:
+        json.dump(index, file, indent=2)
+
+    print(f"Saved visualization index to: {index_path}")
+    print(f"Saved frame overlays to:      {frame_dir}")
+    print(f"Saved example panels to:      {example_dir}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/analysis/visualize_2d_fn_cases.sh

@@ -0,0 +1,43 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ANALYSIS_REPORT=${ANALYSIS_REPORT:-/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_2d/mono3d_fp_fn-roi0/analysis_report.json}
+CONFIG_PATH=${CONFIG_PATH:-eval_tools/configs/eval_config_yolov26s-roi0.yaml}
+OUTPUT_BASE=${OUTPUT_BASE:-/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_2d/mono3d_fp_fn-roi0}
+TOP_K=${TOP_K:-1000}
+
+FN_ERROR_TYPES=(localization missing)
+FP_ERROR_TYPES=(localization background)
+CLASSES=(car)
+
+for mode in fn fp; do
+  if [[ "${mode}" == "fn" ]]; then
+    ERROR_TYPES=("${FN_ERROR_TYPES[@]}")
+  else
+    ERROR_TYPES=("${FP_ERROR_TYPES[@]}")
+  fi
+
+  for error_type in "${ERROR_TYPES[@]}"; do
+    for class_name in "${CLASSES[@]}"; do
+      output_dir="${OUTPUT_BASE}/${mode}_${error_type}_vis_${class_name}"
+
+      echo "============================================================"
+      echo "Running visualization"
+      echo "  mode:       ${mode}"
+      echo "  error_type: ${error_type}"
+      echo "  class:      ${class_name}"
+      echo "  output:     ${output_dir}"
+      echo "============================================================"
+
+      python eval_tools/analysis/visualize_2d_fn_cases.py \
+        --analysis-report "${ANALYSIS_REPORT}" \
+        --config "${CONFIG_PATH}" \
+        --mode "${mode}" \
+        --error-types "${error_type}" \
+        --classes "${class_name}" \
+        --top-k "${TOP_K}" \
+        --output-dir "${output_dir}"
+    done
+  done
+done

eval_tools/analysis/visualize_3d_badcases.py

@@ -0,0 +1,723 @@
+#!/usr/bin/env python3
+"""
+Visualize 3D bad cases from analyze_3d_badcases.py results.
+
+This script focuses on matched 3D samples and renders:
+  - full-frame overlays with GT / active detections / highlighted badcases
+  - per-example panels with crop, simple BEV, and a metrics sidebar
+  - an index.json for downstream browsing
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import sys
+from collections import Counter, defaultdict
+from pathlib import Path
+
+import cv2
+import numpy as np
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from eval_tools.analysis.analyze_2d_fp_fn import build_config, class_name, parse_class_ids
+from eval_tools.evaluator.evaluator import Evaluator
+
+
+BOX_COLORS = {
+    "gt_all": (80, 220, 80),
+    "det_all": (150, 150, 150),
+    "target_gt": (40, 40, 255),
+    "target_det": (0, 215, 255),
+    "title_bg": (30, 30, 30),
+    "bev_gt": (40, 220, 80),
+    "bev_det": (0, 215, 255),
+}
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Visualize 3D bad cases from analysis_report.json."
+    )
+    parser.add_argument(
+        "--analysis-report",
+        type=str,
+        required=True,
+        help="Path to analysis_report.json generated by analyze_3d_badcases.py",
+    )
+    parser.add_argument("--config", type=str, required=True, help="Path to YAML evaluation config")
+    parser.add_argument("--det-path", type=str, help="Detection results root directory")
+    parser.add_argument("--gt-path", type=str, help="Ground-truth labels root directory")
+    parser.add_argument("--path-depth", type=int, choices=[1, 2], help="Directory depth")
+    parser.add_argument(
+        "--det-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Detection file format",
+    )
+    parser.add_argument(
+        "--gt-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Ground-truth file format",
+    )
+    parser.add_argument("--img-width", type=int, help="Image width")
+    parser.add_argument("--img-height", type=int, help="Image height")
+    parser.add_argument(
+        "--coord-system",
+        type=str,
+        choices=["camera", "ego"],
+        help="Coordinate system used by the parser/evaluator",
+    )
+    parser.add_argument(
+        "--iou-threshold",
+        type=float,
+        help="IoU threshold used for evaluator loading",
+    )
+    parser.add_argument(
+        "--conf-threshold",
+        type=float,
+        help="Confidence threshold for active detections shown on overlays",
+    )
+    parser.add_argument(
+        "--metrics",
+        nargs="+",
+        default=["longitudinal_error"],
+        help="Metrics to visualize from badcase_examples.",
+    )
+    parser.add_argument(
+        "--classes",
+        nargs="+",
+        default=None,
+        help="Optional class filter, e.g. car suv pedestrian",
+    )
+    parser.add_argument(
+        "--min-confidence",
+        type=float,
+        default=None,
+        help="Minimum confidence for badcase examples.",
+    )
+    parser.add_argument(
+        "--max-confidence",
+        type=float,
+        default=None,
+        help="Maximum confidence for badcase examples.",
+    )
+    parser.add_argument(
+        "--min-iou",
+        type=float,
+        default=None,
+        help="Minimum IoU for badcase examples.",
+    )
+    parser.add_argument(
+        "--max-iou",
+        type=float,
+        default=None,
+        help="Maximum IoU for badcase examples.",
+    )
+    parser.add_argument(
+        "--top-k",
+        type=int,
+        default=200,
+        help="Maximum number of examples to visualize after filtering",
+    )
+    parser.add_argument(
+        "--top-k-per-distance-bin",
+        type=int,
+        default=0,
+        help="Optional cap per longitudinal distance bin before applying --top-k. 0 disables bin-wise capping.",
+    )
+    parser.add_argument(
+        "--dedup-frame",
+        action="store_true",
+        help="Keep at most one example per case/frame/class/metric combination",
+    )
+    parser.add_argument(
+        "--group-by-distance-bin",
+        action="store_true",
+        help="Render and save outputs separately for each longitudinal distance bin.",
+    )
+    parser.add_argument(
+        "--line-thickness",
+        type=int,
+        default=2,
+        help="Base line thickness for non-highlight boxes",
+    )
+    parser.add_argument(
+        "--crop-scale",
+        type=float,
+        default=1.8,
+        help="Expand crop window around GT/det union box by this factor",
+    )
+    parser.add_argument(
+        "--jpeg-quality",
+        type=int,
+        default=92,
+        help="JPEG quality for saved visualizations",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="Output directory. Defaults to sibling 3d_vis_<report_name>",
+    )
+    return parser.parse_args()
+
+
+def normalize_token_set(values):
+    if not values:
+        return None
+    return {str(v).strip().lower() for v in values if str(v).strip()}
+
+
+def filter_examples(report, args):
+    pools = []
+    for metric_name in args.metrics:
+        pools.extend(report.get("badcase_examples", {}).get(metric_name, []))
+
+    class_filter = normalize_token_set(args.classes)
+    metric_filter = normalize_token_set(args.metrics)
+
+    filtered = []
+    for item in pools:
+        if metric_filter and str(item.get("metric_name", "")).lower() not in metric_filter:
+            continue
+        if class_filter and str(item.get("class_name", "")).lower() not in class_filter:
+            continue
+        confidence = item.get("confidence")
+        iou = item.get("iou")
+        if args.min_confidence is not None and (confidence is None or float(confidence) < args.min_confidence):
+            continue
+        if args.max_confidence is not None and (confidence is None or float(confidence) > args.max_confidence):
+            continue
+        if args.min_iou is not None and (iou is None or float(iou) < args.min_iou):
+            continue
+        if args.max_iou is not None and (iou is None or float(iou) > args.max_iou):
+            continue
+        filtered.append(item)
+
+    filtered.sort(
+        key=lambda item: (
+            float(item.get("metric_value_display", 0.0)),
+            float(item.get("confidence", 0.0)),
+            float(item.get("iou", 0.0)),
+        ),
+        reverse=True,
+    )
+
+    if args.dedup_frame:
+        deduped = []
+        seen = set()
+        for item in filtered:
+            key = (
+                item.get("case_name"),
+                item.get("frame_name"),
+                item.get("class_name"),
+                item.get("metric_name"),
+            )
+            if key in seen:
+                continue
+            seen.add(key)
+            deduped.append(item)
+        filtered = deduped
+
+    if args.top_k_per_distance_bin and args.top_k_per_distance_bin > 0:
+        kept = []
+        counts = Counter()
+        for item in filtered:
+            distance_bin = item.get("distance_bin") or "unbucketed"
+            if counts[distance_bin] >= args.top_k_per_distance_bin:
+                continue
+            kept.append(item)
+            counts[distance_bin] += 1
+        filtered = kept
+
+    if args.top_k is not None:
+        filtered = filtered[: args.top_k]
+    return filtered
+
+
+def bbox_to_int(bbox):
+    return [int(round(float(v))) for v in bbox]
+
+
+def draw_box(image, bbox, color, label=None, thickness=2):
+    x1, y1, x2, y2 = bbox_to_int(bbox)
+    cv2.rectangle(image, (x1, y1), (x2, y2), color, thickness, cv2.LINE_AA)
+    if label:
+        (tw, th), _ = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.55, 1)
+        y_text = max(0, y1 - th - 8)
+        cv2.rectangle(image, (x1, y_text), (x1 + tw + 8, y_text + th + 8), color, -1)
+        cv2.putText(
+            image,
+            label,
+            (x1 + 4, y_text + th + 2),
+            cv2.FONT_HERSHEY_SIMPLEX,
+            0.55,
+            (255, 255, 255),
+            1,
+            cv2.LINE_AA,
+        )
+
+
+def add_header(image, text):
+    h, w = image.shape[:2]
+    overlay = image.copy()
+    cv2.rectangle(overlay, (0, 0), (w, 42), BOX_COLORS["title_bg"], -1)
+    cv2.addWeighted(overlay, 0.55, image, 0.45, 0, image)
+    cv2.putText(
+        image,
+        text,
+        (10, 28),
+        cv2.FONT_HERSHEY_SIMPLEX,
+        0.7,
+        (255, 255, 255),
+        2,
+        cv2.LINE_AA,
+    )
+
+
+def make_crop(image, boxes, scale=1.8):
+    h, w = image.shape[:2]
+    valid = [bbox for bbox in boxes if bbox]
+    if not valid:
+        return image.copy(), (0, 0)
+
+    x1 = min(float(b[0]) for b in valid)
+    y1 = min(float(b[1]) for b in valid)
+    x2 = max(float(b[2]) for b in valid)
+    y2 = max(float(b[3]) for b in valid)
+
+    cx = 0.5 * (x1 + x2)
+    cy = 0.5 * (y1 + y2)
+    bw = max(32.0, (x2 - x1) * scale)
+    bh = max(32.0, (y2 - y1) * scale)
+
+    crop_x1 = max(0, int(round(cx - bw / 2)))
+    crop_y1 = max(0, int(round(cy - bh / 2)))
+    crop_x2 = min(w, int(round(cx + bw / 2)))
+    crop_y2 = min(h, int(round(cy + bh / 2)))
+    return image[crop_y1:crop_y2, crop_x1:crop_x2].copy(), (crop_x1, crop_y1)
+
+
+def shift_box(box, off_x, off_y):
+    if not box:
+        return None
+    return [
+        float(box[0]) - off_x,
+        float(box[1]) - off_y,
+        float(box[2]) - off_x,
+        float(box[3]) - off_y,
+    ]
+
+
+def draw_crop_panel(image, example, crop_scale):
+    gt_bbox = example.get("gt_bbox")
+    det_bbox = example.get("det_bbox")
+    crop, (off_x, off_y) = make_crop(image, [gt_bbox, det_bbox], scale=crop_scale)
+
+    gt_local = shift_box(gt_bbox, off_x, off_y)
+    det_local = shift_box(det_bbox, off_x, off_y)
+    if gt_local:
+        draw_box(
+            crop,
+            gt_local,
+            BOX_COLORS["target_gt"],
+            label=f"GT {example['class_name']}",
+            thickness=3,
+        )
+    if det_local:
+        draw_box(
+            crop,
+            det_local,
+            BOX_COLORS["target_det"],
+            label=f"Det {example['class_name']} {float(example.get('confidence', 0.0)):.2f}",
+            thickness=3,
+        )
+
+    add_header(
+        crop,
+        (
+            f"crop | {example['class_name']} | {example['metric_name']}="
+            f"{float(example.get('metric_value_display', 0.0)):.3f}{example.get('metric_unit', '')}"
+        ),
+    )
+    return crop
+
+
+def create_bev_panel(example, coord_system="camera", width=480, height=320, max_depth_m=100.0, max_lateral_m=30.0):
+    panel = np.full((height, width, 3), 245, dtype=np.uint8)
+
+    def project(point3d):
+        if not point3d or len(point3d) < 3:
+            return None
+        if coord_system == "camera":
+            x = float(point3d[0])
+            z = float(point3d[2])
+        else:
+            x = float(point3d[1])
+            z = float(point3d[0])
+        px = int(round((x + max_lateral_m) / (2.0 * max_lateral_m) * (width - 1)))
+        py = int(round((1.0 - max(0.0, min(z, max_depth_m)) / max_depth_m) * (height - 1)))
+        return px, py
+
+    for depth in range(0, int(max_depth_m) + 1, 10):
+        y = int(round((1.0 - depth / max_depth_m) * (height - 1)))
+        cv2.line(panel, (0, y), (width - 1, y), (225, 225, 225), 1, cv2.LINE_AA)
+        cv2.putText(panel, f"{depth}m", (6, max(14, y - 4)), cv2.FONT_HERSHEY_SIMPLEX, 0.4, (90, 90, 90), 1, cv2.LINE_AA)
+
+    for lat in range(-int(max_lateral_m), int(max_lateral_m) + 1, 10):
+        x = int(round((lat + max_lateral_m) / (2.0 * max_lateral_m) * (width - 1)))
+        cv2.line(panel, (x, 0), (x, height - 1), (232, 232, 232), 1, cv2.LINE_AA)
+
+    center_x = int(round((0.0 + max_lateral_m) / (2.0 * max_lateral_m) * (width - 1)))
+    cv2.line(panel, (center_x, 0), (center_x, height - 1), (180, 180, 180), 2, cv2.LINE_AA)
+
+    gt_pt = project(example.get("gt_center_3d"))
+    det_pt = project(example.get("det_center_3d"))
+    if gt_pt:
+        cv2.circle(panel, gt_pt, 7, BOX_COLORS["bev_gt"], -1, cv2.LINE_AA)
+        draw_heading_arrow(panel, gt_pt, float(example.get("gt_rotation_rad", 0.0)), BOX_COLORS["bev_gt"])
+    if det_pt:
+        cv2.circle(panel, det_pt, 7, BOX_COLORS["bev_det"], -1, cv2.LINE_AA)
+        draw_heading_arrow(panel, det_pt, float(example.get("det_rotation_rad", 0.0)), BOX_COLORS["bev_det"])
+    if gt_pt and det_pt:
+        cv2.line(panel, gt_pt, det_pt, (80, 80, 80), 2, cv2.LINE_AA)
+
+    add_header(panel, "simple BEV | GT=green | Det=orange")
+    return panel
+
+
+def draw_heading_arrow(canvas, anchor, rotation_rad, color, length_px=28):
+    dx = math.sin(rotation_rad) * length_px
+    dy = -math.cos(rotation_rad) * length_px
+    end_point = (int(round(anchor[0] + dx)), int(round(anchor[1] + dy)))
+    cv2.arrowedLine(canvas, anchor, end_point, color, 2, cv2.LINE_AA, tipLength=0.25)
+
+
+def add_sidebar(panel, example):
+    h, _ = panel.shape[:2]
+    sidebar = np.full((h, 420, 3), 28, dtype=np.uint8)
+    lines = [
+        f"case: {example.get('case_name')}",
+        f"frame: {example.get('frame_name')}",
+        f"class: {example.get('class_name')}",
+        f"metric: {example.get('metric_name')}",
+        f"metric_display: {example.get('metric_value_display')} {example.get('metric_unit', '')}",
+        f"conf: {example.get('confidence')}",
+        f"iou: {example.get('iou')}",
+        f"distance_z_m: {example.get('distance_longitudinal_m')}",
+        f"distance_x_m: {example.get('distance_lateral_m')}",
+        f"distance_bin: {example.get('distance_bin')}",
+        f"lateral_bin: {example.get('lateral_bin')}",
+        f"lat_err_m: {example.get('lateral_error_m')}",
+        f"long_err_m: {example.get('longitudinal_error_m')}",
+        f"long_rel_err: {example.get('longitudinal_relative_error')}",
+        f"heading_deg: {example.get('heading_error_deg')}",
+        f"heading_relaxed_deg: {example.get('heading_error_relaxed_deg')}",
+        f"is_reversal: {example.get('is_reversal')}",
+        f"gt_id: {example.get('gt_id')}",
+    ]
+
+    y = 36
+    for line in lines:
+        cv2.putText(
+            sidebar,
+            str(line),
+            (12, y),
+            cv2.FONT_HERSHEY_SIMPLEX,
+            0.56,
+            (235, 235, 235),
+            1,
+            cv2.LINE_AA,
+        )
+        y += 28
+
+    return np.hstack([panel, sidebar])
+
+
+def resize_to_height(image, target_height):
+    h, w = image.shape[:2]
+    if h == target_height:
+        return image
+    scale = target_height / max(h, 1)
+    return cv2.resize(image, (max(1, int(round(w * scale))), target_height))
+
+
+def combine_panels(full_image, crop_image, bev_image, example):
+    target_h = max(full_image.shape[0], crop_image.shape[0], bev_image.shape[0])
+    full_resized = resize_to_height(full_image, target_h)
+    crop_resized = resize_to_height(crop_image, target_h)
+    bev_resized = resize_to_height(bev_image, target_h)
+    panel = np.hstack([full_resized, crop_resized, bev_resized])
+    return add_sidebar(panel, example)
+
+
+def find_pair_map(config):
+    evaluator = Evaluator(
+        config=config,
+        iou_threshold=float(config.get("matching", {}).get("iou_threshold", 0.5)),
+        num_workers=1,
+        save_detailed_matches=False,
+    )
+    dataset_cfg = config["dataset"]
+    image_cfg = config["image"]
+    evaluator.load_data_from_paths(
+        det_root=dataset_cfg["det_path"],
+        gt_root=dataset_cfg["gt_path"],
+        img_width=image_cfg.get("width", 1920),
+        img_height=image_cfg.get("height", 1080),
+        path_depth=dataset_cfg.get("path_depth", 1),
+        det_format=dataset_cfg.get("det_format", "auto"),
+        gt_format=dataset_cfg.get("gt_format", "auto"),
+    )
+
+    pair_map = {}
+    for pair in evaluator.image_pairs:
+        level1_name = pair.get("level1_name")
+        if level1_name:
+            case_key = f"{level1_name}/{pair['case']}"
+        else:
+            case_key = pair["case"]
+        pair_map[(case_key, pair["frame"])] = pair
+    return pair_map, evaluator
+
+
+def find_image_path(pair):
+    gt_file = Path(pair["gt_file"])
+    case_dir = gt_file.parent.parent
+    images_dir = case_dir / "images"
+    stem = gt_file.stem
+    for suffix in (".png", ".jpg", ".jpeg", ".bmp"):
+        candidate = images_dir / f"{stem}{suffix}"
+        if candidate.exists():
+            return candidate
+    matches = list(images_dir.glob(f"{stem}.*"))
+    return matches[0] if matches else None
+
+
+def render_frame_overlay(image, gts, active_dets, frame_examples, class_ids, line_thickness):
+    canvas = image.copy()
+    selected_class_ids = set(class_ids)
+
+    for gt in gts:
+        if gt["label"] not in selected_class_ids:
+            continue
+        draw_box(
+            canvas,
+            gt["bbox_2d"],
+            BOX_COLORS["gt_all"],
+            label=f"GT {class_name(gt['label'])}",
+            thickness=line_thickness,
+        )
+
+    for det in active_dets:
+        if det["label"] not in selected_class_ids:
+            continue
+        conf = float(det.get("confidence", 0.0))
+        draw_box(
+            canvas,
+            det["bbox_2d"],
+            BOX_COLORS["det_all"],
+            label=f"Det {class_name(det['label'])} {conf:.2f}",
+            thickness=line_thickness,
+        )
+
+    for idx, example in enumerate(frame_examples, 1):
+        if example.get("gt_bbox"):
+            draw_box(
+                canvas,
+                example["gt_bbox"],
+                BOX_COLORS["target_gt"],
+                label=f"GT#{idx} {example['class_name']}",
+                thickness=max(3, line_thickness + 1),
+            )
+        if example.get("det_bbox"):
+            label = (
+                f"Det#{idx} {example['class_name']} "
+                f"{float(example.get('confidence', 0.0)):.2f} "
+                f"{example['metric_name']}={float(example.get('metric_value_display', 0.0)):.2f}"
+            )
+            draw_box(
+                canvas,
+                example["det_bbox"],
+                BOX_COLORS["target_det"],
+                label=label,
+                thickness=max(3, line_thickness + 1),
+            )
+
+    headline = (
+        f"3D badcase visualization | examples={len(frame_examples)} | "
+        f"GT=green Det=gray TargetGT=red TargetDet=orange"
+    )
+    add_header(canvas, headline)
+    return canvas
+
+
+def ensure_dir(path):
+    path.mkdir(parents=True, exist_ok=True)
+    return path
+
+
+def resolve_output_dirs(output_dir, distance_bin=None, group_by_distance_bin=False):
+    if group_by_distance_bin:
+        safe_bin = sanitize_token(distance_bin or "unbucketed")
+        base_dir = output_dir / "distance_bins" / safe_bin
+    else:
+        base_dir = output_dir
+    frame_dir = ensure_dir(base_dir / "frames")
+    example_dir = ensure_dir(base_dir / "examples")
+    return base_dir, frame_dir, example_dir
+
+
+def sanitize_token(value):
+    return str(value).replace("/", "__").replace("\\", "__").replace(" ", "_")
+
+
+def default_output_dir(report_path):
+    report_path = Path(report_path)
+    return report_path.parent / f"3d_vis_{report_path.stem}"
+
+
+def main():
+    args = parse_args()
+    with open(args.analysis_report, "r") as file:
+        report = json.load(file)
+
+    config = build_config(args)
+    class_ids = parse_class_ids(args.classes) if args.classes else parse_class_ids(report["metadata"]["classes"])
+    filtered_examples = filter_examples(report, args)
+
+    if not filtered_examples:
+        print("No examples matched the current filters.")
+        return
+
+    pair_map, evaluator = find_pair_map(config)
+    output_dir = Path(args.output_dir) if args.output_dir else default_output_dir(args.analysis_report)
+
+    by_frame = defaultdict(list)
+    for item in filtered_examples:
+        group_distance_bin = item.get("distance_bin") if args.group_by_distance_bin else None
+        by_frame[(item["case_name"], item["frame_name"], group_distance_bin)].append(item)
+
+    index = {
+        "analysis_report": str(Path(args.analysis_report).resolve()),
+        "num_examples": len(filtered_examples),
+        "num_frames": len(by_frame),
+        "metrics": args.metrics,
+        "classes": [class_name(cid) for cid in class_ids],
+        "group_by_distance_bin": bool(args.group_by_distance_bin),
+        "top_k_per_distance_bin": int(args.top_k_per_distance_bin),
+        "distance_bins": {},
+        "frames": [],
+    }
+
+    conf_threshold = float(
+        config.get("metrics_3d", {}).get(
+            "conf_threshold",
+            config.get("metrics_2d", {}).get("conf_threshold", 0.5),
+        )
+    )
+
+    saved_frame_dirs = set()
+    saved_example_dirs = set()
+
+    for frame_idx, ((case_name, frame_name, distance_bin), frame_examples) in enumerate(by_frame.items(), 1):
+        pair = pair_map.get((case_name, frame_name))
+        if pair is None:
+            print(f"Warning: failed to locate pair for {case_name}/{frame_name}, skipping")
+            continue
+
+        image_path = find_image_path(pair)
+        if image_path is None or not image_path.exists():
+            print(f"Warning: image not found for {case_name}/{frame_name}, skipping")
+            continue
+
+        image = cv2.imread(str(image_path))
+        if image is None:
+            print(f"Warning: failed to read image: {image_path}")
+            continue
+
+        gts = Evaluator._parse_ground_truths_for_pair(pair, evaluator.coord_system)
+        dets = Evaluator._parse_detections_for_pair(pair, evaluator.coord_system)
+        active_dets = [det for det in dets if float(det.get("confidence", 0.0)) >= conf_threshold]
+
+        frame_overlay = render_frame_overlay(
+            image,
+            gts,
+            active_dets,
+            frame_examples,
+            class_ids,
+            line_thickness=args.line_thickness,
+        )
+
+        base_dir, frame_dir, example_dir = resolve_output_dirs(
+            output_dir,
+            distance_bin=distance_bin,
+            group_by_distance_bin=args.group_by_distance_bin,
+        )
+        saved_frame_dirs.add(str(frame_dir))
+        saved_example_dirs.add(str(example_dir))
+
+        frame_rel = Path(frame_dir.relative_to(output_dir)) / (
+            f"{frame_idx:04d}_{sanitize_token(case_name)}_{sanitize_token(frame_name)}.jpg"
+        )
+        frame_path = output_dir / frame_rel
+        cv2.imwrite(
+            str(frame_path),
+            frame_overlay,
+            [int(cv2.IMWRITE_JPEG_QUALITY), int(args.jpeg_quality)],
+        )
+
+        frame_entry = {
+            "case_name": case_name,
+            "frame_name": frame_name,
+            "distance_bin": distance_bin,
+            "image_path": str(image_path),
+            "frame_visualization": str(frame_rel),
+            "num_examples": len(frame_examples),
+            "examples": [],
+        }
+        distance_key = distance_bin or "all"
+        index["distance_bins"].setdefault(distance_key, {"num_frames": 0, "num_examples": 0})
+        index["distance_bins"][distance_key]["num_frames"] += 1
+        index["distance_bins"][distance_key]["num_examples"] += len(frame_examples)
+
+        for ex_idx, example in enumerate(frame_examples, 1):
+            crop_image = draw_crop_panel(image.copy(), example, crop_scale=args.crop_scale)
+            bev_image = create_bev_panel(example, coord_system=evaluator.coord_system)
+            panel = combine_panels(frame_overlay.copy(), crop_image, bev_image, example)
+            rel = Path(example_dir.relative_to(output_dir)) / (
+                f"{frame_idx:04d}_{ex_idx:02d}_"
+                f"{sanitize_token(case_name)}_{sanitize_token(frame_name)}_"
+                f"{sanitize_token(example['class_name'])}_{sanitize_token(example['metric_name'])}.jpg"
+            )
+            panel_path = output_dir / rel
+            cv2.imwrite(
+                str(panel_path),
+                panel,
+                [int(cv2.IMWRITE_JPEG_QUALITY), int(args.jpeg_quality)],
+            )
+
+            example_record = dict(example)
+            example_record["visualization"] = str(rel)
+            frame_entry["examples"].append(example_record)
+
+        index["frames"].append(frame_entry)
+
+    index_path = output_dir / "index.json"
+    with open(index_path, "w") as file:
+        json.dump(index, file, indent=2)
+
+    print(f"Saved visualization index to: {index_path}")
+    print(f"Saved frame overlays to:      {', '.join(sorted(saved_frame_dirs))}")
+    print(f"Saved example panels to:      {', '.join(sorted(saved_example_dirs))}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/analysis/visualize_3d_badcases.sh

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ANALYSIS_REPORT=${ANALYSIS_REPORT:-/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_3d/analysis_report.json}
+CONFIG_PATH=${CONFIG_PATH:-eval_tools/configs/eval_config_yolov26s-roi0.yaml}
+OUTPUT_BASE=${OUTPUT_BASE:-$(dirname "${ANALYSIS_REPORT}")/visualizations_distance}
+TOP_K=${TOP_K:-300}
+TOP_K_PER_DISTANCE_BIN=${TOP_K_PER_DISTANCE_BIN:-50}
+
+METRICS=(longitudinal_error heading_error reversal)
+CLASSES=(car suv van bus truck pedestrian bicycle)
+PYTHON_BIN=${PYTHON_BIN:-/deeplearning_team/ydong/dongying/miniconda/envs/dev/bin/python}
+
+for metric_name in "${METRICS[@]}"; do
+  for class_name in "${CLASSES[@]}"; do
+    output_dir="${OUTPUT_BASE}/${metric_name}_vis_${class_name}"
+
+    echo "============================================================"
+    echo "Running 3D visualization"
+    echo "  metric: ${metric_name}"
+    echo "  class:  ${class_name}"
+    echo "  output: ${output_dir}"
+    echo "============================================================"
+
+    "${PYTHON_BIN}" eval_tools/analysis/visualize_3d_badcases.py \
+      --analysis-report "${ANALYSIS_REPORT}" \
+      --config "${CONFIG_PATH}" \
+      --metrics "${metric_name}" \
+      --classes "${class_name}" \
+      --top-k "${TOP_K}" \
+      --top-k-per-distance-bin "${TOP_K_PER_DISTANCE_BIN}" \
+      --group-by-distance-bin \
+      --output-dir "${output_dir}"
+  done
+done