Initial commit: photonAI MZM MLP baseline only

Made-with: Cursor

.gitignore

@@ -0,0 +1,32 @@
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+.env
+.venv
+venv/
+ENV/
+.idea/
+.vscode/
+*.log
+.DS_Store
+.pytest_cache/
+results/run_*/
+data/*.txt
+!data/.gitkeep

README.md

@@ -0,0 +1,191 @@
+# MZM 器件性能 MLP 回归基线（PyTorch）
+
+本项目实现一个**多输出回归**基线模型：用 **8 个器件/偏置参数**预测 **3 个射频性能指标**。代码面向科研复现：可配置 YAML、固定随机种子、训练集拟合标准化器、完整日志与可视化产物。
+
+## 项目简介
+
+- **任务类型**：监督学习，多输出回归（非分类）。
+- **输入（8 维）**：工艺与偏置相关参数。
+- **输出（3 维）**：`BW_3dB`、`IL`、`V_pi`。
+- **模型**：原生 PyTorch MLP，可选 BatchNorm / Dropout / 残差（同维时相加）。
+- **损失**：默认在**标准化后的输出空间**使用加权 `SmoothL1Loss`（Huber）；可选加权 MSE。
+- **v1 目标**：先把数据清洗、划分、训练、评估、日志与可视化流程跑通；**不引入 physics loss**。
+
+## 数据格式
+
+数据为 **txt 或 csv**，每行 **11 个逗号分隔的浮点数**，无表头（txt）或表头与下列字段一致（csv）。
+
+| 顺序 | 列名 | 含义 | 作为 |
+| --- | --- | --- | --- |
+| 1 | `PN_offset` | PN 偏移 | 输入 |
+| 2 | `Bias_V` | 偏置电压 | 输入 |
+| 3 | `Core_width` | 芯区宽度 | 输入 |
+| 4 | `P+_width` | P+ 区宽度 | 输入 |
+| 5 | `N+_width` | N+ 区宽度 | 输入 |
+| 6 | `P_width` | P 区宽度 | 输入 |
+| 7 | `N_width` | N 区宽度 | 输入 |
+| 8 | `Phase_length` | 相位区长度 | 输入 |
+| 9 | `BW_3dB` | 3 dB 带宽 | 目标 |
+| 10 | `IL` | 插入损耗 | 目标 |
+| 11 | `V_pi` | 半波电压 | 目标 |
+
+- 自动忽略空行与行首行尾空格。
+- 每行必须恰好 **11 列**，否则整文件解析失败并给出错误行号提示。
+
+## TXT 数据清洗流程（以 V_pi 为准）
+
+本仓库约定：**txt 每行从左到右第 11 个逗号分隔浮点数**即半波电压 **`V_pi`**（与表头列名一致）。清洗时以该列为**物理可信区间**的主门控，避免异常仿真/标注污染训练。
+
+建议按以下顺序理解流水线（与 `src/preprocess.py` 中 `clean_dataframe` 实现一致）：
+
+1. **解析与建表**：读取 txt → 校验每行 11 列 → 转为 `float` → 构建 `DataFrame`（最后一列为 `V_pi`）。
+2. **（可选）去重**：`remove_duplicate_rows: true` 时删除 11 列完全相同的重复行。
+3. **V_pi 区间门控（主清洗）**：默认启用 `filter_v_pi_range: true`，仅保留  
+   `v_pi_min <= V_pi <= v_pi_max`（默认 **`[0, 500]`**）。**区间之外整行剔除**。  
+   该步骤专门针对「以最后一列 `V_pi` 为正常范围」的需求。
+4. **（可选）严格正电压**：`remove_nonpositive_vpi: true` 时，在区间过滤之后再删除 `V_pi <= 0`（若需保留 `V_pi = 0` 且仍在 `[0,500]` 内，请保持为 `false`）。
+5. **后续步骤**：train/val/test 划分、（可选）训练集离群策略、仅在训练集上拟合 `StandardScaler` 等，与原先一致。
+
+清洗前会在日志与 `data_report.md` 中报告：给定 `[v_pi_min, v_pi_max]` 下 **`V_pi` 越界行数**、重复样本、同输入异输出等统计，便于核对。
+
+## 环境要求
+
+- Python **3.10+**（已在 3.13 下通过冒烟测试）。
+- 推荐使用虚拟环境。
+
+### 安装依赖
+
+```bash
+cd /path/to/photonAI
+python -m venv .venv
+source .venv/bin/activate   # Windows 使用 .venv\Scripts\activate
+pip install -U pip
+pip install -r requirements.txt
+```
+
+## 放置数据
+
+1. 将原始 txt（例如仓库根目录下的 `Sim_MZM_dataset.txt`）复制或软链接到 `data/dataset.txt`。
+2. 或在 `configs/default.yaml` 中修改 `data_path` 为绝对路径或相对项目根目录的路径。
+
+若路径不存在，程序会给出明确报错，不会静默失败。
+
+## 训练
+
+```bash
+python -m src.main train --config configs/default.yaml
+```
+
+或使用脚本：
+
+```bash
+bash scripts/train.sh
+```
+
+训练会在 `results/run_时间戳/` 下生成：
+
+- `config_snapshot.yaml`：本次运行配置快照。
+- `split_indices.json`：对**清洗后**样本行的 train/val/test 索引，便于 `eval` 完全复现划分。
+- `x_scaler.pkl` / `y_scaler.pkl`：`StandardScaler`，推理阶段用于反标准化。
+- `data_report.md` / `data_stats.csv`：数据统计与清洗说明。
+- `cleaning_meta.json`：清洗与划分元信息。
+- `train_log.csv`：逐 epoch 的 train/val loss 与学习率。
+- `checkpoints/best.pt`、`checkpoints/last.pt`：最优与最后一轮权重。
+- 训练结束后：`metrics.csv`、`summary.json`、`summary.md`、`test_predictions.csv`、`figures/*.png`。
+
+**说明（损失列）**：`metrics.csv` / `summary.*` 中的 `loss` 与 `*_loss` 均在**标准化输出空间**按训练准则（Huber / 加权 MSE）计算；物理量空间以 **MAE / RMSE / R²** 为主指标。
+
+## 评估（复现划分与 scaler）
+
+在**同一数据文件**与 `config_snapshot.yaml` 前提下，可仅运行评估：
+
+```bash
+python -m src.main eval --config configs/default.yaml --run-dir results/run_YYYYMMDD_HHMMSS
+```
+
+若不指定 `--run-dir`，将在 `configs/default.yaml` 的 `output_dir`（默认 `results`）下自动选择**最近修改时间**的 `run_*` 目录。
+
+```bash
+bash scripts/eval.sh --run-dir results/run_某次训练
+```
+
+## 推理
+
+输入文件需包含上述 **8 个输入列**（csv 带表头，或 8 列无表头 txt）。
+
+```bash
+python -m src.main infer --config configs/default.yaml --input path/to/inputs.csv --output path/to/preds.csv
+```
+
+脚本封装：
+
+```bash
+bash scripts/infer.sh path/to/inputs.csv --run-dir results/run_某次训练 --output preds.csv
+```
+
+输出列为 8 个输入 + `pred_BW_3dB`、`pred_IL`、`pred_V_pi`（**物理量空间**，已反标准化）。
+
+## 测试
+
+```bash
+pip install pytest
+pytest -q tests/test_smoke.py
+```
+
+## 配置说明（`configs/default.yaml`）
+
+主要字段：
+
+- **数据与清洗**：`data_path`、`remove_duplicate_rows`、**`filter_v_pi_range` / `v_pi_min` / `v_pi_max`**（默认按 **`V_pi ∈ [0, 500]`** 剔除越界行，对应 txt **第 11 列**）、`remove_nonpositive_vpi`、`outlier_strategy`（`none` / `iqr` / `zscore` / `quantile_clip`）及 `outlier_apply_to`（`targets` / `all`）。
+- **划分**：`split_ratios`、`random_seed`；先 shuffle 再切分；**仅在训练子集**上拟合标准化器；离群阈值（若启用）也在训练子集上统计。
+- **模型**：`hidden_dims`、`batchnorm`、`dropout`、`residual`。
+- **训练**：`AdamW`、`lr`、`weight_decay`、`batch_size`、`epochs`、早停 `early_stopping_patience`。
+- **调度器**：`cosine`（默认）或 `plateau`。
+- **损失**：`huber`（默认）或 `weighted_mse`，`target_weights` 长度须为 3。
+
+默认策略刻意**不删除**仅因统计极端的样本（`outlier_strategy: none`），但在报告中给出极端值计数；**默认以 `V_pi` 物理区间 `[0,500]` 删除越界行**；`remove_nonpositive_vpi` 默认为 `false`，以便与「0 属于合法下界」一致，需要时可改为 `true`。
+
+## 项目结构
+
+```text
+.
+├── README.md
+├── requirements.txt
+├── .gitignore
+├── configs
+│   └── default.yaml
+├── data
+├── reports
+├── results
+├── scripts
+│   ├── train.sh
+│   ├── eval.sh
+│   └── infer.sh
+├── src
+│   ├── __init__.py
+│   ├── config.py
+│   ├── utils.py
+│   ├── data.py
+│   ├── preprocess.py
+│   ├── model.py
+│   ├── losses.py
+│   ├── metrics.py
+│   ├── trainer.py
+│   ├── evaluate.py
+│   ├── infer.py
+│   ├── plots.py
+│   └── main.py
+└── tests
+    └── test_smoke.py
+```
+
+## 后续可扩展方向
+
+- **Physics-informed loss**：在标准化空间外叠加与器件物理相关的软约束。
+- **PINN / 解析近似混合**：将部分输出与简化解析模型对齐。
+- **结构搜索**：在 `hidden_dims`、残差块、Bayesian 优化超参等方向扩展。
+- **不确定度**：深度集成、MC Dropout、浅层高斯过程等。
+
+## 许可证与引用
+
+若用于论文，请在方法部分说明数据处理、划分方式与随机种子；并引用本仓库或内部项目号（自行补充）。

configs/default.yaml

@@ -0,0 +1,64 @@
+# 默认配置：MZM MLP 多输出回归基线
+# 将数据 txt 放到 data/ 下并修改 data_path，或保持路径指向你的文件
+
+data_path: data/dataset.txt
+
+split_ratios: [0.7, 0.15, 0.15]  # train, val, test；可改为 [0.8, 0.1, 0.1]
+random_seed: 42
+
+remove_duplicate_rows: true
+
+# 异常值处理策略：none | iqr | zscore | quantile_clip
+# 默认仅报告极端值，不删除；物理上不可信的 V_pi 由下方区间门控剔除
+outlier_strategy: none
+# 启用非 none 策略时，在训练子集上拟合阈值；iqr/zscore 仅删训练集离群行；quantile_clip 按训练分位数 winsorize
+outlier_apply_to: targets  # targets | all
+outlier_config:
+  iqr_k: 1.5
+  zscore_threshold: 4.0
+  quantile_lower: 0.001
+  quantile_upper: 0.999
+
+# 以 txt 第 11 列（列名 V_pi）为物理门控：仅保留闭区间 [v_pi_min, v_pi_max] 内样本
+filter_v_pi_range: true
+v_pi_min: 0.0
+v_pi_max: 500.0
+
+# 在区间过滤之后，是否再剔除 V_pi<=0；若需保留 V_pi=0（仍在 [0,500] 内），请设为 false
+remove_nonpositive_vpi: false
+
+model:
+  input_dim: 8
+  hidden_dims: [200, 300, 350, 300, 200]
+  output_dim: 3
+  batchnorm: false
+  dropout: 0.0
+  residual: false
+
+optimizer:
+  name: adamw
+  lr: 0.001
+  weight_decay: 0.0001
+
+scheduler:
+  type: cosine  # cosine | plateau
+  plateau_factor: 0.5
+  plateau_patience: 10
+  plateau_min_lr: 1.0e-6
+
+training:
+  batch_size: 128
+  epochs: 300
+  early_stopping_patience: 30
+  num_workers: 0
+
+loss:
+  type: huber  # huber | weighted_mse
+  huber_delta: 1.0
+  target_weights: [1.0, 1.0, 1.0]
+
+# 总输出目录；每次训练会在其下创建 run_时间戳/
+output_dir: results
+
+# 评估/推理时若未指定 run_dir，可填最近一次 run 的路径（可选）
+last_run_dir: null

requirements.txt

@@ -0,0 +1,9 @@
+torch>=2.0.0
+numpy>=1.24.0
+pandas>=2.0.0
+scikit-learn>=1.3.0
+matplotlib>=3.7.0
+seaborn>=0.12.0
+pyyaml>=6.0
+tqdm>=4.65.0
+pytest>=7.4.0

scripts/eval.sh

@@ -0,0 +1,5 @@
+#!/usr/bin/env bash
+set -euo pipefail
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+cd "$ROOT"
+python -m src.main eval --config configs/default.yaml "$@"

scripts/infer.sh

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+set -euo pipefail
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+cd "$ROOT"
+if [[ $# -lt 1 ]]; then
+  echo "用法: $0 <input.csv|txt> [--run-dir path] [--output path]" >&2
+  exit 1
+fi
+python -m src.main infer --config configs/default.yaml --input "$1" "${@:2}"

scripts/train.sh

@@ -0,0 +1,5 @@
+#!/usr/bin/env bash
+set -euo pipefail
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+cd "$ROOT"
+python -m src.main train --config configs/default.yaml

src/__init__.py

@@ -0,0 +1,3 @@
+"""MZM MLP 回归项目包。"""
+
+__version__ = "1.0.0"

src/config.py

@@ -0,0 +1,164 @@
+"""YAML 配置加载与校验。"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any, List, Optional
+
+import yaml
+
+
+@dataclass
+class ModelConfig:
+    input_dim: int = 8
+    hidden_dims: List[int] = field(default_factory=lambda: [200, 300, 350, 300, 200])
+    output_dim: int = 3
+    batchnorm: bool = False
+    dropout: float = 0.0
+    residual: bool = False
+
+
+@dataclass
+class OptimizerConfig:
+    name: str = "adamw"
+    lr: float = 1e-3
+    weight_decay: float = 1e-4
+
+
+@dataclass
+class SchedulerConfig:
+    type: str = "cosine"  # cosine | plateau
+    plateau_factor: float = 0.5
+    plateau_patience: int = 10
+    plateau_min_lr: float = 1e-6
+
+
+@dataclass
+class TrainingConfig:
+    batch_size: int = 128
+    epochs: int = 300
+    early_stopping_patience: int = 30
+    num_workers: int = 0
+
+
+@dataclass
+class LossConfig:
+    type: str = "huber"  # huber | weighted_mse
+    huber_delta: float = 1.0
+    target_weights: List[float] = field(default_factory=lambda: [1.0, 1.0, 1.0])
+
+
+@dataclass
+class OutlierConfig:
+    iqr_k: float = 1.5
+    zscore_threshold: float = 4.0
+    quantile_lower: float = 0.001
+    quantile_upper: float = 0.999
+
+
+@dataclass
+class AppConfig:
+    data_path: str
+    split_ratios: List[float]
+    random_seed: int
+    remove_duplicate_rows: bool
+    outlier_strategy: str
+    outlier_config: OutlierConfig
+    outlier_apply_to: str  # targets | all
+    remove_nonpositive_vpi: bool
+    filter_v_pi_range: bool
+    v_pi_min: float
+    v_pi_max: float
+    model: ModelConfig
+    optimizer: OptimizerConfig
+    scheduler: SchedulerConfig
+    training: TrainingConfig
+    loss: LossConfig
+    output_dir: str
+    last_run_dir: Optional[str] = None
+
+    @staticmethod
+    def from_dict(raw: dict[str, Any]) -> "AppConfig":
+        m = raw.get("model", {})
+        o = raw.get("optimizer", {})
+        s = raw.get("scheduler", {})
+        t = raw.get("training", {})
+        l = raw.get("loss", {})
+        oc = raw.get("outlier_config", {})
+        return AppConfig(
+            data_path=str(raw["data_path"]),
+            split_ratios=list(raw["split_ratios"]),
+            random_seed=int(raw["random_seed"]),
+            remove_duplicate_rows=bool(raw["remove_duplicate_rows"]),
+            outlier_strategy=str(raw.get("outlier_strategy", "none")),
+            outlier_config=OutlierConfig(
+                iqr_k=float(oc.get("iqr_k", 1.5)),
+                zscore_threshold=float(oc.get("zscore_threshold", 4.0)),
+                quantile_lower=float(oc.get("quantile_lower", 0.001)),
+                quantile_upper=float(oc.get("quantile_upper", 0.999)),
+            ),
+            outlier_apply_to=str(raw.get("outlier_apply_to", "targets")),
+            remove_nonpositive_vpi=bool(raw.get("remove_nonpositive_vpi", False)),
+            filter_v_pi_range=bool(raw.get("filter_v_pi_range", True)),
+            v_pi_min=float(raw.get("v_pi_min", 0.0)),
+            v_pi_max=float(raw.get("v_pi_max", 500.0)),
+            model=ModelConfig(
+                input_dim=int(m.get("input_dim", 8)),
+                hidden_dims=list(m.get("hidden_dims", [200, 300, 350, 300, 200])),
+                output_dim=int(m.get("output_dim", 3)),
+                batchnorm=bool(m.get("batchnorm", False)),
+                dropout=float(m.get("dropout", 0.0)),
+                residual=bool(m.get("residual", False)),
+            ),
+            optimizer=OptimizerConfig(
+                name=str(o.get("name", "adamw")),
+                lr=float(o.get("lr", 1e-3)),
+                weight_decay=float(o.get("weight_decay", 1e-4)),
+            ),
+            scheduler=SchedulerConfig(
+                type=str(s.get("type", "cosine")),
+                plateau_factor=float(s.get("plateau_factor", 0.5)),
+                plateau_patience=int(s.get("plateau_patience", 10)),
+                plateau_min_lr=float(s.get("plateau_min_lr", 1e-6)),
+            ),
+            training=TrainingConfig(
+                batch_size=int(t.get("batch_size", 128)),
+                epochs=int(t.get("epochs", 300)),
+                early_stopping_patience=int(t.get("early_stopping_patience", 30)),
+                num_workers=int(t.get("num_workers", 0)),
+            ),
+            loss=LossConfig(
+                type=str(l.get("type", "huber")),
+                huber_delta=float(l.get("huber_delta", 1.0)),
+                target_weights=[float(x) for x in l.get("target_weights", [1.0, 1.0, 1.0])],
+            ),
+            output_dir=str(raw.get("output_dir", "results")),
+            last_run_dir=raw.get("last_run_dir"),
+        )
+
+
+def load_config(path: str | Path) -> AppConfig:
+    """从 YAML 文件加载配置并做基本校验。"""
+    path = Path(path)
+    if not path.is_file():
+        raise FileNotFoundError(f"配置文件不存在: {path}")
+    with path.open("r", encoding="utf-8") as f:
+        raw = yaml.safe_load(f)
+    if not isinstance(raw, dict):
+        raise ValueError("YAML 根节点必须是字典")
+    cfg = AppConfig.from_dict(raw)
+    sr = cfg.split_ratios
+    if len(sr) != 3:
+        raise ValueError("split_ratios 必须为长度为 3 的列表 [train, val, test]")
+    if abs(sum(sr) - 1.0) > 1e-6:
+        raise ValueError(f"split_ratios 之和必须为 1，当前为 {sum(sr)}")
+    if cfg.outlier_strategy not in ("none", "iqr", "zscore", "quantile_clip"):
+        raise ValueError(f"未知 outlier_strategy: {cfg.outlier_strategy}")
+    if cfg.outlier_apply_to not in ("targets", "all"):
+        raise ValueError("outlier_apply_to 必须为 targets 或 all")
+    if len(cfg.loss.target_weights) != 3:
+        raise ValueError("loss.target_weights 长度必须为 3")
+    if cfg.filter_v_pi_range and cfg.v_pi_min >= cfg.v_pi_max:
+        raise ValueError("启用 filter_v_pi_range 时须满足 v_pi_min < v_pi_max")
+    return cfg

src/data.py

@@ -0,0 +1,155 @@
+"""原始 txt 数据加载、字段定义与清洗前质量检查。"""
+
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+
+logger = logging.getLogger(__name__)
+
+# 与论文/数据说明一致的物理列名（含 +）
+INPUT_COLUMNS = [
+    "PN_offset",
+    "Bias_V",
+    "Core_width",
+    "P+_width",
+    "N+_width",
+    "P_width",
+    "N_width",
+    "Phase_length",
+]
+TARGET_COLUMNS = ["BW_3dB", "IL", "V_pi"]
+ALL_COLUMNS = INPUT_COLUMNS + TARGET_COLUMNS
+EXPECTED_COLS = 11
+# txt 行内从左到右第 11 个字段即 V_pi，与 DataFrame 最后一列一致，用作物理可信区间清洗门控
+V_PI_TXT_1BASED_INDEX = 11
+
+
+def load_raw_txt(path: str | Path) -> pd.DataFrame:
+    """
+    从 txt 读取数据：逗号分隔、11 列浮点；跳过空行与纯空白行。
+
+    Raises:
+        FileNotFoundError: 文件不存在
+        ValueError: 行格式或列数不合法
+    """
+    path = Path(path)
+    if not path.is_file():
+        raise FileNotFoundError(
+            f"数据文件不存在: {path.resolve()}。请将 11 列逗号分隔的 txt 放到该路径，"
+            f"或修改 configs/default.yaml 中的 data_path。"
+        )
+
+    rows: list[list[float]] = []
+    bad_lines: list[tuple[int, str]] = []
+
+    with path.open("r", encoding="utf-8") as f:
+        for line_no, raw in enumerate(f, start=1):
+            line = raw.strip()
+            if not line:
+                continue
+            parts = [p.strip() for p in line.split(",")]
+            if len(parts) != EXPECTED_COLS:
+                bad_lines.append((line_no, f"列数={len(parts)}，期望 {EXPECTED_COLS}"))
+                continue
+            try:
+                vals = [float(p) for p in parts]
+            except ValueError as e:
+                bad_lines.append((line_no, str(e)))
+                continue
+            rows.append(vals)
+
+    if bad_lines:
+        preview = "; ".join(f"行{k}:{msg}" for k, msg in bad_lines[:5])
+        if len(bad_lines) > 5:
+            preview += f"; ... 共 {len(bad_lines)} 行有问题"
+        raise ValueError(f"数据解析失败（{path}）。{preview}")
+
+    if not rows:
+        raise ValueError(f"文件为空或无非空数据行: {path}")
+
+    df = pd.DataFrame(rows, columns=ALL_COLUMNS)
+    logger.info(
+        "已加载 %d 行，%d 列（第 %d 列为 V_pi，用于物理区间门控）",
+        len(df),
+        len(df.columns),
+        V_PI_TXT_1BASED_INDEX,
+    )
+    return df
+
+
+def basic_statistics(df: pd.DataFrame) -> pd.DataFrame:
+    """返回 describe() 风格的汇总（含 count/mean/std/min/max）。"""
+    return df.describe().T
+
+
+def _extreme_report(series: pd.Series, name: str, z: float = 5.0) -> dict:
+    """单序列极端值统计：|z-score|>z 的个数（相对自身均值方差）。"""
+    s = series.astype(float)
+    mu, sig = float(s.mean()), float(s.std(ddof=0))
+    if sig < 1e-12:
+        return {"column": name, "z_threshold": z, "extreme_count": 0}
+    zscores = (s - mu) / sig
+    extreme = int((zscores.abs() > z).sum())
+    return {"column": name, "z_threshold": z, "extreme_count": extreme, "min": float(s.min()), "max": float(s.max())}
+
+
+def quality_report_before_clean(
+    df: pd.DataFrame,
+    v_pi_min: float = 0.0,
+    v_pi_max: float = 500.0,
+) -> dict:
+    """
+    清洗前数据质量报告：重复行、同输入异输出、V_pi 相对给定区间的越界计数、V_pi<=0、目标极端值。
+    不修改 DataFrame。
+    """
+    n = len(df)
+    dup_mask = df.duplicated(keep=False)
+    n_dup_rows = int(dup_mask.sum())
+    # 若启用去重，将删除的“重复出现行数” = 总行数 - 去重后行数
+    rows_removed_if_dedupe = int(n - df.drop_duplicates().shape[0])
+
+    def _n_unique_target_rows(sub: pd.DataFrame) -> int:
+        return sub[TARGET_COLUMNS].drop_duplicates().shape[0]
+
+    same_x_diff_y = 0
+    for _, sub in df.groupby(INPUT_COLUMNS, dropna=False):
+        if len(sub) <= 1:
+            continue
+        if _n_unique_target_rows(sub) > 1:
+            same_x_diff_y += len(sub)
+
+    vpi_nonpositive = int((df["V_pi"] <= 0).sum())
+    vpi_series = df["V_pi"].astype(float)
+    v_pi_out_of_range_count = int(((vpi_series < v_pi_min) | (vpi_series > v_pi_max)).sum())
+
+    target_extremes = [_extreme_report(df[c], c) for c in TARGET_COLUMNS]
+
+    report = {
+        "n_rows": n,
+        "duplicate_row_mask_count": n_dup_rows,
+        "rows_removed_if_drop_duplicates": rows_removed_if_dedupe,
+        "rows_with_same_inputs_differing_outputs": same_x_diff_y,
+        "v_pi_gate_inclusive_range": {"min": v_pi_min, "max": v_pi_max},
+        "v_pi_out_of_range_count": v_pi_out_of_range_count,
+        "v_pi_nonpositive_count": vpi_nonpositive,
+        "target_extreme_z5": target_extremes,
+    }
+    return report
+
+
+def summarize_for_console(df: pd.DataFrame, q: dict) -> str:
+    """简短人类可读摘要。"""
+    gate = q.get("v_pi_gate_inclusive_range", {})
+    lo, hi = gate.get("min", 0.0), gate.get("max", 500.0)
+    lines = [
+        f"行数={len(df)}",
+        f"去重可删除行数={q['rows_removed_if_drop_duplicates']}",
+        f"同输入异输出涉及行数={q['rows_with_same_inputs_differing_outputs']}",
+        f"V_pi 越界 [ {lo}, {hi} ] 行数={q.get('v_pi_out_of_range_count', 'n/a')}",
+        f"V_pi<=0 行数={q['v_pi_nonpositive_count']}",
+    ]
+    return "; ".join(lines)

src/evaluate.py

@@ -0,0 +1,142 @@
+"""加载最优模型并在各划分上评估，导出 CSV / JSON。"""
+
+from __future__ import annotations
+
+import csv
+import json
+import logging
+from pathlib import Path
+from dataclasses import asdict
+from typing import Dict, Tuple
+
+import numpy as np
+import pandas as pd
+import torch
+import torch.nn as nn
+
+from src.config import AppConfig
+from src.data import INPUT_COLUMNS, TARGET_COLUMNS
+from src.losses import build_loss
+from src.metrics import (
+    FullMetricsReport,
+    compute_full_report,
+    report_to_flat_dict,
+)
+from src.model import MLPRegressor
+from src.preprocess import ProcessedDataBundle
+from src.trainer import evaluate_loss_loader, load_weights
+
+logger = logging.getLogger(__name__)
+
+
+@torch.no_grad()
+def predict_all(
+    model: nn.Module,
+    loader: torch.utils.data.DataLoader,
+    device: torch.device,
+) -> Tuple[np.ndarray, np.ndarray]:
+    model.eval()
+    preds, trues = [], []
+    for xb, yb in loader:
+        xb = xb.to(device)
+        pr = model(xb).detach().cpu().numpy()
+        yt = yb.numpy()
+        preds.append(pr)
+        trues.append(yt)
+    return np.concatenate(preds, axis=0), np.concatenate(trues, axis=0)
+
+
+def evaluate_split(
+    model: nn.Module,
+    criterion: nn.Module,
+    loader: torch.utils.data.DataLoader,
+    device: torch.device,
+    y_scaler,
+    split_name: str,
+) -> Tuple[FullMetricsReport, FullMetricsReport, float]:
+    """返回 (标准化空间报告, 物理空间报告, 平均损失)。"""
+    loss = evaluate_loss_loader(model, loader, criterion, device)
+    pred_n, true_n = predict_all(model, loader, device)
+    pred_p = y_scaler.inverse_transform(pred_n)
+    true_p = y_scaler.inverse_transform(true_n)
+    rep_n = compute_full_report(split_name, loss, true_n, pred_n, TARGET_COLUMNS)
+    rep_p = compute_full_report(split_name, loss, true_p, pred_p, TARGET_COLUMNS)
+    return rep_n, rep_p, loss
+
+
+def run_full_evaluation(
+    cfg: AppConfig,
+    bundle: ProcessedDataBundle,
+    run_dir: Path,
+    device: torch.device,
+) -> Tuple[nn.Module, Dict]:
+    """载入 best.pt，在 train/val/test 上评估并写 metrics.csv 与 summary.json；返回模型与摘要。"""
+    model = MLPRegressor(
+        input_dim=cfg.model.input_dim,
+        hidden_dims=cfg.model.hidden_dims,
+        output_dim=cfg.model.output_dim,
+        batchnorm=cfg.model.batchnorm,
+        dropout=cfg.model.dropout,
+        residual=cfg.model.residual,
+    ).to(device)
+    ckpt_best = run_dir / "checkpoints" / "best.pt"
+    load_weights(model, ckpt_best, device)
+
+    criterion = build_loss(cfg.loss).to(device)
+    y_scaler = bundle.y_scaler
+
+    rows = []
+    summary: Dict = {"splits": {}}
+
+    for name, loader in (
+        ("train", bundle.train_loader),
+        ("val", bundle.val_loader),
+        ("test", bundle.test_loader),
+    ):
+        rep_n, rep_p, loss = evaluate_split(
+            model, criterion, loader, device, y_scaler, name
+        )
+        summary["splits"][name] = {
+            "loss": loss,
+            "normalized": asdict(rep_n),
+            "physical": asdict(rep_p),
+        }
+        row = report_to_flat_dict(rep_n, rep_p)
+        row["split"] = name
+        rows.append(row)
+
+    metrics_path = run_dir / "metrics.csv"
+    if rows:
+        with metrics_path.open("w", newline="", encoding="utf-8") as f:
+            writer = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
+            writer.writeheader()
+            writer.writerows(rows)
+    (run_dir / "summary.json").write_text(
+        json.dumps(summary, indent=2, ensure_ascii=False, default=str),
+        encoding="utf-8",
+    )
+    logger.info("已写入 %s 与 %s", metrics_path, run_dir / "summary.json")
+    return model, summary
+
+
+def export_test_predictions_csv(
+    bundle: ProcessedDataBundle,
+    model: nn.Module,
+    device: torch.device,
+    path: Path,
+) -> None:
+    """导出测试集物理空间真值、预测与误差。"""
+    model.eval()
+    pred_n, true_n = predict_all(model, bundle.test_loader, device)
+    pred_p = bundle.y_scaler.inverse_transform(pred_n)
+    true_p = bundle.y_scaler.inverse_transform(true_n)
+    err = pred_p - true_p
+    cols: Dict[str, np.ndarray] = {}
+    for j, name in enumerate(INPUT_COLUMNS):
+        cols[name] = bundle.X_test_raw[:, j]
+    for i, name in enumerate(TARGET_COLUMNS):
+        cols[f"true_{name}"] = true_p[:, i]
+        cols[f"pred_{name}"] = pred_p[:, i]
+        cols[f"err_{name}"] = err[:, i]
+    pd.DataFrame(cols).to_csv(path, index=False)
+    logger.info("已写入 %s", path)

src/infer.py

@@ -0,0 +1,87 @@
+"""离线推理：从 txt/csv 读取 8 维输入，输出反标准化后的三目标预测。"""
+
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+from typing import Optional
+
+import pandas as pd
+import torch
+
+from src.config import AppConfig, load_config
+from src.data import INPUT_COLUMNS, TARGET_COLUMNS
+from src.model import MLPRegressor
+from src.preprocess import load_scalers
+from src.trainer import load_weights
+
+logger = logging.getLogger(__name__)
+
+
+def _read_inputs_table(path: Path) -> pd.DataFrame:
+    """读取 8 列输入：支持逗号分隔 txt 或 csv。"""
+    path = Path(path)
+    if not path.is_file():
+        raise FileNotFoundError(f"输入文件不存在: {path}")
+    if path.suffix.lower() in {".csv"}:
+        df = pd.read_csv(path)
+    else:
+        df = pd.read_csv(path, header=None, names=INPUT_COLUMNS)
+    missing = [c for c in INPUT_COLUMNS if c not in df.columns]
+    if missing:
+        raise ValueError(f"输入缺少列: {missing}；需要列 {INPUT_COLUMNS}")
+    return df[INPUT_COLUMNS].astype(float)
+
+
+def run_inference(
+    cfg: AppConfig,
+    run_dir: Path,
+    input_path: Path,
+    output_csv: Path,
+    device: torch.device,
+) -> None:
+    """
+    载入 best 模型与 scaler，对输入表进行批量推理并写出 CSV（物理量空间）。
+    """
+    X = _read_inputs_table(Path(input_path)).to_numpy(dtype=np.float32)
+    X_scaler, y_scaler = load_scalers(run_dir)
+    Xn = X_scaler.transform(X)
+
+    model = MLPRegressor(
+        input_dim=cfg.model.input_dim,
+        hidden_dims=cfg.model.hidden_dims,
+        output_dim=cfg.model.output_dim,
+        batchnorm=cfg.model.batchnorm,
+        dropout=cfg.model.dropout,
+        residual=cfg.model.residual,
+    ).to(device)
+    load_weights(model, run_dir / "checkpoints" / "best.pt", device)
+
+    model.eval()
+    with torch.no_grad():
+        pred_n = model(torch.from_numpy(Xn).float().to(device)).cpu().numpy()
+    pred_p = y_scaler.inverse_transform(pred_n)
+
+    out = pd.DataFrame(X, columns=INPUT_COLUMNS)
+    for j, name in enumerate(TARGET_COLUMNS):
+        out[f"pred_{name}"] = pred_p[:, j]
+    output_csv = Path(output_csv)
+    output_csv.parent.mkdir(parents=True, exist_ok=True)
+    out.to_csv(output_csv, index=False)
+    logger.info("推理完成，写入 %s", output_csv)
+
+
+def infer_cli(
+    config_path: str,
+    run_dir: Path,
+    input_path: str,
+    output_csv: Optional[str] = None,
+) -> None:
+    cfg = load_config(config_path)
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    out = (
+        Path(output_csv)
+        if output_csv is not None
+        else run_dir / "inference_output.csv"
+    )
+    run_inference(cfg, run_dir, Path(input_path), out, device)

src/losses.py

@@ -0,0 +1,62 @@
+"""多输出回归损失：Huber（SmoothL1）与加权 MSE。"""
+
+from __future__ import annotations
+
+from typing import List
+
+import torch
+import torch.nn as nn
+
+
+class WeightedHuberLoss(nn.Module):
+    """
+    在标准化输出空间对每个目标维度加权 SmoothL1（Huber）。
+
+    PyTorch 的 SmoothL1Loss 对应 beta 参数等价于 Huber 的 delta。
+    """
+
+    def __init__(self, delta: float = 1.0, weights: List[float] | None = None) -> None:
+        super().__init__()
+        self.delta = float(delta)
+        self.register_buffer(
+            "w",
+            torch.tensor(weights if weights is not None else [1.0, 1.0, 1.0], dtype=torch.float32),
+        )
+        self._base = nn.SmoothL1Loss(reduction="none", beta=self.delta)
+
+    def forward(self, pred: torch.Tensor, target: torch.Tensor) -> torch.Tensor:
+        """
+        Args:
+            pred, target: shape (N, 3)
+        Returns:
+            标量损失（加权平均）
+        """
+        elem = self._base(pred, target)
+        w = self.w.to(pred.device).view(1, -1)
+        return (elem * w).sum(dim=1).mean()
+
+
+class WeightedMSELoss(nn.Module):
+    """逐维加权 MSE，再对 batch 平均。"""
+
+    def __init__(self, weights: List[float] | None = None) -> None:
+        super().__init__()
+        self.register_buffer(
+            "w",
+            torch.tensor(weights if weights is not None else [1.0, 1.0, 1.0], dtype=torch.float32),
+        )
+
+    def forward(self, pred: torch.Tensor, target: torch.Tensor) -> torch.Tensor:
+        err2 = (pred - target) ** 2
+        w = self.w.to(pred.device).view(1, -1)
+        return (err2 * w).sum(dim=1).mean()
+
+
+def build_loss(cfg_loss) -> nn.Module:
+    """根据配置构造损失函数。"""
+    w = list(cfg_loss.target_weights)
+    if cfg_loss.type == "huber":
+        return WeightedHuberLoss(delta=cfg_loss.huber_delta, weights=w)
+    if cfg_loss.type == "weighted_mse":
+        return WeightedMSELoss(weights=w)
+    raise ValueError(f"未知 loss.type: {cfg_loss.type}")

src/main.py

@@ -0,0 +1,186 @@
+"""统一 CLI：train / eval / infer。"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import shutil
+import sys
+from pathlib import Path
+
+import torch
+
+from src.config import load_config
+from src.data import load_raw_txt, quality_report_before_clean, summarize_for_console
+from src.evaluate import export_test_predictions_csv, run_full_evaluation
+from src.model import MLPRegressor
+from src.plots import generate_all_figures
+from src.preprocess import prepare_training_data, rebuild_bundle_for_eval
+from src.trainer import fit
+from src.utils import (
+    find_latest_run,
+    make_run_dir,
+    resolve_path,
+    set_global_seed,
+    setup_logging,
+)
+
+logger = logging.getLogger("mzm")
+
+
+def _project_root() -> Path:
+    return Path(__file__).resolve().parents[1]
+
+
+def _resolve_cfg_path(path: str) -> Path:
+    p = Path(path)
+    if p.is_file():
+        return p.resolve()
+    cand = _project_root() / path
+    if cand.is_file():
+        return cand.resolve()
+    raise FileNotFoundError(f"找不到配置文件: {path}")
+
+
+def _resolve_run_dir(cfg_path: Path, explicit: str | None, output_dir: str | None) -> Path:
+    if explicit:
+        rd = Path(explicit).resolve()
+        if not rd.is_dir():
+            raise FileNotFoundError(f"run_dir 不存在: {rd}")
+        return rd
+    base = Path(output_dir) if output_dir else None
+    if base is None:
+        cfg = load_config(cfg_path)
+        base = Path(cfg.output_dir)
+    return find_latest_run(base)
+
+
+def _write_summary_md(run_dir: Path, summary: dict) -> None:
+    lines = ["# 评估摘要", ""]
+    for split, block in summary.get("splits", {}).items():
+        lines.append(f"## {split}")
+        lines.append("")
+        lines.append(
+            f"- **损失（标准化输出空间 Huber/MSE 准则）**: {block['loss']:.6f}"
+        )
+        for space, label in ("normalized", "标准化空间"), ("physical", "物理量空间"):
+            sub = block[space]
+            lines.append(f"- **{label}** — 平均 MAE: {sub['overall_mae']:.6f}；"
+                         f"平均 RMSE: {sub['overall_rmse']:.6f}；整体 R²: {sub['overall_r2']:.6f}")
+            for t, vals in sub["per_target"].items():
+                lines.append(
+                    f"  - `{t}`: MAE={vals['mae']:.6f}, RMSE={vals['rmse']:.6f}, R²={vals['r2']:.6f}"
+                )
+        lines.append("")
+    (run_dir / "summary.md").write_text("\n".join(lines), encoding="utf-8")
+
+
+def cmd_train(args: argparse.Namespace) -> None:
+    cfg_path = _resolve_cfg_path(args.config)
+    cfg = load_config(cfg_path)
+    set_global_seed(cfg.random_seed)
+
+    run_dir = make_run_dir(resolve_path(cfg.output_dir, _project_root()))
+    shutil.copy2(cfg_path, run_dir / "config_snapshot.yaml")
+    setup_logging(run_dir / "training.log")
+
+    data_path = resolve_path(cfg.data_path, _project_root())
+    df = load_raw_txt(data_path)
+    q = quality_report_before_clean(df, cfg.v_pi_min, cfg.v_pi_max)
+    logger.info("数据质量(清洗前): %s", summarize_for_console(df, q))
+
+    bundle = prepare_training_data(df, cfg, run_dir)
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    model = MLPRegressor(
+        input_dim=cfg.model.input_dim,
+        hidden_dims=cfg.model.hidden_dims,
+        output_dim=cfg.model.output_dim,
+        batchnorm=cfg.model.batchnorm,
+        dropout=cfg.model.dropout,
+        residual=cfg.model.residual,
+    ).to(device)
+
+    history = fit(model, cfg, bundle.train_loader, bundle.val_loader, run_dir, device)
+    model_eval, summary = run_full_evaluation(cfg, bundle, run_dir, device)
+    export_test_predictions_csv(
+        bundle, model_eval, device, run_dir / "test_predictions.csv"
+    )
+    _write_summary_md(run_dir, summary)
+    generate_all_figures(cfg, bundle, run_dir, history, device)
+    logger.info("训练与评估完成，结果目录: %s", run_dir)
+
+
+def cmd_eval(args: argparse.Namespace) -> None:
+    cfg_path = _resolve_cfg_path(args.config)
+    run_dir = _resolve_run_dir(cfg_path, args.run_dir, args.output_dir)
+    snap = run_dir / "config_snapshot.yaml"
+    cfg = load_config(snap if snap.is_file() else cfg_path)
+    set_global_seed(cfg.random_seed)
+    setup_logging(run_dir / "eval.log")
+
+    data_path = resolve_path(cfg.data_path, _project_root())
+    df = load_raw_txt(data_path)
+    bundle = rebuild_bundle_for_eval(df, cfg, run_dir)
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    model_eval, summary = run_full_evaluation(cfg, bundle, run_dir, device)
+    export_test_predictions_csv(
+        bundle, model_eval, device, run_dir / "test_predictions.csv"
+    )
+    _write_summary_md(run_dir, summary)
+    logger.info("评估完成，已更新 %s 下 metrics/summary/test_predictions", run_dir)
+
+
+def cmd_infer(args: argparse.Namespace) -> None:
+    cfg_path = _resolve_cfg_path(args.config)
+    run_dir = _resolve_run_dir(cfg_path, args.run_dir, args.output_dir)
+    snap = run_dir / "config_snapshot.yaml"
+    cfg = load_config(snap if snap.is_file() else cfg_path)
+    set_global_seed(cfg.random_seed)
+    setup_logging(None)
+
+    from src.infer import run_inference
+
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    out = Path(args.output) if args.output else run_dir / "inference_output.csv"
+    run_inference(cfg, run_dir, Path(args.input), out, device)
+    logger.info("推理完成: %s", out)
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(description="MZM MLP 训练 / 评估 / 推理")
+    sub = p.add_subparsers(dest="command", required=True)
+
+    pt = sub.add_parser("train", help="训练模型")
+    pt.add_argument("--config", type=str, default="configs/default.yaml")
+    pt.set_defaults(func=cmd_train)
+
+    pe = sub.add_parser("eval", help="在 train/val/test 上重新评估并导出 CSV")
+    pe.add_argument("--config", type=str, default="configs/default.yaml")
+    pe.add_argument("--run-dir", type=str, default=None, help="指定某次训练输出目录")
+    pe.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="在未指定 run-dir 时用于搜索最新 run_* 的根目录",
+    )
+    pe.set_defaults(func=cmd_eval)
+
+    pi = sub.add_parser("infer", help="批量推理")
+    pi.add_argument("--config", type=str, default="configs/default.yaml")
+    pi.add_argument("--input", type=str, required=True, help="8 列输入 csv/txt")
+    pi.add_argument("--output", type=str, default=None, help="输出预测 csv 路径")
+    pi.add_argument("--run-dir", type=str, default=None)
+    pi.add_argument("--output-dir", type=str, default=None)
+    pi.set_defaults(func=cmd_infer)
+
+    return p
+
+
+def main(argv: list[str] | None = None) -> None:
+    parser = build_parser()
+    args = parser.parse_args(argv)
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main(sys.argv[1:])

src/metrics.py

@@ -0,0 +1,95 @@
+"""MAE / RMSE / R²，支持逐维与整体平均。"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Dict, List, Tuple
+
+import numpy as np
+
+
+def _r2_score(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    """单变量或多变量整体 R²（按 sklearn 定义在最后一维聚合）。"""
+    y_true = np.asarray(y_true, dtype=np.float64)
+    y_pred = np.asarray(y_pred, dtype=np.float64)
+    ss_res = np.sum((y_true - y_pred) ** 2)
+    ss_tot = np.sum((y_true - np.mean(y_true, axis=0)) ** 2)
+    if ss_tot < 1e-15:
+        return float("nan")
+    return float(1.0 - ss_res / ss_tot)
+
+
+def per_output_metrics(
+    y_true: np.ndarray, y_pred: np.ndarray, names: List[str]
+) -> Dict[str, Dict[str, float]]:
+    """对每个输出维度计算 MAE、RMSE、R2。"""
+    y_true = np.asarray(y_true, dtype=np.float64)
+    y_pred = np.asarray(y_pred, dtype=np.float64)
+    out: Dict[str, Dict[str, float]] = {}
+    for i, name in enumerate(names):
+        yt = y_true[:, i]
+        yp = y_pred[:, i]
+        mae = float(np.mean(np.abs(yt - yp)))
+        rmse = float(np.sqrt(np.mean((yt - yp) ** 2)))
+        ss_res = float(np.sum((yt - yp) ** 2))
+        ss_tot = float(np.sum((yt - np.mean(yt)) ** 2))
+        r2 = float(1.0 - ss_res / ss_tot) if ss_tot > 1e-15 else float("nan")
+        out[name] = {"mae": mae, "rmse": rmse, "r2": r2}
+    return out
+
+
+def overall_avg_mae_rmse(
+    y_true: np.ndarray, y_pred: np.ndarray
+) -> Tuple[float, float]:
+    """三输出维度上 MAE/RMSE 的简单算术平均。"""
+    per = per_output_metrics(y_true, y_pred, ["t0", "t1", "t2"])
+    mae = float(np.mean([per[k]["mae"] for k in per]))
+    rmse = float(np.mean([per[k]["rmse"] for k in per]))
+    return mae, rmse
+
+
+@dataclass
+class FullMetricsReport:
+    split: str
+    loss: float
+    per_target: Dict[str, Dict[str, float]]
+    overall_mae: float
+    overall_rmse: float
+    overall_r2: float
+
+
+def compute_full_report(
+    split: str,
+    loss: float,
+    y_true: np.ndarray,
+    y_pred: np.ndarray,
+    target_names: List[str],
+) -> FullMetricsReport:
+    per = per_output_metrics(y_true, y_pred, target_names)
+    mae, rmse = overall_avg_mae_rmse(y_true, y_pred)
+    r2 = _r2_score(y_true, y_pred)
+    return FullMetricsReport(
+        split=split,
+        loss=float(loss),
+        per_target=per,
+        overall_mae=mae,
+        overall_rmse=rmse,
+        overall_r2=float(r2),
+    )
+
+
+def report_to_flat_dict(
+    norm: FullMetricsReport,
+    phys: FullMetricsReport,
+) -> Dict[str, float | str]:
+    """展平为 CSV 一行友好的字典。"""
+    row: Dict[str, float | str] = {"split": norm.split}
+    for prefix, rep in (("norm", norm), ("phys", phys)):
+        row[f"{prefix}_loss"] = rep.loss
+        row[f"{prefix}_overall_mae"] = rep.overall_mae
+        row[f"{prefix}_overall_rmse"] = rep.overall_rmse
+        row[f"{prefix}_overall_r2"] = rep.overall_r2
+        for tname, vals in rep.per_target.items():
+            for mname, v in vals.items():
+                row[f"{prefix}_{tname}_{mname}"] = v
+    return row

src/model.py

@@ -0,0 +1,61 @@
+"""可配置 MLP 回归模型。"""
+
+from __future__ import annotations
+
+from typing import List
+
+import torch
+import torch.nn as nn
+
+
+def kaiming_init_module(m: nn.Module) -> None:
+    """对 Linear 使用 Kaiming uniform（ReLU），偏置置零。"""
+    if isinstance(m, nn.Linear):
+        nn.init.kaiming_uniform_(m.weight, nonlinearity="relu")
+        if m.bias is not None:
+            nn.init.zeros_(m.bias)
+    elif isinstance(m, nn.BatchNorm1d):
+        nn.init.ones_(m.weight)
+        nn.init.zeros_(m.bias)
+
+
+class MLPRegressor(nn.Module):
+    """
+    多层感知机回归：输入 8 维，输出 3 维。
+
+    可选 BatchNorm1d、Dropout、以及在相邻层维度相等时的残差相加。
+    """
+
+    def __init__(
+        self,
+        input_dim: int,
+        hidden_dims: List[int],
+        output_dim: int,
+        batchnorm: bool = False,
+        dropout: float = 0.0,
+        residual: bool = False,
+    ) -> None:
+        super().__init__()
+        self.residual = residual
+        dims = [input_dim] + list(hidden_dims) + [output_dim]
+        self._hidden_blocks = nn.ModuleList()
+        for i in range(len(dims) - 2):
+            in_d, out_d = dims[i], dims[i + 1]
+            seq_layers: list[nn.Module] = [nn.Linear(in_d, out_d)]
+            if batchnorm:
+                seq_layers.append(nn.BatchNorm1d(out_d))
+            seq_layers.append(nn.ReLU(inplace=True))
+            if dropout and dropout > 0:
+                seq_layers.append(nn.Dropout(p=dropout))
+            self._hidden_blocks.append(nn.Sequential(*seq_layers))
+        self._head = nn.Linear(dims[-2], dims[-1])
+        self.apply(kaiming_init_module)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        h = x
+        for block in self._hidden_blocks:
+            inp = h
+            h = block(inp)
+            if self.residual and inp.shape[-1] == h.shape[-1]:
+                h = h + inp
+        return self._head(h)

src/plots.py

@@ -0,0 +1,134 @@
+"""训练曲线、预测散点图与残差图。"""
+
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+
+import matplotlib.pyplot as plt
+import numpy as np
+import seaborn as sns
+import torch
+
+from src.config import AppConfig
+from src.data import TARGET_COLUMNS
+from src.evaluate import predict_all
+from src.model import MLPRegressor
+from src.preprocess import ProcessedDataBundle
+from src.trainer import TrainHistory, load_weights
+
+logger = logging.getLogger(__name__)
+
+
+def plot_loss_curves(history: TrainHistory, out_path: Path) -> None:
+    """绘制 train/val loss 曲线。"""
+    sns.set_theme(style="whitegrid", context="talk")
+    fig, ax = plt.subplots(figsize=(8, 5))
+    ax.plot(history.epoch, history.train_loss, label="Train loss", linewidth=2)
+    ax.plot(history.epoch, history.val_loss, label="Val loss", linewidth=2)
+    ax.set_xlabel("Epoch")
+    ax.set_ylabel("Loss (normalized target space)")
+    ax.set_title("Training / Validation Loss")
+    ax.legend()
+    fig.tight_layout()
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    fig.savefig(out_path, dpi=150)
+    plt.close(fig)
+    logger.info("已保存 %s", out_path)
+
+
+def plot_scatter_true_pred(
+    y_true: np.ndarray,
+    y_pred: np.ndarray,
+    target_name: str,
+    out_path: Path,
+    title_suffix: str = "Test set (physical units)",
+) -> None:
+    """单目标预测 vs 真值散点图。"""
+    sns.set_theme(style="whitegrid", context="talk")
+    fig, ax = plt.subplots(figsize=(6, 6))
+    ax.scatter(y_true, y_pred, alpha=0.35, edgecolors="none", s=18)
+    lims = [
+        min(y_true.min(), y_pred.min()),
+        max(y_true.max(), y_pred.max()),
+    ]
+    ax.plot(lims, lims, "r--", linewidth=1.5, label="Ideal")
+    ax.set_xlabel(f"True {target_name}")
+    ax.set_ylabel(f"Predicted {target_name}")
+    ax.set_title(f"{target_name}: Pred vs True ({title_suffix})")
+    ax.legend()
+    fig.tight_layout()
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    fig.savefig(out_path, dpi=150)
+    plt.close(fig)
+
+
+def plot_residuals(
+    y_true: np.ndarray,
+    y_pred: np.ndarray,
+    target_name: str,
+    out_path: Path,
+    title_suffix: str = "Test set (physical units)",
+) -> None:
+    """单目标残差直方图 + 预测值横轴散点。"""
+    resid = y_pred - y_true
+    sns.set_theme(style="whitegrid", context="talk")
+    fig, axes = plt.subplots(1, 2, figsize=(11, 4))
+    axes[0].hist(resid, bins=40, color="#4C72B0", alpha=0.85)
+    axes[0].set_title(f"{target_name}: Residual histogram")
+    axes[0].set_xlabel("Pred - True")
+    axes[0].set_ylabel("Count")
+    axes[1].scatter(y_pred, resid, alpha=0.35, s=16, edgecolors="none")
+    axes[1].axhline(0.0, color="r", linestyle="--", linewidth=1.2)
+    axes[1].set_xlabel(f"Predicted {target_name}")
+    axes[1].set_ylabel("Residual")
+    axes[1].set_title(f"{target_name}: Residual vs Pred ({title_suffix})")
+    fig.tight_layout()
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    fig.savefig(out_path, dpi=150)
+    plt.close(fig)
+
+
+def generate_all_figures(
+    cfg: AppConfig,
+    bundle: ProcessedDataBundle,
+    run_dir: Path,
+    history: TrainHistory,
+    device: torch.device,
+) -> None:
+    """
+    生成 loss 曲线与测试集上各目标的散点图、残差图。
+    依赖 run_dir/checkpoints/best.pt。
+    """
+    fig_dir = run_dir / "figures"
+    fig_dir.mkdir(parents=True, exist_ok=True)
+    plot_loss_curves(history, fig_dir / "loss_curve.png")
+
+    model = MLPRegressor(
+        input_dim=cfg.model.input_dim,
+        hidden_dims=cfg.model.hidden_dims,
+        output_dim=cfg.model.output_dim,
+        batchnorm=cfg.model.batchnorm,
+        dropout=cfg.model.dropout,
+        residual=cfg.model.residual,
+    ).to(device)
+    load_weights(model, run_dir / "checkpoints" / "best.pt", device)
+
+    pred_n, true_n = predict_all(model, bundle.test_loader, device)
+    pred_p = bundle.y_scaler.inverse_transform(pred_n)
+    true_p = bundle.y_scaler.inverse_transform(true_n)
+
+    for i, name in enumerate(TARGET_COLUMNS):
+        plot_scatter_true_pred(
+            true_p[:, i],
+            pred_p[:, i],
+            name,
+            fig_dir / f"scatter_{name}.png",
+        )
+        plot_residuals(
+            true_p[:, i],
+            pred_p[:, i],
+            name,
+            fig_dir / f"residual_{name}.png",
+        )
+    logger.info("所有图像已写入 %s", fig_dir)

src/preprocess.py

@@ -0,0 +1,481 @@
+"""清洗、划分、标准化与 DataLoader 构建。"""
+
+from __future__ import annotations
+
+import json
+import logging
+import pickle
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Dict, List, Optional, Tuple
+
+import numpy as np
+import pandas as pd
+import torch
+from sklearn.preprocessing import StandardScaler
+from torch.utils.data import DataLoader, TensorDataset
+
+from src.config import AppConfig
+from src.data import ALL_COLUMNS, INPUT_COLUMNS, TARGET_COLUMNS, quality_report_before_clean
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class ProcessedDataBundle:
+    """训练用张量与 DataLoader，以及划分后的 numpy（含测试集原始物理量用于导出）。"""
+
+    train_loader: DataLoader
+    val_loader: DataLoader
+    test_loader: DataLoader
+    X_train: np.ndarray
+    X_val: np.ndarray
+    X_test: np.ndarray
+    y_train: np.ndarray
+    y_val: np.ndarray
+    y_test: np.ndarray
+    X_test_raw: np.ndarray
+    y_test_raw: np.ndarray
+    X_scaler: StandardScaler
+    y_scaler: StandardScaler
+    feature_names: List[str]
+    target_names: List[str]
+
+
+def _mask_outliers_iqr(
+    values: np.ndarray, col_names: List[str], k: float
+) -> np.ndarray:
+    """返回 True 表示该行在任一选定列上超出训练集 IQR 范围（基于传入的 values 统计）。"""
+    mask = np.zeros(len(values), dtype=bool)
+    for j, _ in enumerate(col_names):
+        col = values[:, j]
+        q1, q3 = np.percentile(col, [25, 75])
+        iqr = q3 - q1
+        lo, hi = q1 - k * iqr, q3 + k * iqr
+        mask |= (col < lo) | (col > hi)
+    return mask
+
+
+def _mask_outliers_zscore(values: np.ndarray, threshold: float) -> np.ndarray:
+    mask = np.zeros(len(values), dtype=bool)
+    for j in range(values.shape[1]):
+        col = values[:, j]
+        mu, sig = col.mean(), col.std(ddof=0)
+        if sig < 1e-12:
+            continue
+        z = np.abs((col - mu) / sig)
+        mask |= z > threshold
+    return mask
+
+
+def _winsorize_train_apply_all(
+    train: np.ndarray,
+    val: np.ndarray,
+    test: np.ndarray,
+    ql: float,
+    qu: float,
+) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    """按训练集分位数对 train/val/test 同步裁剪（列方向）。"""
+    lo = np.quantile(train, ql, axis=0)
+    hi = np.quantile(train, qu, axis=0)
+    def clip_arr(a: np.ndarray) -> np.ndarray:
+        return np.clip(a, lo, hi)
+    return clip_arr(train), clip_arr(val), clip_arr(test)
+
+
+def clean_dataframe(
+    df: pd.DataFrame,
+    cfg: AppConfig,
+    report_lines: List[str],
+) -> pd.DataFrame:
+    """
+    清洗流程（顺序固定，便于复现与审计）：
+
+    1. 可选：完全重复行去重。
+    2. 可选：以 **最后一列对应字段 V_pi**（txt 第 11 个逗号分隔字段）为门控，仅保留
+       ``v_pi_min <= V_pi <= v_pi_max``（默认 [0, 500]）。
+    3. 可选：再移除 ``V_pi <= 0``（与区间门控独立，由配置控制）。
+    """
+    out = df.copy()
+    n0 = len(out)
+    if cfg.remove_duplicate_rows:
+        out = out.drop_duplicates()
+        report_lines.append(f"去完全重复行: {n0} -> {len(out)}")
+    if cfg.filter_v_pi_range:
+        n1 = len(out)
+        lo, hi = float(cfg.v_pi_min), float(cfg.v_pi_max)
+        mask = (out["V_pi"] >= lo) & (out["V_pi"] <= hi)
+        out = out[mask].reset_index(drop=True)
+        report_lines.append(
+            f"V_pi 物理区间过滤 [{lo}, {hi}]（txt 第 11 列 / 列名 V_pi）: {n1} -> {len(out)}"
+        )
+    if cfg.remove_nonpositive_vpi:
+        n2 = len(out)
+        out = out[out["V_pi"] > 0].reset_index(drop=True)
+        report_lines.append(f"移除 V_pi<=0: {n2} -> {len(out)}")
+    if len(out) == 0:
+        raise ValueError(
+            "清洗后样本数为 0：请检查 V_pi 区间配置、数据源或是否过度去重。"
+        )
+    return out
+
+
+def stratified_split_indices(
+    n: int,
+    ratios: List[float],
+    seed: int,
+) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    """返回 train/val/test 的整数索引（先 shuffle 再按比例切分）。"""
+    rng = np.random.default_rng(seed)
+    idx = np.arange(n)
+    rng.shuffle(idx)
+    tr, va, te = ratios
+    n_test = int(round(n * te))
+    n_val = int(round(n * va))
+    n_train = n - n_val - n_test
+    if n_train <= 0 or n_val <= 0 or n_test <= 0:
+        raise ValueError(
+            f"划分后样本过少: train={n_train}, val={n_val}, test={n_test}，请调整比例或数据量"
+        )
+    i_train = idx[:n_train]
+    i_val = idx[n_train : n_train + n_val]
+    i_test = idx[n_train + n_val :]
+    return i_train, i_val, i_test
+
+
+def apply_train_only_outliers(
+    X_train: np.ndarray,
+    y_train: np.ndarray,
+    X_val: np.ndarray,
+    y_val: np.ndarray,
+    X_test: np.ndarray,
+    y_test: np.ndarray,
+    cfg: AppConfig,
+    report_lines: List[str],
+) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
+    """
+    仅在训练集上估计阈值：
+    - iqr/zscore: 从训练集删除离群行（val/test 不动）
+    - quantile_clip: 对 train/val/test 同步 winsorize（阈值来自 train）
+    """
+    strat = cfg.outlier_strategy
+    if strat == "none":
+        report_lines.append("outlier_strategy=none：不对数值做裁剪/删除（除配置项外）。")
+        return X_train, y_train, X_val, y_val, X_test, y_test
+
+    cols = cfg.outlier_apply_to
+    if cols == "all":
+        train_mat = np.hstack([X_train, y_train])
+        val_mat = np.hstack([X_val, y_val])
+        test_mat = np.hstack([X_test, y_test])
+        names = INPUT_COLUMNS + TARGET_COLUMNS
+    else:
+        train_mat = y_train.copy()
+        val_mat = y_val.copy()
+        test_mat = y_test.copy()
+        names = TARGET_COLUMNS
+
+    if strat == "quantile_clip":
+        ql = cfg.outlier_config.quantile_lower
+        qu = cfg.outlier_config.quantile_upper
+        tr2, va2, te2 = _winsorize_train_apply_all(train_mat, val_mat, test_mat, ql, qu)
+        report_lines.append(
+            f"quantile_clip: 按训练集分位数 [{ql}, {qu}] 对 {cols} 列 winsorize。"
+        )
+        if cols == "all":
+            d = len(INPUT_COLUMNS)
+            X_train, y_train = tr2[:, :d], tr2[:, d:]
+            X_val, y_val = va2[:, :d], va2[:, d:]
+            X_test, y_test = te2[:, :d], te2[:, d:]
+        else:
+            y_train, y_val, y_test = tr2, va2, te2
+        return X_train, y_train, X_val, y_val, X_test, y_test
+
+    if strat == "iqr":
+        mask = _mask_outliers_iqr(train_mat, names, cfg.outlier_config.iqr_k)
+    elif strat == "zscore":
+        mask = _mask_outliers_zscore(train_mat, cfg.outlier_config.zscore_threshold)
+    else:
+        raise ValueError(f"未知 outlier_strategy: {strat}")
+
+    removed = int(mask.sum())
+    kept = ~mask
+    X_train, y_train = X_train[kept], y_train[kept]
+    report_lines.append(
+        f"{strat}: 在训练子集上检测 {cols} 离群，删除训练行 {removed}，保留 {len(X_train)}。"
+    )
+    return X_train, y_train, X_val, y_val, X_test, y_test
+
+
+def build_dataloaders(
+    X_train: np.ndarray,
+    y_train: np.ndarray,
+    X_val: np.ndarray,
+    y_val: np.ndarray,
+    X_test: np.ndarray,
+    y_test: np.ndarray,
+    batch_size: int,
+    num_workers: int,
+) -> Tuple[DataLoader, DataLoader, DataLoader]:
+    def to_loader(X: np.ndarray, y: np.ndarray, shuffle: bool) -> DataLoader:
+        ds = TensorDataset(
+            torch.from_numpy(X).float(),
+            torch.from_numpy(y).float(),
+        )
+        return DataLoader(
+            ds,
+            batch_size=batch_size,
+            shuffle=shuffle,
+            num_workers=num_workers,
+            pin_memory=False,
+        )
+
+    return (
+        to_loader(X_train, y_train, shuffle=True),
+        to_loader(X_val, y_val, shuffle=False),
+        to_loader(X_test, y_test, shuffle=False),
+    )
+
+
+def save_scalers(
+    X_scaler: StandardScaler,
+    y_scaler: StandardScaler,
+    run_dir: Path,
+) -> None:
+    with (run_dir / "x_scaler.pkl").open("wb") as f:
+        pickle.dump(X_scaler, f)
+    with (run_dir / "y_scaler.pkl").open("wb") as f:
+        pickle.dump(y_scaler, f)
+
+
+def load_scalers(run_dir: Path) -> Tuple[StandardScaler, StandardScaler]:
+    with (run_dir / "x_scaler.pkl").open("rb") as f:
+        X_scaler = pickle.load(f)
+    with (run_dir / "y_scaler.pkl").open("rb") as f:
+        y_scaler = pickle.load(f)
+    return X_scaler, y_scaler
+
+
+def save_split_indices(
+    run_dir: Path,
+    i_train: np.ndarray,
+    i_val: np.ndarray,
+    i_test: np.ndarray,
+) -> None:
+    """保存对清洗后矩阵行的划分索引，便于 eval 阶段完全复现。"""
+    payload = {
+        "train": i_train.astype(int).tolist(),
+        "val": i_val.astype(int).tolist(),
+        "test": i_test.astype(int).tolist(),
+    }
+    (run_dir / "split_indices.json").write_text(
+        json.dumps(payload, indent=2), encoding="utf-8"
+    )
+
+
+def load_split_indices(run_dir: Path) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    path = run_dir / "split_indices.json"
+    if not path.is_file():
+        raise FileNotFoundError(
+            f"未找到 {path}。请使用本仓库训练产生的 run 目录，或先完成一次训练。"
+        )
+    data = json.loads(path.read_text(encoding="utf-8"))
+    return (
+        np.asarray(data["train"], dtype=int),
+        np.asarray(data["val"], dtype=int),
+        np.asarray(data["test"], dtype=int),
+    )
+
+
+def rebuild_bundle_for_eval(
+    df: pd.DataFrame,
+    cfg: AppConfig,
+    run_dir: Path,
+) -> ProcessedDataBundle:
+    """
+    与训练阶段相同的清洗、划分与离群处理，但使用已保存的 StandardScaler 仅做 transform。
+    用于独立 eval / infer 流程，避免重新拟合 scaler 造成分布偏移。
+    """
+    report_lines: List[str] = []
+    cleaned = clean_dataframe(df, cfg, report_lines)
+    X = cleaned[INPUT_COLUMNS].to_numpy(dtype=np.float64)
+    y = cleaned[TARGET_COLUMNS].to_numpy(dtype=np.float64)
+    i_tr, i_va, i_te = load_split_indices(run_dir)
+    for name, idx in ("train", i_tr), ("val", i_va), ("test", i_te):
+        if len(idx) == 0 or int(idx.max()) >= len(X) or int(idx.min()) < 0:
+            raise ValueError(
+                f"split_indices.json 与当前数据不兼容（{name} 索引越界或为空）。"
+                f"请确认 data_path 指向与训练相同的清洗后样本空间。"
+            )
+
+    X_train, y_train = X[i_tr], y[i_tr]
+    X_val, y_val = X[i_va], y[i_va]
+    X_test, y_test = X[i_te], y[i_te]
+    X_train, y_train, X_val, y_val, X_test, y_test = apply_train_only_outliers(
+        X_train, y_train, X_val, y_val, X_test, y_test, cfg, report_lines
+    )
+
+    X_scaler, y_scaler = load_scalers(run_dir)
+    X_train_s = X_scaler.transform(X_train)
+    y_train_s = y_scaler.transform(y_train)
+    X_val_s = X_scaler.transform(X_val)
+    y_val_s = y_scaler.transform(y_val)
+    X_test_s = X_scaler.transform(X_test)
+    y_test_s = y_scaler.transform(y_test)
+
+    train_loader, val_loader, test_loader = build_dataloaders(
+        X_train_s,
+        y_train_s,
+        X_val_s,
+        y_val_s,
+        X_test_s,
+        y_test_s,
+        cfg.training.batch_size,
+        cfg.training.num_workers,
+    )
+
+    return ProcessedDataBundle(
+        train_loader=train_loader,
+        val_loader=val_loader,
+        test_loader=test_loader,
+        X_train=X_train_s,
+        X_val=X_val_s,
+        X_test=X_test_s,
+        y_train=y_train_s,
+        y_val=y_val_s,
+        y_test=y_test_s,
+        X_test_raw=X_test,
+        y_test_raw=y_test,
+        X_scaler=X_scaler,
+        y_scaler=y_scaler,
+        feature_names=list(INPUT_COLUMNS),
+        target_names=list(TARGET_COLUMNS),
+    )
+
+
+def write_data_report_md(
+    path: Path,
+    raw_quality: dict,
+    report_lines: List[str],
+    basic_stats_before: pd.DataFrame,
+    basic_stats_after: pd.DataFrame,
+) -> None:
+    lines = [
+        "# 数据与清洗报告",
+        "",
+        "## 清洗前质量摘要（JSON）",
+        "```json",
+        json.dumps(raw_quality, indent=2, ensure_ascii=False, default=str),
+        "```",
+        "",
+        "## 清洗步骤",
+        "\n".join(f"- {x}" for x in report_lines),
+        "",
+        "## 清洗前 describe（CSV 文本块）",
+        "```text",
+        basic_stats_before.to_csv(),
+        "```",
+        "",
+        "## 清洗后 describe（CSV 文本块）",
+        "```text",
+        basic_stats_after.to_csv(),
+        "```",
+        "",
+    ]
+    path.write_text("\n".join(lines), encoding="utf-8")
+
+
+def prepare_training_data(
+    df: pd.DataFrame,
+    cfg: AppConfig,
+    run_dir: Path,
+) -> ProcessedDataBundle:
+    """
+    完整预处理流水线：质量报告 -> 清洗 -> 划分 -> 训练集离群处理 -> 标准化 -> DataLoader。
+    将 data_report.md 与 cleaning 元数据写入 run_dir。
+    """
+    report_lines: List[str] = []
+    raw_q = quality_report_before_clean(df, cfg.v_pi_min, cfg.v_pi_max)
+    stats_before = df.describe().T
+
+    cleaned = clean_dataframe(df, cfg, report_lines)
+    stats_after = cleaned.describe().T
+
+    X = cleaned[INPUT_COLUMNS].to_numpy(dtype=np.float64)
+    y = cleaned[TARGET_COLUMNS].to_numpy(dtype=np.float64)
+
+    i_tr, i_va, i_te = stratified_split_indices(len(X), cfg.split_ratios, cfg.random_seed)
+    save_split_indices(run_dir, i_tr, i_va, i_te)
+    X_train, y_train = X[i_tr], y[i_tr]
+    X_val, y_val = X[i_va], y[i_va]
+    X_test, y_test = X[i_te], y[i_te]
+    report_lines.append(
+        f"划分 train/val/test = {cfg.split_ratios}，样本数 "
+        f"{len(X_train)}/{len(X_val)}/{len(X_test)}"
+    )
+
+    X_train, y_train, X_val, y_val, X_test, y_test = apply_train_only_outliers(
+        X_train, y_train, X_val, y_val, X_test, y_test, cfg, report_lines
+    )
+
+    X_scaler = StandardScaler()
+    y_scaler = StandardScaler()
+    X_train_s = X_scaler.fit_transform(X_train)
+    y_train_s = y_scaler.fit_transform(y_train)
+    X_val_s = X_scaler.transform(X_val)
+    y_val_s = y_scaler.transform(y_val)
+    X_test_s = X_scaler.transform(X_test)
+    y_test_s = y_scaler.transform(y_test)
+
+    save_scalers(X_scaler, y_scaler, run_dir)
+
+    meta = {
+        "raw_quality": raw_q,
+        "cleaning_steps": report_lines,
+        "split_ratios": cfg.split_ratios,
+        "n_train": int(len(X_train_s)),
+        "n_val": int(len(X_val_s)),
+        "n_test": int(len(X_test_s)),
+    }
+    (run_dir / "cleaning_meta.json").write_text(
+        json.dumps(meta, indent=2, ensure_ascii=False, default=str),
+        encoding="utf-8",
+    )
+    write_data_report_md(
+        run_dir / "data_report.md",
+        raw_q,
+        report_lines,
+        stats_before,
+        stats_after,
+    )
+    stats_after.to_csv(run_dir / "data_stats.csv", encoding="utf-8")
+    logger.info("预处理完成：%s", run_dir / "data_report.md")
+
+    train_loader, val_loader, test_loader = build_dataloaders(
+        X_train_s,
+        y_train_s,
+        X_val_s,
+        y_val_s,
+        X_test_s,
+        y_test_s,
+        cfg.training.batch_size,
+        cfg.training.num_workers,
+    )
+
+    return ProcessedDataBundle(
+        train_loader=train_loader,
+        val_loader=val_loader,
+        test_loader=test_loader,
+        X_train=X_train_s,
+        X_val=X_val_s,
+        X_test=X_test_s,
+        y_train=y_train_s,
+        y_val=y_val_s,
+        y_test=y_test_s,
+        X_test_raw=X_test,
+        y_test_raw=y_test,
+        X_scaler=X_scaler,
+        y_scaler=y_scaler,
+        feature_names=list(INPUT_COLUMNS),
+        target_names=list(TARGET_COLUMNS),
+    )

src/trainer.py

@@ -0,0 +1,188 @@
+"""训练循环、早停、调度器与 checkpoint。"""
+
+from __future__ import annotations
+
+import csv
+import logging
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Dict, List, Optional, Tuple
+
+import torch
+import torch.nn as nn
+from torch.optim import AdamW
+from torch.optim.lr_scheduler import CosineAnnealingLR, ReduceLROnPlateau
+from tqdm import tqdm
+
+from src.config import AppConfig
+from src.losses import build_loss
+from src.model import MLPRegressor
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class TrainHistory:
+    epoch: List[int]
+    train_loss: List[float]
+    val_loss: List[float]
+    lr: List[float]
+
+
+def _move_batch(
+    batch: Tuple[torch.Tensor, torch.Tensor], device: torch.device
+) -> Tuple[torch.Tensor, torch.Tensor]:
+    x, y = batch
+    return x.to(device), y.to(device)
+
+
+def train_one_epoch(
+    model: nn.Module,
+    loader: torch.utils.data.DataLoader,
+    criterion: nn.Module,
+    optimizer: torch.optim.Optimizer,
+    device: torch.device,
+) -> float:
+    model.train()
+    total, n = 0.0, 0
+    for batch in loader:
+        xb, yb = _move_batch(batch, device)
+        optimizer.zero_grad(set_to_none=True)
+        pred = model(xb)
+        loss = criterion(pred, yb)
+        loss.backward()
+        optimizer.step()
+        total += float(loss.detach().cpu()) * xb.size(0)
+        n += xb.size(0)
+    return total / max(n, 1)
+
+
+@torch.no_grad()
+def evaluate_loss_loader(
+    model: nn.Module,
+    loader: torch.utils.data.DataLoader,
+    criterion: nn.Module,
+    device: torch.device,
+) -> float:
+    model.eval()
+    total, n = 0.0, 0
+    for batch in loader:
+        xb, yb = _move_batch(batch, device)
+        pred = model(xb)
+        loss = criterion(pred, yb)
+        total += float(loss.detach().cpu()) * xb.size(0)
+        n += xb.size(0)
+    return total / max(n, 1)
+
+
+def build_optimizer_and_scheduler(
+    model: nn.Module, cfg: AppConfig
+) -> Tuple[AdamW, object]:
+    opt = AdamW(
+        model.parameters(),
+        lr=cfg.optimizer.lr,
+        weight_decay=cfg.optimizer.weight_decay,
+    )
+    if cfg.scheduler.type == "cosine":
+        sched: torch.optim.lr_scheduler._LRScheduler = CosineAnnealingLR(
+            opt, T_max=cfg.training.epochs, eta_min=cfg.scheduler.plateau_min_lr
+        )
+    elif cfg.scheduler.type == "plateau":
+        sched = ReduceLROnPlateau(
+            opt,
+            mode="min",
+            factor=cfg.scheduler.plateau_factor,
+            patience=cfg.scheduler.plateau_patience,
+            min_lr=cfg.scheduler.plateau_min_lr,
+        )
+    else:
+        raise ValueError(f"未知 scheduler.type: {cfg.scheduler.type}")
+    return opt, sched
+
+
+def fit(
+    model: nn.Module,
+    cfg: AppConfig,
+    train_loader: torch.utils.data.DataLoader,
+    val_loader: torch.utils.data.DataLoader,
+    run_dir: Path,
+    device: torch.device,
+) -> TrainHistory:
+    """
+    训练模型：早停依据验证集损失；保存 best / last 权重到 run_dir/checkpoints。
+    同步写入 train_log.csv。
+    """
+    criterion = build_loss(cfg.loss).to(device)
+    optimizer, scheduler = build_optimizer_and_scheduler(model, cfg)
+    ckpt_dir = run_dir / "checkpoints"
+    ckpt_dir.mkdir(parents=True, exist_ok=True)
+    log_path = run_dir / "train_log.csv"
+
+    best_val = float("inf")
+    best_epoch = -1
+    patience_left = cfg.training.early_stopping_patience
+
+    hist = TrainHistory(epoch=[], train_loss=[], val_loss=[], lr=[])
+
+    with log_path.open("w", newline="", encoding="utf-8") as fcsv:
+        writer = csv.writer(fcsv)
+        writer.writerow(["epoch", "train_loss", "val_loss", "lr", "best_val"])
+
+        for epoch in range(1, cfg.training.epochs + 1):
+            tr_loss = train_one_epoch(model, train_loader, criterion, optimizer, device)
+            va_loss = evaluate_loss_loader(model, val_loader, criterion, device)
+
+            if cfg.scheduler.type == "cosine":
+                scheduler.step()
+            elif cfg.scheduler.type == "plateau":
+                scheduler.step(va_loss)
+
+            lr_now = float(optimizer.param_groups[0]["lr"])
+            hist.epoch.append(epoch)
+            hist.train_loss.append(tr_loss)
+            hist.val_loss.append(va_loss)
+            hist.lr.append(lr_now)
+
+            improved = va_loss + 1e-12 < best_val
+            if improved:
+                best_val = va_loss
+                best_epoch = epoch
+                patience_left = cfg.training.early_stopping_patience
+                torch.save(
+                    {"epoch": epoch, "model_state": model.state_dict(), "val_loss": va_loss},
+                    ckpt_dir / "best.pt",
+                )
+            else:
+                patience_left -= 1
+
+            writer.writerow([epoch, tr_loss, va_loss, lr_now, best_val])
+            fcsv.flush()
+
+            logger.info(
+                "Epoch %d | train_loss=%.6f val_loss=%.6f | best_val=%.6f @%d",
+                epoch,
+                tr_loss,
+                va_loss,
+                best_val,
+                best_epoch,
+            )
+
+            torch.save(
+                {"epoch": epoch, "model_state": model.state_dict(), "val_loss": va_loss},
+                ckpt_dir / "last.pt",
+            )
+
+            if patience_left <= 0:
+                logger.info("早停触发于 epoch %d，最佳 epoch=%d", epoch, best_epoch)
+                break
+
+    return hist
+
+
+def load_weights(model: nn.Module, ckpt_path: Path, device: torch.device) -> None:
+    """从 checkpoint 载入 model_state。"""
+    try:
+        ckpt = torch.load(ckpt_path, map_location=device, weights_only=False)
+    except TypeError:
+        ckpt = torch.load(ckpt_path, map_location=device)
+    model.load_state_dict(ckpt["model_state"])

src/utils.py

@@ -0,0 +1,97 @@
+"""随机种子、日志与路径工具。"""
+
+from __future__ import annotations
+
+import logging
+import os
+import random
+import sys
+from datetime import datetime
+from pathlib import Path
+from typing import Optional
+
+import numpy as np
+import torch
+
+
+def set_global_seed(seed: int) -> None:
+    """设置 random / numpy / torch 与 cuDNN 行为以保证可复现性。"""
+    random.seed(seed)
+    np.random.seed(seed)
+    torch.manual_seed(seed)
+    if torch.cuda.is_available():
+        torch.cuda.manual_seed_all(seed)
+    torch.backends.cudnn.deterministic = True
+    torch.backends.cudnn.benchmark = False
+    # 部分算子仍可能非确定，但满足常规科研复现需求
+    os.environ.setdefault("CUBLAS_WORKSPACE_CONFIG", ":4096:8")
+    try:
+        torch.use_deterministic_algorithms(True, warn_only=True)
+    except (TypeError, AttributeError):
+        pass
+
+
+def make_run_dir(base_output: str | Path) -> Path:
+    """在 output_dir 下创建带时间戳的 run 目录。"""
+    base = Path(base_output)
+    base.mkdir(parents=True, exist_ok=True)
+    stamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+    run_dir = base / f"run_{stamp}"
+    run_dir.mkdir(parents=True, exist_ok=False)
+    (run_dir / "figures").mkdir(exist_ok=True)
+    (run_dir / "checkpoints").mkdir(exist_ok=True)
+    return run_dir
+
+
+def setup_logging(
+    log_file: Optional[Path] = None,
+    level: int = logging.INFO,
+) -> logging.Logger:
+    """
+    配置根 logger：控制台简洁格式，可选文件完整日志。
+    """
+    root = logging.getLogger()
+    root.handlers.clear()
+    root.setLevel(level)
+
+    fmt_console = logging.Formatter("%(levelname)s %(message)s")
+    ch = logging.StreamHandler(sys.stdout)
+    ch.setLevel(level)
+    ch.setFormatter(fmt_console)
+    root.addHandler(ch)
+
+    if log_file is not None:
+        log_file.parent.mkdir(parents=True, exist_ok=True)
+        fh = logging.FileHandler(log_file, encoding="utf-8")
+        fh.setLevel(logging.DEBUG)
+        fh.setFormatter(
+            logging.Formatter("%(asctime)s %(levelname)s [%(name)s] %(message)s")
+        )
+        root.addHandler(fh)
+
+    return logging.getLogger("mzm")
+
+
+def find_latest_run(output_dir: str | Path) -> Path:
+    """返回 output_dir 下按修改时间最近的一个 run_* 目录。"""
+    base = Path(output_dir)
+    if not base.is_dir():
+        raise FileNotFoundError(f"输出目录不存在: {base}")
+    runs = sorted(
+        [p for p in base.iterdir() if p.is_dir() and p.name.startswith("run_")],
+        key=lambda p: p.stat().st_mtime,
+        reverse=True,
+    )
+    if not runs:
+        raise FileNotFoundError(f"在 {base} 下未找到 run_* 子目录，请先训练模型。")
+    return runs[0]
+
+
+def resolve_path(path: str | Path, base: Optional[Path] = None) -> Path:
+    """将路径解析为绝对路径；若提供 base，则相对 base 解析。"""
+    p = Path(path)
+    if p.is_absolute():
+        return p
+    if base is not None:
+        return (base / p).resolve()
+    return p.resolve()

tests/test_smoke.py

@@ -0,0 +1,108 @@
+"""最小冒烟测试：模型 shape 与单 epoch 训练不报错。"""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import numpy as np
+import torch
+import yaml
+
+from src.config import load_config
+from src.data import load_raw_txt
+from src.model import MLPRegressor
+from src.preprocess import prepare_training_data
+from src.trainer import fit
+
+
+def _write_synthetic_txt(path: Path, n: int = 64) -> None:
+    rng = np.random.default_rng(0)
+    x = rng.normal(size=(n, 8))
+    y = np.zeros((n, 3))
+    y[:, 0] = rng.normal(size=n)
+    y[:, 1] = rng.normal(size=n)
+    # 第 11 列 V_pi 落在默认物理门控 [0, 500] 内
+    y[:, 2] = rng.uniform(1.0, 400.0, size=n)
+    mat = np.hstack([x, y])
+    lines = [",".join(str(v) for v in row) for row in mat]
+    path.write_text("\n".join(lines), encoding="utf-8")
+
+
+def test_mlp_forward_shape() -> None:
+    m = MLPRegressor(8, [16, 16], 3, batchnorm=False, dropout=0.0, residual=False)
+    x = torch.randn(5, 8)
+    y = m(x)
+    assert y.shape == (5, 3)
+
+
+def test_one_epoch_training_pipeline(tmp_path: Path) -> None:
+    data_txt = tmp_path / "data.txt"
+    _write_synthetic_txt(data_txt, n=80)
+
+    cfg_dict = {
+        "data_path": str(data_txt),
+        "split_ratios": [0.7, 0.15, 0.15],
+        "random_seed": 1,
+        "remove_duplicate_rows": False,
+        "outlier_strategy": "none",
+        "outlier_apply_to": "targets",
+        "outlier_config": {
+            "iqr_k": 1.5,
+            "zscore_threshold": 4.0,
+            "quantile_lower": 0.001,
+            "quantile_upper": 0.999,
+        },
+        "remove_nonpositive_vpi": False,
+        "filter_v_pi_range": True,
+        "v_pi_min": 0.0,
+        "v_pi_max": 500.0,
+        "model": {
+            "input_dim": 8,
+            "hidden_dims": [32, 32],
+            "output_dim": 3,
+            "batchnorm": False,
+            "dropout": 0.0,
+            "residual": False,
+        },
+        "optimizer": {"name": "adamw", "lr": 0.01, "weight_decay": 0.0},
+        "scheduler": {
+            "type": "cosine",
+            "plateau_factor": 0.5,
+            "plateau_patience": 10,
+            "plateau_min_lr": 1e-6,
+        },
+        "training": {
+            "batch_size": 16,
+            "epochs": 1,
+            "early_stopping_patience": 1,
+            "num_workers": 0,
+        },
+        "loss": {"type": "huber", "huber_delta": 1.0, "target_weights": [1.0, 1.0, 1.0]},
+        "output_dir": str(tmp_path / "results"),
+    }
+    cfg_path = tmp_path / "cfg.yaml"
+    cfg_path.write_text(yaml.safe_dump(cfg_dict), encoding="utf-8")
+
+    cfg = load_config(cfg_path)
+    df = load_raw_txt(data_txt)
+
+    run_dir = tmp_path / "run0"
+    run_dir.mkdir()
+    (run_dir / "figures").mkdir()
+    (run_dir / "checkpoints").mkdir()
+
+    bundle = prepare_training_data(df, cfg, run_dir)
+    device = torch.device("cpu")
+    model = MLPRegressor(
+        input_dim=cfg.model.input_dim,
+        hidden_dims=cfg.model.hidden_dims,
+        output_dim=cfg.model.output_dim,
+        batchnorm=cfg.model.batchnorm,
+        dropout=cfg.model.dropout,
+        residual=cfg.model.residual,
+    ).to(device)
+    fit(model, cfg, bundle.train_loader, bundle.val_loader, run_dir, device)
+    assert (run_dir / "checkpoints" / "best.pt").is_file()
+    meta = json.loads((run_dir / "cleaning_meta.json").read_text(encoding="utf-8"))
+    assert meta["n_train"] > 0