AI重构
This commit is contained in:
452
PIPELINE_GUIDE.md
Normal file
452
PIPELINE_GUIDE.md
Normal file
@@ -0,0 +1,452 @@
|
||||
# DronePlanning 处理 Pipeline 说明
|
||||
|
||||
本文档说明当前后端从自然语言输入到计划 JSON 输出的完整处理流程，以及可自定义修改点。
|
||||
|
||||
## 1. 总体流程
|
||||
|
||||
入口为 `POST /generate_plan`，主链路由 `GenerationOrchestrator` 编排：
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
api[main.py /generate_plan] --> gen[py_tree_generator.generate]
|
||||
gen --> orch[GenerationOrchestrator]
|
||||
orch --> s1[Stage1 TaskUnderstanding]
|
||||
orch --> s2[Stage2 ContextBinding]
|
||||
orch --> s3[Stage3 BTPlanning]
|
||||
orch --> s4[Stage4 ValidateAndPostprocess]
|
||||
s4 --> out[返回 py_tree JSON]
|
||||
```
|
||||
|
||||
对应代码：
|
||||
|
||||
- `backend_service/src/main.py`
|
||||
- `backend_service/src/py_tree_generator.py`
|
||||
- `backend_service/src/pipeline/orchestrator.py`
|
||||
- `backend_service/src/pipeline/stages.py`
|
||||
|
||||
## 2. Stage1：任务理解（TaskUnderstanding）
|
||||
|
||||
功能：
|
||||
|
||||
- 场景分类：`simple / scene1 / scene4`
|
||||
- 意图推断：`intent_type`
|
||||
- 风险标记：`risk_flags`（例如是否需要人工确认、是否涉及相对方位）
|
||||
|
||||
核心代码：
|
||||
|
||||
- `backend_service/src/llm/gateway.py::classify_scene()`
|
||||
- `backend_service/src/pipeline/stages.py::_infer_intent_type()`
|
||||
- `backend_service/src/pipeline/stages.py::_extract_risk_flags()`
|
||||
- 数据契约：`backend_service/src/pipeline/contracts.py::TaskUnderstanding`
|
||||
|
||||
说明：
|
||||
|
||||
- 分类模型可启用 thinking（由 `STAGE1_ENABLE_THINKING` 控制）。
|
||||
- 该阶段不依赖节点字典，避免循环依赖。
|
||||
|
||||
### 2.1 输入格式
|
||||
|
||||
`stage1_task_understanding(user_prompt: str)` 仅接受原始用户指令字符串。
|
||||
|
||||
示例输入：
|
||||
|
||||
```json
|
||||
{
|
||||
"user_prompt": "无人机当前在地面，到广场查找绿色公交车，找到后拍照。"
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 输出格式（TaskUnderstanding）
|
||||
|
||||
```json
|
||||
{
|
||||
"scene_mode": "scene4",
|
||||
"intent_type": "search_and_photo",
|
||||
"requires_relative_target": false,
|
||||
"entities": {
|
||||
"raw_prompt": "无人机当前在地面，到广场查找绿色公交车，找到后拍照。"
|
||||
},
|
||||
"risk_flags": [],
|
||||
"constraints": {}
|
||||
}
|
||||
```
|
||||
|
||||
字段说明：
|
||||
|
||||
- `scene_mode`: `simple | scene1 | scene4`
|
||||
- `intent_type`: 当前规则推导出的任务意图标签
|
||||
- `requires_relative_target`: 是否检测到相对方位需求
|
||||
- `entities`: 当前为轻量占位（最小包含 `raw_prompt`）
|
||||
- `risk_flags`: 风险标记列表（如 `needs_manual_confirmation`）
|
||||
- `constraints`: 约束占位（当前为空对象）
|
||||
|
||||
## 3. Stage2：上下文绑定（ContextBinding）
|
||||
|
||||
功能：
|
||||
|
||||
- 按场景动态决定检索范围（location/pattern/rules）
|
||||
- 从多知识库并行检索并汇总
|
||||
- 相对目标提取（`relative_refs`）
|
||||
- 预计算航点（`precomputed_waypoints`，MVP）
|
||||
- 基于意图与风险推导 `required_actions`
|
||||
|
||||
核心代码：
|
||||
|
||||
- `backend_service/src/retrieval/retriever.py::UnifiedRetriever`
|
||||
- `backend_service/src/retrieval/adapters/chroma_adapter.py`
|
||||
- `backend_service/src/llm/tool_runtime.py`
|
||||
- `backend_service/src/pipeline/stages.py::stage2_context_binding()`
|
||||
- 数据契约：`backend_service/src/pipeline/contracts.py::ContextBinding`
|
||||
|
||||
多知识库策略：
|
||||
|
||||
- 主集合：`location_kb`、`pattern_kb`、`rules_kb`
|
||||
- 兼容集合：`drone_docs`
|
||||
- 若主集合无结果，会尝试从 `drone_docs` + `kb_type` 过滤回退查询
|
||||
|
||||
### 3.1 输入格式
|
||||
|
||||
该阶段接收：
|
||||
|
||||
- `user_prompt: str`
|
||||
- Stage1 输出 `TaskUnderstanding`
|
||||
|
||||
示例输入：
|
||||
|
||||
```json
|
||||
{
|
||||
"user_prompt": "无人机当前在空中，去广场南边40米，持续监控5分钟，发现人就拍照。",
|
||||
"understanding": {
|
||||
"scene_mode": "scene4",
|
||||
"intent_type": "patrol_or_monitor",
|
||||
"requires_relative_target": false,
|
||||
"risk_flags": [],
|
||||
"entities": {"raw_prompt": "无人机当前在空中，去广场南边40米，持续监控5分钟，发现人就拍照。"},
|
||||
"constraints": {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 输出格式（ContextBinding）
|
||||
|
||||
```json
|
||||
{
|
||||
"location_context": "地点：广场，坐标(x=120,y=30,z=0)...",
|
||||
"pattern_context": "示例：先到达命名地点，再监控，再条件触发拍照...",
|
||||
"rules_context": "",
|
||||
"citations": {
|
||||
"location": ["..."],
|
||||
"pattern": ["..."],
|
||||
"rules": []
|
||||
},
|
||||
"resolved_refs": {},
|
||||
"precomputed_waypoints": [
|
||||
{"x": 160.0, "y": 30.0, "z": 10.0}
|
||||
],
|
||||
"relative_refs": [],
|
||||
"required_actions": [
|
||||
"Sequence",
|
||||
"fly_to_waypoint",
|
||||
"loiter",
|
||||
"object_detect",
|
||||
"object_detected",
|
||||
"take_photos"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
字段说明：
|
||||
|
||||
- `*_context`: 分知识域拼接后的文本上下文
|
||||
- `citations`: 每个知识域的原始命中文档片段
|
||||
- `precomputed_waypoints`: Stage2 静态可解析时的预计算坐标
|
||||
- `relative_refs`: 相对目标结构化描述
|
||||
- `required_actions`: 后续用于节点裁剪注入的动作白名单
|
||||
|
||||
## 4. Stage3：BT 生成（BTPlanning）
|
||||
|
||||
功能：
|
||||
|
||||
- 组装 prompt（骨架 + 节点裁剪 + 示例 + 规则 + 检索结果）
|
||||
- 调用对应模型生成严格 JSON
|
||||
- 解析模型响应（含 reasoning 提取）
|
||||
|
||||
核心代码：
|
||||
|
||||
- `backend_service/src/prompting/composer.py::PromptComposer`
|
||||
- `backend_service/src/llm/gateway.py::generate_json()`
|
||||
- `backend_service/src/llm/response_parser.py`
|
||||
- `backend_service/src/pipeline/stages.py::stage3_bt_planning()`
|
||||
- 数据契约：`backend_service/src/pipeline/contracts.py::BTDraft`
|
||||
|
||||
关键行为：
|
||||
|
||||
- simple 模式与复杂模式使用不同客户端/模型配置
|
||||
- Stage3 强制关闭 thinking，强制 `response_format=json_object`
|
||||
- 节点定义采用裁剪注入（非全量注入）
|
||||
|
||||
### 4.1 输入格式
|
||||
|
||||
该阶段接收：
|
||||
|
||||
- `user_prompt: str`
|
||||
- `TaskUnderstanding`
|
||||
- `ContextBinding`
|
||||
|
||||
核心输入（示例）：
|
||||
|
||||
```json
|
||||
{
|
||||
"scene_mode": "scene4",
|
||||
"intent_type": "search_and_photo",
|
||||
"required_actions": ["Sequence", "fly_to_waypoint", "rotate_search", "object_detected", "take_photos"],
|
||||
"risk_flags": [],
|
||||
"context_blocks": {
|
||||
"location": "地点：广场...",
|
||||
"pattern": "示例：先到达再搜索...",
|
||||
"rules": ""
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4.2 输出格式（BTDraft）
|
||||
|
||||
```json
|
||||
{
|
||||
"system_prompt": "...(裁剪后的系统提示词)...",
|
||||
"user_prompt": "原始指令 + 参考知识增强段",
|
||||
"allowed_nodes": {
|
||||
"actions": ["fly_to_waypoint", "rotate_search", "take_photos"],
|
||||
"conditions": ["object_detected"]
|
||||
},
|
||||
"llm_raw_json": {
|
||||
"root": {
|
||||
"type": "Sequence",
|
||||
"name": "Sequence",
|
||||
"children": [
|
||||
{"type": "action", "name": "fly_to_waypoint", "params": {"x": 120, "y": 30, "z": 10, "acceptance_radius": 2}},
|
||||
{"type": "action", "name": "rotate_search", "params": {"target_class": "bus"}},
|
||||
{"type": "condition", "name": "object_detected", "params": {"target_class": "bus"}},
|
||||
{"type": "action", "name": "take_photos", "params": {"target_class": "bus", "track_time": 8}}
|
||||
]
|
||||
}
|
||||
},
|
||||
"reasoning_text": null,
|
||||
"final_prompt": "=== System Prompt === ... === User Prompt === ..."
|
||||
}
|
||||
```
|
||||
|
||||
字段说明：
|
||||
|
||||
- `system_prompt/user_prompt`: 实际发给模型的提示词
|
||||
- `allowed_nodes`: 节点裁剪结果（用于调试与复盘）
|
||||
- `llm_raw_json`: 模型返回并解析后的原始计划 JSON
|
||||
- `reasoning_text`: 可选推理文本（若模型返回）
|
||||
- `final_prompt`: 完整组合记录（便于离线排查）
|
||||
|
||||
## 5. Stage4：校验与后处理（ValidateAndPostprocess）
|
||||
|
||||
功能：
|
||||
|
||||
- JSON Schema 校验（simple / complex）
|
||||
- 注入 `plan_id`、`visualization_url`、`final_prompt`
|
||||
- 保存推理链与历史记录
|
||||
- 在复杂场景下注入 `context.relative_refs`（及可选 `context.resolved_refs`）
|
||||
|
||||
核心代码：
|
||||
|
||||
- `backend_service/src/validation/validator.py`
|
||||
- `backend_service/src/validation/schema_provider.py`
|
||||
- `backend_service/src/pipeline/stages.py::stage4_validate_and_postprocess()`
|
||||
- `backend_service/src/py_tree_generator.py::render_visualization()`
|
||||
- `backend_service/src/py_tree_generator.py::_save_history()`
|
||||
|
||||
### 5.1 输入格式
|
||||
|
||||
该阶段接收：
|
||||
|
||||
- `user_prompt: str`
|
||||
- `TaskUnderstanding`
|
||||
- `ContextBinding`
|
||||
- `BTDraft`
|
||||
|
||||
其中主载荷来自 `BTDraft.llm_raw_json`。
|
||||
|
||||
### 5.2 输出格式（最终 API 返回）
|
||||
|
||||
复杂模式示例：
|
||||
|
||||
```json
|
||||
{
|
||||
"root": {
|
||||
"type": "Sequence",
|
||||
"name": "Sequence",
|
||||
"children": [
|
||||
{"type": "action", "name": "fly_to_waypoint", "params": {"x": 120, "y": 30, "z": 10, "acceptance_radius": 2}},
|
||||
{"type": "action", "name": "rotate_search", "params": {"target_class": "bus"}},
|
||||
{"type": "condition", "name": "object_detected", "params": {"target_class": "bus"}},
|
||||
{"type": "action", "name": "take_photos", "params": {"target_class": "bus", "track_time": 8}}
|
||||
]
|
||||
},
|
||||
"context": {
|
||||
"relative_refs": [
|
||||
{"anchor": "front_building", "relation": "left", "distance_m": 20.0}
|
||||
],
|
||||
"resolved_refs": {
|
||||
"strategy": "backend_static_resolution",
|
||||
"waypoints": [{"x": 120.0, "y": 30.0, "z": 10.0}]
|
||||
}
|
||||
},
|
||||
"plan_id": "6a924d0d-f1a7-4ef1-a9fa-31f73f3115ce",
|
||||
"visualization_url": "/static/py_tree.png",
|
||||
"final_prompt": "=== System Prompt === ... === User Prompt === ..."
|
||||
}
|
||||
```
|
||||
|
||||
simple 模式示例：
|
||||
|
||||
```json
|
||||
{
|
||||
"root": {
|
||||
"type": "action",
|
||||
"name": "move_direction",
|
||||
"params": {"direction": "north", "distance": 50}
|
||||
},
|
||||
"plan_id": "0f0e5e5f-b9a2-4e6f-95f5-c95e9e6280a5",
|
||||
"visualization_url": "/static/py_tree.png",
|
||||
"final_prompt": "=== System Prompt === ... === User Prompt === ..."
|
||||
}
|
||||
```
|
||||
|
||||
字段说明：
|
||||
|
||||
- `root`: 通过 schema 校验后的行为树根节点
|
||||
- `context`: 仅在复杂模式且命中相对目标时追加
|
||||
- `plan_id`: 每次生成唯一 ID
|
||||
- `visualization_url`: 最新可视化图访问路径
|
||||
- `final_prompt`: 生成时使用的完整提示词记录
|
||||
|
||||
## 6. 数据入库（RAG Ingestion）逻辑
|
||||
|
||||
入库脚本：
|
||||
|
||||
- `tools/rag/ingest.py`
|
||||
|
||||
行为：
|
||||
|
||||
- 扫描 `tools/rag/knowledge_base/`
|
||||
- 根据子目录推断 `kb_type`（location/pattern/rules）
|
||||
- 同时写入：
|
||||
- `drone_docs`（兼容）
|
||||
- `location_kb` / `pattern_kb` / `rules_kb`（新检索路径）
|
||||
|
||||
## 7. 可自定义修改点（推荐按优先级）
|
||||
|
||||
### 7.1 场景与意图逻辑
|
||||
|
||||
可改文件：
|
||||
|
||||
- `backend_service/src/pipeline/stages.py`
|
||||
|
||||
可改内容：
|
||||
|
||||
- `_infer_intent_type()`：扩展意图类别
|
||||
- `_extract_risk_flags()`：新增风险规则
|
||||
- `_derive_required_actions()`：调整规则推导的动作集合
|
||||
|
||||
### 7.2 Prompt 策略
|
||||
|
||||
可改文件：
|
||||
|
||||
- `backend_service/src/prompting/composer.py`
|
||||
- `backend_service/src/prompts/prompt_manifest.yaml`
|
||||
- `backend_service/src/prompts/partials/*`
|
||||
|
||||
可改内容：
|
||||
|
||||
- 骨架片段选择
|
||||
- 节点裁剪规则（当前上限 30）
|
||||
- 示例注入策略（何时注入 extra examples）
|
||||
- 用户侧检索增强格式
|
||||
|
||||
### 7.3 模型路由与推理参数
|
||||
|
||||
可改文件：
|
||||
|
||||
- `backend_service/src/llm/gateway.py`
|
||||
|
||||
可改内容：
|
||||
|
||||
- 分类模型与生成模型分流策略
|
||||
- `temperature`、`max_tokens`、重试次数
|
||||
- thinking 开关策略（Stage1/Stage3）
|
||||
|
||||
### 7.4 检索策略
|
||||
|
||||
可改文件：
|
||||
|
||||
- `backend_service/src/retrieval/retriever.py`
|
||||
- `backend_service/src/retrieval/adapters/chroma_adapter.py`
|
||||
- `tools/rag/ingest.py`
|
||||
|
||||
可改内容：
|
||||
|
||||
- 检索并发策略与 `top_k`
|
||||
- 回退策略（主集合与兼容集合）
|
||||
- kb_type 划分方式
|
||||
- 文档切分与 metadata 设计
|
||||
|
||||
### 7.5 相对目标解析
|
||||
|
||||
可改文件：
|
||||
|
||||
- `backend_service/src/pipeline/stages.py`
|
||||
- `backend_service/src/llm/tool_runtime.py`
|
||||
|
||||
可改内容：
|
||||
|
||||
- `relative_refs` 抽取规则
|
||||
- 静态解析能力（何时生成 `resolved_refs`）
|
||||
- 与 UAV 端协议字段兼容策略
|
||||
|
||||
### 7.6 校验与输出协议
|
||||
|
||||
可改文件：
|
||||
|
||||
- `backend_service/src/validation/validator.py`
|
||||
- `backend_service/src/validation/schema_provider.py`
|
||||
- `backend_service/src/py_tree_generator.py`（schema 来源）
|
||||
|
||||
可改内容：
|
||||
|
||||
- simple/complex schema 约束强度
|
||||
- 顶层 `context` 字段的可选校验
|
||||
- 失败错误信息与恢复策略
|
||||
|
||||
## 8. 关键环境变量
|
||||
|
||||
- `ORIN_IP`
|
||||
- `OPENAI_API_KEY`
|
||||
- `CLASSIFIER_MODEL` / `SIMPLE_MODEL` / `COMPLEX_MODEL`
|
||||
- `CLASSIFIER_BASE_URL` / `SIMPLE_BASE_URL` / `COMPLEX_BASE_URL`
|
||||
- `STAGE1_ENABLE_THINKING`
|
||||
- `ENABLE_REASONING_CAPTURE`
|
||||
- `REASONING_PREVIEW_LINES`
|
||||
|
||||
## 9. 自定义改造建议（实践顺序）
|
||||
|
||||
1. 先改 Stage1 规则推导（低风险，收益快）
|
||||
2. 再改 PromptComposer 的裁剪与注入（控制长度与稳定性）
|
||||
3. 再改检索策略（top_k、回退、metadata）
|
||||
4. 最后改 schema 与输出协议（需联动执行端）
|
||||
|
||||
## 10. 变更后最小回归清单
|
||||
|
||||
每次改造后至少验证：
|
||||
|
||||
1. simple 指令：返回 `root.action`，且无 children
|
||||
2. scene4 指令：有复合树结构，JSON 可解析
|
||||
3. relative 指令：复杂模式下出现 `context.relative_refs`
|
||||
4. `/generate_plan` 不变更接口字段（兼容外部调用）
|
||||
5. `python tools/rag/ingest.py` 可完成入库（或输出可定位错误）
|
||||
|
||||
Reference in New Issue
Block a user