新增说明
This commit is contained in:
28
README.md
28
README.md
@@ -19,6 +19,7 @@
|
||||
│ │ │ └── classifier_prompt.txt # 指令简单/复杂分类提示词
|
||||
│ │ ├── ...
|
||||
│ ├── generated_visualizations/ # 存放最新生成的py_tree可视化图像
|
||||
│ ├── generated_reasoning_content/ # 存放最新推理链Markdown(<plan_id>.md)
|
||||
│ └── requirements.txt # 后端服务的Python依赖
|
||||
│
|
||||
├── tools/
|
||||
@@ -26,7 +27,8 @@
|
||||
│ ├── knowledge_base/ # 【处理后】存放build_knowledge_base.py生成的.ndjson文件
|
||||
│ ├── vector_store/ # 【数据库】存放最终的ChromaDB向量数据库
|
||||
│ ├── build_knowledge_base.py # 【步骤1】用于将原始数据转换为自然语言知识
|
||||
│ └── ingest.py # 【步骤2】用于将自然语言知识摄入向量数据库
|
||||
│ ├── ingest.py # 【步骤2】用于将自然语言知识摄入向量数据库
|
||||
│ └── test_llama_server.py # 直接调用本地8081端口llama-server,支持 --system / --system-file
|
||||
│
|
||||
├── / # ROS2接口定义 (保持不变)
|
||||
└── docs/
|
||||
@@ -95,6 +97,10 @@
|
||||
|
||||
通用API Key:`OPENAI_API_KEY`
|
||||
|
||||
推理链捕获相关:
|
||||
- `ENABLE_REASONING_CAPTURE`:是否允许模型返回含有 <think> 的原文以便捕获推理链;默认 true。
|
||||
- `REASONING_PREVIEW_LINES`:在后端日志中打印推理链预览的行数;默认 20。
|
||||
|
||||
示例:
|
||||
```bash
|
||||
export CLASSIFIER_MODEL="qwen2.5-1.8b-instruct"
|
||||
@@ -104,6 +110,10 @@ export CLASSIFIER_BASE_URL="http://$ORIN_IP:8081/v1"
|
||||
export SIMPLE_BASE_URL="http://$ORIN_IP:8081/v1"
|
||||
export COMPLEX_BASE_URL="http://$ORIN_IP:8081/v1"
|
||||
export OPENAI_API_KEY="sk-no-key-required"
|
||||
|
||||
# 推理链捕获(可选)
|
||||
export ENABLE_REASONING_CAPTURE=true # 默认已为true;如需关闭,设置为 false
|
||||
export REASONING_PREVIEW_LINES=30 # 调整日志预览行数
|
||||
```
|
||||
|
||||
### 测试简单模式
|
||||
@@ -117,6 +127,22 @@ python test_api.py
|
||||
|
||||
示例输入:“简单模式,起飞” 或 “起飞到10米”。返回结果为简单JSON(无 `root`):包含 `mode`、`action`、`plan_id`、`visualization_url`。
|
||||
|
||||
### 直接调用 llama-server(绕过后端)
|
||||
|
||||
当仅需测试本地 8081 端口的推理服务(OpenAI 兼容接口)时,可使用内置脚本:
|
||||
|
||||
```bash
|
||||
python tools/test_llama_server.py \
|
||||
--system-file backend_service/src/prompts/system_prompt.txt \
|
||||
--user "起飞到10米然后降落" \
|
||||
--base-url "http://127.0.0.1:8081/v1" \
|
||||
--verbose
|
||||
```
|
||||
|
||||
说明:
|
||||
- 支持 `--system` 或 `--system-file` 自定义提示词文件;`--system-file` 优先。
|
||||
- 默认解析 OpenAI 风格返回,若包含 `<think>` 推理内容会显示在输出中(具体取决于模型和服务配置)。
|
||||
|
||||
---
|
||||
|
||||
## 工作流程
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
好的,我现在需要处理用户的任务指令,生成一个符合要求的行为树JSON。首先,我得仔细阅读用户的指令和参考知识,确保理解所有要求。
|
||||
|
||||
用户的要求是:无人机起飞后,先移动至学生宿舍,搜索并锁定任一红色的球,在对其环绕三十两圈后进行打击操作。参考知识中提到学生宿舍的坐标是(x:5, y:3, z:2)。不过,这里有个问题,用户提到的“红色的球”可能不在学生宿舍区域,但根据参考知识,学生宿舍的坐标是(5,3,2),而其他地点如跷跷板和A地的坐标不同。不过用户可能希望无人机先飞到学生宿舍,然后在附近搜索球。
|
||||
|
||||
首先,我需要确保所有坐标符合物理约束。学生宿舍的坐标是x=5,y=3,z=2,这在允许的范围内。接下来,无人机需要起飞,然后飞往学生宿舍。这里可能需要使用fly_to_waypoint动作,将坐标设置为学生宿舍的坐标。
|
||||
|
||||
接下来是搜索红色球。用户提到“搜索并锁定任一红色的球”,这里可能需要使用search_pattern或者object_detect。不过,用户可能希望先飞到学生宿舍区域,然后进行搜索。根据参考知识,学生宿舍的坐标是(5,3,2),所以可能需要先飞到那里,然后进行搜索。不过,用户可能希望在学生宿舍附近搜索,所以可能需要使用search_pattern,或者先悬停再检测。
|
||||
|
||||
不过,用户提到“搜索并锁定”,可能需要先使用search_pattern来搜索,或者直接使用object_detect。但object_detect需要无人机在目标附近悬停,而search_pattern是移动搜索。根据任务描述,可能先飞到学生宿舍,然后进行搜索,所以可能需要使用search_pattern,或者先飞到学生宿舍,然后使用object_detect。
|
||||
|
||||
不过,用户可能希望先飞到学生宿舍,然后在该区域搜索红色球。所以可能需要先fly_to_waypoint到学生宿舍,然后使用search_pattern或者object_detect。但根据参考知识,学生宿舍的坐标是(5,3,2),而红色球可能在附近,所以可能需要使用search_pattern,或者直接使用object_detect,但需要确认位置。
|
||||
|
||||
不过,用户可能希望先飞到学生宿舍,然后在该区域进行搜索。因此,可能需要先fly_to_waypoint到学生宿舍,然后使用search_pattern来搜索红色球。或者,如果红色球的位置已知,可能直接使用object_detect。但根据用户指令,可能需要先搜索,所以使用search_pattern更合适。
|
||||
|
||||
接下来,找到红色球后,需要环绕三十两圈。这里的“三十两圈”可能是笔误,应该是“三十圈”或者“三圈”?但用户写的是“三十两圈”,可能是指半径30米,绕两圈?或者可能是“三十圈”?需要根据中文习惯判断。通常,“三十两圈”可能是指半径30米,绕两圈,或者可能是“三十圈”?这里可能需要进一步确认,但根据用户给出的参考知识,可能需要将半径设为30米,绕两圈。或者,用户可能想表达的是“三十圈”?但中文中“三十两圈”不太常见,可能是“三十圈”或“三圈”。不过,根据用户给出的参考知识中的“环绕三十两圈”,可能是指半径30米,绕两圈。因此,在orbit_around_target中设置radius=30,laps=2。
|
||||
|
||||
然后进行打击操作,使用strike_target动作。
|
||||
|
||||
同时,必须包含安全监控,使用Parallel节点,包含电池和GPS监控,以及紧急处理流程。
|
||||
|
||||
现在,构建行为树的结构:
|
||||
|
||||
根节点是Parallel,包含主任务和安全监控。主任务是一个Sequence,包含起飞、飞往学生宿舍、搜索球、环绕、打击、降落。安全监控是Selector,包含电池、GPS条件,以及紧急处理Sequence。
|
||||
|
||||
在飞往学生宿舍时,使用fly_to_waypoint,参数x=5,y=3,z=2。然后进行搜索,使用search_pattern,可能以学生宿舍为中心,半径适当,比如50米,搜索红色球。或者,可能需要先悬停,使用object_detect。但用户可能希望先搜索,所以使用search_pattern,参数center_x=5,center_y=3,radius=50,target_class=ball,description=红色球。不过,用户提到的是红色球,可能需要调整参数。
|
||||
|
||||
接下来,环绕目标,使用orbit_around_target,参数center_x=5(假设球在学生宿舍附近),但需要确认。或者,可能在搜索后,锁定球的位置,然后环绕。但根据用户指令,可能需要先飞到学生宿舍,然后搜索球,所以search_pattern的中心是学生宿舍的坐标,半径50米,搜索红色球。然后,环绕该球,使用orbit_around_target,radius=30,laps=2。然后进行打击。
|
||||
|
||||
现在,检查所有参数是否符合约束:
|
||||
|
||||
fly_to_waypoint的坐标是学生宿舍的(5,3,2),z=2,符合约束。
|
||||
|
||||
search_pattern的参数:pattern_type=spiral,center_x=5,center_y=3,center_z=2(学生宿舍的坐标),radius=50,target_class=ball,description=红色球。这符合约束。
|
||||
|
||||
然后,环绕该球,使用orbit_around_target,参数center_x=5(假设球在学生宿舍附近),但需要确认。或者,可能在搜索后,球的位置被确认,所以使用orbit_around_target,中心是搜索到的球的位置。但根据用户指令,可能需要在搜索后,锁定球的位置,然后环绕。因此,在search_pattern后,可能需要使用object_detect来确认球的位置,但用户可能希望直接环绕。
|
||||
|
||||
不过,用户可能希望在搜索后,锁定球的位置,然后环绕。因此,在search_pattern后,可能需要使用object_detect来确认球的位置,但根据任务描述,可能需要直接环绕。
|
||||
|
||||
因此,在search_pattern后,可能需要使用orbit_around_target,参数center_x=5(假设球在学生宿舍附近),但需要确认。或者,可能在搜索后,球的位置被确认,所以使用orbit_around_target,中心是球的位置。
|
||||
|
||||
但根据用户指令,可能需要先飞到学生宿舍,然后在该区域搜索球,所以search_pattern的中心是学生宿舍的坐标,半径50米,搜索红色球。然后,环绕该球,使用orbit_around_target,参数center_x=5(假设球在学生宿舍附近),radius=30,laps=2。
|
||||
|
||||
然后进行打击操作。
|
||||
|
||||
现在,构建行为树的结构:
|
||||
|
||||
根节点是Parallel,包含主任务Sequence和安全监控Selector。
|
||||
|
||||
主任务Sequence的步骤:
|
||||
|
||||
1. preflight_checks,检查级别为comprehensive。
|
||||
2. takeoff,altitude=2.0。
|
||||
3. fly_to_waypoint到学生宿舍的坐标(5,3,2),acceptance_radius=2.0。
|
||||
4. search_pattern,pattern_type=spiral,center_x=5,center_y=3,center_z=2,radius=50,target_class=ball,description=红色球。
|
||||
5. orbit_around_target,参数center_x=5(假设球在学生宿舍附近),但需要确认。或者,可能在搜索后,球的位置被确认,所以使用orbit_around_target,中心是球的位置。但根据用户指令,可能需要在搜索后,锁定球的位置,然后环绕。因此,在search_pattern后,可能需要使用object_detect来确认球的位置,但用户可能希望直接环绕。
|
||||
|
||||
因此,在search_pattern后,可能需要使用orbit_around_target,参数center_x=5(假设球在学生宿舍附近),radius=30,laps=2。
|
||||
|
||||
然后进行strike_target操作。
|
||||
|
||||
最后,land,mode=home。
|
||||
|
||||
同时,安全监控Selector包含电池条件(threshold=0.35)和GPS条件(min_satellites=8),以及紧急处理Sequence。
|
||||
|
||||
现在,检查所有参数是否符合约束:
|
||||
|
||||
- fly_to_waypoint的坐标是学生宿舍的(5,3,2),z=2,符合约束。
|
||||
- search_pattern的参数:center_x=5,center_y=3,center_z=2,radius=50,符合约束。
|
||||
- orbit_around_target的参数:center_x=5(假设球在学生宿舍附近),但需要确认。或者,可能在搜索后,球的位置被确认,所以使用orbit_around_target,中心是球的位置。但根据用户指令,可能需要在搜索后,锁定球的位置,然后环绕。因此,在search_pattern后,可能需要使用object_detect来确认球的位置,但用户可能希望直接环绕。
|
||||
|
||||
因此,在search_pattern后,可能需要使用orbit_around_target,参数center_x=5(假设球在学生宿舍附近),radius=30,laps=2。
|
||||
|
||||
然后进行strike_target操作。
|
||||
|
||||
现在,构建JSON:
|
||||
|
||||
根节点是Parallel,包含主任务Sequence和安全监控Selector。
|
||||
|
||||
主任务Sequence的步骤:
|
||||
|
||||
1. preflight_checks,参数check_level=comprehensive。
|
||||
2. takeoff,参数altitude=2.0。
|
||||
3. fly_to_waypoint到学生宿舍的坐标(5,3,2),acceptance_radius=2.0。
|
||||
4. search_pattern,参数pattern_type=spiral,center_x=5,center_y=3,center_z=2,radius=50,target_class=ball,description=红色球。
|
||||
5. orbit_around_target,参数center_x=5(假设球在学生宿舍附近),但需要确认。或者,可能在搜索后,球的位置被确认,所以使用orbit_around_target,中心是球的位置。但根据用户指令,可能需要在搜索后,锁定球的位置,然后环绕。因此,在search_pattern后,可能需要使用object_detect来确认球的位置,但用户可能希望直接环绕。
|
||||
|
||||
因此,在search_pattern后,可能需要使用orbit_around_target,参数center_x=5(假设球在学生宿舍附近),radius=30,laps=2。
|
||||
|
||||
然后进行strike_target操作。
|
||||
|
||||
最后,land,mode=home。
|
||||
|
||||
同时,安全监控Selector包含电池条件(threshold=0.35)和GPS条件(min_satellites=8),以及紧急处理Sequence。
|
||||
|
||||
现在,检查所有参数是否符合约束:
|
||||
|
||||
- orbit_around_target的参数center_x=5,但需要确认球的位置是否在学生宿舍附近。根据用户指令,可能需要在搜索后,锁定球的位置,所以使用orbit_around_target,中心是球的位置。但根据参考知识,学生宿舍的坐标是(5,3,2),而红色球可能在附近,所以使用orbit_around_target,参数center_x=5(学生宿舍的坐标),radius=30,laps=2。
|
||||
|
||||
这样,参数符合约束。
|
||||
|
||||
现在,构建JSON:
|
||||
|
||||
{
|
||||
"root": {
|
||||
"type": "Parallel",
|
||||
"name": "CompleteStrikeMission",
|
||||
"params": {
|
||||
"policy": "all_success"
|
||||
},
|
||||
"children": [
|
||||
{
|
||||
"type": "Sequence",
|
||||
"name": "MainStrikeSequence",
|
||||
"children": [
|
||||
{
|
||||
"type": "action",
|
||||
"name": "preflight_checks",
|
||||
"params": {
|
||||
"check_level": "comprehensive"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "action",
|
||||
"name": "takeoff",
|
||||
"params": {
|
||||
"altitude": 2.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "action",
|
||||
"name": "fly_to_waypoint",
|
||||
"params": {
|
||||
"x": 5.0,
|
||||
"y": 3.0,
|
||||
"z": 2.0,
|
||||
"acceptance_radius": 2.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "action",
|
||||
"name": "search_pattern",
|
||||
"params": {
|
||||
"pattern_type": "spiral",
|
||||
"center_x": 5.0,
|
||||
"center_y": 3.0,
|
||||
"center_z": 2.0,
|
||||
"radius": 50.0,
|
||||
"target_class": "ball",
|
||||
"description": "红色球",
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "action",
|
||||
"name": "orbit_around_target",
|
||||
"params": {
|
||||
"center_x": 5.0,
|
||||
"center_y": 3.0,
|
||||
"center_z": 2.0,
|
||||
"radius": 30.0,
|
||||
"laps": 2,
|
||||
"clockwise": true,
|
||||
"gimbal_lock": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "action",
|
||||
"name": "strike_target",
|
||||
"params": {
|
||||
"target_class": "ball",
|
||||
"description": "红色球",
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "action",
|
||||
"name": "land",
|
||||
"params": {
|
||||
"mode": "home"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "Selector",
|
||||
"name": "SafetyMonitor",
|
||||
"params": {
|
||||
"memory": true
|
||||
},
|
||||
"children": [
|
||||
{
|
||||
"type": "condition",
|
||||
"name": "battery_above",
|
||||
"params": {
|
||||
"threshold": 0.35
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "condition",
|
||||
"name": "gps_status",
|
||||
"params": {
|
||||
"min_satellites": 8
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "Sequence",
|
||||
"name": "EmergencyHandler",
|
||||
"children": [
|
||||
{
|
||||
"type": "action",
|
||||
"name": "emergency_return",
|
||||
"params": {
|
||||
"reason": "safety_breach"
|
||||
}
|
||||
}
|
||||
,
|
||||
{
|
||||
"type": "action",
|
||||
"name": "land",
|
||||
"params": {
|
||||
"mode": "home"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 200 KiB After Width: | Height: | Size: 208 KiB |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1,11 +1,10 @@
|
||||
你是一个无人机简单指令执行规划器。你的任务:当用户给出“简单指令”(单一原子动作即可完成)时,输出一个严格的JSON对象。
|
||||
你是一个无人机简单指令执行规划器。你的任务:输出一个严格的JSON对象。
|
||||
|
||||
输出要求(必须遵守):
|
||||
- 只输出一个JSON对象,不要任何解释或多余文本。
|
||||
- JSON结构:
|
||||
{"mode":"simple","action":{"name":"<action_name>","params":{...}}}
|
||||
- <action_name> 与参数定义、取值范围,必须与“复杂模式”提示词(system_prompt.txt)中的定义完全一致。
|
||||
- 简单模式下不包含任何行为树结构与安全监控并行,仅输出单一原子动作。
|
||||
- 不包含任何行为树结构与安全监控并行,仅输出单一原子动作。
|
||||
|
||||
示例:
|
||||
- “起飞到10米” → {"mode":"simple","action":{"name":"takeoff","params":{"altitude":10.0}}}
|
||||
@@ -21,13 +20,13 @@
|
||||
{"name": "fly_to_waypoint", "description": "导航至一个指定坐标点。使用相对坐标系(x,y,z),单位为米。", "params": {"x": "float", "y": "float", "z": "float", "acceptance_radius": "float, 可选,默认2.0"}},
|
||||
{"name": "move_direction", "description": "按指定方向直线移动。方向可为绝对方位或相对机体朝向。", "params": {"direction": "string: north|south|east|west|forward|backward|left|right", "distance": "float[1,10000], 可选, 不指定则持续移动"}},
|
||||
{"name": "orbit_around_point", "description": "以给定中心点为中心,等速圆周飞行指定圈数。", "params": {"center_x": "float", "center_y": "float", "center_z": "float", "radius": "float[5,1000]", "laps": "int[1,20]", "clockwise": "boolean, 可选, 默认true", "speed_mps": "float[0.5,15], 可选", "gimbal_lock": "boolean, 可选, 默认true"}},
|
||||
{"name": "orbit_around_target", "description": "以目标为中心,等速圆周飞行指定圈数(需已有目标)。", "params": {"target_class": "string, 见复杂模式定义列表", "description": "string, 可选", "radius": "float[5,1000]", "laps": "int[1,20]", "clockwise": "boolean, 可选, 默认true", "speed_mps": "float[0.5,15], 可选", "gimbal_lock": "boolean, 可选, 默认true"}},
|
||||
{"name": "orbit_around_target", "description": "以目标为中心,等速圆周飞行指定圈数(需已有目标)。", "params": {"target_class": "string, 取值同object_detect列表", "description": "string, 可选", "radius": "float[5,1000]", "laps": "int[1,20]", "clockwise": "boolean, 可选, 默认true", "speed_mps": "float[0.5,15], 可选", "gimbal_lock": "boolean, 可选, 默认true"}},
|
||||
{"name": "loiter", "description": "在当前位置上空悬停一段时间或直到条件触发。", "params": {"duration": "float, 可选[1,600]", "until_condition": "string, 可选"}},
|
||||
{"name": "object_detect", "description": "识别特定目标对象。", "params": {"target_class": "string, 见复杂模式定义列表", "description": "string, 可选", "count": "int, 可选, 默认1"}},
|
||||
{"name": "object_detect", "description": "识别特定目标对象。一般是用户提到的需要检测的目标;如果用户给出了需要探索的目标的优先级,比如蓝色球危险性大于红色球大于绿色球,需要检测最危险的球,此处应给出检测优先级,描述应当为 '蓝>红>绿'", "params": {"target_class": "string, 要识别的目标类别,必须为以下值之一: balloon,person, bicycle, car, motorcycle, airplane, bus, train, truck, boat, traffic_light, fire_hydrant, stop_sign, parking_meter, bench, bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe, backpack, umbrella, handbag, tie, suitcase, frisbee, skis, snowboard, sports_ball, kite, baseball_bat, baseball_glove, skateboard, surfboard, tennis_racket, bottle, wine_glass, cup, fork, knife, spoon, bowl, banana, apple, sandwich, orange, broccoli, carrot, hot_dog, pizza, donut, cake, chair, couch, potted_plant, bed, dining_table, toilet, tv, laptop, mouse, remote, keyboard, cell_phone, microwave, oven, toaster, sink, refrigerator, book, clock, vase, scissors, teddy_bear, hair_drier, toothbrush", "description": "string, 可选", "count": "int, 可选, 默认1"}},
|
||||
{"name": "strike_target", "description": "对已识别目标进行打击。", "params": {"target_class": "string", "description": "string, 可选", "count": "int, 可选, 默认1"}},
|
||||
{"name": "battle_damage_assessment", "description": "战损评估。", "params": {"target_class": "string", "assessment_time": "float[5-60], 默认15.0"}},
|
||||
{"name": "search_pattern", "description": "按模式搜索。", "params": {"pattern_type": "string: spiral|grid", "center_x": "float", "center_y": "float", "center_z": "float", "radius": "float[5,1000]", "target_class": "string", "description": "string, 可选", "count": "int, 可选, 默认1"}},
|
||||
{"name": "track_object", "description": "持续跟踪目标。", "params": {"target_class": "string, 见复杂模式定义列表", "description": "string, 可选", "track_time": "float[1,600], 默认30.0", "min_confidence": "float[0.5-1.0], 默认0.7", "safe_distance": "float[2-50], 默认10.0"}},
|
||||
{"name": "track_object", "description": "持续跟踪目标。", "params": {"target_class": "string, 取值同object_detect列表", "description": "string, 可选", "track_time": "float[1,600], 默认30.0", "min_confidence": "float[0.5-1.0], 默认0.7", "safe_distance": "float[2-50], 默认10.0"}},
|
||||
{"name": "deliver_payload", "description": "投放物资。", "params": {"payload_type": "string", "release_altitude": "float[2,100], 默认5.0"}},
|
||||
{"name": "preflight_checks", "description": "飞行前系统自检。", "params": {"check_level": "string: basic|comprehensive"}},
|
||||
{"name": "emergency_return", "description": "执行紧急返航程序。", "params": {"reason": "string"}}
|
||||
@@ -53,7 +52,6 @@
|
||||
- orbit_around_target.radius: [5, 1000]
|
||||
- orbit_around_point/target.laps: [1, 20]
|
||||
- orbit_around_point/target.speed_mps: [0.5, 15]
|
||||
- 电池阈值等同复杂模式(如需涉及)
|
||||
- 若参考知识提供坐标,必须使用并裁剪到约束范围内
|
||||
|
||||
—— 口令转化规则(环绕类)——
|
||||
|
||||
@@ -85,8 +85,8 @@
|
||||
"name": "object_detect",
|
||||
"description": "在当前视野范围内识别特定目标对象。适用于定点检测,无人机应在目标大致位置悬停或保持稳定姿态。",
|
||||
"params": {
|
||||
"target_class": "string, 要识别的目标类别,必须为以下值之一: person, bicycle, car, motorcycle, airplane, bus, train, truck, boat, traffic_light, fire_hydrant, stop_sign, parking_meter, bench, bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe, backpack, umbrella, handbag, tie, suitcase, frisbee, skis, snowboard, sports_ball, kite, baseball_bat, baseball_glove, skateboard, surfboard, tennis_racket, bottle, wine_glass, cup, fork, knife, spoon, bowl, banana, apple, sandwich, orange, broccoli, carrot, hot_dog, pizza, donut, cake, chair, couch, potted_plant, bed, dining_table, toilet, tv, laptop, mouse, remote, keyboard, cell_phone, microwave, oven, toaster, sink, refrigerator, book, clock, vase, scissors, teddy_bear, hair_drier, toothbrush",
|
||||
"description": "string, 可选,目标属性描述(如颜色、状态等)",
|
||||
"target_class": "string, 要识别的目标类别,必须为以下值之一: balloon,person, bicycle, car, motorcycle, airplane, bus, train, truck, boat, traffic_light, fire_hydrant, stop_sign, parking_meter, bench, bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe, backpack, umbrella, handbag, tie, suitcase, frisbee, skis, snowboard, sports_ball, kite, baseball_bat, baseball_glove, skateboard, surfboard, tennis_racket, bottle, wine_glass, cup, fork, knife, spoon, bowl, banana, apple, sandwich, orange, broccoli, carrot, hot_dog, pizza, donut, cake, chair, couch, potted_plant, bed, dining_table, toilet, tv, laptop, mouse, remote, keyboard, cell_phone, microwave, oven, toaster, sink, refrigerator, book, clock, vase, scissors, teddy_bear, hair_drier, toothbrush",
|
||||
"description": "string, 可选,目标属性描述(如颜色、状态等),一般是用户提到的需要检测的目标;如果用户给出了需要探索的目标的优先级,比如蓝色球危险性大于红色球大于绿色球,需要检测最危险的球,此处应给出检测优先级,描述应当为 '蓝>红>绿'",
|
||||
"count": "int, 可选,需要检测的目标个数,默认1"
|
||||
}
|
||||
},
|
||||
|
||||
@@ -201,7 +201,7 @@ def _generate_pytree_schema(allowed_actions: set, allowed_conditions: set) -> di
|
||||
"sandwich", "orange", "broccoli", "carrot", "hot_dog", "pizza", "donut", "cake", "chair",
|
||||
"couch", "potted_plant", "bed", "dining_table", "toilet", "tv", "laptop", "mouse", "remote",
|
||||
"keyboard", "cell_phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book",
|
||||
"clock", "vase", "scissors", "teddy_bear", "hair_drier", "toothbrush"
|
||||
"clock", "vase", "scissors", "teddy_bear", "hair_drier", "toothbrush","balloon"
|
||||
]
|
||||
|
||||
# 递归节点定义
|
||||
@@ -548,6 +548,16 @@ class PyTreeGenerator:
|
||||
# Updated output directory for visualizations
|
||||
self.vis_dir = os.path.abspath(os.path.join(self.base_dir, '..', 'generated_visualizations'))
|
||||
os.makedirs(self.vis_dir, exist_ok=True)
|
||||
# Reasoning content output directory (Markdown files)
|
||||
self.reasoning_dir = os.path.abspath(os.path.join(self.base_dir, '..', 'generated_reasoning_content'))
|
||||
os.makedirs(self.reasoning_dir, exist_ok=True)
|
||||
# 控制是否允许模型返回含 <think> 的原文(不强制JSON),以便提取推理链
|
||||
self.enable_reasoning_capture = os.getenv("ENABLE_REASONING_CAPTURE", "true").lower() in ("1", "true", "yes")
|
||||
# 终端预览的最大行数
|
||||
try:
|
||||
self.reasoning_preview_lines = int(os.getenv("REASONING_PREVIEW_LINES", "20"))
|
||||
except Exception:
|
||||
self.reasoning_preview_lines = 20
|
||||
# 加载提示词:复杂模式复用现有 system_prompt.txt;简单模式与分类器独立提示词
|
||||
self.complex_prompt = self._load_prompt("system_prompt.txt")
|
||||
self.simple_prompt = self._load_prompt("simple_mode_prompt.txt")
|
||||
@@ -661,16 +671,46 @@ class PyTreeGenerator:
|
||||
# 简单/复杂分流到不同模型与提示词
|
||||
client = self.simple_llm_client if mode == "simple" else self.complex_llm_client
|
||||
model_name = self.simple_model if mode == "simple" else self.complex_model
|
||||
response = client.chat.completions.create(
|
||||
model=model_name,
|
||||
messages=[
|
||||
# 根据是否捕获推理链来决定是否强制JSON响应
|
||||
response_kwargs = {
|
||||
"model": model_name,
|
||||
"messages": [
|
||||
{"role": "system", "content": use_prompt},
|
||||
{"role": "user", "content": final_user_prompt}
|
||||
],
|
||||
temperature=0.1 if mode == "complex" else 0.0,
|
||||
response_format={"type": "json_object"}
|
||||
)
|
||||
pytree_str = response.choices[0].message.content
|
||||
"temperature": 0.1 if mode == "complex" else 0.0,
|
||||
}
|
||||
if not self.enable_reasoning_capture:
|
||||
response_kwargs["response_format"] = {"type": "json_object"}
|
||||
response = client.chat.completions.create(**response_kwargs)
|
||||
# 兼容可能存在的 reasoning_content 字段
|
||||
try:
|
||||
msg = response.choices[0].message
|
||||
msg_content = getattr(msg, "content", None)
|
||||
msg_reasoning = getattr(msg, "reasoning_content", None)
|
||||
except Exception:
|
||||
msg = response.choices[0]["message"] if isinstance(response.choices[0], dict) else None
|
||||
msg_content = (msg or {}).get("content") if isinstance(msg, dict) else None
|
||||
msg_reasoning = (msg or {}).get("reasoning_content") if isinstance(msg, dict) else None
|
||||
|
||||
combined_text = ""
|
||||
if isinstance(msg_reasoning, str) and msg_reasoning.strip():
|
||||
# 将 reasoning_content 包装为 <think>,便于统一解析
|
||||
combined_text += f"<think>\n{msg_reasoning}\n</think>\n"
|
||||
if isinstance(msg_content, str) and msg_content.strip():
|
||||
combined_text += msg_content
|
||||
pytree_str = combined_text if combined_text else (msg_content or "")
|
||||
|
||||
# 提取 <think> 推理链内容(若存在)
|
||||
reasoning_text = None
|
||||
try:
|
||||
think_match = re.search(r"<think>([\s\S]*?)</think>", pytree_str)
|
||||
if think_match:
|
||||
reasoning_text = think_match.group(1).strip()
|
||||
# 去除推理文本后再尝试解析JSON
|
||||
pytree_str = re.sub(r"<think>[\s\S]*?</think>", "", pytree_str).strip()
|
||||
except Exception:
|
||||
reasoning_text = None
|
||||
# 单独捕获JSON解析错误并打印原始响应
|
||||
try:
|
||||
pytree_dict = json.loads(pytree_str)
|
||||
@@ -702,6 +742,25 @@ class PyTreeGenerator:
|
||||
pytree_dict['visualization_url'] = f"/static/{vis_filename}"
|
||||
except Exception as e:
|
||||
logging.warning(f"简单模式可视化失败: {e}")
|
||||
|
||||
# 保存推理链(若有)
|
||||
try:
|
||||
if reasoning_text:
|
||||
reasoning_path = os.path.join(self.reasoning_dir, f"{plan_id}.md")
|
||||
with open(reasoning_path, 'w', encoding='utf-8') as rf:
|
||||
rf.write(reasoning_text)
|
||||
logging.info(f"📝 推理链已保存: {reasoning_path}")
|
||||
# 终端预览(最多N行)
|
||||
try:
|
||||
lines = reasoning_text.splitlines()
|
||||
preview = "\n".join(lines[: self.reasoning_preview_lines])
|
||||
logging.info("🧠 推理链预览(前%d行):\n%s", self.reasoning_preview_lines, preview)
|
||||
except Exception:
|
||||
pass
|
||||
else:
|
||||
logging.info("未在模型输出中发现 <think> 推理链片段。若需捕获,请设置 ENABLE_REASONING_CAPTURE=true 以放宽JSON强制格式。")
|
||||
except Exception as e:
|
||||
logging.warning(f"保存推理链Markdown失败: {e}")
|
||||
return pytree_dict
|
||||
|
||||
# 复杂模式回退:若模型误返回简单结构,则自动包装为含安全监控的行为树
|
||||
@@ -754,6 +813,25 @@ class PyTreeGenerator:
|
||||
vis_path = os.path.join(self.vis_dir, vis_filename)
|
||||
_visualize_pytree(pytree_dict['root'], os.path.splitext(vis_path)[0])
|
||||
pytree_dict['visualization_url'] = f"/static/{vis_filename}"
|
||||
|
||||
# 保存推理链(若有)
|
||||
try:
|
||||
if reasoning_text:
|
||||
reasoning_path = os.path.join(self.reasoning_dir, f"{plan_id}.md")
|
||||
with open(reasoning_path, 'w', encoding='utf-8') as rf:
|
||||
rf.write(reasoning_text)
|
||||
logging.info(f"📝 推理链已保存: {reasoning_path}")
|
||||
# 终端预览(最多N行)
|
||||
try:
|
||||
lines = reasoning_text.splitlines()
|
||||
preview = "\n".join(lines[: self.reasoning_preview_lines])
|
||||
logging.info("🧠 推理链预览(前%d行):\n%s", self.reasoning_preview_lines, preview)
|
||||
except Exception:
|
||||
pass
|
||||
else:
|
||||
logging.info("未在模型输出中发现 <think> 推理链片段。若需捕获,请设置 ENABLE_REASONING_CAPTURE=true 以放宽JSON强制格式。")
|
||||
except Exception as e:
|
||||
logging.warning(f"保存推理链Markdown失败: {e}")
|
||||
return pytree_dict
|
||||
else:
|
||||
# 打印未通过验证的Pytree以便排查
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -12,7 +12,7 @@ BASE_URL = "http://127.0.0.1:8000"
|
||||
ENDPOINT = "/generate_plan"
|
||||
|
||||
# The user prompt we will send for the test
|
||||
TEST_PROMPT = "飞到学生宿舍"
|
||||
TEST_PROMPT = "已知目标检测红色气球危险性高于蓝色气球高于绿色气球,飞往搜索区搜索并锁定危险性最高的气球,对其跟踪30秒后进行打击操作"
|
||||
|
||||
def test_generate_plan():
|
||||
"""
|
||||
|
||||
174
tools/test_llama_server.py
Normal file
174
tools/test_llama_server.py
Normal file
@@ -0,0 +1,174 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
import os
|
||||
import sys
|
||||
import json
|
||||
import argparse
|
||||
from typing import Any, Dict
|
||||
|
||||
import requests
|
||||
|
||||
|
||||
def build_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="调用本地 llama-server (OpenAI兼容) 进行推理,支持自定义系统/用户提示词"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--base-url",
|
||||
default=os.getenv("SIMPLE_BASE_URL", "http://127.0.0.1:8081/v1"),
|
||||
help="llama-server 的基础URL(默认: http://127.0.0.1:8081/v1,或环境变量 SIMPLE_BASE_URL)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--model",
|
||||
default=os.getenv("SIMPLE_MODEL", "local-model"),
|
||||
help="模型名称(默认: local-model,或环境变量 SIMPLE_MODEL)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--system",
|
||||
default="You are a helpful assistant.",
|
||||
help="系统提示词(system role)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--system-file",
|
||||
default=None,
|
||||
help="系统提示词文件路径(txt);若提供,则覆盖 --system 的字符串",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--user",
|
||||
default=None,
|
||||
help="用户提示词(user role);若不传则从交互式输入读取",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--temperature",
|
||||
type=float,
|
||||
default=0.2,
|
||||
help="采样温度(默认: 0.2)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-tokens",
|
||||
type=int,
|
||||
default=4096,
|
||||
help="最大生成Token数(默认: 4096)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--timeout",
|
||||
type=float,
|
||||
default=120.0,
|
||||
help="HTTP超时时间秒(默认: 120)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose",
|
||||
action="store_true",
|
||||
help="打印完整返回JSON",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def call_llama_server(
|
||||
base_url: str,
|
||||
model: str,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
temperature: float,
|
||||
max_tokens: int,
|
||||
timeout: float,
|
||||
) -> Dict[str, Any]:
|
||||
endpoint = base_url.rstrip("/") + "/chat/completions"
|
||||
headers: Dict[str, str] = {"Content-Type": "application/json"}
|
||||
|
||||
# 兼容需要API Key的代理/服务(llama-server通常不强制)
|
||||
api_key = os.getenv("OPENAI_API_KEY")
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
|
||||
payload: Dict[str, Any] = {
|
||||
"model": model,
|
||||
"messages": [
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": user_prompt},
|
||||
],
|
||||
"temperature": temperature,
|
||||
"max_tokens": max_tokens,
|
||||
}
|
||||
|
||||
resp = requests.post(endpoint, headers=headers, data=json.dumps(payload), timeout=timeout)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = build_args()
|
||||
|
||||
user_prompt = args.user
|
||||
if not user_prompt:
|
||||
try:
|
||||
user_prompt = input("请输入用户提示词: ")
|
||||
except KeyboardInterrupt:
|
||||
print("\n已取消。")
|
||||
sys.exit(1)
|
||||
|
||||
# 解析系统提示词:优先使用 --system-file
|
||||
system_prompt = args.system
|
||||
if args.system_file:
|
||||
try:
|
||||
with open(args.system_file, "r", encoding="utf-8") as f:
|
||||
system_prompt = f.read()
|
||||
except Exception as e:
|
||||
print("\n❌ 读取系统提示词文件失败:")
|
||||
print(str(e))
|
||||
sys.exit(1)
|
||||
|
||||
try:
|
||||
print("--- llama-server 推理 ---")
|
||||
print(f"Base URL: {args.base_url}")
|
||||
print(f"Model: {args.model}")
|
||||
if args.system_file:
|
||||
print(f"System(from file): {args.system_file}")
|
||||
else:
|
||||
print(f"System: {system_prompt}")
|
||||
print(f"User: {user_prompt}")
|
||||
|
||||
data = call_llama_server(
|
||||
base_url=args.base_url,
|
||||
model=args.model,
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=user_prompt,
|
||||
temperature=args.temperature,
|
||||
max_tokens=args.max_tokens,
|
||||
timeout=args.timeout,
|
||||
)
|
||||
|
||||
if args.verbose:
|
||||
print("\n完整返回JSON:")
|
||||
print(json.dumps(data, ensure_ascii=False, indent=2))
|
||||
|
||||
# 尝试按OpenAI兼容格式提取assistant内容
|
||||
content = None
|
||||
try:
|
||||
content = data["choices"][0]["message"]["content"]
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if content is not None:
|
||||
print("\n模型输出:")
|
||||
print(content)
|
||||
else:
|
||||
# 兜底打印
|
||||
print("\n无法按OpenAI兼容字段解析内容,原始返回如下:")
|
||||
print(json.dumps(data, ensure_ascii=False))
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
print("\n❌ 请求失败:请确认 llama-server 已在 8081 端口启动并可访问。")
|
||||
print(f"详情: {e}")
|
||||
sys.exit(2)
|
||||
except Exception as e:
|
||||
print("\n❌ 发生未预期的错误:")
|
||||
print(str(e))
|
||||
sys.exit(3)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
Binary file not shown.
Reference in New Issue
Block a user