huangfu/DronePlanning
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

AI重构

Browse Source
This commit is contained in:
huangfu
2026-02-20 23:04:09 +08:00
parent 43d69d5a99
commit 5d8412bcb6
244 changed files with 9304 additions and 1463 deletions
Download Patch File Download Diff File Expand all files Collapse all files

231
logs/embedding_model.log
View File

@@ -11,7 +11,7 @@ main: binding port with default address family
main: HTTP server is listening, hostname: 0.0.0.0, port: 8090, http threads: 15
main: loading model
srv load_model: loading model '/home/huangfukk/models/gguf/Qwen3/Qwen3-Embedding-4B/Qwen3-Embedding-4B-Q5_K_M.gguf'
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 4060 Ti) - 2767 MiB free
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 4060 Ti) - 15225 MiB free
llama_model_loader: loaded meta data with 36 key-value pairs and 398 tensors from /home/huangfukk/models/gguf/Qwen3/Qwen3-Embedding-4B/Qwen3-Embedding-4B-Q5_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen3
@@ -120,6 +120,7 @@ print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
srv log_server_r: request: GET /health 127.0.0.1 503
load_tensors: offloading 36 repeating layers to GPU
load_tensors: offloaded 36/37 layers to GPU
load_tensors: CUDA0 model buffer size = 2445.68 MiB
@@ -220,7 +221,227 @@ How are you?<|im_end|>
'
main: server is listening on http://0.0.0.0:8090 - starting the main loop
srv update_slots: all slots are idle
srv operator(): operator(): cleaning up before exit...
Received second interrupt, terminating immediately.
srv log_server_r: request: GET /health 127.0.0.1 200
srv operator(): operator(): cleaning up before exit...
slot launch_slot_: id 0 | task 0 | processing task
slot update_slots: id 0 | task 0 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 38
slot update_slots: id 0 | task 0 | kv cache rm [0, end)
slot update_slots: id 0 | task 0 | prompt processing progress, n_past = 38, n_tokens = 38, progress = 1.000000
slot update_slots: id 0 | task 0 | prompt done, n_past = 38, n_tokens = 38
slot release: id 0 | task 0 | stop processing: n_past = 38, truncated = 0
slot launch_slot_: id 0 | task 1 | processing task
slot update_slots: id 0 | task 1 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 50
slot update_slots: id 0 | task 1 | kv cache rm [4, end)
slot update_slots: id 0 | task 1 | prompt processing progress, n_past = 50, n_tokens = 46, progress = 0.920000
slot update_slots: id 0 | task 1 | prompt done, n_past = 50, n_tokens = 46
slot release: id 0 | task 1 | stop processing: n_past = 50, truncated = 0
slot launch_slot_: id 0 | task 2 | processing task
slot update_slots: id 0 | task 2 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 142
slot update_slots: id 0 | task 2 | kv cache rm [4, end)
slot update_slots: id 0 | task 2 | prompt processing progress, n_past = 142, n_tokens = 138, progress = 0.971831
slot update_slots: id 0 | task 2 | prompt done, n_past = 142, n_tokens = 138
slot release: id 0 | task 2 | stop processing: n_past = 142, truncated = 0
slot launch_slot_: id 0 | task 3 | processing task
slot update_slots: id 0 | task 3 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 27
slot update_slots: id 0 | task 3 | kv cache rm [0, end)
slot update_slots: id 0 | task 3 | prompt processing progress, n_past = 27, n_tokens = 27, progress = 1.000000
slot update_slots: id 0 | task 3 | prompt done, n_past = 27, n_tokens = 27
slot release: id 0 | task 3 | stop processing: n_past = 27, truncated = 0
slot launch_slot_: id 0 | task 4 | processing task
slot update_slots: id 0 | task 4 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 20
slot update_slots: id 0 | task 4 | kv cache rm [0, end)
slot update_slots: id 0 | task 4 | prompt processing progress, n_past = 20, n_tokens = 20, progress = 1.000000
slot update_slots: id 0 | task 4 | prompt done, n_past = 20, n_tokens = 20
slot release: id 0 | task 4 | stop processing: n_past = 20, truncated = 0
slot launch_slot_: id 0 | task 5 | processing task
slot update_slots: id 0 | task 5 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 17
slot update_slots: id 0 | task 5 | kv cache rm [0, end)
slot update_slots: id 0 | task 5 | prompt processing progress, n_past = 17, n_tokens = 17, progress = 1.000000
slot update_slots: id 0 | task 5 | prompt done, n_past = 17, n_tokens = 17
slot release: id 0 | task 5 | stop processing: n_past = 17, truncated = 0
slot launch_slot_: id 0 | task 6 | processing task
slot update_slots: id 0 | task 6 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 10
slot update_slots: id 0 | task 6 | kv cache rm [0, end)
slot update_slots: id 0 | task 6 | prompt processing progress, n_past = 10, n_tokens = 10, progress = 1.000000
slot update_slots: id 0 | task 6 | prompt done, n_past = 10, n_tokens = 10
slot release: id 0 | task 6 | stop processing: n_past = 10, truncated = 0
slot launch_slot_: id 0 | task 7 | processing task
slot update_slots: id 0 | task 7 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 17
slot update_slots: id 0 | task 7 | kv cache rm [0, end)
slot update_slots: id 0 | task 7 | prompt processing progress, n_past = 17, n_tokens = 17, progress = 1.000000
slot update_slots: id 0 | task 7 | prompt done, n_past = 17, n_tokens = 17
slot release: id 0 | task 7 | stop processing: n_past = 17, truncated = 0
slot launch_slot_: id 0 | task 8 | processing task
slot update_slots: id 0 | task 8 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 18
slot update_slots: id 0 | task 8 | kv cache rm [0, end)
slot update_slots: id 0 | task 8 | prompt processing progress, n_past = 18, n_tokens = 18, progress = 1.000000
slot update_slots: id 0 | task 8 | prompt done, n_past = 18, n_tokens = 18
slot release: id 0 | task 8 | stop processing: n_past = 18, truncated = 0
slot launch_slot_: id 0 | task 9 | processing task
slot update_slots: id 0 | task 9 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 23
slot update_slots: id 0 | task 9 | kv cache rm [0, end)
slot update_slots: id 0 | task 9 | prompt processing progress, n_past = 23, n_tokens = 23, progress = 1.000000
slot update_slots: id 0 | task 9 | prompt done, n_past = 23, n_tokens = 23
slot release: id 0 | task 9 | stop processing: n_past = 23, truncated = 0
slot launch_slot_: id 0 | task 10 | processing task
slot update_slots: id 0 | task 10 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 25
slot update_slots: id 0 | task 10 | kv cache rm [0, end)
slot update_slots: id 0 | task 10 | prompt processing progress, n_past = 25, n_tokens = 25, progress = 1.000000
slot update_slots: id 0 | task 10 | prompt done, n_past = 25, n_tokens = 25
slot release: id 0 | task 10 | stop processing: n_past = 25, truncated = 0
slot launch_slot_: id 0 | task 11 | processing task
slot update_slots: id 0 | task 11 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 24
slot update_slots: id 0 | task 11 | kv cache rm [0, end)
slot update_slots: id 0 | task 11 | prompt processing progress, n_past = 24, n_tokens = 24, progress = 1.000000
slot update_slots: id 0 | task 11 | prompt done, n_past = 24, n_tokens = 24
slot release: id 0 | task 11 | stop processing: n_past = 24, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot launch_slot_: id 0 | task 24 | processing task
slot update_slots: id 0 | task 24 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 38
slot update_slots: id 0 | task 24 | kv cache rm [0, end)
slot update_slots: id 0 | task 24 | prompt processing progress, n_past = 38, n_tokens = 38, progress = 1.000000
slot update_slots: id 0 | task 24 | prompt done, n_past = 38, n_tokens = 38
slot release: id 0 | task 24 | stop processing: n_past = 38, truncated = 0
slot launch_slot_: id 0 | task 25 | processing task
slot update_slots: id 0 | task 25 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 50
slot update_slots: id 0 | task 25 | kv cache rm [4, end)
slot update_slots: id 0 | task 25 | prompt processing progress, n_past = 50, n_tokens = 46, progress = 0.920000
slot update_slots: id 0 | task 25 | prompt done, n_past = 50, n_tokens = 46
slot release: id 0 | task 25 | stop processing: n_past = 50, truncated = 0
slot launch_slot_: id 0 | task 26 | processing task
slot update_slots: id 0 | task 26 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 142
slot update_slots: id 0 | task 26 | kv cache rm [4, end)
slot update_slots: id 0 | task 26 | prompt processing progress, n_past = 142, n_tokens = 138, progress = 0.971831
slot update_slots: id 0 | task 26 | prompt done, n_past = 142, n_tokens = 138
slot release: id 0 | task 26 | stop processing: n_past = 142, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot launch_slot_: id 0 | task 30 | processing task
slot update_slots: id 0 | task 30 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 27
slot update_slots: id 0 | task 30 | kv cache rm [0, end)
slot update_slots: id 0 | task 30 | prompt processing progress, n_past = 27, n_tokens = 27, progress = 1.000000
slot update_slots: id 0 | task 30 | prompt done, n_past = 27, n_tokens = 27
slot release: id 0 | task 30 | stop processing: n_past = 27, truncated = 0
slot launch_slot_: id 0 | task 31 | processing task
slot update_slots: id 0 | task 31 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 20
slot update_slots: id 0 | task 31 | kv cache rm [0, end)
slot update_slots: id 0 | task 31 | prompt processing progress, n_past = 20, n_tokens = 20, progress = 1.000000
slot update_slots: id 0 | task 31 | prompt done, n_past = 20, n_tokens = 20
slot release: id 0 | task 31 | stop processing: n_past = 20, truncated = 0
slot launch_slot_: id 0 | task 32 | processing task
slot update_slots: id 0 | task 32 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 17
slot update_slots: id 0 | task 32 | kv cache rm [0, end)
slot update_slots: id 0 | task 32 | prompt processing progress, n_past = 17, n_tokens = 17, progress = 1.000000
slot update_slots: id 0 | task 32 | prompt done, n_past = 17, n_tokens = 17
slot release: id 0 | task 32 | stop processing: n_past = 17, truncated = 0
slot launch_slot_: id 0 | task 33 | processing task
slot update_slots: id 0 | task 33 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 10
slot update_slots: id 0 | task 33 | kv cache rm [0, end)
slot update_slots: id 0 | task 33 | prompt processing progress, n_past = 10, n_tokens = 10, progress = 1.000000
slot update_slots: id 0 | task 33 | prompt done, n_past = 10, n_tokens = 10
slot release: id 0 | task 33 | stop processing: n_past = 10, truncated = 0
slot launch_slot_: id 0 | task 34 | processing task
slot update_slots: id 0 | task 34 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 17
slot update_slots: id 0 | task 34 | kv cache rm [0, end)
slot update_slots: id 0 | task 34 | prompt processing progress, n_past = 17, n_tokens = 17, progress = 1.000000
slot update_slots: id 0 | task 34 | prompt done, n_past = 17, n_tokens = 17
slot release: id 0 | task 34 | stop processing: n_past = 17, truncated = 0
slot launch_slot_: id 0 | task 35 | processing task
slot update_slots: id 0 | task 35 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 18
slot update_slots: id 0 | task 35 | kv cache rm [0, end)
slot update_slots: id 0 | task 35 | prompt processing progress, n_past = 18, n_tokens = 18, progress = 1.000000
slot update_slots: id 0 | task 35 | prompt done, n_past = 18, n_tokens = 18
slot release: id 0 | task 35 | stop processing: n_past = 18, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot launch_slot_: id 0 | task 42 | processing task
slot update_slots: id 0 | task 42 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 23
slot update_slots: id 0 | task 42 | kv cache rm [0, end)
slot update_slots: id 0 | task 42 | prompt processing progress, n_past = 23, n_tokens = 23, progress = 1.000000
slot update_slots: id 0 | task 42 | prompt done, n_past = 23, n_tokens = 23
slot release: id 0 | task 42 | stop processing: n_past = 23, truncated = 0
slot launch_slot_: id 0 | task 43 | processing task
slot update_slots: id 0 | task 43 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 25
slot update_slots: id 0 | task 43 | kv cache rm [0, end)
slot update_slots: id 0 | task 43 | prompt processing progress, n_past = 25, n_tokens = 25, progress = 1.000000
slot update_slots: id 0 | task 43 | prompt done, n_past = 25, n_tokens = 25
slot release: id 0 | task 43 | stop processing: n_past = 25, truncated = 0
slot launch_slot_: id 0 | task 44 | processing task
slot update_slots: id 0 | task 44 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 24
slot update_slots: id 0 | task 44 | kv cache rm [0, end)
slot update_slots: id 0 | task 44 | prompt processing progress, n_past = 24, n_tokens = 24, progress = 1.000000
slot update_slots: id 0 | task 44 | prompt done, n_past = 24, n_tokens = 24
slot release: id 0 | task 44 | stop processing: n_past = 24, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot launch_slot_: id 0 | task 48 | processing task
slot update_slots: id 0 | task 48 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 2
slot update_slots: id 0 | task 48 | kv cache rm [0, end)
slot update_slots: id 0 | task 48 | prompt processing progress, n_past = 2, n_tokens = 2, progress = 1.000000
slot update_slots: id 0 | task 48 | prompt done, n_past = 2, n_tokens = 2
slot release: id 0 | task 48 | stop processing: n_past = 2, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot launch_slot_: id 0 | task 50 | processing task
slot update_slots: id 0 | task 50 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 12
slot update_slots: id 0 | task 50 | kv cache rm [0, end)
slot update_slots: id 0 | task 50 | prompt processing progress, n_past = 12, n_tokens = 12, progress = 1.000000
slot update_slots: id 0 | task 50 | prompt done, n_past = 12, n_tokens = 12
slot release: id 0 | task 50 | stop processing: n_past = 12, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot launch_slot_: id 0 | task 52 | processing task
slot update_slots: id 0 | task 52 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 17
slot update_slots: id 0 | task 52 | kv cache rm [3, end)
slot update_slots: id 0 | task 52 | prompt processing progress, n_past = 17, n_tokens = 14, progress = 0.823529
slot update_slots: id 0 | task 52 | prompt done, n_past = 17, n_tokens = 14
slot release: id 0 | task 52 | stop processing: n_past = 17, truncated = 0
slot launch_slot_: id 0 | task 53 | processing task
slot update_slots: id 0 | task 53 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 17
slot update_slots: id 0 | task 53 | need to evaluate at least 1 token for each active slot, n_past = 17, n_prompt_tokens = 17
slot update_slots: id 0 | task 53 | kv cache rm [16, end)
slot update_slots: id 0 | task 53 | prompt processing progress, n_past = 17, n_tokens = 1, progress = 0.058824
slot update_slots: id 0 | task 53 | prompt done, n_past = 17, n_tokens = 1
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot release: id 0 | task 53 | stop processing: n_past = 17, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot launch_slot_: id 0 | task 56 | processing task
slot update_slots: id 0 | task 56 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 16
slot update_slots: id 0 | task 56 | kv cache rm [5, end)
slot update_slots: id 0 | task 56 | prompt processing progress, n_past = 16, n_tokens = 11, progress = 0.687500
slot update_slots: id 0 | task 56 | prompt done, n_past = 16, n_tokens = 11
slot release: id 0 | task 56 | stop processing: n_past = 16, truncated = 0
slot launch_slot_: id 0 | task 59 | processing task
slot update_slots: id 0 | task 59 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 16
slot update_slots: id 0 | task 59 | need to evaluate at least 1 token for each active slot, n_past = 16, n_prompt_tokens = 16
slot update_slots: id 0 | task 59 | kv cache rm [15, end)
slot update_slots: id 0 | task 59 | prompt processing progress, n_past = 16, n_tokens = 1, progress = 0.062500
slot update_slots: id 0 | task 59 | prompt done, n_past = 16, n_tokens = 1
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot release: id 0 | task 59 | stop processing: n_past = 16, truncated = 0
slot launch_slot_: id 0 | task 57 | processing task
slot update_slots: id 0 | task 57 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 16
slot update_slots: id 0 | task 57 | need to evaluate at least 1 token for each active slot, n_past = 16, n_prompt_tokens = 16
slot update_slots: id 0 | task 57 | kv cache rm [15, end)
slot update_slots: id 0 | task 57 | prompt processing progress, n_past = 16, n_tokens = 1, progress = 0.062500
slot update_slots: id 0 | task 57 | prompt done, n_past = 16, n_tokens = 1
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot release: id 0 | task 57 | stop processing: n_past = 16, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot launch_slot_: id 0 | task 62 | processing task
slot update_slots: id 0 | task 62 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 15
slot update_slots: id 0 | task 62 | kv cache rm [5, end)
slot update_slots: id 0 | task 62 | prompt processing progress, n_past = 15, n_tokens = 10, progress = 0.666667
slot update_slots: id 0 | task 62 | prompt done, n_past = 15, n_tokens = 10
slot release: id 0 | task 62 | stop processing: n_past = 15, truncated = 0
slot launch_slot_: id 0 | task 64 | processing task
slot update_slots: id 0 | task 64 | new prompt, n_ctx_slot = 4096, n_keep = 0, n_prompt_tokens = 15
slot update_slots: id 0 | task 64 | need to evaluate at least 1 token for each active slot, n_past = 15, n_prompt_tokens = 15
slot update_slots: id 0 | task 64 | kv cache rm [14, end)
slot update_slots: id 0 | task 64 | prompt processing progress, n_past = 15, n_tokens = 1, progress = 0.066667
slot update_slots: id 0 | task 64 | prompt done, n_past = 15, n_tokens = 1
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200
slot release: id 0 | task 64 | stop processing: n_past = 15, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/embeddings 127.0.0.1 200

56
logs/fastapi.log
View File

@@ -1,15 +1,47 @@
2026-01-20 09:41:43,111 - INFO - Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2026-01-20 09:41:43,260 - INFO - 成功找到节点定义JSON代码块
2026-01-20 09:41:43,260 - INFO - 成功解析出动作节点: ['approach_target', 'deliver_payload', 'fly_sequence', 'fly_to_waypoint', 'land', 'loiter', 'manual_confirmation', 'move_direction', 'object_detect', 'return_emergency', 'rotate', 'rotate_search', 'search_pattern', 'system_checks', 'take_photos', 'takeoff', 'track_object']
2026-01-20 09:41:43,260 - INFO - 成功解析出条件节点: ['at_waypoint', 'object_detected']
INFO: Started server process [34239]
2026-02-20 22:25:24,889 - ERROR - 提示词文件未找到 -> classifier_prompt.txt
2026-02-20 22:25:24,979 - INFO - Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2026-02-20 22:25:25,151 - INFO - 成功找到节点定义JSON代码块
2026-02-20 22:25:25,152 - INFO - 成功解析出动作节点: ['approach_target', 'deliver_payload', 'fly_sequence', 'fly_to_waypoint', 'land', 'loiter', 'manual_confirmation', 'move_direction', 'object_detect', 'return_emergency', 'rotate', 'rotate_search', 'search_pattern', 'system_checks', 'take_photos', 'takeoff', 'track_object']
2026-02-20 22:25:25,152 - INFO - 成功解析出条件节点: ['at_waypoint', 'object_detected']
2026-02-20 22:25:25,152 - WARNING - 集合 location_kb 不可用: Collection [location_kb] does not exist
2026-02-20 22:25:25,152 - WARNING - 集合 pattern_kb 不可用: Collection [pattern_kb] does not exist
2026-02-20 22:25:25,153 - WARNING - 集合 rules_kb 不可用: Collection [rules_kb] does not exist
INFO: Started server process [10575]
INFO: Waiting for application startup.
2026-01-20 09:41:43,263 - INFO - WebSocket event loop configured.
2026-02-20 22:25:25,155 - INFO - WebSocket event loop configured.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO: 127.0.0.1:44013 - "GET /docs HTTP/1.1" 200 OK
INFO: Shutting down
INFO: Waiting for application shutdown.
2026-01-20 09:42:11,885 - INFO - Backend service shutting down.
INFO: Application shutdown complete.
INFO: Finished server process [34239]
INFO: 127.0.0.1:45093 - "GET /docs HTTP/1.1" 200 OK
INFO: 127.0.0.1:46985 - "GET /docs HTTP/1.1" 200 OK
2026-02-20 22:29:37,482 - INFO - 接收到用户请求: 无人机当前在空中，往北飞50米
2026-02-20 22:29:38,939 - INFO - HTTP Request: POST http://localhost:8081/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-20 22:29:40,302 - INFO - HTTP Request: POST http://localhost:8081/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-20 22:29:42,389 - INFO - ✅ 任务树可视化成功
2026-02-20 22:29:42,389 - INFO - 图形已保存到: /home/huangfukk/DronePlanning/backend_service/generated_visualizations/py_tree.png
2026-02-20 22:29:42,389 - INFO - 💾 历史记录已保存: /home/huangfukk/DronePlanning/backend_service/src/../history/20260220_222942_plan.json
2026-02-20 22:29:42,389 - INFO - ✅ 成功生成并验证了Pytree（Pipeline）
INFO: 127.0.0.1:44861 - "POST /generate_plan HTTP/1.1" 200 OK
2026-02-20 22:30:15,774 - INFO - 接收到用户请求: 无人机当前在地面，到广场查找穿红色衣服的人，找到后拍照
2026-02-20 22:30:16,805 - INFO - HTTP Request: POST http://localhost:8081/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-20 22:30:29,109 - INFO - HTTP Request: POST http://localhost:8081/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-20 22:30:29,161 - INFO - ✅ 任务树可视化成功
2026-02-20 22:30:29,161 - INFO - 图形已保存到: /home/huangfukk/DronePlanning/backend_service/generated_visualizations/py_tree.png
2026-02-20 22:30:29,161 - INFO - 💾 历史记录已保存: /home/huangfukk/DronePlanning/backend_service/src/../history/20260220_223029_plan.json
2026-02-20 22:30:29,161 - INFO - ✅ 成功生成并验证了Pytree（Pipeline）
INFO: 127.0.0.1:43927 - "POST /generate_plan HTTP/1.1" 200 OK
2026-02-20 22:32:13,176 - INFO - 接收到用户请求: 无人机当前在地面，去面前大楼左边20米巡查并拍照
2026-02-20 22:32:14,331 - INFO - HTTP Request: POST http://localhost:8081/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-20 22:32:30,714 - INFO - HTTP Request: POST http://localhost:8081/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-20 22:32:30,764 - INFO - ✅ 任务树可视化成功
2026-02-20 22:32:30,764 - INFO - 图形已保存到: /home/huangfukk/DronePlanning/backend_service/generated_visualizations/py_tree.png
2026-02-20 22:32:30,765 - INFO - 💾 历史记录已保存: /home/huangfukk/DronePlanning/backend_service/src/../history/20260220_223230_plan.json
2026-02-20 22:32:30,765 - INFO - ✅ 成功生成并验证了Pytree（Pipeline）
INFO: 127.0.0.1:45399 - "POST /generate_plan HTTP/1.1" 200 OK
2026-02-20 22:36:36,485 - INFO - 接收到用户请求: 无人机当前在地面，到广场查找绿色公交车，找到就拍照
2026-02-20 22:36:37,548 - INFO - HTTP Request: POST http://localhost:8081/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-20 22:36:48,200 - INFO - HTTP Request: POST http://localhost:8081/v1/chat/completions "HTTP/1.1 200 OK"
2026-02-20 22:36:48,248 - INFO - ✅ 任务树可视化成功
2026-02-20 22:36:48,248 - INFO - 图形已保存到: /home/huangfukk/DronePlanning/backend_service/generated_visualizations/py_tree.png
2026-02-20 22:36:48,248 - INFO - 💾 历史记录已保存: /home/huangfukk/DronePlanning/backend_service/src/../history/20260220_223648_plan.json
2026-02-20 22:36:48,248 - INFO - ✅ 成功生成并验证了Pytree（Pipeline）
INFO: 127.0.0.1:47209 - "POST /generate_plan HTTP/1.1" 200 OK

145
logs/inference_model.log
View File

@@ -11,7 +11,7 @@ main: binding port with default address family
main: HTTP server is listening, hostname: 0.0.0.0, port: 8081, http threads: 15
main: loading model
srv load_model: loading model '/home/huangfukk/models/gguf/Qwen3/Qwen3-4B/Qwen3-4B-Q5_K_M.gguf'
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 4060 Ti) - 2789 MiB free
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 4060 Ti) - 15225 MiB free
llama_model_loader: loaded meta data with 28 key-value pairs and 398 tensors from /home/huangfukk/models/gguf/Qwen3/Qwen3-4B/Qwen3-4B-Q5_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen3
@@ -112,6 +112,7 @@ print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
srv log_server_r: request: GET /health 127.0.0.1 503
load_tensors: offloading 36 repeating layers to GPU
load_tensors: offloaded 36/37 layers to GPU
load_tensors: CUDA0 model buffer size = 2445.68 MiB
@@ -119,8 +120,8 @@ load_tensors: CPU_Mapped model buffer size = 304.29 MiB
..........................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 8192
llama_context: n_ctx_per_seq = 8192
llama_context: n_ctx = 16384
llama_context: n_ctx_per_seq = 16384
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
@@ -128,12 +129,12 @@ llama_context: flash_attn = 0
llama_context: kv_unified = false
llama_context: freq_base = 1000000.0
llama_context: freq_scale = 1
llama_context: n_ctx_per_seq (8192) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
llama_context: n_ctx_per_seq (16384) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
llama_context: CPU output buffer size = 0.58 MiB
llama_kv_cache_unified: CUDA0 KV buffer size = 1152.00 MiB
llama_kv_cache_unified: size = 1152.00 MiB ( 8192 cells, 36 layers, 1/1 seqs), K (f16): 576.00 MiB, V (f16): 576.00 MiB
llama_context: CUDA0 compute buffer size = 606.03 MiB
llama_context: CUDA_Host compute buffer size = 25.01 MiB
llama_kv_cache_unified: CUDA0 KV buffer size = 2304.00 MiB
llama_kv_cache_unified: size = 2304.00 MiB ( 16384 cells, 36 layers, 1/1 seqs), K (f16): 1152.00 MiB, V (f16): 1152.00 MiB
llama_context: CUDA0 compute buffer size = 1086.01 MiB
llama_context: CUDA_Host compute buffer size = 41.01 MiB
llama_context: graph nodes = 1410
llama_context: graph splits = 4 (with bs=512), 3 (with bs=1)
common_init_from_params: added <|endoftext|> logit bias = -inf
@@ -141,10 +142,10 @@ common_init_from_params: added <|im_end|> logit bias = -inf
common_init_from_params: added <|fim_pad|> logit bias = -inf
common_init_from_params: added <|repo_name|> logit bias = -inf
common_init_from_params: added <|file_sep|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 8192
common_init_from_params: setting dry_penalty_last_n to ctx_size = 16384
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
srv init: initializing slots, n_slots = 1
slot init: id 0 | task -1 | new slot n_ctx_slot = 8192
slot init: id 0 | task -1 | new slot n_ctx_slot = 16384
main: model loaded
main: chat template, chat_template: {%- if tools %}
{{- '<|im_start|>system\n' }}
@@ -242,8 +243,122 @@ How are you?<|im_end|>
'
main: server is listening on http://0.0.0.0:8081 - starting the main loop
srv update_slots: all slots are idle
srv operator(): operator(): cleaning up before exit...
ed second interrupt, terminating immediately.
srv operator(): operator(): cleaning up before exit...
srv log_server_r: request: GET /health 127.0.0.1 200
srv operator(): operator(): cleaning up before exit...
srv log_server_r: request: GET /v1/models 127.0.0.1 200
srv params_from_: Chat format: Content-only
slot launch_slot_: id 0 | task 0 | processing task
slot update_slots: id 0 | task 0 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 1569
slot update_slots: id 0 | task 0 | kv cache rm [0, end)
slot update_slots: id 0 | task 0 | prompt processing progress, n_past = 1569, n_tokens = 1569, progress = 1.000000
slot update_slots: id 0 | task 0 | prompt done, n_past = 1569, n_tokens = 1569
slot release: id 0 | task 0 | stop processing: n_past = 1578, truncated = 0
slot print_timing: id 0 | task 0 |
prompt eval time = 1012.37 ms / 1569 tokens ( 0.65 ms per token, 1549.82 tokens per second)
eval time = 258.11 ms / 10 tokens ( 25.81 ms per token, 38.74 tokens per second)
total time = 1270.48 ms / 1579 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv params_from_: Chat format: Content-only
slot launch_slot_: id 0 | task 11 | processing task
slot update_slots: id 0 | task 11 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 1611
slot update_slots: id 0 | task 11 | kv cache rm [5, end)
slot update_slots: id 0 | task 11 | prompt processing progress, n_past = 1611, n_tokens = 1606, progress = 0.996896
slot update_slots: id 0 | task 11 | prompt done, n_past = 1611, n_tokens = 1606
slot release: id 0 | task 11 | stop processing: n_past = 1636, truncated = 0
slot print_timing: id 0 | task 11 |
prompt eval time = 613.51 ms / 1606 tokens ( 0.38 ms per token, 2617.71 tokens per second)
eval time = 653.35 ms / 26 tokens ( 25.13 ms per token, 39.80 tokens per second)
total time = 1266.86 ms / 1632 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv params_from_: Chat format: Content-only
slot launch_slot_: id 0 | task 38 | processing task
slot update_slots: id 0 | task 38 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 1574
slot update_slots: id 0 | task 38 | kv cache rm [5, end)
slot update_slots: id 0 | task 38 | prompt processing progress, n_past = 1574, n_tokens = 1569, progress = 0.996823
slot update_slots: id 0 | task 38 | prompt done, n_past = 1574, n_tokens = 1569
slot release: id 0 | task 38 | stop processing: n_past = 1584, truncated = 0
slot print_timing: id 0 | task 38 |
prompt eval time = 768.98 ms / 1569 tokens ( 0.49 ms per token, 2040.36 tokens per second)
eval time = 254.75 ms / 11 tokens ( 23.16 ms per token, 43.18 tokens per second)
total time = 1023.74 ms / 1580 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv params_from_: Chat format: Content-only
slot launch_slot_: id 0 | task 50 | processing task
slot update_slots: id 0 | task 50 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 4929
slot update_slots: id 0 | task 50 | kv cache rm [3, end)
slot update_slots: id 0 | task 50 | prompt processing progress, n_past = 2051, n_tokens = 2048, progress = 0.415500
slot update_slots: id 0 | task 50 | kv cache rm [2051, end)
slot update_slots: id 0 | task 50 | prompt processing progress, n_past = 4099, n_tokens = 2048, progress = 0.831000
slot update_slots: id 0 | task 50 | kv cache rm [4099, end)
slot update_slots: id 0 | task 50 | prompt processing progress, n_past = 4929, n_tokens = 830, progress = 0.999391
slot update_slots: id 0 | task 50 | prompt done, n_past = 4929, n_tokens = 830
slot release: id 0 | task 50 | stop processing: n_past = 5254, truncated = 0
slot print_timing: id 0 | task 50 |
prompt eval time = 2682.34 ms / 4926 tokens ( 0.54 ms per token, 1836.46 tokens per second)
eval time = 9502.10 ms / 326 tokens ( 29.15 ms per token, 34.31 tokens per second)
total time = 12184.44 ms / 5252 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv params_from_: Chat format: Content-only
slot launch_slot_: id 0 | task 379 | processing task
slot update_slots: id 0 | task 379 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 1573
slot update_slots: id 0 | task 379 | kv cache rm [3, end)
slot update_slots: id 0 | task 379 | prompt processing progress, n_past = 1573, n_tokens = 1570, progress = 0.998093
slot update_slots: id 0 | task 379 | prompt done, n_past = 1573, n_tokens = 1570
slot release: id 0 | task 379 | stop processing: n_past = 1583, truncated = 0
slot print_timing: id 0 | task 379 |
prompt eval time = 862.21 ms / 1570 tokens ( 0.55 ms per token, 1820.91 tokens per second)
eval time = 284.82 ms / 11 tokens ( 25.89 ms per token, 38.62 tokens per second)
total time = 1147.03 ms / 1581 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv params_from_: Chat format: Content-only
slot launch_slot_: id 0 | task 391 | processing task
slot update_slots: id 0 | task 391 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 7208
slot update_slots: id 0 | task 391 | kv cache rm [3, end)
slot update_slots: id 0 | task 391 | prompt processing progress, n_past = 2051, n_tokens = 2048, progress = 0.284129
slot update_slots: id 0 | task 391 | kv cache rm [2051, end)
slot update_slots: id 0 | task 391 | prompt processing progress, n_past = 4099, n_tokens = 2048, progress = 0.568258
slot update_slots: id 0 | task 391 | kv cache rm [4099, end)
slot update_slots: id 0 | task 391 | prompt processing progress, n_past = 6147, n_tokens = 2048, progress = 0.852386
slot update_slots: id 0 | task 391 | kv cache rm [6147, end)
slot update_slots: id 0 | task 391 | prompt processing progress, n_past = 7208, n_tokens = 1061, progress = 0.999584
slot update_slots: id 0 | task 391 | prompt done, n_past = 7208, n_tokens = 1061
slot release: id 0 | task 391 | stop processing: n_past = 7577, truncated = 0
slot print_timing: id 0 | task 391 |
prompt eval time = 4810.49 ms / 7205 tokens ( 0.67 ms per token, 1497.77 tokens per second)
eval time = 11443.72 ms / 370 tokens ( 30.93 ms per token, 32.33 tokens per second)
total time = 16254.21 ms / 7575 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv params_from_: Chat format: Content-only
slot launch_slot_: id 0 | task 765 | processing task
slot update_slots: id 0 | task 765 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 1572
slot update_slots: id 0 | task 765 | kv cache rm [3, end)
slot update_slots: id 0 | task 765 | prompt processing progress, n_past = 1572, n_tokens = 1569, progress = 0.998092
slot update_slots: id 0 | task 765 | prompt done, n_past = 1572, n_tokens = 1569
slot release: id 0 | task 765 | stop processing: n_past = 1582, truncated = 0
slot print_timing: id 0 | task 765 |
prompt eval time = 826.51 ms / 1569 tokens ( 0.53 ms per token, 1898.34 tokens per second)
eval time = 227.62 ms / 11 tokens ( 20.69 ms per token, 48.33 tokens per second)
total time = 1054.13 ms / 1580 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv params_from_: Chat format: Content-only
slot launch_slot_: id 0 | task 777 | processing task
slot update_slots: id 0 | task 777 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 4927
slot update_slots: id 0 | task 777 | kv cache rm [3, end)
slot update_slots: id 0 | task 777 | prompt processing progress, n_past = 2051, n_tokens = 2048, progress = 0.415669
slot update_slots: id 0 | task 777 | kv cache rm [2051, end)
slot update_slots: id 0 | task 777 | prompt processing progress, n_past = 4099, n_tokens = 2048, progress = 0.831338
slot update_slots: id 0 | task 777 | kv cache rm [4099, end)
slot update_slots: id 0 | task 777 | prompt processing progress, n_past = 4927, n_tokens = 828, progress = 0.999391
slot update_slots: id 0 | task 777 | prompt done, n_past = 4927, n_tokens = 828
slot release: id 0 | task 777 | stop processing: n_past = 5246, truncated = 0
slot print_timing: id 0 | task 777 |
prompt eval time = 2382.24 ms / 4924 tokens ( 0.48 ms per token, 2066.96 tokens per second)
eval time = 8172.57 ms / 320 tokens ( 25.54 ms per token, 39.16 tokens per second)
total time = 10554.82 ms / 5244 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 127.0.0.1 200

3
logs/services.pid Normal file
View File

@@ -0,0 +1,3 @@
10395
10396
10573

2
logs/services_vllm.pid Normal file
View File

@@ -0,0 +1,2 @@
43538
43539

150
logs/vllm_inference_model.log Normal file
View File

@@ -0,0 +1,150 @@
[DEBUG] conda_env=VLLM
[DEBUG] CONDA_PREFIX=/home/huangfukk/miniconda3/envs/VLLM
[DEBUG] which python: /home/huangfukk/miniconda3/envs/VLLM/bin/python
[DEBUG] which vllm: /home/huangfukk/miniconda3/envs/VLLM/bin/vllm
0.13.0
[DEBUG] python: 3.12.12 | packaged by Anaconda, Inc. | (main, Oct 21 2025, 20:16:04) [GCC 11.2.0]
[DEBUG] torch: 2.9.0+cu128 cuda: 12.8 available: True
(APIServer pid=43538) INFO 01-20 16:25:05 [api_server.py:1351] vLLM API server version 0.13.0
(APIServer pid=43538) INFO 01-20 16:25:05 [utils.py:253] non-default args: {'model_tag': '/home/huangfukk/models/AWQ/Qwen3-4B-Instruct-2507', 'host': '0.0.0.0', 'port': 8081, 'uvicorn_log_level': 'debug', 'chat_template': '/home/huangfukk/DronePlanning/tools/vllm_templates/qwen3_xml_tool.jinja', 'chat_template_content_format': 'string', 'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_xml', 'model': '/home/huangfukk/models/AWQ/Qwen3-4B-Instruct-2507', 'trust_remote_code': True, 'max_model_len': 16384, 'served_model_name': ['local-model'], 'max_parallel_loading_workers': 4, 'gpu_memory_utilization': 0.5, 'max_num_seqs': 1}
(APIServer pid=43538) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=43538) INFO 01-20 16:25:05 [model.py:514] Resolved architecture: Qwen3ForCausalLM
(APIServer pid=43538) INFO 01-20 16:25:05 [model.py:1661] Using max model len 16384
(APIServer pid=43538) INFO 01-20 16:25:05 [awq_marlin.py:162] The model is convertible to awq_marlin during runtime. Using awq_marlin kernel.
(APIServer pid=43538) WARNING 01-20 16:25:05 [parallel.py:595] max_parallel_loading_workers is currently not supported and will be ignored.
(APIServer pid=43538) INFO 01-20 16:25:05 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=2048.
(APIServer pid=43538) WARNING 01-20 16:25:06 [interface.py:465] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:10 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='/home/huangfukk/models/AWQ/Qwen3-4B-Instruct-2507', speculative_config=None, tokenizer='/home/huangfukk/models/AWQ/Qwen3-4B-Instruct-2507', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=16384, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=awq_marlin, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=local-model, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 2, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:11 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://192.168.31.62:43504 backend=nccl
[W120 16:25:21.543804205 socket.cpp:209] [c10d] The hostname of the client socket cannot be retrieved. err=-3
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:21 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=43785) WARNING 01-20 16:25:21 [interface.py:465] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:22 [gpu_model_runner.py:3562] Starting to load model /home/huangfukk/models/AWQ/Qwen3-4B-Instruct-2507...
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:22 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=43785)
Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=43785)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:03<00:00, 3.71s/it]
(EngineCore_DP0 pid=43785)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:03<00:00, 3.71s/it]
(EngineCore_DP0 pid=43785)
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:26 [default_loader.py:308] Loading weights took 2.68 seconds
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:27 [gpu_model_runner.py:3659] Model loading took 2.5630 GiB memory and 3.642749 seconds
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:34 [backends.py:643] Using cache directory: /home/huangfukk/.cache/vllm/torch_compile_cache/d651665fcd/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:34 [backends.py:703] Dynamo bytecode transform time: 7.23 s
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:39 [backends.py:226] Directly load the compiled graph(s) for compile range (1, 2048) from the cache, took 1.264 s
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:39 [monitor.py:34] torch.compile takes 8.49 s in total
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:40 [gpu_worker.py:375] Available KV cache memory: 5.26 GiB
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:40 [kv_cache_utils.py:1291] GPU KV cache size: 38,288 tokens
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:40 [kv_cache_utils.py:1296] Maximum concurrency for 16,384 tokens per request: 2.34x
(EngineCore_DP0 pid=43785)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 2/2 [00:00<00:00, 21.01it/s]
(EngineCore_DP0 pid=43785)
Capturing CUDA graphs (decode, FULL): 0%| | 0/1 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 1/1 [00:00<00:00, 12.71it/s]
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:40 [gpu_model_runner.py:4587] Graph capturing finished in 1 secs, took 0.02 GiB
(EngineCore_DP0 pid=43785) INFO 01-20 16:25:40 [core.py:259] init engine (profile, create kv cache, warmup model) took 13.61 seconds
(APIServer pid=43538) INFO 01-20 16:25:41 [api_server.py:1099] Supported tasks: ['generate']
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] Using supplied chat template: {%- if tools %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_start|>system
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- if messages[0].role == 'system' -%}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {{ messages[0].content }}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- else -%}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] You are a helpful assistant.
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- endif -%}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312]
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] # Tools
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] You may call one or more functions to assist with the user query.
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] The function signatures are provided below:
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <tools>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- for tool in tools %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {{ tool | tojson }}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- endfor %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] </tools>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312]
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] When you need to call a tool, respond with XML tags in this exact format:
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <tool_call>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <function=tool_name>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <parameter=param1>value1</parameter>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <parameter=param2>value2</parameter>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] </function>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] </tool_call>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_end|>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- else %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- if messages[0].role == 'system' -%}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_start|>system
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {{ messages[0].content }}<|im_end|>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- endif -%}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- endif %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312]
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- for message in messages %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_start|>{{ message.role }}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {{ message.content }}<|im_end|>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- elif message.role == "assistant" %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_start|>assistant
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- if message.content %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {{ message.content }}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- endif %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_end|>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- elif message.role == "tool" %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_start|>user
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <tool_response>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {{ message.content }}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] </tool_response>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_end|>
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- endif %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- endfor %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312]
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- if add_generation_prompt %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] <|im_start|>assistant
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] {%- endif %}
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312]
(APIServer pid=43538) WARNING 01-20 16:25:42 [utils.py:312] It is different from official chat template '/home/huangfukk/models/AWQ/Qwen3-4B-Instruct-2507'. This discrepancy may lead to performance degradation.
(APIServer pid=43538) WARNING 01-20 16:25:42 [model.py:1487] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(APIServer pid=43538) INFO 01-20 16:25:42 [serving_responses.py:201] Using default chat sampling params from model: {'temperature': 0.7, 'top_k': 20, 'top_p': 0.8}
(APIServer pid=43538) INFO 01-20 16:25:42 [serving_engine.py:270] "auto" tool choice has been enabled.
(APIServer pid=43538) INFO 01-20 16:25:42 [serving_engine.py:270] "auto" tool choice has been enabled.
(APIServer pid=43538) INFO 01-20 16:25:42 [serving_chat.py:137] Using default chat sampling params from model: {'temperature': 0.7, 'top_k': 20, 'top_p': 0.8}
(APIServer pid=43538) INFO 01-20 16:25:42 [serving_completion.py:77] Using default completion sampling params from model: {'temperature': 0.7, 'top_k': 20, 'top_p': 0.8}
(APIServer pid=43538) INFO 01-20 16:25:42 [serving_engine.py:270] "auto" tool choice has been enabled.
(APIServer pid=43538) INFO 01-20 16:25:42 [serving_chat.py:137] Using default chat sampling params from model: {'temperature': 0.7, 'top_k': 20, 'top_p': 0.8}
(APIServer pid=43538) INFO 01-20 16:25:42 [api_server.py:1425] Starting vLLM API server 0 on http://0.0.0.0:8081
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:38] Available routes are:
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /openapi.json, Methods: GET, HEAD
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /docs, Methods: GET, HEAD
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: GET, HEAD
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /redoc, Methods: GET, HEAD
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /tokenize, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /detokenize, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /inference/v1/generate, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /pause, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /resume, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /is_paused, Methods: GET
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /metrics, Methods: GET
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /health, Methods: GET
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /load, Methods: GET
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/models, Methods: GET
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /version, Methods: GET
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/responses, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/messages, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/chat/completions, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/completions, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/audio/transcriptions, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/audio/translations, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /ping, Methods: GET
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /ping, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /invocations, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /classify, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/embeddings, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /score, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/score, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /rerank, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v1/rerank, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /v2/rerank, Methods: POST
(APIServer pid=43538) INFO 01-20 16:25:42 [launcher.py:46] Route: /pooling, Methods: POST