主要改了 vendor/esaxx-rs/build.rs：只在 crt-static 目标下才启用 static_crt(true)，用来修复 Windows 下常见的 MSVC 运行库冲突
This commit is contained in:
273265088@qq.com
+60
-8
@@ -20,6 +20,9 @@ aha [COMMAND] [OPTIONS]
|
||||
| `--download-retries <DOWNLOAD_RETRIES>` | Download retry count | 3 |
|
||||
| `--gguf-path <GGUF_PATH>` | Local GGUF weight | - |
|
||||
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight | - |
|
||||
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path | - |
|
||||
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX | - |
|
||||
| `--artifact-format <ARTIFACT_FORMAT>` | Model artifact format (`auto|safetensors|gguf|onnx`) | auto |
|
||||
| `-h, --help` | Display help information | - |
|
||||
| `-V, --version` | Display version number | - |
|
||||
|
||||
@@ -46,6 +49,9 @@ aha cli [OPTIONS] --model <MODEL>
|
||||
| `--download-retries <DOWNLOAD_RETRIES>` | Download retry count | 3 |
|
||||
| `--gguf-path <GGUF_PATH>` | Local GGUF weight | - |
|
||||
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight | - |
|
||||
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path | - |
|
||||
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX | - |
|
||||
| `--artifact-format <ARTIFACT_FORMAT>` | Artifact format (`auto|safetensors|gguf|onnx`) | auto |
|
||||
|
||||
**Examples:**
|
||||
|
||||
@@ -64,6 +70,11 @@ aha -m qwen3vl-2b
|
||||
|
||||
# use gguf-path and mmproj-path
|
||||
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
|
||||
|
||||
# run service with ONNX artifact
|
||||
aha cli -m qwen3-embedding-0.6b --artifact-format onnx \
|
||||
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
|
||||
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
|
||||
```
|
||||
|
||||
### run - Direct model inference
|
||||
@@ -72,7 +83,9 @@ Run model inference directly without starting an HTTP service. Suitable for one-
|
||||
|
||||
**Syntax:**
|
||||
```bash
|
||||
aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>]
|
||||
aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] \
|
||||
[--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>] \
|
||||
[--onnx-path <ONNX_PATH>] [--tokenizer-dir <TOKENIZER_DIR>] [--artifact-format <ARTIFACT_FORMAT>]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
@@ -82,9 +95,12 @@ aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] [--weight-p
|
||||
| `-m, --model <MODEL>` | Model type (required) | - |
|
||||
| `-i, --input <INPUT>` | Input text or file path (model-specific interpretation, supports 1-2 parameters: input1: prompt text, input2: file path) | - |
|
||||
| `-o, --output <OUTPUT>` | Output file path (optional, auto-generated if not specified) | - |
|
||||
| `--weight-path <WEIGHT_PATH>` | Local model weight path (required when using non-GGUF models) | - |
|
||||
| `--weight-path <WEIGHT_PATH>` | Local safetensors model directory (required when `--artifact-format safetensors`) | - |
|
||||
| `--gguf-path <GGUF_PATH>` | Local GGUF model weight path（required when using GGUF models） | - |
|
||||
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight path（optional，If not specified, the module will not be loaded） | - |
|
||||
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path (required when `--artifact-format onnx`) | - |
|
||||
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX (optional but recommended) | - |
|
||||
| `--artifact-format <ARTIFACT_FORMAT>` | Artifact format (`auto|safetensors|gguf|onnx`) | auto |
|
||||
**Examples:**
|
||||
|
||||
```bash
|
||||
@@ -112,6 +128,9 @@ aha run -m fun-asr-nano-2512 -i "语音转写：" -i "audio.wav" --weight-path /
|
||||
# qwen3 text generation (single input)
|
||||
aha run -m qwen3-0.6b -i "你好" --weight-path /path/to/model
|
||||
|
||||
# qwen3 GGUF text generation (single input)
|
||||
aha run -m qwen3-0.6b -i "hello" --artifact-format gguf --gguf-path /path/to/Qwen3-0.6B-Q8_0.gguf
|
||||
|
||||
# qwen2.5vl image understanding (two inputs: prompt text + image file)
|
||||
aha run -m qwen2.5vl-3b -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model
|
||||
|
||||
@@ -125,6 +144,11 @@ aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
|
||||
aha run -m qwen3.5-gguf -i 提取图片中的文本 -i https://ai.bdstatic.com/file/C56CC9B274CF460CA33
|
||||
63E59ECD94423 --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
|
||||
|
||||
# Qwen3.5 ONNX text-only generation
|
||||
aha run -m qwen3.5-0.8b -i "hello" --artifact-format onnx \
|
||||
--onnx-path /path/to/Qwen3.5-0.8B-ONNX \
|
||||
--tokenizer-dir /path/to/Qwen3.5-0.8B-ONNX
|
||||
|
||||
```
|
||||
|
||||
### serv - Start service
|
||||
@@ -133,7 +157,9 @@ Start HTTP service with a model. The `--weight-path` is optional - if not specif
|
||||
|
||||
**Syntax:**
|
||||
```bash
|
||||
aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>]
|
||||
aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] \
|
||||
[--mmproj-path <MMPROJ_PATH>] [--onnx-path <ONNX_PATH>] [--tokenizer-dir <TOKENIZER_DIR>] \
|
||||
[--artifact-format <ARTIFACT_FORMAT>]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
@@ -147,6 +173,9 @@ aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <G
|
||||
| `--allow-remote-shutdown` | Allow remote shutdown requests (not recommended) | false |
|
||||
| `--gguf-path <GGUF_PATH>` | Local GGUF model weight path（required when using GGUF models） | - |
|
||||
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight path（optional，If not specified, the module will not be loaded） | - |
|
||||
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path（required when using ONNX models） | - |
|
||||
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX | - |
|
||||
| `--artifact-format <ARTIFACT_FORMAT>` | Artifact format (`auto|safetensors|gguf|onnx`) | auto |
|
||||
|
||||
**Examples:**
|
||||
|
||||
@@ -165,6 +194,11 @@ aha serv -m qwen3vl-2b -a 0.0.0.0
|
||||
|
||||
# Enable remote shutdown (not recommended for production)
|
||||
aha serv -m qwen3vl-2b --allow-remote-shutdown
|
||||
|
||||
# Start ONNX embedding service
|
||||
aha serv -m qwen3-embedding-0.6b --artifact-format onnx \
|
||||
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
|
||||
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
|
||||
```
|
||||
|
||||
### ps - List running services
|
||||
@@ -408,6 +442,20 @@ After the service starts, the following API endpoints are available:
|
||||
- **Format**: OpenAI Chat Completion format
|
||||
- **Streaming Support**: No
|
||||
|
||||
### Embeddings Endpoint
|
||||
- **Endpoint**: `POST /embeddings` or `POST /v1/embeddings`
|
||||
- **Function**: Text embedding generation
|
||||
- **Supported Models**: Qwen3-Embedding family
|
||||
- **Format**: OpenAI embeddings format
|
||||
- **Streaming Support**: No
|
||||
|
||||
### Rerank Endpoint
|
||||
- **Endpoint**: `POST /rerank` or `POST /v1/rerank`
|
||||
- **Function**: Query-document reranking
|
||||
- **Supported Models**: Qwen3-Reranker family
|
||||
- **Format**: Rerank JSON response (`results[index,relevance_score,document]`)
|
||||
- **Streaming Support**: No
|
||||
|
||||
### Shutdown Endpoint
|
||||
- **Endpoint**: `POST /shutdown`
|
||||
- **Function**: Gracefully shut down the server
|
||||
@@ -429,15 +477,19 @@ aha -m qwen3vl-2b
|
||||
|
||||
## Notes
|
||||
|
||||
1. **serv subcommand requires `--weight-path`**: Since the `serv` subcommand does not download models, you must specify the path to an already downloaded model via `--weight-path`.
|
||||
1. **Local-path rule for GGUF/ONNX**: GGUF and ONNX artifacts are local-path only; use `--gguf-path` or `--onnx-path`. Remote download management is only for safetensors models.
|
||||
|
||||
2. **Download retry mechanism**: By default, retries 3 times, waiting 2 seconds after each failure before retrying. You can adjust the retry count with `--download-retries`.
|
||||
2. **Artifact selection**: `--artifact-format auto` uses model default; you can force `safetensors|gguf|onnx` explicitly.
|
||||
|
||||
3. **Default save directory**: Models are saved to `~/.aha/` directory by default, which can be customized via `--save-dir` or `-s` parameter.
|
||||
3. **Tokenizer directory**: For GGUF/ONNX, if tokenizer files are not colocated with model files, set `--tokenizer-dir`.
|
||||
|
||||
4. **Port occupation**: Ensure the specified port is not occupied before starting the service. The default port is 10100.
|
||||
4. **Download retry mechanism**: By default, retries 3 times, waiting 2 seconds after each failure before retrying. You can adjust the retry count with `--download-retries`.
|
||||
|
||||
5. **Permission issues**: If saving to a system directory (such as `/data/models`), ensure you have the corresponding write permissions.
|
||||
5. **Default save directory**: Models are saved to `~/.aha/` directory by default, which can be customized via `--save-dir` or `-s` parameter.
|
||||
|
||||
6. **Port occupation**: Ensure the specified port is not occupied before starting the service. The default port is 10100.
|
||||
|
||||
7. **Permission issues**: If saving to a system directory (such as `/data/models`), ensure you have the corresponding write permissions.
|
||||
|
||||
## Getting Help
|
||||
|
||||
|
||||
Reference in New Issue
Block a user