tckm.yan/aha
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

主要改了 vendor/esaxx-rs/build.rs：只在 crt-static 目标下才启用 static_crt(true)，用来修复 Windows 下常见的 MSVC 运行库冲突

Browse Source
This commit is contained in:
273265088@qq.com
2026-03-25 16:05:00 +08:00
parent b8d8732c3d
commit a74b67c93f
60 changed files with 16216 additions and 349 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/changelog.md
+23
View File
@@ -5,6 +5,29 @@ All notable changes to aha will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### 2026-03-25
- Added unified multi-artifact loading via `LoadSpec` (`safetensors` / `gguf` / `onnx`) across CLI, API, and service entrypoints.
- Added CLI options:
- `--artifact-format` (`auto|safetensors|gguf|onnx`)
- `--onnx-path`
- `--tokenizer-dir`
- Added ONNX runtime helper layer with repository-local `lib/onnxruntime.dll` auto-discovery on Windows.
- Enabled ONNX runtime path for:
- `qwen3` text generation (dynamic cache-aware decode path)
- `qwen3_embedding` (real session init + embedding)
- `qwen3_reranker` (reusing embedding-similarity backend)
- `qwen3.5` text/image generation (vision encoder path; video/audio explicitly rejected)
- Enabled GGUF runtime path for `qwen3-0.6b` by reusing `candle_transformers::quantized_qwen3`.
- Enabled GGUF runtime path for:
- `qwen3_embedding` (token embedding + mean pooling + normalization)
- `qwen3_reranker` (reusing embedding-similarity backend on top of GGUF embedding)
- Added reusable GGUF text bootstrap helpers in `models/common/gguf.rs` and reused them in `qwen3` / `qwen3.5`.
- Added/updated validation tests:
- `test_load_spec`
- `test_qwen3_multi_format`
- `test_qwen3_embedding_multi_format`
- `test_qwen3_reranker_multi_format`
### v0.2.3 (2026-03-18)
- add DeepSeek-OCR-2
docs/changelog.zh-CN.md
+23
View File
@@ -5,6 +5,29 @@
格式基于 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.0.0/)，
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/spec/v2.0.0.html)。
### 2026-03-25
- 新增统一多制品加载抽象 `LoadSpec`，在 CLI / API / service 三入口支持 `safetensors` / `gguf` / `onnx`。
- 新增 CLI 参数：
- `--artifact-format`（`auto|safetensors|gguf|onnx`）
- `--onnx-path`
- `--tokenizer-dir`
- 新增 ONNX 运行时通用层，并在 Windows 上支持仓库内 `lib/onnxruntime.dll` 自动发现。
- 完成以下模型族 ONNX 运行路径接入：
- `qwen3` 文本生成（动态 cache 适配解码路径）
- `qwen3_embedding`（真实 session 初始化与 embedding）
- `qwen3_reranker`（复用 embedding-similarity 后端）
- `qwen3.5` 文本+图片生成（接入 vision encoder；视频/音频显式拒绝）
- 新增 `qwen3-0.6b` 的 GGUF 运行路径（复用 `candle_transformers::quantized_qwen3`）。
- 新增以下模型族的 GGUF 运行路径：
- `qwen3_embedding`（token embedding + mean pooling + normalization）
- `qwen3_reranker`（在 GGUF embedding 之上复用 embedding-similarity 后端）
- 在 `models/common/gguf.rs` 新增通用 GGUF 文本引导 helper，并复用到 `qwen3` / `qwen3.5`。
- 新增/更新验证测试：
- `test_load_spec`
- `test_qwen3_multi_format`
- `test_qwen3_embedding_multi_format`
- `test_qwen3_reranker_multi_format`
### v0.2.3 (2026-03-18)
- 新增 DeepSeek-OCR-2
docs/cli.md
+60 -8
View File
@@ -20,6 +20,9 @@ aha [COMMAND] [OPTIONS]
| `--download-retries <DOWNLOAD_RETRIES>` | Download retry count | 3 |
| `--gguf-path <GGUF_PATH>` | Local GGUF weight | - |
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight | - |
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX | - |
| `--artifact-format <ARTIFACT_FORMAT>` | Model artifact format (`auto|safetensors|gguf|onnx`) | auto |
| `-h, --help` | Display help information | - |
| `-V, --version` | Display version number | - |
@@ -46,6 +49,9 @@ aha cli [OPTIONS] --model <MODEL>
| `--download-retries <DOWNLOAD_RETRIES>` | Download retry count | 3 |
| `--gguf-path <GGUF_PATH>` | Local GGUF weight | - |
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight | - |
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX | - |
| `--artifact-format <ARTIFACT_FORMAT>` | Artifact format (`auto|safetensors|gguf|onnx`) | auto |
**Examples:**
@@ -64,6 +70,11 @@ aha -m qwen3vl-2b
# use gguf-path and mmproj-path
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# run service with ONNX artifact
aha cli -m qwen3-embedding-0.6b --artifact-format onnx \
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
```
### run - Direct model inference
@@ -72,7 +83,9 @@ Run model inference directly without starting an HTTP service. Suitable for one-
**Syntax:**
```bash
aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>]
aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] \
[--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>] \
[--onnx-path <ONNX_PATH>] [--tokenizer-dir <TOKENIZER_DIR>] [--artifact-format <ARTIFACT_FORMAT>]
```
**Options:**
@@ -82,9 +95,12 @@ aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] [--weight-p
| `-m, --model <MODEL>` | Model type (required) | - |
| `-i, --input <INPUT>` | Input text or file path (model-specific interpretation, supports 1-2 parameters: input1: prompt text, input2: file path) | - |
| `-o, --output <OUTPUT>` | Output file path (optional, auto-generated if not specified) | - |
| `--weight-path <WEIGHT_PATH>` | Local model weight path (required when using non-GGUF models) | - |
| `--weight-path <WEIGHT_PATH>` | Local safetensors model directory (required when `--artifact-format safetensors`) | - |
| `--gguf-path <GGUF_PATH>` | Local GGUF model weight path（required when using GGUF models） | - |
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight path（optional，If not specified, the module will not be loaded） | - |
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path (required when `--artifact-format onnx`) | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX (optional but recommended) | - |
| `--artifact-format <ARTIFACT_FORMAT>` | Artifact format (`auto|safetensors|gguf|onnx`) | auto |
**Examples:**
```bash
@@ -112,6 +128,9 @@ aha run -m fun-asr-nano-2512 -i "语音转写：" -i "audio.wav" --weight-path /
# qwen3 text generation (single input)
aha run -m qwen3-0.6b -i "你好" --weight-path /path/to/model
# qwen3 GGUF text generation (single input)
aha run -m qwen3-0.6b -i "hello" --artifact-format gguf --gguf-path /path/to/Qwen3-0.6B-Q8_0.gguf
# qwen2.5vl image understanding (two inputs: prompt text + image file)
aha run -m qwen2.5vl-3b -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model
@@ -125,6 +144,11 @@ aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
aha run -m qwen3.5-gguf -i 提取图片中的文本 -i https://ai.bdstatic.com/file/C56CC9B274CF460CA33
63E59ECD94423 --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# Qwen3.5 ONNX text-only generation
aha run -m qwen3.5-0.8b -i "hello" --artifact-format onnx \
--onnx-path /path/to/Qwen3.5-0.8B-ONNX \
--tokenizer-dir /path/to/Qwen3.5-0.8B-ONNX
```
### serv - Start service
@@ -133,7 +157,9 @@ Start HTTP service with a model. The `--weight-path` is optional - if not specif
**Syntax:**
```bash
aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>]
aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] \
[--mmproj-path <MMPROJ_PATH>] [--onnx-path <ONNX_PATH>] [--tokenizer-dir <TOKENIZER_DIR>] \
[--artifact-format <ARTIFACT_FORMAT>]
```
**Options:**
@@ -147,6 +173,9 @@ aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <G
| `--allow-remote-shutdown` | Allow remote shutdown requests (not recommended) | false |
| `--gguf-path <GGUF_PATH>` | Local GGUF model weight path（required when using GGUF models） | - |
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight path（optional，If not specified, the module will not be loaded） | - |
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path（required when using ONNX models） | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX | - |
| `--artifact-format <ARTIFACT_FORMAT>` | Artifact format (`auto|safetensors|gguf|onnx`) | auto |
**Examples:**
@@ -165,6 +194,11 @@ aha serv -m qwen3vl-2b -a 0.0.0.0
# Enable remote shutdown (not recommended for production)
aha serv -m qwen3vl-2b --allow-remote-shutdown
# Start ONNX embedding service
aha serv -m qwen3-embedding-0.6b --artifact-format onnx \
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
```
### ps - List running services
@@ -408,6 +442,20 @@ After the service starts, the following API endpoints are available:
- **Format**: OpenAI Chat Completion format
- **Streaming Support**: No
### Embeddings Endpoint
- **Endpoint**: `POST /embeddings` or `POST /v1/embeddings`
- **Function**: Text embedding generation
- **Supported Models**: Qwen3-Embedding family
- **Format**: OpenAI embeddings format
- **Streaming Support**: No
### Rerank Endpoint
- **Endpoint**: `POST /rerank` or `POST /v1/rerank`
- **Function**: Query-document reranking
- **Supported Models**: Qwen3-Reranker family
- **Format**: Rerank JSON response (`results[index,relevance_score,document]`)
- **Streaming Support**: No
### Shutdown Endpoint
- **Endpoint**: `POST /shutdown`
- **Function**: Gracefully shut down the server
@@ -429,15 +477,19 @@ aha -m qwen3vl-2b
## Notes
1. **serv subcommand requires `--weight-path`**: Since the `serv` subcommand does not download models, you must specify the path to an already downloaded model via `--weight-path`.
1. **Local-path rule for GGUF/ONNX**: GGUF and ONNX artifacts are local-path only; use `--gguf-path` or `--onnx-path`. Remote download management is only for safetensors models.
2. **Download retry mechanism**: By default, retries 3 times, waiting 2 seconds after each failure before retrying. You can adjust the retry count with `--download-retries`.
2. **Artifact selection**: `--artifact-format auto` uses model default; you can force `safetensors|gguf|onnx` explicitly.
3. **Default save directory**: Models are saved to `~/.aha/` directory by default, which can be customized via `--save-dir` or `-s` parameter.
3. **Tokenizer directory**: For GGUF/ONNX, if tokenizer files are not colocated with model files, set `--tokenizer-dir`.
4. **Port occupation**: Ensure the specified port is not occupied before starting the service. The default port is 10100.
4. **Download retry mechanism**: By default, retries 3 times, waiting 2 seconds after each failure before retrying. You can adjust the retry count with `--download-retries`.
5. **Permission issues**: If saving to a system directory (such as `/data/models`), ensure you have the corresponding write permissions.
5. **Default save directory**: Models are saved to `~/.aha/` directory by default, which can be customized via `--save-dir` or `-s` parameter.
6. **Port occupation**: Ensure the specified port is not occupied before starting the service. The default port is 10100.
7. **Permission issues**: If saving to a system directory (such as `/data/models`), ensure you have the corresponding write permissions.
## Getting Help
docs/cli.zh-CN.md
+60 -8
View File
@@ -20,6 +20,9 @@ aha [COMMAND] [OPTIONS]
| `--download-retries <DOWNLOAD_RETRIES>` | 下载重试次数 | 3 |
| `--gguf-path <GGUF_PATH>` | 本地 GGUF 模型权重 | - |
| `--mmproj-path <MMPROJ_PATH>` | 本地 mmproj GGUF 模型权重 | - |
| `--onnx-path <ONNX_PATH>` | 本地 ONNX 模型目录或文件路径 | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | GGUF/ONNX 的 tokenizer/config 目录 | - |
| `--artifact-format <ARTIFACT_FORMAT>` | 制品格式（`auto|safetensors|gguf|onnx`） | auto |
| `-h, --help` | 显示帮助信息 | - |
| `-V, --version` | 显示版本号 | - |
@@ -46,6 +49,9 @@ aha cli [OPTIONS] --model <MODEL>
| `--download-retries <DOWNLOAD_RETRIES>` | 下载重试次数 | 3 |
| `--gguf-path <GGUF_PATH>` | 本地 GGUF 模型权重 | - |
| `--mmproj-path <MMPROJ_PATH>` | 本地 mmproj GGUF 模型权重 | - |
| `--onnx-path <ONNX_PATH>` | 本地 ONNX 模型目录或文件路径 | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | GGUF/ONNX 的 tokenizer/config 目录 | - |
| `--artifact-format <ARTIFACT_FORMAT>` | 制品格式（`auto|safetensors|gguf|onnx`） | auto |
**示例：**
@@ -64,6 +70,11 @@ aha -m qwen3vl-2b
# 指定gguf-path和mmproj-path
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# 使用 ONNX 模型启动服务
aha cli -m qwen3-embedding-0.6b --artifact-format onnx \
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
```
### run - 直接模型推理
@@ -72,7 +83,9 @@ aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmp
**语法：**
```bash
aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>]
aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] \
[--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>] \
[--onnx-path <ONNX_PATH>] [--tokenizer-dir <TOKENIZER_DIR>] [--artifact-format <ARTIFACT_FORMAT>]
```
**选项：**
@@ -82,9 +95,12 @@ aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] [--weight-p
| `-m, --model <MODEL>` | 模型类型（必选） | - |
| `-i, --input <INPUT>` | 输入文本或文件路径（模型特定解释，支持1-2个参数, input1： 提示文本, input2: 文件地址） | - |
| `-o, --output <OUTPUT>` | 输出文件路径（可选，未指定则自动生成） | - |
| `--weight-path <WEIGHT_PATH>` | 本地模型权重路径（使用非GGUF模型时必选） | - |
| `--weight-path <WEIGHT_PATH>` | 本地 safetensors 模型目录（`--artifact-format safetensors` 时必选） | - |
| `--gguf-path <GGUF_PATH>` | 本地GGUF模型权重路径（使用GGUF模型时必选） | - |
| `--mmproj-path <MMPROJ_PATH>` | 本地mmproj GGUF模型权重路径（可选，未指定则不加载该模块） | - |
| `--onnx-path <ONNX_PATH>` | 本地 ONNX 模型目录或文件路径（`--artifact-format onnx` 时必选） | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | GGUF/ONNX 的 tokenizer/config 目录（可选，建议提供） | - |
| `--artifact-format <ARTIFACT_FORMAT>` | 制品格式（`auto|safetensors|gguf|onnx`） | auto |
**示例：**
@@ -113,6 +129,9 @@ aha run -m fun-asr-nano-2512 -i "语音转写：" -i "audio.wav" --weight-path /
# qwen3 文本生成（单个输入）
aha run -m qwen3-0.6b -i "你好" --weight-path /path/to/model
# qwen3 GGUF 文本生成（单个输入）
aha run -m qwen3-0.6b -i "你好" --artifact-format gguf --gguf-path /path/to/Qwen3-0.6B-Q8_0.gguf
# qwen2.5vl 图像理解（两个输入：提示文本 + 图片文件）
aha run -m qwen2.5vl-3b -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model
@@ -125,6 +144,11 @@ aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
# Qwen3.5-GGUF 有mmproj (两个输入：提示文本 + 文件)
aha run -m qwen3.5-gguf -i 提取图片中的文本 -i https://ai.bdstatic.com/file/C56CC9B274CF460CA33
63E59ECD94423 --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# Qwen3.5 ONNX 文本生成（text-only）
aha run -m qwen3.5-0.8b -i "你好" --artifact-format onnx \
--onnx-path /path/to/Qwen3.5-0.8B-ONNX \
--tokenizer-dir /path/to/Qwen3.5-0.8B-ONNX
```
### serv - 启动服务
@@ -133,7 +157,9 @@ aha run -m qwen3.5-gguf -i 提取图片中的文本 -i https://ai.bdstatic.com/f
**语法：**
```bash
aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>]
aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] \
[--mmproj-path <MMPROJ_PATH>] [--onnx-path <ONNX_PATH>] [--tokenizer-dir <TOKENIZER_DIR>] \
[--artifact-format <ARTIFACT_FORMAT>]
```
**选项：**
@@ -147,6 +173,9 @@ aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <G
| `--allow-remote-shutdown` | 允许远程关机请求（不推荐） | false |
| `--gguf-path <GGUF_PATH>` | 本地GGUF模型权重路径（使用GGUF模型时必选） | - |
| `--mmproj-path <MMPROJ_PATH>` | 本地mmproj GGUF模型权重路径（可选，未指定则不加载该模块） | - |
| `--onnx-path <ONNX_PATH>` | 本地 ONNX 模型目录或文件路径（ONNX 模型时必选） | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | GGUF/ONNX 的 tokenizer/config 目录 | - |
| `--artifact-format <ARTIFACT_FORMAT>` | 制品格式（`auto|safetensors|gguf|onnx`） | auto |
**示例：**
@@ -165,6 +194,11 @@ aha serv -m qwen3vl-2b -a 0.0.0.0
# 启用远程关机（不推荐用于生产环境）
aha serv -m qwen3vl-2b --allow-remote-shutdown
# 启动 ONNX embedding 服务
aha serv -m qwen3-embedding-0.6b --artifact-format onnx \
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
```
### ps - 列出运行中的服务
@@ -408,6 +442,20 @@ aha -m qwen3vl-2b -a 0.0.0.0 -p 8080
- **格式**: OpenAI Chat Completion 格式
- **流式支持**: 不支持
### Embeddings 接口
- **端点**: `POST /embeddings` 或 `POST /v1/embeddings`
- **功能**: 文本向量生成
- **支持模型**: Qwen3-Embedding 系列
- **格式**: OpenAI embeddings 格式
- **流式支持**: 不支持
### Rerank 接口
- **端点**: `POST /rerank` 或 `POST /v1/rerank`
- **功能**: query/document 重排打分
- **支持模型**: Qwen3-Reranker 系列
- **格式**: Rerank JSON（`results[index,relevance_score,document]`）
- **流式支持**: 不支持
### 关机接口
- **端点**: `POST /shutdown`
- **功能**: 优雅地关闭服务器
@@ -428,15 +476,19 @@ aha -m qwen3vl-2b
## 注意事项
1. **serv 子命令必须指定 `--weight-path`**：由于 `serv` 子命令不下载模型，必须通过 `--weight-path` 指定已下载的模型路径。
1. **GGUF/ONNX 仅支持本地路径**：请使用 `--gguf-path` 或 `--onnx-path`。自动下载管理仅适用于 safetensors 模型。
2. **下载重试机制**：默认重试 3 次，每次失败后等待 2 秒再重试。可通过 `--download-retries` 调整重试次数。
2. **制品格式选择**：`--artifact-format auto` 使用模型默认格式，也可显式指定 `safetensors|gguf|onnx`。
3. **默认保存目录**：模型默认保存到 `~/.aha/` 目录下，可通过 `--save-dir` 或 `-s` 参数自定义。
3. **tokenizer 目录**：GGUF/ONNX 若未与 tokenizer/config 同目录，请额外指定 `--tokenizer-dir`。
4. **端口占用**：启动服务前确保指定的端口未被占用，默认端口为 10100。
4. **下载重试机制**：默认重试 3 次，每次失败后等待 2 秒再重试。可通过 `--download-retries` 调整重试次数。
5. **权限问题**：如果保存到系统目录（如 `/data/models`），确保有相应的写入权限。
5. **默认保存目录**：模型默认保存到 `~/.aha/` 目录下，可通过 `--save-dir` 或 `-s` 参数自定义。
6. **端口占用**：启动服务前确保指定的端口未被占用，默认端口为 10100。
7. **权限问题**：如果保存到系统目录（如 `/data/models`），确保有相应的写入权限。
## 获取帮助
docs/development.md
+11
View File
@@ -335,6 +335,17 @@ Update the following files:
- `docs/api.md` - Add model to supported models section
- `CHANGELOG.md` - Add entry for new model
## Multi-Artifact Integration Rules
When adding or extending model loaders, follow the current artifact architecture:
1. Use `LoadSpec` (`src/models/core/artifact.rs`) as the only input contract for loading.
2. Keep model identity (`WhichModel`) separate from artifact format (`ArtifactKind`).
3. Register supported formats in `supported_artifacts(model)`; do not expose unsupported formats.
4. Route initialization through family loader registry (`src/models/core/registry.rs`) instead of extending a large `match` in `load_model`.
5. Keep GGUF/ONNX path handling local-only (`gguf_path` / `onnx_path` / `tokenizer_dir`), and avoid adding remote download logic for these formats.
6. Maintain API compatibility of `ModelInstance` capabilities (`generate`, `generate_stream`, `embedding`, `rerank`) when adding new backends.
## Testing
### Unit Tests
docs/development.zh-CN.md
+11
View File
@@ -335,6 +335,17 @@ fn test_newmodel_generate() -> Result<()> {
- `docs/api.md` - 将模型添加到支持的模型部分
- `CHANGELOG.md` - 为新模型添加条目
## 多制品接入规则
为模型新增或扩展加载后端时，请遵循当前制品架构：
1. 统一使用 `LoadSpec`（`src/models/core/artifact.rs`）作为加载输入契约。
2. 保持模型身份（`WhichModel`）与制品格式（`ArtifactKind`）解耦。
3. 在 `supported_artifacts(model)` 中声明支持矩阵，不要提前暴露未实现格式。
4. 通过 family loader registry（`src/models/core/registry.rs`）分发初始化，不再扩展 `load_model` 的大 `match`。
5. GGUF/ONNX 保持本地路径输入（`gguf_path` / `onnx_path` / `tokenizer_dir`），不引入远程下载管理逻辑。
6. 新后端接入时保持 `ModelInstance` 能力接口语义兼容（`generate`、`generate_stream`、`embedding`、`rerank`）。
## 测试
### 单元测试
docs/supported-models.md
+12 -12
View File
@@ -6,24 +6,24 @@ aha supports a growing collection of state-of-the-art AI models across multiple
| Model | Parameters | Description | Use Case | License |
|-------|-----------|-------------|----------|---------|
| **Qwen3-0.6B** | 0.6B | Latest generation | Advanced reasoning | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-0.6B** | 0.6B | Latest generation (safetensors / gguf / onnx) | Advanced reasoning | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **MiniCPM4-0.5B** | 0.5B | Efficient lightweight | Edge deployment | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Embedding
| Model | Parameters | Description | License |
|-------|-----------|-------------|---------|
| **Qwen3-Embedding-0.6B** | 0.6B | Text embedding (safetensors) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-4B** | 4B | Text embedding (safetensors) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-8B** | 8B | Text embedding (safetensors) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-0.6B** | 0.6B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-4B** | 4B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-8B** | 8B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Reranker
| Model | Parameters | Description | License |
|-------|-----------|-------------|---------|
| **Qwen3-Reranker-0.6B** | 0.6B | Text reranking (embedding-similarity baseline, safetensors) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-4B** | 4B | Text reranking (embedding-similarity baseline, safetensors) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-8B** | 8B | Text reranking (embedding-similarity baseline, safetensors) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-0.6B** | 0.6B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-4B** | 4B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-8B** | 8B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Vision & Multimodal
@@ -35,11 +35,11 @@ aha supports a growing collection of state-of-the-art AI models across multiple
| **Qwen3-VL-4B** | 4B | Enhanced multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-VL-8B** | 8B | Enhanced multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-VL-32B** | 32B | Enhanced multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-0.8B** | 0.8B | Native Multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-2B** | 2B | Native Multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-4B** | 4B | Native Multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B** | 9B | Native Multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2** | 9B | Distilled variant (Qwen3.5 family) | [Model license on HF](https://huggingface.co/Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2) |
| **Qwen3.5-0.8B** | 0.8B | Native Multimodal (safetensors / gguf / onnx-text+image, video/audio rejected) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-2B** | 2B | Native Multimodal (safetensors / gguf / onnx-text+image, video/audio rejected) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-4B** | 4B | Native Multimodal (safetensors / gguf / onnx-text+image, video/audio rejected) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B** | 9B | Native Multimodal (safetensors / gguf / onnx-text+image, video/audio rejected) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2** | 9B | Distilled variant (Qwen3.5 family, safetensors / gguf / onnx-text+image, video/audio rejected) | [Model license on HF](https://huggingface.co/Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2) |
### Qwen3.5 GGUF Sources (Runtime Reused)
docs/supported-models.zh-CN.md
+12 -12
View File
@@ -6,24 +6,24 @@ aha 支持多个领域的最先进 AI 模型集合。
| 模型 | 参数量 | 描述 | 使用场景 | 开源协议 |
|------|--------|------|----------|---------|
| **Qwen3-0.6B** | 0.6B | 最新一代 | 高级推理 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-0.6B** | 0.6B | 最新一代（safetensors / gguf / onnx） | 高级推理 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **MiniCPM4-0.5B** | 0.5B | 高效轻量级 | 边缘部署 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Embedding
| 模型 | 参数量 | 描述 | 开源协议 |
|------|--------|------|---------|
| **Qwen3-Embedding-0.6B** | 0.6B | 文本向量（safetensors） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-4B** | 4B | 文本向量（safetensors） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-8B** | 8B | 文本向量（safetensors） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-0.6B** | 0.6B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-4B** | 4B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-8B** | 8B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Reranker
| 模型 | 参数量 | 描述 | 开源协议 |
|------|--------|------|---------|
| **Qwen3-Reranker-0.6B** | 0.6B | 文本重排（基于 embedding 相似度的基线实现，safetensors） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-4B** | 4B | 文本重排（基于 embedding 相似度的基线实现，safetensors） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-8B** | 8B | 文本重排（基于 embedding 相似度的基线实现，safetensors） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-0.6B** | 0.6B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-4B** | 4B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-8B** | 8B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## 视觉与多模态
@@ -35,11 +35,11 @@ aha 支持多个领域的最先进 AI 模型集合。
| **Qwen3-VL-4B** | 4B | 增强多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-VL-8B** | 8B | 增强多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-VL-32B** | 32B | 增强多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-0.8B** | 0.8B | 原生多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-2B** | 2B | 原生多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-4B** | 4B | 原生多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B** | 9B | 原生多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2** | 9B | Qwen3.5 同构蒸馏版本 | [HF 页面许可证](https://huggingface.co/Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2) |
| **Qwen3.5-0.8B** | 0.8B | 原生多模态（safetensors / gguf / onnx 文本+图片，视频/音频显式拒绝） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-2B** | 2B | 原生多模态（safetensors / gguf / onnx 文本+图片，视频/音频显式拒绝） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-4B** | 4B | 原生多模态（safetensors / gguf / onnx 文本+图片，视频/音频显式拒绝） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B** | 9B | 原生多模态（safetensors / gguf / onnx 文本+图片，视频/音频显式拒绝） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2** | 9B | Qwen3.5 同构蒸馏版本（safetensors / gguf / onnx 文本+图片，视频/音频显式拒绝） | [HF 页面许可证](https://huggingface.co/Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2) |
### Qwen3.5 GGUF 仓库来源（复用现有运行时）