tckm.yan/aha
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'pr/happynewplay/41'

Browse Source
This commit is contained in:
jhqxxx
2026-04-06 18:23:23 +08:00
parent ed4811a01f aec47e47af
commit 110a1bc5eb
33 changed files with 11898 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cli.md
+39 -5
View File
@@ -66,6 +66,11 @@ aha cli -m Qwen/Qwen3-VL-2B-Instruct --weight-path /path/to/model
# use gguf-path and mmproj-path
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# run service with ONNX artifact
aha cli -m qwen3-embedding-0.6b --artifact-format onnx \
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
```
### run - Direct model inference
@@ -117,6 +122,9 @@ aha run -m FunAudioLLM/Fun-ASR-Nano-2512 -i "语音转写：" -i "audio.wav" --w
# qwen3 text generation (single input)
aha run -m Qwen/Qwen3-0.6B -i "你好" --weight-path /path/to/model
# qwen3 GGUF text generation (single input)
aha run -m qwen3-0.6b -i "hello" --artifact-format gguf --gguf-path /path/to/Qwen3-0.6B-Q8_0.gguf
# qwen2.5vl image understanding (two inputs: prompt text + image file)
aha run -m Qwen/Qwen2.5-VL-3B-Instruct -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model
@@ -130,6 +138,11 @@ aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
aha run -m qwen3.5-gguf -i 提取图片中的文本 -i https://ai.bdstatic.com/file/C56CC9B274CF460CA33
63E59ECD94423 --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# Qwen3.5 ONNX text-only generation
aha run -m qwen3.5-0.8b -i "hello" --artifact-format onnx \
--onnx-path /path/to/Qwen3.5-0.8B-ONNX \
--tokenizer-dir /path/to/Qwen3.5-0.8B-ONNX
```
### serv - Start service
@@ -154,6 +167,9 @@ aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <G
| `--allow-remote-shutdown` | Allow remote shutdown requests (not recommended) | false |
| `--gguf-path <GGUF_PATH>` | Local GGUF model weight path（required when using GGUF models） | - |
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight path（optional，If not specified, the module will not be loaded） | - |
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path（required when using ONNX models） | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX | - |
| `--artifact-format <ARTIFACT_FORMAT>` | Artifact format (`auto|safetensors|gguf|onnx`) | auto |
**Examples:**
@@ -395,6 +411,20 @@ After the service starts, the following API endpoints are available:
- **Format**: OpenAI Chat Completion format
- **Streaming Support**: No
### Embeddings Endpoint
- **Endpoint**: `POST /embeddings` or `POST /v1/embeddings`
- **Function**: Text embedding generation
- **Supported Models**: Qwen3-Embedding family
- **Format**: OpenAI embeddings format
- **Streaming Support**: No
### Rerank Endpoint
- **Endpoint**: `POST /rerank` or `POST /v1/rerank`
- **Function**: Query-document reranking
- **Supported Models**: Qwen3-Reranker family
- **Format**: Rerank JSON response (`results[index,relevance_score,document]`)
- **Streaming Support**: No
### Shutdown Endpoint
- **Endpoint**: `POST /shutdown`
- **Function**: Gracefully shut down the server
@@ -404,15 +434,19 @@ After the service starts, the following API endpoints are available:
## Notes
1. **serv subcommand requires `--weight-path`**: Since the `serv` subcommand does not download models, you must specify the path to an already downloaded model via `--weight-path`.
1. **Local-path rule for GGUF/ONNX**: GGUF and ONNX artifacts are local-path only; use `--gguf-path` or `--onnx-path`. Remote download management is only for safetensors models.
2. **Download retry mechanism**: By default, retries 3 times, waiting 2 seconds after each failure before retrying. You can adjust the retry count with `--download-retries`.
2. **Artifact selection**: `--artifact-format auto` uses model default; you can force `safetensors|gguf|onnx` explicitly.
3. **Default save directory**: Models are saved to `~/.aha/` directory by default, which can be customized via `--save-dir` or `-s` parameter.
3. **Tokenizer directory**: For GGUF/ONNX, if tokenizer files are not colocated with model files, set `--tokenizer-dir`.
4. **Port occupation**: Ensure the specified port is not occupied before starting the service. The default port is 10100.
4. **Download retry mechanism**: By default, retries 3 times, waiting 2 seconds after each failure before retrying. You can adjust the retry count with `--download-retries`.
5. **Permission issues**: If saving to a system directory (such as `/data/models`), ensure you have the corresponding write permissions.
5. **Default save directory**: Models are saved to `~/.aha/` directory by default, which can be customized via `--save-dir` or `-s` parameter.
6. **Port occupation**: Ensure the specified port is not occupied before starting the service. The default port is 10100.
7. **Permission issues**: If saving to a system directory (such as `/data/models`), ensure you have the corresponding write permissions.
## Getting Help
docs/cli.zh-CN.md
+39 -6
View File
@@ -66,6 +66,11 @@ aha cli -m Qwen/Qwen3-VL-2B-Instruct --weight-path /path/to/model
# 指定gguf-path和mmproj-path
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# 使用 ONNX 模型启动服务
aha cli -m qwen3-embedding-0.6b --artifact-format onnx \
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
```
### run - 直接模型推理
@@ -117,6 +122,9 @@ aha run -m FunAudioLLM/Fun-ASR-Nano-2512 -i "语音转写：" -i "audio.wav" --w
# qwen3 文本生成（单个输入）
aha run -m Qwen/Qwen3-0.6B -i "你好" --weight-path /path/to/model
# qwen3 GGUF 文本生成（单个输入）
aha run -m qwen3-0.6b -i "你好" --artifact-format gguf --gguf-path /path/to/Qwen3-0.6B-Q8_0.gguf
# qwen2.5vl 图像理解（两个输入：提示文本 + 图片文件）
aha run -m Qwen/Qwen2.5-VL-3B-Instruct -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model
@@ -129,6 +137,11 @@ aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
# Qwen3.5-GGUF 有mmproj (两个输入：提示文本 + 文件)
aha run -m qwen3.5-gguf -i 提取图片中的文本 -i https://ai.bdstatic.com/file/C56CC9B274CF460CA33
63E59ECD94423 --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# Qwen3.5 ONNX 文本生成（text-only）
aha run -m qwen3.5-0.8b -i "你好" --artifact-format onnx \
--onnx-path /path/to/Qwen3.5-0.8B-ONNX \
--tokenizer-dir /path/to/Qwen3.5-0.8B-ONNX
```
### serv - 启动服务
@@ -139,7 +152,9 @@ GGUF/ONNX模型必须指定本地文件路径
**语法：**
```bash
aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] [--mmproj-path <MMPROJ_PATH>]
aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <GGUF_PATH>] \
[--mmproj-path <MMPROJ_PATH>] [--onnx-path <ONNX_PATH>] [--tokenizer-dir <TOKENIZER_DIR>] \
[--artifact-format <ARTIFACT_FORMAT>]
```
**选项：**
@@ -396,6 +411,20 @@ aha cli -m Qwen/Qwen3-VL-2B-Instruct -a 0.0.0.0 -p 8080
- **格式**: OpenAI Chat Completion 格式
- **流式支持**: 不支持
### Embeddings 接口
- **端点**: `POST /embeddings` 或 `POST /v1/embeddings`
- **功能**: 文本向量生成
- **支持模型**: Qwen3-Embedding 系列
- **格式**: OpenAI embeddings 格式
- **流式支持**: 不支持
### Rerank 接口
- **端点**: `POST /rerank` 或 `POST /v1/rerank`
- **功能**: query/document 重排打分
- **支持模型**: Qwen3-Reranker 系列
- **格式**: Rerank JSON（`results[index,relevance_score,document]`）
- **流式支持**: 不支持
### 关机接口
- **端点**: `POST /shutdown`
- **功能**: 优雅地关闭服务器
@@ -405,15 +434,19 @@ aha cli -m Qwen/Qwen3-VL-2B-Instruct -a 0.0.0.0 -p 8080
## 注意事项
1. **serv 子命令必须指定 `--weight-path`**：由于 `serv` 子命令不下载模型，必须通过 `--weight-path` 指定已下载的模型路径。
1. **GGUF/ONNX 仅支持本地路径**：请使用 `--gguf-path` 或 `--onnx-path`。自动下载管理仅适用于 safetensors 模型。
2. **下载重试机制**：默认重试 3 次，每次失败后等待 2 秒再重试。可通过 `--download-retries` 调整重试次数。
2. **制品格式选择**：`--artifact-format auto` 使用模型默认格式，也可显式指定 `safetensors|gguf|onnx`。
3. **默认保存目录**：模型默认保存到 `~/.aha/` 目录下，可通过 `--save-dir` 或 `-s` 参数自定义。
3. **tokenizer 目录**：GGUF/ONNX 若未与 tokenizer/config 同目录，请额外指定 `--tokenizer-dir`。
4. **端口占用**：启动服务前确保指定的端口未被占用，默认端口为 10100。
4. **下载重试机制**：默认重试 3 次，每次失败后等待 2 秒再重试。可通过 `--download-retries` 调整重试次数。
5. **权限问题**：如果保存到系统目录（如 `/data/models`），确保有相应的写入权限。
5. **默认保存目录**：模型默认保存到 `~/.aha/` 目录下，可通过 `--save-dir` 或 `-s` 参数自定义。
6. **端口占用**：启动服务前确保指定的端口未被占用，默认端口为 10100。
7. **权限问题**：如果保存到系统目录（如 `/data/models`），确保有相应的写入权限。
## 获取帮助
docs/supported-models.md
+70
View File
@@ -51,6 +51,23 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
| **LFM2.5-1.2B-Instruct** | 1.2B | LiquidAI/LFM2.5-1.2B-Instruct | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) |
## Embedding
| Model | Parameters | Description | License |
|-------|-----------|-------------|---------|
| **Qwen3-Embedding-0.6B** | 0.6B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-4B** | 4B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-8B** | 8B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **all-MiniLM-L6-v2** | 22M | Sentence-transformers embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/LICENSE) |
## Reranker
| Model | Parameters | Description | License |
|-------|-----------|-------------|---------|
| **Qwen3-Reranker-0.6B** | 0.6B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-4B** | 4B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-8B** | 8B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Vision & Multimodal
| Model | Parameters | Model Id | License |
@@ -71,6 +88,8 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
| **GLM-OCR** | 8 | ZhipuAI/GLM-OCR | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
GLM-OCR local artifacts: `safetensors`, `gguf`, `onnx`
## Speech Recognition (ASR)
| Model | Parameters | Language | Model Id | License |
@@ -98,6 +117,57 @@ Models are sourced from:
- [Hugging Face](https://huggingface.co) - Primary model hub
- [ModelScope](https://modelscope.cn) - Chinese model hub
## Registered Repositories (Not Runtime-Integrated Yet)
The following repositories are now cataloged for future integration, but are **not** directly runnable in current `aha` runtime yet:
### MLX / Format-Specific Variants
- Jackrong/MLX-Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-6bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-8bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-6bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-8bit
### Embedding Models
- google/embeddinggemma-300m
- ggml-org/embeddinggemma-300M-GGUF
- onnx-community/embeddinggemma-300m-ONNX
- unsloth/embeddinggemma-300m-GGUF
- onnx-community/Qwen3-Embedding-0.6B-ONNX
- Qwen/Qwen3-Embedding-0.6B-GGUF
- onnx-community/Qwen3-Embedding-4B-ONNX
- Qwen/Qwen3-Embedding-4B-GGUF
- Qwen/Qwen3-Embedding-8B-GGUF
- onnx-community/Qwen3-Embedding-8B-ONNX
- perplexity-ai/pplx-embed-v1-0.6b
- nomic-ai/nomic-embed-text-v2-moe
- nomic-ai/nomic-embed-text-v2-moe-GGUF
- jinaai/jina-embeddings-v5-text-small
- jinaai/jina-embeddings-v5-text-nano
- jinaai/jina-embeddings-v5-text-small-text-matching
- jinaai/jina-embeddings-v5-text-small-text-matching-GGUF
- sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
### Reranker Models
- BAAI/bge-reranker-v2-m3
- ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF
### ONNX Repositories
- onnx-community/GLM-OCR-ONNX
- onnx-community/Qwen3-Reranker-0.6B-ONNX
- onnx-community/Qwen3.5-2B-ONNX
- onnx-community/Qwen3.5-4B-ONNX
- onnx-community/Qwen3.5-0.8B-ONNX
- onnx-community/Qwen3-VL-2B-Instruct-ONNX
- onnx-community/ONNX_Qwen3-Embedding-0.6B
- onnx-community/Nanbeige4.1-3B-ONNX
- onnx-community/Qwen3-Embedding-8B-ONNX
- onnx-community/Qwen3-Embedding-4B-ONNX
- onnx-community/bge-reranker-v2-m3-ONNX
- onnx-community/all-MiniLM-L6-v2-ONNX
## Adding New Models
See [Development Guide](./development.md) for instructions on adding new model integrations.
docs/supported-models.zh-CN.md
+70
View File
@@ -50,6 +50,23 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
| **LFM2-1.2B** | 1.2B | LiquidAI/LFM2-1.2B | [lfm1.0](https://huggingface.co/LiquidAI/LFM2-1.2B/blob/main/LICENSE) |
| **LFM2.5-1.2B-Instruct** | 1.2B | LiquidAI/LFM2.5-1.2B-Instruct | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) |
## Embedding
| 模型 | 参数量 | 描述 | 开源协议 |
|------|--------|------|---------|
| **Qwen3-Embedding-0.6B** | 0.6B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-4B** | 4B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-8B** | 8B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **all-MiniLM-L6-v2** | 22M | sentence-transformers 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/LICENSE) |
## Reranker
| 模型 | 参数量 | 描述 | 开源协议 |
|------|--------|------|---------|
| **Qwen3-Reranker-0.6B** | 0.6B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-4B** | 4B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-8B** | 8B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## 视觉与多模态
| 模型 | 参数量 | 模型id | 开源协议 |
@@ -69,6 +86,8 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
| **DeepSeek-OCR** | 多语言 | deepseek-ai/DeepSeek-OCR <br> deepseek-ai/DeepSeek-OCR-2 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
| **GLM-OCR** | 8 | ZhipuAI/GLM-OCR | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
GLM-OCR 本地制品格式：`safetensors`、`gguf`、`onnx`
## 语音识别 (ASR)
| 模型 | 参数量 | 语言 | 模型id | 开源协议 |
@@ -97,6 +116,57 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
- [Hugging Face](https://huggingface.co) - 主模型中心
- [ModelScope](https://modelscope.cn) - 中文模型中心
## 已收录仓库（当前运行时暂未直接接入）
以下仓库已纳入项目模型目录，但当前 `aha` 运行时尚不能直接推理：
### MLX / 特定格式变体
- Jackrong/MLX-Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-6bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-8bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-6bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-8bit
### Embedding 模型
- google/embeddinggemma-300m
- ggml-org/embeddinggemma-300M-GGUF
- onnx-community/embeddinggemma-300m-ONNX
- unsloth/embeddinggemma-300m-GGUF
- onnx-community/Qwen3-Embedding-0.6B-ONNX
- Qwen/Qwen3-Embedding-0.6B-GGUF
- onnx-community/Qwen3-Embedding-4B-ONNX
- Qwen/Qwen3-Embedding-4B-GGUF
- Qwen/Qwen3-Embedding-8B-GGUF
- onnx-community/Qwen3-Embedding-8B-ONNX
- perplexity-ai/pplx-embed-v1-0.6b
- nomic-ai/nomic-embed-text-v2-moe
- nomic-ai/nomic-embed-text-v2-moe-GGUF
- jinaai/jina-embeddings-v5-text-small
- jinaai/jina-embeddings-v5-text-nano
- jinaai/jina-embeddings-v5-text-small-text-matching
- jinaai/jina-embeddings-v5-text-small-text-matching-GGUF
- sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
### Reranker 模型
- BAAI/bge-reranker-v2-m3
- ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF
### ONNX 仓库
- onnx-community/GLM-OCR-ONNX
- onnx-community/Qwen3-Reranker-0.6B-ONNX
- onnx-community/Qwen3.5-2B-ONNX
- onnx-community/Qwen3.5-4B-ONNX
- onnx-community/Qwen3.5-0.8B-ONNX
- onnx-community/Qwen3-VL-2B-Instruct-ONNX
- onnx-community/ONNX_Qwen3-Embedding-0.6B
- onnx-community/Nanbeige4.1-3B-ONNX
- onnx-community/Qwen3-Embedding-8B-ONNX
- onnx-community/Qwen3-Embedding-4B-ONNX
- onnx-community/bge-reranker-v2-m3-ONNX
- onnx-community/all-MiniLM-L6-v2-ONNX
## 添加新模型
参见 [开发指南](./development.zh-CN.md) 了解添加新模型集成的说明。