tckm.yan/aha
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update doc

Browse Source
This commit is contained in:
jhqxxx
2026-04-06 19:32:25 +08:00
parent 110a1bc5eb
commit 46ee721d74
13 changed files with 339 additions and 183 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Cargo.lock
Generated
+1 -1
View File
@@ -21,7 +21,7 @@ dependencies = [
[[package]]
name = "aha"
version = "0.2.4"
version = "0.2.5"
dependencies = [
"ahash",
"anyhow",
Cargo.toml
+2 -2
View File
@@ -1,10 +1,10 @@
[package]
name = "aha"
version = "0.2.4"
version = "0.2.5"
edition = "2024"
repository = "https://github.com/jhqxxx/aha"
license = "Apache-2.0"
description = "aha model inference library, now supports Qwen(2.5VL/3/3VL/3.5/ASR), MiniCPM4, VoxCPM/1.5, DeepSeek-OCR/2, Hunyuan-OCR, PaddleOCR-VL/1.5, RMBG2.0, GLM(ASR-Nano-2512/OCR), Fun-ASR-Nano-2512, LFM(2/2.5/2VL/2.5VL)"
description = "aha model inference library, now supports Qwen(2.5VL/3/3VL/3.5/ASR/3Embedding/3Reranker), MiniCPM4, VoxCPM/1.5, DeepSeek-OCR/2, Hunyuan-OCR, PaddleOCR-VL/1.5, RMBG2.0, GLM(ASR-Nano-2512/OCR), Fun-ASR-Nano-2512, LFM(2/2.5/2VL/2.5VL)"
[dependencies]
candle-core = { version = "0.9.2" }
README.md
+6
View File
@@ -35,6 +35,9 @@ aha is a high-performance, cross-platform AI inference engine built with Rust an
| **ASR** | GLM-ASR-Nano, Fun-ASR-Nano, Qwen3-ASR |
| **TTS** | VoxCPM, VoxCPM1.5 |
| **Image** | RMBG-2.0 (background removal) |
| **Embedding** | Qwen3-Embedding, all-MiniLM-L6-v2 |
| **Reranker** | Qwen3-Reranker |
## Why aha?
- **🚀 High-Performance Inference** - Powered by Candle framework for efficient tensor computation and model inference
@@ -46,6 +49,9 @@ aha is a high-performance, cross-platform AI inference engine built with Rust an
- **🧠 Attention Optimization** - Optional Flash Attention support for optimized long sequence processing
## Changelog
### 0.2.5 (2026-04-06)
- add qwen3-embedding/qwen3-reranker/all-minilm-l6-v2
### 2026-04-03
- CLI update: subcommand must be specified
- ChatCompletionParameters add repeat_penalty and repeat_last_n
README.zh-CN.md
+5
View File
@@ -34,6 +34,8 @@ aha 是一款基于 Rust 和 Candle 框架构建的高性能跨平台 AI 推理
| **ASR** | GLM-ASR-Nano, Fun-ASR-Nano, Qwen3-ASR |
| **TTS** | VoxCPM, VoxCPM1.5 |
| **图像** | RMBG-2.0 (背景移除) |
| **嵌入** | Qwen3-Embedding, all-MiniLM-L6-v2 |
| **重排序** | Qwen3-Reranker |
## 为什么选择 aha？
- **🚀 高性能推理** - 基于 Candle 框架，提供高效的张量计算和模型推理
@@ -46,6 +48,9 @@ aha 是一款基于 Rust 和 Candle 框架构建的高性能跨平台 AI 推理
## 更新日志
## Changelog
### 0.2.5 (2026-04-06)
- 添加 qwen3-embedding/qwen3-reranker/all-minilm-l6-v2
### 2026-04-03
- CLI 更新: 必须指定子命令
- ChatCompletionParameters 新增 repeat_penalty 和 repeat_last_n 参数
docs/api.md
+142
View File
@@ -541,6 +541,148 @@ curl http://127.0.0.1:10100/images/remove_background \
Returns the processed image in base64 PNG format.
### Embeddings
Generate text embeddings.
#### Endpoints
```
POST /embeddings
POST /v1/embeddings
```
#### Request Body
| Parameter | Type | Required | Description |
|------|------|------|------|
| `model` | string | No | Model identifier (optional, ignored - uses loaded model) |
| `input` | string or array | Yes | Text or array of texts to embed |
#### Examples
Single text：
```bash
curl http://127.0.0.1:10100/embeddings \
-H "Content-Type: application/json" \
-d '{
"input": "Hello world"
}'
```
Multiple texts：
```bash
curl http://127.0.0.1:10100/embeddings \
-H "Content-Type: application/json" \
-d '{
"input": ["Hello world", "How are you?", "Goodbye"]
}'
```
#### Response
**Success (HTTP 200):**
```json
{
"object": "list",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [0.1, 0.2, 0.3, ...]
}
],
"model": "model-name"
}
```
**Error (HTTP 400):**
```json
{
"error": "embedding input must be a string or an array of strings"
}
```
### Rerank
Re-rank a list of documents according to a query.
#### Endpoint
```
POST /rerank
POST /v1/rerank
```
#### Request Body
| Parameter | Type | Required | Description |
|------|------|------|------|
| `model` | string | No | 模型标识符 |
| `query` | string | Yes | Query text |
| `documents` | array | Yes | Array of document texts to re-rank |
| `top_n` | int | No | Return top N results (optional) |
#### Example
Basic re-ranking：
```bash
curl http://127.0.0.1:10100/rerank \
-H "Content-Type: application/json" \
-d '{
"query": "artificial intelligence",
"documents": [
"Machine learning is a form of artificial intelligence",
"Apple is a fruit",
"Deep learning belongs to the field of artificial intelligence"
]
}'
```
Limit return count：
```bash
curl http://127.0.0.1:10100/rerank \
-H "Content-Type: application/json" \
-d '{
"query": "artificial intelligence",
"documents": [
"Machine learning is a form of artificial intelligence",
"Apple is a fruit",
"Deep learning belongs to the field of artificial intelligence"
],
"top_n": 2
}'
```
#### Response
**Success (HTTP 200):**
```json
{
"object": "list",
"model": "model-name",
"results": [
{
"index": 0,
"relevance_score": 0.95,
"document": "Machine learning is a form of artificial intelligence"
},
{
"index": 2,
"relevance_score": 0.87,
"document": "Deep learning belongs to the field of artificial intelligence"
}
]
}
```
**Error (HTTP 400):**
```json
{
"error": "rerank query cannot be empty"
}
```
#### Parameter Description
| Parameter | Type | Description |
|------|------|-----|
| `model` | string | Model identifier |
| `object` | string | Fixed value: "list" |
| `results` | array | Re-ranked results array |
| `index` | int | Original document index |
| `relevance_score` | f32 | Relevance score (higher is more relevant) |
| `document` | string | Original document text |
### Graceful Shutdown
Gracefully shut down the AHA server. This endpoint initiates a graceful shutdown process that:
docs/api.zh-CN.md
+144 -1
View File
@@ -149,8 +149,10 @@ curl http://127.0.0.1:10100/models
#### 端点
```
POST /chat/completions
POST /chat/completions
POST /v1/chat/completions
```
两个端点使用相同的处理函数并返回相同的响应。`/v1/chat/completions` 路径遵循 OpenAI 的标准 API 约定。
#### 请求体
@@ -544,6 +546,147 @@ curl http://127.0.0.1:10100/images/remove_background \
以base64 PNG 格式返回处理后的图像。
### 嵌入
生成文本嵌入向量。
#### 端点
```
POST /embeddings
POST /v1/embeddings
```
#### 请求体
| 参数 | 类型 | 必需 | 描述 |
|------|------|------|------|
| `model` | string | 否 | 模型标识符 |
| `input` | string 或 array | 是 | 要嵌入的文本或文本数组 |
#### 示例
单个文本：
```bash
curl http://127.0.0.1:10100/embeddings \
-H "Content-Type: application/json" \
-d '{
"input": "Hello world"
}'
```
多个文本：
```bash
curl http://127.0.0.1:10100/embeddings \
-H "Content-Type: application/json" \
-d '{
"input": ["Hello world", "How are you?", "Goodbye"]
}'
```
#### 响应
**成功 (HTTP 200):**
```json
{
"object": "list",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [0.1, 0.2, 0.3, ...]
}
],
"model": "model-name"
}
```
**错误 (HTTP 400):**
```json
{
"error": "embedding input must be a string or an array of strings"
}
```
### 重排
对文档列表根据查询进行重新排序。
#### 端点
```
POST /rerank
POST /v1/rerank
```
#### 请求体
| 参数 | 类型 | 必需 | 描述 |
|------|------|------|------|
| `model` | string | 否 | 模型标识符 |
| `query` | string | 是 | 查询文本 |
| `documents` | array | 是 | 要重排序的文档文本数组 |
| `top_n` | int | 否 | 返回前N个结果（可选） |
#### 示例
基础重排序：
```bash
curl http://127.0.0.1:10100/rerank \
-H "Content-Type: application/json" \
-d '{
"query": "人工智能",
"documents": [
"机器学习是一种人工智能技术",
"苹果是一种水果",
"深度学习属于人工智能领域"
]
}'
```
限制返回数量：
```bash
curl http://127.0.0.1:10100/rerank \
-H "Content-Type: application/json" \
-d '{
"query": "人工智能",
"documents": [
"机器学习是一种人工智能技术",
"苹果是一种水果",
"深度学习属于人工智能领域"
],
"top_n": 2
}'
```
#### 响应
**成功 (HTTP 200):**
```json
{
"object": "list",
"model": "model-name",
"results": [
{
"index": 0,
"relevance_score": 0.95,
"document": "机器学习是一种人工智能技术"
},
{
"index": 2,
"relevance_score": 0.87,
"document": "深度学习属于人工智能领域"
}
]
}
```
**错误 (HTTP 400):**
```json
{
"error": "rerank query cannot be empty"
}
```
#### 字段说明
| 字段 | 类型 | 描述 |
|------|------|-----|
| `model` | string | 模型标识符 |
| `object` | string | 固定值："list" |
| `results` | array | 重排序结果数组 |
| `index` | int | 原始文档索引 |
| `relevance_score` | f32 | 相关性分数（越高越相关） |
| `document` | string | 原始文档文本 |
### 优雅关机
docs/changelog.md
+3
View File
@@ -5,6 +5,9 @@ All notable changes to aha will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### 0.2.5 (2026-04-06)
- add qwen3-embedding/qwen3-reranker/all-minilm-l6-v2
### 2026-04-03
- CLI update: subcommand must be specified
- ChatCompletionParameters add repeat_penalty and repeat_last_n
docs/changelog.zh-CN.md
+3
View File
@@ -5,6 +5,9 @@
格式基于 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.0.0/)，
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/spec/v2.0.0.html)。
### 0.2.5 (2026-04-06)
- 添加 qwen3-embedding/qwen3-reranker/all-minilm-l6-v2
### 2026-04-03
- CLI 更新: 必须指定子命令
- ChatCompletionParameters 新增 repeat_penalty 和 repeat_last_n 参数
docs/cli.md
+1 -26
View File
@@ -67,10 +67,6 @@ aha cli -m Qwen/Qwen3-VL-2B-Instruct --weight-path /path/to/model
# use gguf-path and mmproj-path
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# run service with ONNX artifact
aha cli -m qwen3-embedding-0.6b --artifact-format onnx \
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
```
### run - Direct model inference
@@ -138,10 +134,6 @@ aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
aha run -m qwen3.5-gguf -i 提取图片中的文本 -i https://ai.bdstatic.com/file/C56CC9B274CF460CA33
63E59ECD94423 --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# Qwen3.5 ONNX text-only generation
aha run -m qwen3.5-0.8b -i "hello" --artifact-format onnx \
--onnx-path /path/to/Qwen3.5-0.8B-ONNX \
--tokenizer-dir /path/to/Qwen3.5-0.8B-ONNX
```
@@ -168,8 +160,7 @@ aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <G
| `--gguf-path <GGUF_PATH>` | Local GGUF model weight path（required when using GGUF models） | - |
| `--mmproj-path <MMPROJ_PATH>` | Local mmproj GGUF weight path（optional，If not specified, the module will not be loaded） | - |
| `--onnx-path <ONNX_PATH>` | Local ONNX model directory/file path（required when using ONNX models） | - |
| `--tokenizer-dir <TOKENIZER_DIR>` | Tokenizer/config directory for GGUF/ONNX | - |
| `--artifact-format <ARTIFACT_FORMAT>` | Artifact format (`auto|safetensors|gguf|onnx`) | auto |
| `--config-path <ONNX_PATH>` | extra config path for gguf/onnx | - |
**Examples:**
@@ -432,22 +423,6 @@ After the service starts, the following API endpoints are available:
- **Format**: JSON response
## Notes
1. **Local-path rule for GGUF/ONNX**: GGUF and ONNX artifacts are local-path only; use `--gguf-path` or `--onnx-path`. Remote download management is only for safetensors models.
2. **Artifact selection**: `--artifact-format auto` uses model default; you can force `safetensors|gguf|onnx` explicitly.
3. **Tokenizer directory**: For GGUF/ONNX, if tokenizer files are not colocated with model files, set `--tokenizer-dir`.
4. **Download retry mechanism**: By default, retries 3 times, waiting 2 seconds after each failure before retrying. You can adjust the retry count with `--download-retries`.
5. **Default save directory**: Models are saved to `~/.aha/` directory by default, which can be customized via `--save-dir` or `-s` parameter.
6. **Port occupation**: Ensure the specified port is not occupied before starting the service. The default port is 10100.
7. **Permission issues**: If saving to a system directory (such as `/data/models`), ensure you have the corresponding write permissions.
## Getting Help
```bash
docs/cli.zh-CN.md
-24
View File
@@ -67,10 +67,6 @@ aha cli -m Qwen/Qwen3-VL-2B-Instruct --weight-path /path/to/model
# 指定gguf-path和mmproj-path
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# 使用 ONNX 模型启动服务
aha cli -m qwen3-embedding-0.6b --artifact-format onnx \
--onnx-path /path/to/Qwen3-Embedding-0.6B-ONNX \
--tokenizer-dir /path/to/Qwen3-Embedding-0.6B-ONNX
```
### run - 直接模型推理
@@ -138,10 +134,6 @@ aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
aha run -m qwen3.5-gguf -i 提取图片中的文本 -i https://ai.bdstatic.com/file/C56CC9B274CF460CA33
63E59ECD94423 --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
# Qwen3.5 ONNX 文本生成（text-only）
aha run -m qwen3.5-0.8b -i "你好" --artifact-format onnx \
--onnx-path /path/to/Qwen3.5-0.8B-ONNX \
--tokenizer-dir /path/to/Qwen3.5-0.8B-ONNX
```
### serv - 启动服务
@@ -432,22 +424,6 @@ aha cli -m Qwen/Qwen3-VL-2B-Instruct -a 0.0.0.0 -p 8080
- **格式**: JSON 响应
## 注意事项
1. **GGUF/ONNX 仅支持本地路径**：请使用 `--gguf-path` 或 `--onnx-path`。自动下载管理仅适用于 safetensors 模型。
2. **制品格式选择**：`--artifact-format auto` 使用模型默认格式，也可显式指定 `safetensors|gguf|onnx`。
3. **tokenizer 目录**：GGUF/ONNX 若未与 tokenizer/config 同目录，请额外指定 `--tokenizer-dir`。
4. **下载重试机制**：默认重试 3 次，每次失败后等待 2 秒再重试。可通过 `--download-retries` 调整重试次数。
5. **默认保存目录**：模型默认保存到 `~/.aha/` 目录下，可通过 `--save-dir` 或 `-s` 参数自定义。
6. **端口占用**：启动服务前确保指定的端口未被占用，默认端口为 10100。
7. **权限问题**：如果保存到系统目录（如 `/data/models`），确保有相应的写入权限。
## 获取帮助
```bash
docs/supported-models.md
+15 -65
View File
@@ -7,6 +7,7 @@ Available models:
Model ID Owner type Download
--------------------------------------------------------------------------------
sentence-transformers/all-MiniLM-L6-v2 sentence-transformers embedding ✔
LiquidAI/LFM2-1.2B LiquidAI llm ✔
LiquidAI/LFM2.5-1.2B-Instruct LiquidAI llm ✔
LiquidAI/LFM2.5-VL-1.6B LiquidAI vlm ✔
@@ -14,9 +15,9 @@ LiquidAI/LFM2-VL-1.6B LiquidAI vlm ✔
OpenBMB/MiniCPM4-0.5B OpenBMB llm ✔
Qwen/Qwen2.5-VL-3B-Instruct Qwen vlm ✔
Qwen/Qwen2.5-VL-7B-Instruct Qwen vlm
Qwen/Qwen3-0.6B Qwen llm ✔
Qwen/Qwen3-1.7B Qwen llm
Qwen/Qwen3-4B Qwen llm
Qwen/Qwen3-0.6B Qwen llm ✔
Qwen/Qwen3-1.7B Qwen llm ✔
Qwen/Qwen3-4B Qwen llm ✔
Qwen/Qwen3.5-0.8B Qwen vlm ✔
Qwen/Qwen3.5-2B Qwen vlm
Qwen/Qwen3.5-4B Qwen vlm
@@ -24,6 +25,12 @@ Qwen/Qwen3.5-9B Qwen vlm
qwen3.5-gguf none vlm
Qwen/Qwen3-ASR-0.6B Qwen asr ✔
Qwen/Qwen3-ASR-1.7B Qwen asr
Qwen/Qwen3-Embedding-0.6B Qwen embedding ✔
Qwen/Qwen3-Embedding-4B Qwen embedding
Qwen/Qwen3-Embedding-8B Qwen embedding
Qwen/Qwen3-Reranker-0.6B Qwen reranker ✔
Qwen/Qwen3-Reranker-4B Qwen reranker
Qwen/Qwen3-Reranker-8B Qwen reranker
Qwen/Qwen3-VL-2B-Instruct Qwen vlm ✔
Qwen/Qwen3-VL-4B-Instruct Qwen vlm
Qwen/Qwen3-VL-8B-Instruct Qwen vlm
@@ -53,20 +60,16 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
## Embedding
| Model | Parameters | Description | License |
| Model | Parameters | Model Id | License |
|-------|-----------|-------------|---------|
| **Qwen3-Embedding-0.6B** | 0.6B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-4B** | 4B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-8B** | 8B | Text embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **all-MiniLM-L6-v2** | 22M | Sentence-transformers embedding (safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/LICENSE) |
| **Qwen3-Embedding** | 0.6B <br> 4B <br> 8B| Qwen/Qwen3-Embedding-0.6B <br> Qwen/Qwen3-Embedding-4B <br> Qwen/Qwen3-Embedding-8B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **all-MiniLM-L6-v2** | 91M | sentence-transformers/all-MiniLM-L6-v2 | [Apache 2.0](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/LICENSE) |
## Reranker
| Model | Parameters | Description | License |
| Model | Parameters | Model Id | License |
|-------|-----------|-------------|---------|
| **Qwen3-Reranker-0.6B** | 0.6B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-4B** | 4B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-8B** | 8B | Text reranking (embedding-similarity baseline, safetensors / gguf / onnx) | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker** | 0.6B <br> 4B <br> 8B| Qwen/Qwen3-Reranker-0.6B <br> Qwen/Qwen3-Reranker-4B <br> Qwen/Qwen3-Reranker-8B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Vision & Multimodal
@@ -88,8 +91,6 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
| **GLM-OCR** | 8 | ZhipuAI/GLM-OCR | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
GLM-OCR local artifacts: `safetensors`, `gguf`, `onnx`
## Speech Recognition (ASR)
| Model | Parameters | Language | Model Id | License |
@@ -117,57 +118,6 @@ Models are sourced from:
- [Hugging Face](https://huggingface.co) - Primary model hub
- [ModelScope](https://modelscope.cn) - Chinese model hub
## Registered Repositories (Not Runtime-Integrated Yet)
The following repositories are now cataloged for future integration, but are **not** directly runnable in current `aha` runtime yet:
### MLX / Format-Specific Variants
- Jackrong/MLX-Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-6bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-8bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-6bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-8bit
### Embedding Models
- google/embeddinggemma-300m
- ggml-org/embeddinggemma-300M-GGUF
- onnx-community/embeddinggemma-300m-ONNX
- unsloth/embeddinggemma-300m-GGUF
- onnx-community/Qwen3-Embedding-0.6B-ONNX
- Qwen/Qwen3-Embedding-0.6B-GGUF
- onnx-community/Qwen3-Embedding-4B-ONNX
- Qwen/Qwen3-Embedding-4B-GGUF
- Qwen/Qwen3-Embedding-8B-GGUF
- onnx-community/Qwen3-Embedding-8B-ONNX
- perplexity-ai/pplx-embed-v1-0.6b
- nomic-ai/nomic-embed-text-v2-moe
- nomic-ai/nomic-embed-text-v2-moe-GGUF
- jinaai/jina-embeddings-v5-text-small
- jinaai/jina-embeddings-v5-text-nano
- jinaai/jina-embeddings-v5-text-small-text-matching
- jinaai/jina-embeddings-v5-text-small-text-matching-GGUF
- sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
### Reranker Models
- BAAI/bge-reranker-v2-m3
- ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF
### ONNX Repositories
- onnx-community/GLM-OCR-ONNX
- onnx-community/Qwen3-Reranker-0.6B-ONNX
- onnx-community/Qwen3.5-2B-ONNX
- onnx-community/Qwen3.5-4B-ONNX
- onnx-community/Qwen3.5-0.8B-ONNX
- onnx-community/Qwen3-VL-2B-Instruct-ONNX
- onnx-community/ONNX_Qwen3-Embedding-0.6B
- onnx-community/Nanbeige4.1-3B-ONNX
- onnx-community/Qwen3-Embedding-8B-ONNX
- onnx-community/Qwen3-Embedding-4B-ONNX
- onnx-community/bge-reranker-v2-m3-ONNX
- onnx-community/all-MiniLM-L6-v2-ONNX
## Adding New Models
See [Development Guide](./development.md) for instructions on adding new model integrations.
docs/supported-models.zh-CN.md
+15 -64
View File
@@ -7,6 +7,7 @@ Available models:
Model ID Owner type Download
--------------------------------------------------------------------------------
sentence-transformers/all-MiniLM-L6-v2 sentence-transformers embedding ✔
LiquidAI/LFM2-1.2B LiquidAI llm ✔
LiquidAI/LFM2.5-1.2B-Instruct LiquidAI llm ✔
LiquidAI/LFM2.5-VL-1.6B LiquidAI vlm ✔
@@ -14,9 +15,9 @@ LiquidAI/LFM2-VL-1.6B LiquidAI vlm ✔
OpenBMB/MiniCPM4-0.5B OpenBMB llm ✔
Qwen/Qwen2.5-VL-3B-Instruct Qwen vlm ✔
Qwen/Qwen2.5-VL-7B-Instruct Qwen vlm
Qwen/Qwen3-0.6B Qwen llm ✔
Qwen/Qwen3-1.7B Qwen llm
Qwen/Qwen3-4B Qwen llm
Qwen/Qwen3-0.6B Qwen llm ✔
Qwen/Qwen3-1.7B Qwen llm ✔
Qwen/Qwen3-4B Qwen llm ✔
Qwen/Qwen3.5-0.8B Qwen vlm ✔
Qwen/Qwen3.5-2B Qwen vlm
Qwen/Qwen3.5-4B Qwen vlm
@@ -24,6 +25,12 @@ Qwen/Qwen3.5-9B Qwen vlm
qwen3.5-gguf none vlm
Qwen/Qwen3-ASR-0.6B Qwen asr ✔
Qwen/Qwen3-ASR-1.7B Qwen asr
Qwen/Qwen3-Embedding-0.6B Qwen embedding ✔
Qwen/Qwen3-Embedding-4B Qwen embedding
Qwen/Qwen3-Embedding-8B Qwen embedding
Qwen/Qwen3-Reranker-0.6B Qwen reranker ✔
Qwen/Qwen3-Reranker-4B Qwen reranker
Qwen/Qwen3-Reranker-8B Qwen reranker
Qwen/Qwen3-VL-2B-Instruct Qwen vlm ✔
Qwen/Qwen3-VL-4B-Instruct Qwen vlm
Qwen/Qwen3-VL-8B-Instruct Qwen vlm
@@ -52,20 +59,16 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
## Embedding
| 模型 | 参数量 | 描述 | 开源协议 |
| 模型 | 参数量 | 模型id | 开源协议 |
|------|--------|------|---------|
| **Qwen3-Embedding-0.6B** | 0.6B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-4B** | 4B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Embedding-8B** | 8B | 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **all-MiniLM-L6-v2** | 22M | sentence-transformers 文本向量（safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/LICENSE) |
| **Qwen3-Embedding** | 0.6B <br> 4B <br> 8B| Qwen/Qwen3-Embedding-0.6B <br> Qwen/Qwen3-Embedding-4B <br> Qwen/Qwen3-Embedding-8B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **all-MiniLM-L6-v2** | 91M | sentence-transformers/all-MiniLM-L6-v2 | [Apache 2.0](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/LICENSE) |
## Reranker
| 模型 | 参数量 | 描述 | 开源协议 |
| 模型 | 参数量 | 模型id | 开源协议 |
|------|--------|------|---------|
| **Qwen3-Reranker-0.6B** | 0.6B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-4B** | 4B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker-8B** | 8B | 文本重排（embedding-similarity 基线，safetensors / gguf / onnx） | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-Reranker** | 0.6B <br> 4B <br> 8B| Qwen/Qwen3-Reranker-0.6B <br> Qwen/Qwen3-Reranker-4B <br> Qwen/Qwen3-Reranker-8B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## 视觉与多模态
@@ -86,7 +89,6 @@ ZhipuAI/GLM-OCR ZhipuAI ocr ✔
| **DeepSeek-OCR** | 多语言 | deepseek-ai/DeepSeek-OCR <br> deepseek-ai/DeepSeek-OCR-2 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
| **GLM-OCR** | 8 | ZhipuAI/GLM-OCR | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
GLM-OCR 本地制品格式：`safetensors`、`gguf`、`onnx`
## 语音识别 (ASR)
@@ -116,57 +118,6 @@ GLM-OCR 本地制品格式：`safetensors`、`gguf`、`onnx`
- [Hugging Face](https://huggingface.co) - 主模型中心
- [ModelScope](https://modelscope.cn) - 中文模型中心
## 已收录仓库（当前运行时暂未直接接入）
以下仓库已纳入项目模型目录，但当前 `aha` 运行时尚不能直接推理：
### MLX / 特定格式变体
- Jackrong/MLX-Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-6bit
- Jackrong/MLX-Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-8bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-4bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-6bit
- Jackrong/MLX-Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2-8bit
### Embedding 模型
- google/embeddinggemma-300m
- ggml-org/embeddinggemma-300M-GGUF
- onnx-community/embeddinggemma-300m-ONNX
- unsloth/embeddinggemma-300m-GGUF
- onnx-community/Qwen3-Embedding-0.6B-ONNX
- Qwen/Qwen3-Embedding-0.6B-GGUF
- onnx-community/Qwen3-Embedding-4B-ONNX
- Qwen/Qwen3-Embedding-4B-GGUF
- Qwen/Qwen3-Embedding-8B-GGUF
- onnx-community/Qwen3-Embedding-8B-ONNX
- perplexity-ai/pplx-embed-v1-0.6b
- nomic-ai/nomic-embed-text-v2-moe
- nomic-ai/nomic-embed-text-v2-moe-GGUF
- jinaai/jina-embeddings-v5-text-small
- jinaai/jina-embeddings-v5-text-nano
- jinaai/jina-embeddings-v5-text-small-text-matching
- jinaai/jina-embeddings-v5-text-small-text-matching-GGUF
- sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
### Reranker 模型
- BAAI/bge-reranker-v2-m3
- ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF
### ONNX 仓库
- onnx-community/GLM-OCR-ONNX
- onnx-community/Qwen3-Reranker-0.6B-ONNX
- onnx-community/Qwen3.5-2B-ONNX
- onnx-community/Qwen3.5-4B-ONNX
- onnx-community/Qwen3.5-0.8B-ONNX
- onnx-community/Qwen3-VL-2B-Instruct-ONNX
- onnx-community/ONNX_Qwen3-Embedding-0.6B
- onnx-community/Nanbeige4.1-3B-ONNX
- onnx-community/Qwen3-Embedding-8B-ONNX
- onnx-community/Qwen3-Embedding-4B-ONNX
- onnx-community/bge-reranker-v2-m3-ONNX
- onnx-community/all-MiniLM-L6-v2-ONNX
## 添加新模型
参见 [开发指南](./development.zh-CN.md) 了解添加新模型集成的说明。
src/lib.rs
+2
View File
@@ -5,3 +5,5 @@ pub mod params;
pub mod position_embed;
pub mod tokenizer;
pub mod utils;
pub use candle_core::Device;