tckm.yan/aha
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update WhichModel enum

Browse Source
This commit is contained in:
jhqxxx
2026-03-31 18:45:01 +08:00
parent c5551e7fb8
commit 741869d6dd
44 changed files with 691 additions and 812 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+11 -8
View File
@@ -26,6 +26,9 @@ aha is a high-performance, cross-platform AI inference engine built with Rust an
## Changelog ## Changelog
### 2026-03-31 ### 2026-03-31
- aha model name use modelscope id replace
- update WhichModel
- Usage add time info
- dependencies delete aha_openai_dive,chrono - dependencies delete aha_openai_dive,chrono
### v0.2.5 (2026-03-30) ### v0.2.5 (2026-03-30)
@@ -86,23 +89,23 @@ cargo build --release --features ffmpeg
aha list aha list
# Download model only # Download model only
aha download -m qwen3asr-0.6b aha download -m Qwen/Qwen3-ASR-0.6B
# Download model and start service # Download model and start service
aha -m qwen3asr-0.6b aha -m Qwen/Qwen3-ASR-0.6B
# Run inference directly (without starting service) # Run inference directly (without starting service)
aha run -m qwen3asr-0.6b -i "audio.wav" aha run -m Qwen/Qwen3-ASR-0.6B -i "audio.wav"
# Start service only (model already downloaded) # Start service only (model already downloaded)
aha serv -m qwen3asr-0.6b -p 10100 aha serv -m Qwen/Qwen3-ASR-0.6B -p 10100
``` ```
### Chat ### Chat
```bash ```bash
aha serv -m qwen3-0.6b -p 10100 aha serv -m Qwen/Qwen3-0.6B -p 10100
``` ```
Then use the unified (OpenAI-compatible) API: Then use the unified (OpenAI-compatible) API:
@@ -111,7 +114,7 @@ Then use the unified (OpenAI-compatible) API:
curl http://localhost:10100/chat/completions \ curl http://localhost:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [{"role": "user", "content": "Hello!"}], "messages": [{"role": "user", "content": "Hello!"}],
"stream": false "stream": false
} }
@@ -122,8 +125,8 @@ curl http://localhost:10100/chat/completions \
| Category | Models | | Category | Models |
|----------|--------| |----------|--------|
| **Text** | Qwen3, MiniCPM4, <br> LFM2-1.2B, LFM2.5-1.2B-Instruct | | **Text** | Qwen3, MiniCPM4, <br> LFM2, LFM2.5 |
| **Vision** | Qwen2.5-VL, Qwen3-VL, Qwen3.5, <br> LFM2.5-VL-1.6B, LFM2-VL-1.6B | | **Vision** | Qwen2.5-VL, Qwen3-VL, Qwen3.5, <br> LFM2.5-VL, LFM2-VL |
| **OCR** | DeepSeek-OCR, DeepSeek-OCR-2 , <br> PaddleOCR-VL, PaddleOCR-VL1.5, <br> Hunyuan-OCR, GLM-OCR | | **OCR** | DeepSeek-OCR, DeepSeek-OCR-2 , <br> PaddleOCR-VL, PaddleOCR-VL1.5, <br> Hunyuan-OCR, GLM-OCR |
| **ASR** | GLM-ASR-Nano, Fun-ASR-Nano, Qwen3-ASR | | **ASR** | GLM-ASR-Nano, Fun-ASR-Nano, Qwen3-ASR |
| **Audio** | VoxCPM, VoxCPM1.5 | | **Audio** | VoxCPM, VoxCPM1.5 |
README.zh-CN.md
+13 -10
View File
@@ -25,8 +25,12 @@
aha 是一款基于 Rust 和 Candle 框架构建的高性能跨平台 AI 推理引擎。将最先进的 AI 模型带到您的本地机器——无需 API 密钥，无需云依赖，纯粹、快速的 AI，直接在您的硬件上运行。 aha 是一款基于 Rust 和 Candle 框架构建的高性能跨平台 AI 推理引擎。将最先进的 AI 模型带到您的本地机器——无需 API 密钥，无需云依赖，纯粹、快速的 AI，直接在您的硬件上运行。
## 更新日志 ## 更新日志
### 2026-03-31 ### 2026-03-31
- 删除 aha_openai_dive,chrono 依赖 - aha模型名称使用 modelscope id 替换
- 更新 WhichModel 枚举
- Usage 增加时间信息
- 删除 aha_openai_dive, chrono 依赖
### v0.2.5 (2026-03-30) ### v0.2.5 (2026-03-30)
- 新增 LFM2.5VL-1.6B - 新增 LFM2.5VL-1.6B
@@ -50,7 +54,6 @@ aha 是一款基于 Rust 和 Candle 框架构建的高性能跨平台 AI 推理
### 2026-03-16 ### 2026-03-16
- 增加 Qwen3.5 mmproj - 增加 Qwen3.5 mmproj
**[查看完整更新日志](docs/changelog.zh-CN.md)** → **[查看完整更新日志](docs/changelog.zh-CN.md)** →
## 快速开始 ## 快速开始
@@ -87,23 +90,23 @@ cargo build --release --features ffmpeg
aha list aha list
# 仅下载模型 # 仅下载模型
aha download -m qwen3asr-0.6b aha download -m Qwen/Qwen3-ASR-0.6B
# 下载模型并启动服务 # 下载模型并启动服务
aha -m qwen3asr-0.6b aha -m Qwen/Qwen3-ASR-0.6B
# 直接运行推理（无需启动服务） # 直接运行推理（无需启动服务）
aha run -m qwen3asr-0.6b -i "audio.wav" aha run -m Qwen/Qwen3-ASR-0.6B -i "audio.wav"
# 仅启动服务（模型已下载） # 仅启动服务（模型已下载）
aha serv -m qwen3asr-0.6b -p 10100 aha serv -m Qwen/Qwen3-ASR-0.6B -p 10100
``` ```
### 对话 ### 对话
```bash ```bash
aha serv -m qwen3-0.6b -p 10100 aha serv -m Qwen/Qwen3-0.6B -p 10100
``` ```
然后使用统一(兼容 OpenAI)的 API： 然后使用统一(兼容 OpenAI)的 API：
@@ -112,7 +115,7 @@ aha serv -m qwen3-0.6b -p 10100
curl http://localhost:10100/chat/completions \ curl http://localhost:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [{"role": "user", "content": "你好！"}], "messages": [{"role": "user", "content": "你好！"}],
"stream": false "stream": false
}' }'
@@ -123,8 +126,8 @@ curl http://localhost:10100/chat/completions \
| 类别 | 模型 | | 类别 | 模型 |
|------|------| |------|------|
| **文本** | Qwen3, MiniCPM4, <br> LFM2-1.2B, LFM2.5-1.2B-Instruct | | **文本** | Qwen3, MiniCPM4, <br> LFM2, LFM2.5 |
| **视觉** | Qwen2.5-VL, Qwen3-VL, Qwen3.5 <br> LFM2.5-VL-1.6B, LFM2-VL-1.6B | | **视觉** | Qwen2.5-VL, Qwen3-VL, Qwen3.5 <br> LFM2.5-VL, LFM2-VL |
| **OCR** | DeepSeek-OCR, DeepSeek-OCR-2 , <br> PaddleOCR-VL, PaddleOCR-VL1.5, <br> Hunyuan-OCR, GLM-OCR | | **OCR** | DeepSeek-OCR, DeepSeek-OCR-2 , <br> PaddleOCR-VL, PaddleOCR-VL1.5, <br> Hunyuan-OCR, GLM-OCR |
| **ASR** | GLM-ASR-Nano, Fun-ASR-Nano,Qwen3-ASR | | **ASR** | GLM-ASR-Nano, Fun-ASR-Nano,Qwen3-ASR |
| **音频** | VoxCPM, VoxCPM1.5 | | **音频** | VoxCPM, VoxCPM1.5 |
docs/api.md
+22 -44
View File
@@ -15,7 +15,7 @@ http://127.0.0.1:10100
You can customize this when starting the service: You can customize this when starting the service:
```bash ```bash
aha cli -m qwen3-0.6b -a 0.0.0.0 -p 8080 aha cli -m Qwen/Qwen3-0.6B -a 0.0.0.0 -p 8080
``` ```
### Authentication ### Authentication
@@ -109,7 +109,7 @@ GET /models
"object": "list", "object": "list",
"data": [ "data": [
{ {
"id": "qwen3-0.6b", "id": "Qwen/Qwen3-0.6B",
"object": "model", "object": "model",
"created": null, "created": null,
"owned_by": "Qwen" "owned_by": "Qwen"
@@ -132,7 +132,7 @@ GET /models
|-------|------|-------------| |-------|------|-------------|
| `object` | string | Fixed value: "list" | | `object` | string | Fixed value: "list" |
| `data` | array | Array of model objects (currently contains one loaded model) | | `data` | array | Array of model objects (currently contains one loaded model) |
| `id` | string | Model identifier in kebab-case (e.g., "qwen3-0.6b") | | `id` | string | Model identifier in kebab-case (e.g., "Qwen/Qwen3-0.6B") |
| `object` | string | Fixed value: "model" | | `object` | string | Fixed value: "model" |
| `created` | integer\|null | Unix timestamp (currently null) | | `created` | integer\|null | Unix timestamp (currently null) |
| `owned_by` | string | Model owner/organization name | | `owned_by` | string | Model owner/organization name |
@@ -156,7 +156,7 @@ POST /chat/completions
| Parameter | Type | Required | Description | | Parameter | Type | Required | Description |
|-----------|------|----------|-------------| |-----------|------|----------|-------------|
| `model` | string | Yes | Model identifier (e.g., "qwen3-0.6b") | | `model` | string | Yes | Model identifier (e.g., "Qwen/Qwen3-0.6B") |
| `messages` | array | Yes | Array of message objects | | `messages` | array | Yes | Array of message objects |
| `temperature` | number | No | Sampling temperature (0-2, default: 1) | | `temperature` | number | No | Sampling temperature (0-2, default: 1) |
| `top_p` | number | No | Nucleus sampling (0-1, default: 1) | | `top_p` | number | No | Nucleus sampling (0-1, default: 1) |
@@ -197,7 +197,7 @@ Supported content types:
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "user", "content": "Hello!"} {"role": "user", "content": "Hello!"}
] ]
@@ -210,7 +210,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "system", "content": "You are a helpful assistant."}, {"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain Rust in one sentence."} {"role": "user", "content": "Explain Rust in one sentence."}
@@ -226,7 +226,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3vl-2b", "model": "Qwen/Qwen3-VL-2B-Instruct",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -245,7 +245,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "deepseek-ocr", "model": "deepseek-ai/DeepSeek-OCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -264,7 +264,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "glm-asr-nano-2512", "model": "ZhipuAI/GLM-ASR-Nano-2512",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -286,7 +286,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "user", "content": "Tell me a story"} {"role": "user", "content": "Tell me a story"}
], ],
@@ -312,7 +312,7 @@ data: [DONE]
"id": "chatcmpl-123", "id": "chatcmpl-123",
"object": "chat.completion", "object": "chat.completion",
"created": 1677652288, "created": 1677652288,
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"choices": [ "choices": [
{ {
"index": 0, "index": 0,
@@ -329,13 +329,6 @@ data: [DONE]
} }
``` ```
#### Supported Models
- Text: `qwen3-0.6b`, `minicpm4-0.5b`
- Vision: `qwen2.5vl-3b`, `qwen2.5vl-7b`, `qwen3vl-2b`, `qwen3vl-4b`, `qwen3vl-8b`, `qwen3vl-32b`
- OCR: `deepseek-ocr`, `hunyuan-ocr`, `paddleocr-vl`
- ASR: `glm-asr-nano-2512`, `fun-asr-nano-2512`, `qwen3asr-0.6b`, `qwen3asr-1.7b`
### Audio Speech ### Audio Speech
Generate speech from text (Text-to-Speech). Generate speech from text (Text-to-Speech).
@@ -349,7 +342,7 @@ POST /audio/speech
| Parameter | Type | Required | Description | | Parameter | Type | Required | Description |
|-----------|------|----------|-------------| |-----------|------|----------|-------------|
| `model` | string | Yes | Model identifier (e.g., "voxcpm1.5") | | `model` | string | Yes | Model identifier (e.g., "OpenBMB/VoxCPM-1.5") |
| `messages` | array | Yes | Array of message objects | | `messages` | array | Yes | Array of message objects |
#### Example #### Example
@@ -358,7 +351,7 @@ POST /audio/speech
curl http://127.0.0.1:10100/audio/speech \ curl http://127.0.0.1:10100/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "voxcpm1.5", "model": "OpenBMB/VoxCPM-1.5",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -375,10 +368,6 @@ curl http://127.0.0.1:10100/audio/speech \
Returns audio data in base64 WAV format. Returns audio data in base64 WAV format.
#### Supported Models
- `voxcpm`, `voxcpm1.5`
### Audio Transcriptions ### Audio Transcriptions
Transcribe audio files to text (Automatic Speech Recognition). Transcribe audio files to text (Automatic Speech Recognition).
@@ -433,7 +422,7 @@ Both endpoints use the same handler and return identical responses. The `/v1/aud
curl -X POST http://127.0.0.1:10100/audio/transcriptions \ curl -X POST http://127.0.0.1:10100/audio/transcriptions \
-H "Authorization: Bearer NO_NEED" \ -H "Authorization: Bearer NO_NEED" \
-F file="@./audio.wav" \ -F file="@./audio.wav" \
-F model="qwen3asr-0.6b" -F model="Qwen/Qwen3-ASR-0.6B"
``` ```
**With language specification:** **With language specification:**
@@ -442,7 +431,7 @@ curl -X POST http://127.0.0.1:10100/audio/transcriptions \
curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \ curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \
-H "Authorization: Bearer NO_NEED" \ -H "Authorization: Bearer NO_NEED" \
-F file="@./chinese_audio.wav" \ -F file="@./chinese_audio.wav" \
-F model="qwen3asr-0.6b" \ -F model="Qwen/Qwen3-ASR-0.6B" \
-F language="zh" -F language="zh"
``` ```
@@ -452,7 +441,7 @@ curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \
curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \ curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \
-H "Authorization: Bearer NO_NEED" \ -H "Authorization: Bearer NO_NEED" \
-F file="@./audio.wav" \ -F file="@./audio.wav" \
-F model="qwen3asr-0.6b" \ -F model="Qwen/Qwen3-ASR-0.6B" \
-F temperature="0.0" -F temperature="0.0"
``` ```
@@ -490,13 +479,6 @@ curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \
} }
``` ```
#### Supported Models
- `qwen3asr-0.6b`
- `qwen3asr-1.7b`
- `glm-asr-nano-2512`
- `fun-asr-nano-2512`
#### File Upload Limit #### File Upload Limit
Maximum audio file size: 100 MB Maximum audio file size: 100 MB
@@ -514,7 +496,7 @@ POST /images/remove_background
| Parameter | Type | Required | Description | | Parameter | Type | Required | Description |
|-----------|------|----------|-------------| |-----------|------|----------|-------------|
| `model` | string | Yes | Model identifier (e.g., "rmbg2.0") | | `model` | string | Yes | Model identifier (e.g., "AI-ModelScope/RMBG-2.0") |
| `messages` | array | Yes | Array of message objects | | `messages` | array | Yes | Array of message objects |
#### Example #### Example
@@ -525,7 +507,7 @@ POST /images/remove_background
curl http://127.0.0.1:10100/images/remove_background \ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "AI-ModelScope/RMBG-2.0",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -543,7 +525,7 @@ curl http://127.0.0.1:10100/images/remove_background \
curl http://127.0.0.1:10100/images/remove_background \ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "AI-ModelScope/RMBG-2.0",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -559,10 +541,6 @@ curl http://127.0.0.1:10100/images/remove_background \
Returns the processed image in base64 PNG format. Returns the processed image in base64 PNG format.
#### Supported Models
- `rmbg2.0`
### Graceful Shutdown ### Graceful Shutdown
Gracefully shut down the AHA server. This endpoint initiates a graceful shutdown process that: Gracefully shut down the AHA server. This endpoint initiates a graceful shutdown process that:
@@ -605,7 +583,7 @@ When remote shutdown is not allowed:
By default, the shutdown endpoint only allows requests from localhost (127.0.0.1). To enable remote shutdown, start the server with the `--allow-remote-shutdown` flag: By default, the shutdown endpoint only allows requests from localhost (127.0.0.1). To enable remote shutdown, start the server with the `--allow-remote-shutdown` flag:
```bash ```bash
aha serv -m qwen3-0.6b --allow-remote-shutdown aha serv -m Qwen/Qwen3-0.6B --allow-remote-shutdown
``` ```
**Warning:** Enabling remote shutdown is not recommended for production use unless properly secured. **Warning:** Enabling remote shutdown is not recommended for production use unless properly secured.
@@ -671,7 +649,7 @@ client = OpenAI(
) )
response = client.chat.completions.create( response = client.chat.completions.create(
model="qwen3-0.6b", model="Qwen/Qwen3-0.6B",
messages=[ messages=[
{"role": "user", "content": "Hello!"} {"role": "user", "content": "Hello!"}
] ]
@@ -691,7 +669,7 @@ const client = new OpenAI({
}); });
const response = await client.chat.completions.create({ const response = await client.chat.completions.create({
model: 'qwen3-0.6b', model: 'Qwen/Qwen3-0.6B',
messages: [{ role: 'user', content: 'Hello!' }] messages: [{ role: 'user', content: 'Hello!' }]
}); });
docs/api.zh-CN.md
+22 -40
View File
@@ -15,7 +15,7 @@ http://127.0.0.1:10100
您可以在启动服务时自定义： 您可以在启动服务时自定义：
```bash ```bash
aha cli -m qwen3-0.6b -a 0.0.0.0 -p 8080 aha cli -m Qwen/Qwen3-0.6B -a 0.0.0.0 -p 8080
``` ```
### 身份验证 ### 身份验证
@@ -109,7 +109,7 @@ GET /models
"object": "list", "object": "list",
"data": [ "data": [
{ {
"id": "qwen3-0.6b", "id": "Qwen/Qwen3-0.6B",
"object": "model", "object": "model",
"created": null, "created": null,
"owned_by": "Qwen" "owned_by": "Qwen"
@@ -132,7 +132,7 @@ GET /models
|------|------|------| |------|------|------|
| `object` | string | 固定值："list" | | `object` | string | 固定值："list" |
| `data` | array | 模型对象数组（当前仅包含一个已加载的模型） | | `data` | array | 模型对象数组（当前仅包含一个已加载的模型） |
| `id` | string | 模型标识符（kebab-case，如 "qwen3-0.6b"） | | `id` | string | 模型标识符（kebab-case，如 "Qwen/Qwen3-0.6B"） |
| `object` | string | 固定值："model" | | `object` | string | 固定值："model" |
| `created` | integer\|null | Unix 时间戳（当前为 null） | | `created` | integer\|null | Unix 时间戳（当前为 null） |
| `owned_by` | string | 模型所有者/组织名称 | | `owned_by` | string | 模型所有者/组织名称 |
@@ -156,7 +156,7 @@ POST /chat/completions
| 参数 | 类型 | 必需 | 描述 | | 参数 | 类型 | 必需 | 描述 |
|------|------|------|------| |------|------|------|------|
| `model` | string | 是 | 模型标识符（如 "qwen3-0.6b"） | | `model` | string | 是 | 模型标识符（如 "Qwen/Qwen3-0.6B"） |
| `messages` | array | 是 | 消息对象数组 | | `messages` | array | 是 | 消息对象数组 |
| `temperature` | number | 否 | 采样温度（0-2，默认：1） | | `temperature` | number | 否 | 采样温度（0-2，默认：1） |
| `top_p` | number | 否 | 核采样（0-1，默认：1） | | `top_p` | number | 否 | 核采样（0-1，默认：1） |
@@ -197,7 +197,7 @@ POST /chat/completions
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "user", "content": "你好！"} {"role": "user", "content": "你好！"}
] ]
@@ -210,7 +210,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "system", "content": "你是一个有用的助手。"}, {"role": "system", "content": "你是一个有用的助手。"},
{"role": "user", "content": "用一句话解释 Rust。"} {"role": "user", "content": "用一句话解释 Rust。"}
@@ -226,7 +226,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3vl-2b", "model": "Qwen/Qwen3-VL-2B-Instruct",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -245,7 +245,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "deepseek-ocr", "model": "deepseek-ai/DeepSeek-OCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -264,7 +264,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "glm-asr-nano-2512", "model": "ZhipuAI/GLM-ASR-Nano-2512",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -286,7 +286,7 @@ curl http://127.0.0.1:10100/chat/completions \
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "user", "content": "给我讲个故事"} {"role": "user", "content": "给我讲个故事"}
], ],
@@ -312,7 +312,7 @@ data: [DONE]
"id": "chatcmpl-123", "id": "chatcmpl-123",
"object": "chat.completion", "object": "chat.completion",
"created": 1677652288, "created": 1677652288,
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"choices": [ "choices": [
{ {
"index": 0, "index": 0,
@@ -329,12 +329,6 @@ data: [DONE]
} }
``` ```
#### 支持的模型
- 文本：`qwen3-0.6b`、`minicpm4-0.5b`
- 视觉：`qwen2.5vl-3b`、`qwen2.5vl-7b`、`qwen3vl-2b`、`qwen3vl-4b`、`qwen3vl-8b`、`qwen3vl-32b`
- OCR：`deepseek-ocr`、`hunyuan-ocr`、`paddleocr-vl`
- ASR：`glm-asr-nano-2512`、`fun-asr-nano-2512`、`qwen3asr-0.6b`、`qwen3asr-1.7b`
### 语音生成 ### 语音生成
@@ -349,7 +343,7 @@ POST /audio/speech
| 参数 | 类型 | 必需 | 描述 | | 参数 | 类型 | 必需 | 描述 |
|------|------|------|------| |------|------|------|------|
| `model` | string | 是 | 模型标识符（如 "voxcpm1.5"） | | `model` | string | 是 | 模型标识符（如 "OpenBMB/VoxCPM-0.5B1.5"） |
| `messages` | array | 是 | 消息对象数组 | | `messages` | array | 是 | 消息对象数组 |
#### 示例 #### 示例
@@ -358,7 +352,7 @@ POST /audio/speech
curl http://127.0.0.1:10100/audio/speech \ curl http://127.0.0.1:10100/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "voxcpm1.5", "model": "OpenBMB/VoxCPM-0.5B1.5",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -375,9 +369,6 @@ curl http://127.0.0.1:10100/audio/speech \
以 base64 WAV 格式返回音频数据。 以 base64 WAV 格式返回音频数据。
#### 支持的模型
- `voxcpm`、`voxcpm1.5`
### 语音转写 ### 语音转写
@@ -433,7 +424,7 @@ POST /v1/audio/transcriptions
curl -X POST http://127.0.0.1:10100/audio/transcriptions \ curl -X POST http://127.0.0.1:10100/audio/transcriptions \
-H "Authorization: Bearer NO_NEED" \ -H "Authorization: Bearer NO_NEED" \
-F file="@./audio.wav" \ -F file="@./audio.wav" \
-F model="qwen3asr-0.6b" -F model="Qwen/Qwen3-ASR-0.6B"
``` ```
**指定语言：** **指定语言：**
@@ -442,7 +433,7 @@ curl -X POST http://127.0.0.1:10100/audio/transcriptions \
curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \ curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \
-H "Authorization: Bearer NO_NEED" \ -H "Authorization: Bearer NO_NEED" \
-F file="@./chinese_audio.wav" \ -F file="@./chinese_audio.wav" \
-F model="qwen3asr-0.6b" \ -F model="Qwen/Qwen3-ASR-0.6B" \
-F language="zh" -F language="zh"
``` ```
@@ -452,7 +443,7 @@ curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \
curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \ curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \
-H "Authorization: Bearer NO_NEED" \ -H "Authorization: Bearer NO_NEED" \
-F file="@./audio.wav" \ -F file="@./audio.wav" \
-F model="qwen3asr-0.6b" \ -F model="Qwen/Qwen3-ASR-0.6B" \
-F temperature="0.0" -F temperature="0.0"
``` ```
@@ -490,12 +481,6 @@ curl -X POST http://127.0.0.1:10100/v1/audio/transcriptions \
} }
``` ```
#### 支持的模型
- `qwen3asr-0.6b`
- `qwen3asr-1.7b`
- `glm-asr-nano-2512`
- `fun-asr-nano-2512`
#### 文件上传限制 #### 文件上传限制
@@ -514,7 +499,7 @@ POST /images/remove_background
| 参数 | 类型 | 必需 | 描述 | | 参数 | 类型 | 必需 | 描述 |
|------|------|------|------| |------|------|------|------|
| `model` | string | 是 | 模型标识符（如 "rmbg2.0"） | | `model` | string | 是 | 模型标识符（如 "AI-ModelScope/RMBG-2.0"） |
| `messages` | array | 是 | 消息对象数组 | | `messages` | array | 是 | 消息对象数组 |
#### 示例 #### 示例
@@ -525,7 +510,7 @@ POST /images/remove_background
curl http://127.0.0.1:10100/images/remove_background \ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "AI-ModelScope/RMBG-2.0",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -543,7 +528,7 @@ curl http://127.0.0.1:10100/images/remove_background \
curl http://127.0.0.1:10100/images/remove_background \ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "AI-ModelScope/RMBG-2.0",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -559,9 +544,6 @@ curl http://127.0.0.1:10100/images/remove_background \
以base64 PNG 格式返回处理后的图像。 以base64 PNG 格式返回处理后的图像。
#### 支持的模型
- `rmbg2.0`
### 优雅关机 ### 优雅关机
@@ -605,7 +587,7 @@ POST /shutdown
默认情况下，关机端点仅允许来自 localhost (127.0.0.1) 的请求。要启用远程关闭，请使用 `--allow-remote-shutdown` 标志启动服务器： 默认情况下，关机端点仅允许来自 localhost (127.0.0.1) 的请求。要启用远程关闭，请使用 `--allow-remote-shutdown` 标志启动服务器：
```bash ```bash
aha serv -m qwen3-0.6b --allow-remote-shutdown aha serv -m Qwen/Qwen3-0.6B --allow-remote-shutdown
``` ```
**警告：** 除非有适当的安全措施，否则不建议在生产环境中启用远程关闭。 **警告：** 除非有适当的安全措施，否则不建议在生产环境中启用远程关闭。
@@ -671,7 +653,7 @@ client = OpenAI(
) )
response = client.chat.completions.create( response = client.chat.completions.create(
model="qwen3-0.6b", model="Qwen/Qwen3-0.6B",
messages=[ messages=[
{"role": "user", "content": "你好！"} {"role": "user", "content": "你好！"}
] ]
@@ -691,7 +673,7 @@ const client = new OpenAI({
}); });
const response = await client.chat.completions.create({ const response = await client.chat.completions.create({
model: 'qwen3-0.6b', model: 'Qwen/Qwen3-0.6B',
messages: [{ role: 'user', content: '你好！' }] messages: [{ role: 'user', content: '你好！' }]
}); });
docs/changelog.md
+3
View File
@@ -6,6 +6,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### 2026-03-31 ### 2026-03-31
- aha model name use modelscope id replace
- update WhichModel enum
- Usage add time info
- dependencies delete aha_openai_dive,chrono - dependencies delete aha_openai_dive,chrono
### v0.2.5 (2026-03-30) ### v0.2.5 (2026-03-30)
docs/changelog.zh-CN.md
+4 -1
View File
@@ -6,7 +6,10 @@
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/spec/v2.0.0.html)。 本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/spec/v2.0.0.html)。
### 2026-03-31 ### 2026-03-31
- 删除 aha_openai_dive,chrono 依赖 - aha模型名称使用 modelscope id 替换
- 更新 WhichModel 枚举
- Usage 增加时间信息
- 删除 aha_openai_dive, chrono 依赖
### v0.2.5 (2026-03-30) ### v0.2.5 (2026-03-30)
- 新增 LFM2.5VL-1.6B - 新增 LFM2.5VL-1.6B
docs/cli.md
+35 -55
View File
@@ -51,16 +51,16 @@ aha cli [OPTIONS] --model <MODEL>
```bash ```bash
# Download model and start service (default port 10100) # Download model and start service (default port 10100)
aha cli -m qwen3vl-2b aha cli -m Qwen/Qwen3-VL-2B-Instruct
# Specify port and save directory # Specify port and save directory
aha cli -m qwen3vl-2b -p 8080 --save-dir /data/models aha cli -m Qwen/Qwen3-VL-2B-Instruct -p 8080 --save-dir /data/models
# Use local model (skip download) # Use local model (skip download)
aha cli -m qwen3vl-2b --weight-path /path/to/model aha cli -m Qwen/Qwen3-VL-2B-Instruct --weight-path /path/to/model
# Backward compatible way (equivalent to cli subcommand) # Backward compatible way (equivalent to cli subcommand)
aha -m qwen3vl-2b aha -m Qwen/Qwen3-VL-2B-Instruct
# use gguf-path and mmproj-path # use gguf-path and mmproj-path
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
@@ -89,34 +89,34 @@ aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] [--weight-p
```bash ```bash
# VoxCPM1.5 text-to-speech (single input) # VoxCPM1.5 text-to-speech (single input)
aha run -m voxcpm1.5 -i "太阳当空照" -o output.wav --weight-path /path/to/model aha run -m OpenBMB/VoxCPM1.5 -i "太阳当空照" -o output.wav --weight-path /path/to/model
# VoxCPM1.5 read input from file (single input) # VoxCPM1.5 read input from file (single input)
aha run -m voxcpm1.5 -i "file://./input.txt" --weight-path /path/to/model aha run -m OpenBMB/VoxCPM1.5 -i "file://./input.txt" --weight-path /path/to/model
# MiniCPM4 text generation (single input) # MiniCPM4 text generation (single input)
aha run -m minicpm4-0.5b -i "你好" --weight-path /path/to/model aha run -m OpenBMB/MiniCPM4-0.5B -i "你好" --weight-path /path/to/model
# DeepSeek OCR image recognition (single input) # DeepSeek OCR image recognition (single input)
aha run -m deepseek-ocr -i "image.jpg" --weight-path /path/to/model aha run -m deepseek-ai/DeepSeek-OCR -i "image.jpg" --weight-path /path/to/model
# RMBG2.0 background removal (single input) # RMBG2.0 background removal (single input)
aha run -m RMBG2.0 -i "photo.png" -o "no_bg.png" --weight-path /path/to/model aha run -m AI-ModelScope/RMBG-2.0 -i "photo.png" -o "no_bg.png" --weight-path /path/to/model
# GLM-ASR speech recognition (two inputs: prompt text + audio file) # GLM-ASR speech recognition (two inputs: prompt text + audio file)
aha run -m glm-asr-nano-2512 -i "请转写这段音频" -i "audio.wav" --weight-path /path/to/model aha run -m ZhipuAI/GLM-ASR-Nano-2512 -i "请转写这段音频" -i "audio.wav" --weight-path /path/to/model
# Fun-ASR speech recognition (two inputs: prompt text + audio file) # Fun-ASR speech recognition (two inputs: prompt text + audio file)
aha run -m fun-asr-nano-2512 -i "语音转写：" -i "audio.wav" --weight-path /path/to/model aha run -m FunAudioLLM/Fun-ASR-Nano-2512 -i "语音转写：" -i "audio.wav" --weight-path /path/to/model
# qwen3 text generation (single input) # qwen3 text generation (single input)
aha run -m qwen3-0.6b -i "你好" --weight-path /path/to/model aha run -m Qwen/Qwen3-0.6B -i "你好" --weight-path /path/to/model
# qwen2.5vl image understanding (two inputs: prompt text + image file) # qwen2.5vl image understanding (two inputs: prompt text + image file)
aha run -m qwen2.5vl-3b -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model aha run -m Qwen/Qwen2.5-VL-3B-Instruct -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model
# Qwen3-ASR speech recognition (single input: audio file) # Qwen3-ASR speech recognition (single input: audio file)
aha run -m qwen3asr-0.6b -i "audio.wav" --weight-path /path/to/model aha run -m Qwen/Qwen3-ASR-0.6B -i "audio.wav" --weight-path /path/to/model
# Qwen3.5-GGUF without mmproj (single input: prompt text) # Qwen3.5-GGUF without mmproj (single input: prompt text)
aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
@@ -152,19 +152,19 @@ aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <G
```bash ```bash
# Start service with default model path (~/.aha/{model_id}) # Start service with default model path (~/.aha/{model_id})
aha serv -m qwen3vl-2b aha serv -m Qwen/Qwen3-VL-2B-Instruct
# Start service with local model # Start service with local model
aha serv -m qwen3vl-2b --weight-path /path/to/model aha serv -m Qwen/Qwen3-VL-2B-Instruct --weight-path /path/to/model
# Start with specified port # Start with specified port
aha serv -m qwen3vl-2b -p 8080 aha serv -m Qwen/Qwen3-VL-2B-Instruct -p 8080
# Specify listen address # Specify listen address
aha serv -m qwen3vl-2b -a 0.0.0.0 aha serv -m Qwen/Qwen3-VL-2B-Instruct -a 0.0.0.0
# Enable remote shutdown (not recommended for production) # Enable remote shutdown (not recommended for production)
aha serv -m qwen3vl-2b --allow-remote-shutdown aha serv -m Qwen/Qwen3-VL-2B-Instruct --allow-remote-shutdown
``` ```
### ps - List running services ### ps - List running services
@@ -229,16 +229,16 @@ aha download [OPTIONS] --model <MODEL>
```bash ```bash
# Download model to default directory # Download model to default directory
aha download -m qwen3vl-2b aha download -m Qwen/Qwen3-VL-2B-Instruct
# Specify save directory # Specify save directory
aha download -m qwen3vl-2b -s /data/models aha download -m Qwen/Qwen3-VL-2B-Instruct -s /data/models
# Specify download retry count # Specify download retry count
aha download -m qwen3vl-2b --download-retries 5 aha download -m Qwen/Qwen3-VL-2B-Instruct --download-retries 5
# Download MiniCPM4-0.5B model # Download MiniCPM4-0.5B model
aha download -m minicpm4-0.5b -s models aha download -m OpenBMB/MiniCPM4-0.5B -s models
``` ```
### delete - Delete downloaded model ### delete - Delete downloaded model
@@ -260,10 +260,10 @@ aha delete [OPTIONS] --model <MODEL>
```bash ```bash
# Delete RMBG2.0 model from default location # Delete RMBG2.0 model from default location
aha delete -m rmbg2.0 aha delete -m AI-ModelScope/RMBG-2.0
# Delete Qwen3-VL-2B model # Delete Qwen3-VL-2B model
aha delete --model qwen3vl-2b aha delete --model Qwen/Qwen3-VL-2B-Instruct
``` ```
**Behavior:** **Behavior:**
@@ -311,12 +311,12 @@ Example:
```json ```json
[ [
{ {
"name": "qwen3vl-2b", "name": "Qwen/Qwen3-VL-2B-Instruct",
"model_id": "Qwen/Qwen3-VL-2B-Instruct", "model_id": "Qwen/Qwen3-VL-2B-Instruct",
"type": "llm" "type": "llm"
}, },
{ {
"name": "deepseek-ocr", "name": "deepseek-ai/DeepSeek-OCR",
"model_id": "deepseek-ai/DeepSeek-OCR", "model_id": "deepseek-ai/DeepSeek-OCR",
"type": "ocr" "type": "ocr"
} }
@@ -328,27 +328,7 @@ Example:
- `ocr`: Optical Character Recognition models - `ocr`: Optical Character Recognition models
- `asr`: Automatic Speech Recognition models - `asr`: Automatic Speech Recognition models
- `image`: Image processing models - `image`: Image processing models
- `tts`： Text to speech
## Supported Models
| Model ID | Model Name | Description |
|----------|------------|-------------|
| `minicpm4-0.5b` | OpenBMB/MiniCPM4-0.5B | OpenBMB MiniCPM4 0.5B model |
| `qwen2.5vl-3b` | Qwen/Qwen2.5-VL-3B-Instruct | Qwen 2.5 VL 3B model |
| `qwen2.5vl-7b` | Qwen/Qwen2.5-VL-7B-Instruct | Qwen 2.5 VL 7B model |
| `qwen3-0.6b` | Qwen/Qwen3-0.6B | Qwen 3 0.6B model |
| `qwen3vl-2b` | Qwen/Qwen3-VL-2B-Instruct | Qwen 3 VL 2B model |
| `qwen3vl-4b` | Qwen/Qwen3-VL-4B-Instruct | Qwen 3 VL 4B model |
| `qwen3vl-8b` | Qwen/Qwen3-VL-8B-Instruct | Qwen 3 VL 8B model |
| `qwen3vl-32b` | Qwen/Qwen3-VL-32B-Instruct | Qwen 3 VL 32B model |
| `deepseek-ocr` | deepseek-ai/DeepSeek-OCR | DeepSeek OCR model |
| `hunyuan-ocr` | Tencent-Hunyuan/HunyuanOCR | Tencent Hunyuan OCR model |
| `paddleocr-vl` | PaddlePaddle/PaddleOCR-VL | Baidu PaddleOCR VL model |
| `RMBG2.0` | AI-ModelScope/RMBG-2.0 | RMBG 2.0 background removal model |
| `voxcpm` | OpenBMB/VoxCPM-0.5B | OpenBMB VoxCPM 0.5B speech synthesis model |
| `voxcpm1.5` | OpenBMB/VoxCPM1.5 | OpenBMB VoxCPM 1.5 speech synthesis model |
| `glm-asr-nano-2512` | ZhipuAI/GLM-ASR-Nano-2512 | Zhipu AI ASR Nano 2512 speech recognition model |
| `fun-asr-nano-2512` | FunAudioLLM/Fun-ASR-Nano-2512 | FunAudioLLM ASR Nano 2512 speech recognition model |
## Common Use Cases ## Common Use Cases
@@ -356,31 +336,31 @@ Example:
```bash ```bash
# One command to download and start service # One command to download and start service
aha -m qwen3vl-2b aha -m Qwen/Qwen3-VL-2B-Instruct
``` ```
### Scenario 2: Start service with existing model ### Scenario 2: Start service with existing model
```bash ```bash
# Assuming model is downloaded to /data/models/Qwen/Qwen3-VL-2B-Instruct # Assuming model is downloaded to /data/models/Qwen/Qwen3-VL-2B-Instruct
aha serv -m qwen3vl-2b --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct aha serv -m Qwen/Qwen3-VL-2B-Instruct --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct
``` ```
### Scenario 3: Pre-download model ### Scenario 3: Pre-download model
```bash ```bash
# Download model to specified directory for later use # Download model to specified directory for later use
aha download -m qwen3vl-2b -s /data/models aha download -m Qwen/Qwen3-VL-2B-Instruct -s /data/models
# Later start with local model # Later start with local model
aha serv -m qwen3vl-2b --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct aha serv -m Qwen/Qwen3-VL-2B-Instruct --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct
``` ```
### Scenario 4: Custom service port and address ### Scenario 4: Custom service port and address
```bash ```bash
# Start service on 0.0.0.0:8080, allow external access # Start service on 0.0.0.0:8080, allow external access
aha -m qwen3vl-2b -a 0.0.0.0 -p 8080 aha -m Qwen/Qwen3-VL-2B-Instruct -a 0.0.0.0 -p 8080
``` ```
## API Endpoints ## API Endpoints
@@ -421,10 +401,10 @@ To maintain compatibility with older versions, the following two usage methods a
```bash ```bash
# New way (recommended) # New way (recommended)
aha cli -m qwen3vl-2b aha cli -m Qwen/Qwen3-VL-2B-Instruct
# Old way (backward compatible) # Old way (backward compatible)
aha -m qwen3vl-2b aha -m Qwen/Qwen3-VL-2B-Instruct
``` ```
## Notes ## Notes
docs/cli.zh-CN.md
+35 -55
View File
@@ -51,16 +51,16 @@ aha cli [OPTIONS] --model <MODEL>
```bash ```bash
# 下载模型并启动服务（默认端口 10100） # 下载模型并启动服务（默认端口 10100）
aha cli -m qwen3vl-2b aha cli -m Qwen/Qwen3-VL-2B-Instruct
# 指定端口和保存目录 # 指定端口和保存目录
aha cli -m qwen3vl-2b -p 8080 --save-dir /data/models aha cli -m Qwen/Qwen3-VL-2B-Instruct -p 8080 --save-dir /data/models
# 使用本地模型（不下载） # 使用本地模型（不下载）
aha cli -m qwen3vl-2b --weight-path /path/to/model aha cli -m Qwen/Qwen3-VL-2B-Instruct --weight-path /path/to/model
# 向后兼容方式（等同于 cli 子命令） # 向后兼容方式（等同于 cli 子命令）
aha -m qwen3vl-2b aha -m Qwen/Qwen3-VL-2B-Instruct
# 指定gguf-path和mmproj-path # 指定gguf-path和mmproj-path
aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf aha cli -m qwen3.5-gguf --gguf-path /path/to/xxx.gguf --mmproj-path /path/to/mmproj-xxx.gguf
@@ -90,34 +90,34 @@ aha run [OPTIONS] --model <MODEL> --input <INPUT> [--input <INPUT2>] [--weight-p
```bash ```bash
# VoxCPM1.5 文字转语音（单个输入） # VoxCPM1.5 文字转语音（单个输入）
aha run -m voxcpm1.5 -i "太阳当空照" -o output.wav --weight-path /path/to/model aha run -m OpenBMB/VoxCPM1.5 -i "太阳当空照" -o output.wav --weight-path /path/to/model
# VoxCPM1.5 从文件读取输入（单个输入） # VoxCPM1.5 从文件读取输入（单个输入）
aha run -m voxcpm1.5 -i "file://./input.txt" --weight-path /path/to/model aha run -m OpenBMB/VoxCPM1.5 -i "file://./input.txt" --weight-path /path/to/model
# MiniCPM4 文本生成（单个输入） # MiniCPM4 文本生成（单个输入）
aha run -m minicpm4-0.5b -i "你好" --weight-path /path/to/model aha run -m OpenBMB/MiniCPM4-0.5B -i "你好" --weight-path /path/to/model
# DeepSeek OCR 图片识别（单个输入） # DeepSeek OCR 图片识别（单个输入）
aha run -m deepseek-ocr -i "image.jpg" --weight-path /path/to/model aha run -m deepseek-ai/DeepSeek-OCR -i "image.jpg" --weight-path /path/to/model
# RMBG2.0 背景移除（单个输入） # RMBG2.0 背景移除（单个输入）
aha run -m RMBG2.0 -i "photo.png" -o "no_bg.png" --weight-path /path/to/model aha run -m AI-ModelScope/RMBG-2.0 -i "photo.png" -o "no_bg.png" --weight-path /path/to/model
# GLM-ASR 语音识别（两个输入：提示文本 + 音频文件） # GLM-ASR 语音识别（两个输入：提示文本 + 音频文件）
aha run -m glm-asr-nano-2512 -i "请转写这段音频" -i "audio.wav" --weight-path /path/to/model aha run -m ZhipuAI/GLM-ASR-Nano-2512 -i "请转写这段音频" -i "audio.wav" --weight-path /path/to/model
# Fun-ASR 语音识别（两个输入：提示文本 + 音频文件） # Fun-ASR 语音识别（两个输入：提示文本 + 音频文件）
aha run -m fun-asr-nano-2512 -i "语音转写：" -i "audio.wav" --weight-path /path/to/model aha run -m FunAudioLLM/Fun-ASR-Nano-2512 -i "语音转写：" -i "audio.wav" --weight-path /path/to/model
# qwen3 文本生成（单个输入） # qwen3 文本生成（单个输入）
aha run -m qwen3-0.6b -i "你好" --weight-path /path/to/model aha run -m Qwen/Qwen3-0.6B -i "你好" --weight-path /path/to/model
# qwen2.5vl 图像理解（两个输入：提示文本 + 图片文件） # qwen2.5vl 图像理解（两个输入：提示文本 + 图片文件）
aha run -m qwen2.5vl-3b -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model aha run -m Qwen/Qwen2.5-VL-3B-Instruct -i "请分析图片并提取所有可见文本内容，按从左到右、从上到下的布局，返回纯文本" -i "image.jpg" --weight-path /path/to/model
# Qwen3-ASR 语音识别（单个输入：音频文件） # Qwen3-ASR 语音识别（单个输入：音频文件）
aha run -m qwen3asr-0.6b -i "audio.wav" --weight-path /path/to/model aha run -m Qwen/Qwen3-ASR-0.6B -i "audio.wav" --weight-path /path/to/model
# Qwen3.5-GGUF 无mmproj (单个输入：提示文本) # Qwen3.5-GGUF 无mmproj (单个输入：提示文本)
aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf aha run -m qwen3.5-gguf -i 你如何看待AI --gguf-path /path/to/xxx.gguf
@@ -152,19 +152,19 @@ aha serv [OPTIONS] --model <MODEL> [--weight-path <WEIGHT_PATH>] [--gguf-path <G
```bash ```bash
# 使用默认模型路径启动服务 (~/.aha/{model_id}) # 使用默认模型路径启动服务 (~/.aha/{model_id})
aha serv -m qwen3vl-2b aha serv -m Qwen/Qwen3-VL-2B-Instruct
# 使用本地模型启动服务 # 使用本地模型启动服务
aha serv -m qwen3vl-2b --weight-path /path/to/model aha serv -m Qwen/Qwen3-VL-2B-Instruct --weight-path /path/to/model
# 指定端口启动 # 指定端口启动
aha serv -m qwen3vl-2b -p 8080 aha serv -m Qwen/Qwen3-VL-2B-Instruct -p 8080
# 指定监听地址 # 指定监听地址
aha serv -m qwen3vl-2b -a 0.0.0.0 aha serv -m Qwen/Qwen3-VL-2B-Instruct -a 0.0.0.0
# 启用远程关机（不推荐用于生产环境） # 启用远程关机（不推荐用于生产环境）
aha serv -m qwen3vl-2b --allow-remote-shutdown aha serv -m Qwen/Qwen3-VL-2B-Instruct --allow-remote-shutdown
``` ```
### ps - 列出运行中的服务 ### ps - 列出运行中的服务
@@ -229,16 +229,16 @@ aha download [OPTIONS] --model <MODEL>
```bash ```bash
# 下载模型到默认目录 # 下载模型到默认目录
aha download -m qwen3vl-2b aha download -m Qwen/Qwen3-VL-2B-Instruct
# 指定保存目录 # 指定保存目录
aha download -m qwen3vl-2b -s /data/models aha download -m Qwen/Qwen3-VL-2B-Instruct -s /data/models
# 指定下载重试次数 # 指定下载重试次数
aha download -m qwen3vl-2b --download-retries 5 aha download -m Qwen/Qwen3-VL-2B-Instruct --download-retries 5
# 下载 MiniCPM4-0.5B 模型 # 下载 MiniCPM4-0.5B 模型
aha download -m minicpm4-0.5b -s models aha download -m OpenBMB/MiniCPM4-0.5B -s models
``` ```
### delete - 删除已下载的模型 ### delete - 删除已下载的模型
@@ -260,10 +260,10 @@ aha delete [OPTIONS] --model <MODEL>
```bash ```bash
# 删除 RMBG2.0 模型 # 删除 RMBG2.0 模型
aha delete -m rmbg2.0 aha delete -m AI-ModelScope/RMBG-2.0
# 删除 Qwen3-VL-2B 模型 # 删除 Qwen3-VL-2B 模型
aha delete --model qwen3vl-2b aha delete --model Qwen/Qwen3-VL-2B-Instruct
``` ```
**行为说明：** **行为说明：**
@@ -311,12 +311,12 @@ aha list -j
```json ```json
[ [
{ {
"name": "qwen3vl-2b", "name": "Qwen/Qwen3-VL-2B-Instruct",
"model_id": "Qwen/Qwen3-VL-2B-Instruct", "model_id": "Qwen/Qwen3-VL-2B-Instruct",
"type": "llm" "type": "llm"
}, },
{ {
"name": "deepseek-ocr", "name": "deepseek-ai/DeepSeek-OCR",
"model_id": "deepseek-ai/DeepSeek-OCR", "model_id": "deepseek-ai/DeepSeek-OCR",
"type": "ocr" "type": "ocr"
} }
@@ -328,27 +328,7 @@ aha list -j
- `ocr`：光学字符识别模型 - `ocr`：光学字符识别模型
- `asr`：自动语音识别模型 - `asr`：自动语音识别模型
- `image`：图像处理模型 - `image`：图像处理模型
- `tts`：语音生成
## 支持的模型
| 模型标识 | 模型名称 | 说明 |
|---------|---------|------|
| `minicpm4-0.5b` | OpenBMB/MiniCPM4-0.5B | 面壁智能 MiniCPM4 0.5B 模型 |
| `qwen2.5vl-3b` | Qwen/Qwen2.5-VL-3B-Instruct | 通义千问 2.5 VL 3B 模型 |
| `qwen2.5vl-7b` | Qwen/Qwen2.5-VL-7B-Instruct | 通义千问 2.5 VL 7B 模型 |
| `qwen3-0.6b` | Qwen/Qwen3-0.6B | 通义千问 3 0.6B 模型 |
| `qwen3vl-2b` | Qwen/Qwen3-VL-2B-Instruct | 通义千问 3 VL 2B 模型 |
| `qwen3vl-4b` | Qwen/Qwen3-VL-4B-Instruct | 通义千问 3 VL 4B 模型 |
| `qwen3vl-8b` | Qwen/Qwen3-VL-8B-Instruct | 通义千问 3 VL 8B 模型 |
| `qwen3vl-32b` | Qwen/Qwen3-VL-32B-Instruct | 通义千问 3 VL 32B 模型 |
| `deepseek-ocr` | deepseek-ai/DeepSeek-OCR | DeepSeek OCR 模型 |
| `hunyuan-ocr` | Tencent-Hunyuan/HunyuanOCR | 腾讯混元 OCR 模型 |
| `paddleocr-vl` | PaddlePaddle/PaddleOCR-VL | 百度飞桨 OCR VL 模型 |
| `RMBG2.0` | AI-ModelScope/RMBG-2.0 | RMBG 2.0 背景移除模型 |
| `voxcpm` | OpenBMB/VoxCPM-0.5B | 面壁智能 VoxCPM 0.5B 语音生成模型 |
| `voxcpm1.5` | OpenBMB/VoxCPM1.5 | 面壁智能 VoxCPM 1.5 语音生成模型 |
| `glm-asr-nano-2512` | ZhipuAI/GLM-ASR-Nano-2512 | 智谱 AI ASR Nano 2512 语音识别模型 |
| `fun-asr-nano-2512` | FunAudioLLM/Fun-ASR-Nano-2512 | 通义百聆 ASR Nano 2512 语音识别模型 |
## 常见使用场景 ## 常见使用场景
@@ -356,31 +336,31 @@ aha list -j
```bash ```bash
# 一条命令下载并启动服务 # 一条命令下载并启动服务
aha -m qwen3vl-2b aha -m Qwen/Qwen3-VL-2B-Instruct
``` ```
### 场景 2：使用已有模型启动服务 ### 场景 2：使用已有模型启动服务
```bash ```bash
# 假设模型已下载到 /data/models/Qwen/Qwen3-VL-2B-Instruct # 假设模型已下载到 /data/models/Qwen/Qwen3-VL-2B-Instruct
aha serv -m qwen3vl-2b --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct aha serv -m Qwen/Qwen3-VL-2B-Instruct --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct
``` ```
### 场景 3：预先下载模型 ### 场景 3：预先下载模型
```bash ```bash
# 下载模型到指定目录，稍后使用 # 下载模型到指定目录，稍后使用
aha download -m qwen3vl-2b -s /data/models aha download -m Qwen/Qwen3-VL-2B-Instruct -s /data/models
# 后续启动时直接使用 # 后续启动时直接使用
aha serv -m qwen3vl-2b --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct aha serv -m Qwen/Qwen3-VL-2B-Instruct --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct
``` ```
### 场景 4：自定义服务端口和地址 ### 场景 4：自定义服务端口和地址
```bash ```bash
# 在 0.0.0.0:8080 启动服务，允许外部访问 # 在 0.0.0.0:8080 启动服务，允许外部访问
aha -m qwen3vl-2b -a 0.0.0.0 -p 8080 aha -m Qwen/Qwen3-VL-2B-Instruct -a 0.0.0.0 -p 8080
``` ```
## API 接口 ## API 接口
@@ -420,10 +400,10 @@ aha -m qwen3vl-2b -a 0.0.0.0 -p 8080
```bash ```bash
# 新方式（推荐） # 新方式（推荐）
aha cli -m qwen3vl-2b aha cli -m Qwen/Qwen3-VL-2B-Instruct
# 旧方式（向后兼容） # 旧方式（向后兼容）
aha -m qwen3vl-2b aha -m Qwen/Qwen3-VL-2B-Instruct
``` ```
## 注意事项 ## 注意事项
docs/concepts.md
+3 -3
View File
@@ -191,13 +191,13 @@ Model loading uses a factory function:
```rust ```rust
pub fn load_model( pub fn load_model(
model_type: &str, model_type: WhichModel,
model_path: &str, model_path: &str,
device: &Device, device: &Device,
) -> Result<Box<dyn GenerateModel>> { ) -> Result<Box<dyn GenerateModel>> {
match model_type { match model_type {
"qwen3vl-2b" => Ok(Box::new(qwen3vl::generate::Qwen3VLGenerate::init(...)?)), WhichModel::Qwen3VL2B => Ok(Box::new(qwen3vl::generate::Qwen3VLGenerate::init(...)?)),
"voxcpm1.5" => Ok(Box::new(voxcpm::generate::VoxCPMGenerate::init(...)?)), WhichModel::VoxCPM1_5 => Ok(Box::new(voxcpm::generate::VoxCPMGenerate::init(...)?)),
// ... other models // ... other models
_ => Err(anyhow!("Unsupported model: {}", model_type)), _ => Err(anyhow!("Unsupported model: {}", model_type)),
} }
docs/concepts.zh-CN.md
+5 -5
View File
@@ -191,15 +191,15 @@ let result = model.generate(prompt, params)?;
```rust ```rust
pub fn load_model( pub fn load_model(
model_type: &str, model_type: WhichModel,
model_path: &str, model_path: &str,
device: &Device, device: &Device,
) -> Result<Box<dyn GenerateModel>> { ) -> Result<Box<dyn GenerateModel>> {
match model_type { match model_type {
"qwen3vl-2b" => Ok(Box::new(qwen3vl::generate::Qwen3VLGenerate::init(...)?)), WhichModel::Qwen3VL2B => Ok(Box::new(qwen3vl::generate::Qwen3VLGenerate::init(...)?)),
"voxcpm1.5" => Ok(Box::new(voxcpm::generate::VoxCPMGenerate::init(...)?)), WhichModel::VoxCPM1_5 => Ok(Box::new(voxcpm::generate::VoxCPMGenerate::init(...)?)),
// ... 其他模型 // ... other models
_ => Err(anyhow!("不支持的模型: {}", model_type)), _ => Err(anyhow!("Unsupported model: {}", model_type)),
} }
} }
``` ```
docs/development.md
+5 -5
View File
@@ -43,7 +43,7 @@ cargo build --features ffmpeg
```bash ```bash
# Run the CLI # Run the CLI
cargo run -- -m qwen3-0.6b cargo run -- -m Qwen/Qwen3-0.6B
# Run tests # Run tests
cargo test cargo test
@@ -52,7 +52,7 @@ cargo test
cargo test test_qwen3vl_generate cargo test test_qwen3vl_generate
# Run with logging # Run with logging
RUST_LOG=debug cargo run -- -m qwen3-0.6b RUST_LOG=debug cargo run -- -m Qwen/Qwen3-0.6B
# Check code without building # Check code without building
cargo check cargo check
@@ -481,7 +481,7 @@ Follow conventional commits:
feat: add NewModel support feat: add NewModel support
fix: correct tensor dimensions in Qwen3VL fix: correct tensor dimensions in Qwen3VL
docs: update installation guide docs: update installation guide
test: add integration test for VoxCPM test: add integration test for NewModel
refactor: simplify model loading logic refactor: simplify model loading logic
perf: improve inference speed by 20% perf: improve inference speed by 20%
``` ```
@@ -535,13 +535,13 @@ cargo publish
Enable debug logging: Enable debug logging:
```bash ```bash
RUST_LOG=debug cargo run -- -m qwen3-0.6b RUST_LOG=debug cargo run -- -m Qwen/Qwen3-0.6B
``` ```
Set specific module logging: Set specific module logging:
```bash ```bash
RUST_LOG=aha::models::qwen3vl=debug cargo run -- -m qwen3-0.6b RUST_LOG=aha::models::qwen3vl=debug cargo run -- -m Qwen/Qwen3-0.6B
``` ```
### Debugging Tests ### Debugging Tests
docs/development.zh-CN.md
+5 -5
View File
@@ -43,7 +43,7 @@ cargo build --features ffmpeg
```bash ```bash
# 运行 CLI # 运行 CLI
cargo run -- -m qwen3-0.6b cargo run -- -m Qwen/Qwen3-0.6B
# 运行测试 # 运行测试
cargo test cargo test
@@ -52,7 +52,7 @@ cargo test
cargo test test_qwen3vl_generate cargo test test_qwen3vl_generate
# 启用日志运行 # 启用日志运行
RUST_LOG=debug cargo run -- -m qwen3-0.6b RUST_LOG=debug cargo run -- -m Qwen/Qwen3-0.6B
# 检查代码而不构建 # 检查代码而不构建
cargo check cargo check
@@ -481,7 +481,7 @@ pub fn generate(&mut self, prompt: &str) -> Result<String> {
feat: 添加 NewModel 支持 feat: 添加 NewModel 支持
fix: 修正 Qwen3VL 中的张量维度 fix: 修正 Qwen3VL 中的张量维度
docs: 更新安装指南 docs: 更新安装指南
test: 添加 VoxCPM 集成测试 test: 添加 NewModel 集成测试
refactor: 简化模型加载逻辑 refactor: 简化模型加载逻辑
perf: 将推理速度提高 20% perf: 将推理速度提高 20%
``` ```
@@ -535,13 +535,13 @@ cargo publish
启用调试日志： 启用调试日志：
```bash ```bash
RUST_LOG=debug cargo run -- -m qwen3-0.6b RUST_LOG=debug cargo run -- -m Qwen/Qwen3-0.6B
``` ```
设置特定模块日志： 设置特定模块日志：
```bash ```bash
RUST_LOG=aha::models::qwen3vl=debug cargo run -- -m qwen3-0.6b RUST_LOG=aha::models::qwen3vl=debug cargo run -- -m Qwen/Qwen3-0.6B
``` ```
### 调试测试 ### 调试测试
docs/getting-started.md
+39 -39
View File
@@ -14,14 +14,14 @@ aha list
```bash ```bash
# Download a small text model to start # Download a small text model to start
aha download -m qwen3-0.6b aha download -m Qwen/Qwen3-0.6B
``` ```
### 3. Start the Service ### 3. Start the Service
```bash ```bash
# Start the HTTP API server # Start the HTTP API server
aha cli -m qwen3-0.6b aha cli -m Qwen/Qwen3-0.6B
``` ```
The service will start on `http://127.0.0.1:10100` The service will start on `http://127.0.0.1:10100`
@@ -34,7 +34,7 @@ In a new terminal:
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "user", "content": "Hello, AHA!"} {"role": "user", "content": "Hello, AHA!"}
] ]
@@ -78,13 +78,13 @@ AHA is a local AI inference engine that:
```bash ```bash
# Start the service # Start the service
aha cli -m qwen3-0.6b aha cli -m Qwen/Qwen3-0.6B
# In another terminal, make a request # In another terminal, make a request
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "system", "content": "You are a helpful assistant."}, {"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain quantum computing in simple terms."} {"role": "user", "content": "Explain quantum computing in simple terms."}
@@ -98,13 +98,13 @@ curl http://127.0.0.1:10100/chat/completions \
```bash ```bash
# Start a vision model # Start a vision model
aha cli -m qwen3vl-2b aha cli -m Qwen/Qwen3-VL-2B-Instruct
# Analyze an image # Analyze an image
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3vl-2b", "model": "Qwen/Qwen3-VL-2B-Instruct",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -122,13 +122,13 @@ curl http://127.0.0.1:10100/chat/completions \
```bash ```bash
# Start an OCR model # Start an OCR model
aha cli -m deepseek-ocr aha cli -m deepseek-ai/DeepSeek-OCR
# Extract text from an image # Extract text from an image
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "deepseek-ocr", "model": "deepseek-ai/DeepSeek-OCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -145,13 +145,13 @@ curl http://127.0.0.1:10100/chat/completions \
```bash ```bash
# Start an ASR model # Start an ASR model
aha cli -m glm-asr-nano-2512 aha cli -m ZhipuAI/GLM-ASR-Nano-2512
# Transcribe audio # Transcribe audio
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "glm-asr-nano-2512", "model": "ZhipuAI/GLM-ASR-Nano-2512",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -168,13 +168,13 @@ curl http://127.0.0.1:10100/chat/completions \
```bash ```bash
# Start a TTS model # Start a TTS model
aha cli -m voxcpm1.5 aha cli -m OpenBMB/VoxCPM1.5
# Generate speech # Generate speech
curl http://127.0.0.1:10100/audio/speech \ curl http://127.0.0.1:10100/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "voxcpm1.5", "model": "OpenBMB/VoxCPM1.5",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -191,13 +191,13 @@ curl http://127.0.0.1:10100/audio/speech \
```bash ```bash
# Start RMBG2.0 model # Start RMBG2.0 model
aha cli -m rmbg2.0 aha cli -m AI-ModelScope/RMBG-2.0
# Remove background from image # Remove background from image
curl http://127.0.0.1:10100/images/remove_background \ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "AI-ModelScope/RMBG-2.0",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -213,7 +213,7 @@ curl http://127.0.0.1:10100/images/remove_background \
```bash ```bash
# Run inference directly without starting HTTP server # Run inference directly without starting HTTP server
aha run -m qwen3-0.6b \ aha run -m Qwen/Qwen3-0.6B \
-i "Write a haiku about AI" \ -i "Write a haiku about AI" \
--weight-path ~/.aha/Qwen/Qwen3-0.6B --weight-path ~/.aha/Qwen/Qwen3-0.6B
``` ```
@@ -224,21 +224,21 @@ aha run -m qwen3-0.6b \
```bash ```bash
# Use port 8080 instead of default 10100 # Use port 8080 instead of default 10100
aha cli -m qwen3-0.6b -p 8080 aha cli -m Qwen/Qwen3-0.6B -p 8080
``` ```
### Bind to All Interfaces ### Bind to All Interfaces
```bash ```bash
# Allow external access (use with caution) # Allow external access (use with caution)
aha cli -m qwen3-0.6b -a 0.0.0.0 -p 8080 aha cli -m Qwen/Qwen3-0.6B -a 0.0.0.0 -p 8080
``` ```
### Use Local Model ### Use Local Model
```bash ```bash
# Skip download, use existing model # Skip download, use existing model
aha serv -m qwen3-0.6b \ aha serv -m Qwen/Qwen3-0.6B \
--weight-path /path/to/model \ --weight-path /path/to/model \
-p 8080 -p 8080
``` ```
@@ -247,7 +247,7 @@ aha serv -m qwen3-0.6b \
```bash ```bash
# Download model to specific directory # Download model to specific directory
aha download -m qwen3vl-2b -s /data/models aha download -m Qwen/Qwen3-VL-2B-Instruct -s /data/models
``` ```
## Streaming Responses ## Streaming Responses
@@ -258,7 +258,7 @@ For chat/completions, using no "stream" field or "stream": true enables streamin
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "user", "content": "Tell me a story"} {"role": "user", "content": "Tell me a story"}
], ],
@@ -269,28 +269,28 @@ curl http://127.0.0.1:10100/chat/completions \
## Model Selection Guide ## Model Selection Guide
### For Text Generation ### For Text Generation
- **qwen3-0.6b**: Fast, lightweight (~1.2 GB) - **Qwen/Qwen3-0.6B**: Fast, lightweight (~1.2 GB)
- **minicpm4-0.5b**: Small, efficient (~1 GB) - **OpenBMB/MiniCPM4-0.5B**: Small, efficient (~1 GB)
### For Vision Tasks ### For Vision Tasks
- **qwen3vl-2b**: Balanced performance (~4 GB) - **Qwen/Qwen3-VL-2B-Instruct**: Balanced performance (~4 GB)
- **qwen3vl-8b**: Better quality (~16 GB) - **Qwen/Qwen3-VL-8B-Instruct**: Better quality (~16 GB)
### For OCR ### For OCR
- **deepseek-ocr**: General purpose - **deepseek-ai/DeepSeek-OCR**: General purpose
- **hunyuan-ocr**: Good for Chinese text - **Tencent-Hunyuan/HunyuanOCR**: Good for Chinese text
- **paddleocr-vl**: Lightweight option - **PaddlePaddle/PaddleOCR-VL**: Lightweight option
### For Speech Recognition ### For Speech Recognition
- **glm-asr-nano-2512**: Fast, accurate - **ZhipuAI/GLM-ASR-Nano-2512**: Fast, accurate
- **fun-asr-nano-2512**: Good for Chinese - **FunAudioLLM/Fun-ASR-Nano-2512**: Good for Chinese
- **qwen3asr-0.6b**: Lightweight - **Qwen/Qwen3-ASR-0.6B**: Lightweight
### For Text-to-Speech ### For Text-to-Speech
- **voxcpm1.5**: High quality Chinese - **OpenBMB/VoxCPM1.5**: High quality Chinese
### For Background Removal ### For Background Removal
- **rmbg2.0**: State-of-the-art results - **AI-ModelScope/RMBG-2.0**: State-of-the-art results
## Tips & Best Practices ## Tips & Best Practices
@@ -298,7 +298,7 @@ curl http://127.0.0.1:10100/chat/completions \
Begin with smaller models to understand the workflow: Begin with smaller models to understand the workflow:
```bash ```bash
aha download -m qwen3-0.6b aha download -m Qwen/Qwen3-0.6B
``` ```
### 2. Use GPU Acceleration ### 2. Use GPU Acceleration
@@ -316,12 +316,12 @@ cargo build --release --features metal
Download models when you have good internet: Download models when you have good internet:
```bash ```bash
aha download -m qwen3vl-2b aha download -m Qwen/Qwen3-VL-2B-Instruct
``` ```
Then use them later without internet: Then use them later without internet:
```bash ```bash
aha serv -m qwen3vl-2b --weight-path ~/.aha/Qwen/Qwen3-VL-2B-Instruct aha serv -m Qwen/Qwen3-VL-2B-Instruct --weight-path ~/.aha/Qwen/Qwen3-VL-2B-Instruct
``` ```
### 4. Manage Disk Space ### 4. Manage Disk Space
@@ -353,21 +353,21 @@ Activity Monitor
```bash ```bash
# Use a different port # Use a different port
aha cli -m qwen3-0.6b -p 8080 aha cli -m Qwen/Qwen3-0.6B -p 8080
``` ```
### Model Download Failed ### Model Download Failed
```bash ```bash
# Retry with more attempts # Retry with more attempts
aha download -m qwen3vl-2b --download-retries 5 aha download -m Qwen/Qwen3-VL-2B-Instruct --download-retries 5
``` ```
### Out of Memory ### Out of Memory
```bash ```bash
# Use a smaller model # Use a smaller model
aha cli -m qwen3-0.6b aha cli -m Qwen/Qwen3-0.6B
``` ```
## Next Steps ## Next Steps
docs/getting-started.zh-CN.md
+39 -39
View File
@@ -14,14 +14,14 @@ aha list
```bash ```bash
# 从下载一个小型文本模型开始 # 从下载一个小型文本模型开始
aha download -m qwen3-0.6b aha download -m Qwen/Qwen3-0.6B
``` ```
### 3. 启动服务 ### 3. 启动服务
```bash ```bash
# 启动 HTTP API 服务器 # 启动 HTTP API 服务器
aha cli -m qwen3-0.6b aha cli -m Qwen/Qwen3-0.6B
``` ```
服务将在 `http://127.0.0.1:10100` 上启动 服务将在 `http://127.0.0.1:10100` 上启动
@@ -34,7 +34,7 @@ aha cli -m qwen3-0.6b
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "user", "content": "你好，AHA！"} {"role": "user", "content": "你好，AHA！"}
], ],
@@ -79,13 +79,13 @@ AHA 是一个本地 AI 推理引擎，具有以下特点：
```bash ```bash
# 启动服务 # 启动服务
aha cli -m qwen3-0.6b aha cli -m Qwen/Qwen3-0.6B
# 在另一个终端中，发起请求 # 在另一个终端中，发起请求
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "system", "content": "你是一个有用的助手。"}, {"role": "system", "content": "你是一个有用的助手。"},
{"role": "user", "content": "用简单的术语解释量子计算。"} {"role": "user", "content": "用简单的术语解释量子计算。"}
@@ -99,13 +99,13 @@ curl http://127.0.0.1:10100/chat/completions \
```bash ```bash
# 启动视觉模型 # 启动视觉模型
aha cli -m qwen3vl-2b aha cli -m Qwen/Qwen3-VL-2B-Instruct
# 分析图像 # 分析图像
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3vl-2b", "model": "Qwen/Qwen3-VL-2B-Instruct",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -123,13 +123,13 @@ curl http://127.0.0.1:10100/chat/completions \
```bash ```bash
# 启动 OCR 模型 # 启动 OCR 模型
aha cli -m deepseek-ocr aha cli -m deepseek-ai/DeepSeek-OCR
# 从图像中提取文本 # 从图像中提取文本
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "deepseek-ocr", "model": "deepseek-ai/DeepSeek-OCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -146,13 +146,13 @@ curl http://127.0.0.1:10100/chat/completions \
```bash ```bash
# 启动 ASR 模型 # 启动 ASR 模型
aha cli -m glm-asr-nano-2512 aha cli -m ZhipuAI/GLM-ASR-Nano-2512
# 转写音频 # 转写音频
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "glm-asr-nano-2512", "model": "ZhipuAI/GLM-ASR-Nano-2512",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -169,13 +169,13 @@ curl http://127.0.0.1:10100/chat/completions \
```bash ```bash
# 启动 TTS 模型 # 启动 TTS 模型
aha cli -m voxcpm1.5 aha cli -m OpenBMB/VoxCPM1.5
# 生成语音 # 生成语音
curl http://127.0.0.1:10100/audio/speech \ curl http://127.0.0.1:10100/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "voxcpm1.5", "model": "OpenBMB/VoxCPM1.5",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -192,13 +192,13 @@ curl http://127.0.0.1:10100/audio/speech \
```bash ```bash
# 启动 RMBG2.0 模型 # 启动 RMBG2.0 模型
aha cli -m rmbg2.0 aha cli -m AI-ModelScope/RMBG-2.0
# 移除图像背景 # 移除图像背景
curl http://127.0.0.1:10100/images/remove_background \ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "AI-ModelScope/RMBG-2.0",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -214,7 +214,7 @@ curl http://127.0.0.1:10100/images/remove_background \
```bash ```bash
# 直接运行推理，无需启动 HTTP 服务器 # 直接运行推理，无需启动 HTTP 服务器
aha run -m qwen3-0.6b \ aha run -m Qwen/Qwen3-0.6B \
-i "写一首关于AI的俳句" \ -i "写一首关于AI的俳句" \
--weight-path ~/.aha/Qwen/Qwen3-0.6B --weight-path ~/.aha/Qwen/Qwen3-0.6B
``` ```
@@ -225,21 +225,21 @@ aha run -m qwen3-0.6b \
```bash ```bash
# 使用端口 8080 而不是默认的 10100 # 使用端口 8080 而不是默认的 10100
aha cli -m qwen3-0.6b -p 8080 aha cli -m Qwen/Qwen3-0.6B -p 8080
``` ```
### 绑定到所有接口 ### 绑定到所有接口
```bash ```bash
# 允许外部访问（请谨慎使用） # 允许外部访问（请谨慎使用）
aha cli -m qwen3-0.6b -a 0.0.0.0 -p 8080 aha cli -m Qwen/Qwen3-0.6B -a 0.0.0.0 -p 8080
``` ```
### 使用本地模型 ### 使用本地模型
```bash ```bash
# 跳过下载，使用现有模型 # 跳过下载，使用现有模型
aha serv -m qwen3-0.6b \ aha serv -m Qwen/Qwen3-0.6B \
--weight-path /path/to/model \ --weight-path /path/to/model \
-p 8080 -p 8080
``` ```
@@ -248,7 +248,7 @@ aha serv -m qwen3-0.6b \
```bash ```bash
# 将模型下载到特定目录 # 将模型下载到特定目录
aha download -m qwen3vl-2b -s /data/models aha download -m Qwen/Qwen3-VL-2B-Instruct -s /data/models
``` ```
## 流式响应 ## 流式响应
@@ -259,7 +259,7 @@ aha download -m qwen3vl-2b -s /data/models
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "Qwen/Qwen3-0.6B",
"messages": [ "messages": [
{"role": "user", "content": "给我讲个故事"} {"role": "user", "content": "给我讲个故事"}
], ],
@@ -270,28 +270,28 @@ curl http://127.0.0.1:10100/chat/completions \
## 模型选择指南 ## 模型选择指南
### 文本生成 ### 文本生成
- **qwen3-0.6b**：快速、轻量级（~1.2 GB） - **Qwen/Qwen3-0.6B**：快速、轻量级（~1.2 GB）
- **minicpm4-0.5b**：小型、高效（~1 GB） - **OpenBMB/MiniCPM4-0.5B**：小型、高效（~1 GB）
### 视觉任务 ### 视觉任务
- **qwen3vl-2b**：平衡性能（~4 GB） - **Qwen/Qwen3-VL-2B-Instruct**：平衡性能（~4 GB）
- **qwen3vl-8b**：更高质量（~16 GB） - **Qwen/Qwen3-VL-8B-Instruct**：更高质量（~16 GB）
### OCR ### OCR
- **deepseek-ocr**：通用 - **deepseek-ai/DeepSeek-OCR**：通用
- **hunyuan-ocr**：适合中文文本 - **Tencent-Hunyuan/HunyuanOCR**：适合中文文本
- **paddleocr-vl**：轻量级选项 - **PaddlePaddle/PaddleOCR-VL**：轻量级选项
### 语音识别 ### 语音识别
- **glm-asr-nano-2512**：快速、准确 - **ZhipuAI/GLM-ASR-Nano-2512**：快速、准确
- **fun-asr-nano-2512**：适合中文 - **FunAudioLLM/Fun-ASR-Nano-2512**：适合中文
- **qwen3asr-0.6b**：轻量级 - **Qwen/Qwen3-ASR-0.6B**：轻量级
### 文本转语音 ### 文本转语音
- **voxcpm1.5**：高质量中文 - **OpenBMB/VoxCPM1.5**：高质量中文
### 背景移除 ### 背景移除
- **rmbg2.0**：最先进的结果 - **AI-ModelScope/RMBG-2.0**：最先进的结果
## 提示与最佳实践 ## 提示与最佳实践
@@ -299,7 +299,7 @@ curl http://127.0.0.1:10100/chat/completions \
从小型模型开始了解工作流程： 从小型模型开始了解工作流程：
```bash ```bash
aha download -m qwen3-0.6b aha download -m Qwen/Qwen3-0.6B
``` ```
### 2. 使用 GPU 加速 ### 2. 使用 GPU 加速
@@ -317,12 +317,12 @@ cargo build --release --features metal
在网络良好时下载模型： 在网络良好时下载模型：
```bash ```bash
aha download -m qwen3vl-2b aha download -m Qwen/Qwen3-VL-2B-Instruct
``` ```
稍后在没有网络的情况下使用： 稍后在没有网络的情况下使用：
```bash ```bash
aha serv -m qwen3vl-2b --weight-path ~/.aha/Qwen/Qwen3-VL-2B-Instruct aha serv -m Qwen/Qwen3-VL-2B-Instruct --weight-path ~/.aha/Qwen/Qwen3-VL-2B-Instruct
``` ```
### 4. 管理磁盘空间 ### 4. 管理磁盘空间
@@ -354,21 +354,21 @@ nvidia-smi # 对于 NVIDIA GPU
```bash ```bash
# 使用不同的端口 # 使用不同的端口
aha cli -m qwen3-0.6b -p 8080 aha cli -m Qwen/Qwen3-0.6B -p 8080
``` ```
### 模型下载失败 ### 模型下载失败
```bash ```bash
# 重试更多次数 # 重试更多次数
aha download -m qwen3vl-2b --download-retries 5 aha download -m Qwen/Qwen3-VL-2B-Instruct --download-retries 5
``` ```
### 内存不足 ### 内存不足
```bash ```bash
# 使用更小的模型 # 使用更小的模型
aha cli -m qwen3-0.6b aha cli -m Qwen/Qwen3-0.6B
``` ```
## 后续步骤 ## 后续步骤
docs/installation.md
+36 -24
View File
@@ -284,24 +284,36 @@ Expected output for `aha list`:
Model Name ModelScope ID Model Name ModelScope ID
----------------------------------------------------------- -----------------------------------------------------------
minicpm4-0.5b OpenBMB/MiniCPM4-0.5B LiquidAI/LFM2-1.2B LiquidAI/LFM2-1.2B ✔
qwen2.5vl-3b Qwen/Qwen2.5-VL-3B-Instruct LiquidAI/LFM2.5-1.2B-Instruct LiquidAI/LFM2.5-1.2B-Instruct ✔
qwen2.5vl-7b Qwen/Qwen2.5-VL-7B-Instruct LiquidAI/LFM2.5-VL-1.6B LiquidAI/LFM2.5-VL-1.6B ✔
qwen3-0.6b Qwen/Qwen3-0.6B LiquidAI/LFM2-VL-1.6B LiquidAI/LFM2-VL-1.6B ✔
qwen3asr-0.6b Qwen/Qwen3-ASR-0.6B OpenBMB/MiniCPM4-0.5B OpenBMB/MiniCPM4-0.5B ✔
qwen3asr-1.7b Qwen/Qwen3-ASR-1.7B Qwen/Qwen2.5-VL-3B-Instruct Qwen/Qwen2.5-VL-3B-Instruct ✔
qwen3vl-4b Qwen/Qwen3-VL-2B-Instruct Qwen/Qwen2.5-VL-7B-Instruct Qwen/Qwen2.5-VL-7B-Instruct
qwen3vl-4b Qwen/Qwen3-VL-4B-Instruct Qwen/Qwen3-0.6B Qwen/Qwen3-0.6B ✔
qwen3vl-8b Qwen/Qwen3-VL-8B-Instruct Qwen/Qwen3.5-0.8B Qwen/Qwen3.5-0.8B ✔
qwen3vl-32b Qwen/Qwen3-VL-32B-Instruct Qwen/Qwen3.5-2B Qwen/Qwen3.5-2B
deepseek-ocr deepseek-ai/DeepSeek-OCR Qwen/Qwen3.5-4B Qwen/Qwen3.5-4B
hunyuan-ocr Tencent-Hunyuan/HunyuanOCR Qwen/Qwen3.5-9B Qwen/Qwen3.5-9B
paddleocr-vl PaddlePaddle/PaddleOCR-VL qwen3.5-gguf qwen3.5-gguf
rmbg2.0 AI-ModelScope/RMBG-2.0 Qwen/Qwen3-ASR-0.6B Qwen/Qwen3-ASR-0.6B ✔
voxcpm OpenBMB/VoxCPM-0.5B Qwen/Qwen3-ASR-1.7B Qwen/Qwen3-ASR-1.7B
voxcpm1.5 OpenBMB/VoxCPM1.5 Qwen/Qwen3-VL-2B-Instruct Qwen/Qwen3-VL-2B-Instruct ✔
glm-asr-nano-2512 ZhipuAI/GLM-ASR-Nano-2512 Qwen/Qwen3-VL-4B-Instruct Qwen/Qwen3-VL-4B-Instruct
fun-asr-nano-2512 FunAudioLLM/Fun-ASR-Nano-2512 Qwen/Qwen3-VL-8B-Instruct Qwen/Qwen3-VL-8B-Instruct
Qwen/Qwen3-VL-32B-Instruct Qwen/Qwen3-VL-32B-Instruct
deepseek-ai/DeepSeek-OCR deepseek-ai/DeepSeek-OCR ✔
deepseek-ai/DeepSeek-OCR-2 deepseek-ai/DeepSeek-OCR-2
Tencent-Hunyuan/HunyuanOCR Tencent-Hunyuan/HunyuanOCR ✔
PaddlePaddle/PaddleOCR-VL PaddlePaddle/PaddleOCR-VL ✔
PaddlePaddle/PaddleOCR-VL-1.5 PaddlePaddle/PaddleOCR-VL-1.5
AI-ModelScope/RMBG-2.0 AI-ModelScope/RMBG-2.0 ✔
OpenBMB/VoxCPM-0.5B OpenBMB/VoxCPM-0.5B ✔
OpenBMB/VoxCPM1.5 OpenBMB/VoxCPM1.5 ✔
ZhipuAI/GLM-ASR-Nano-2512 ZhipuAI/GLM-ASR-Nano-2512 ✔
FunAudioLLM/Fun-ASR-Nano-2512 FunAudioLLM/Fun-ASR-Nano-2512 ✔
ZhipuAI/GLM-OCR ZhipuAI/GLM-OCR
``` ```
@@ -398,18 +410,18 @@ Approximate download sizes for popular models:
| Model | Size | RAM Usage | | Model | Size | RAM Usage |
|-------|------|-----------| |-------|------|-----------|
| qwen3-0.6b | ~1.2 GB | ~2 GB | | Qwen/Qwen3-0.6B | ~1.2 GB | ~2 GB |
| qwen3vl-2b | ~4 GB | ~6 GB | | Qwen/Qwen3-VL-2B-Instruct | ~4 GB | ~6 GB |
| qwen3vl-8b | ~16 GB | ~20 GB | | Qwen/Qwen3-VL-8B-Instruct | ~16 GB | ~20 GB |
| qwen3vl-32b | ~64 GB | ~70 GB | | Qwen/Qwen3-VL-32B-Instruct | ~64 GB | ~70 GB |
## Next Steps ## Next Steps
After successful installation: After successful installation:
1. Read the [Getting Started Guide](./getting-started.md) 1. Read the [Getting Started Guide](./getting-started.md)
2. Download your first model: `aha download -m qwen3-0.6b` 2. Download your first model: `aha download -m Qwen/Qwen3-0.6B`
3. Start the service: `aha cli -m qwen3-0.6b` 3. Start the service: `aha cli -m Qwen/Qwen3-0.6B`
4. Explore the [API Reference](./api.md) 4. Explore the [API Reference](./api.md)
## See Also ## See Also
docs/installation.zh-CN.md
+23 -23
View File
@@ -283,24 +283,24 @@ aha list
Model Name ModelScope ID Model Name ModelScope ID
----------------------------------------------------------- -----------------------------------------------------------
minicpm4-0.5b OpenBMB/MiniCPM4-0.5B OpenBMB/MiniCPM4-0.5B OpenBMB/MiniCPM4-0.5B
qwen2.5vl-3b Qwen/Qwen2.5-VL-3B-Instruct Qwen/Qwen2.5-VL-3B-Instruct Qwen/Qwen2.5-VL-3B-Instruct
qwen2.5vl-7b Qwen/Qwen2.5-VL-7B-Instruct Qwen/Qwen2.5-VL-7B-Instruct Qwen/Qwen2.5-VL-7B-Instruct
qwen3-0.6b Qwen/Qwen3-0.6B Qwen/Qwen3-0.6B Qwen/Qwen3-0.6B
qwen3asr-0.6b Qwen/Qwen3-ASR-0.6B Qwen/Qwen3-ASR-0.6B Qwen/Qwen3-ASR-0.6B
qwen3asr-1.7b Qwen/Qwen3-ASR-1.7B Qwen/Qwen3-ASR-1.7B Qwen/Qwen3-ASR-1.7B
qwen3vl-4b Qwen/Qwen3-VL-2B-Instruct Qwen/Qwen3-VL-4B-Instruct Qwen/Qwen3-VL-2B-Instruct
qwen3vl-4b Qwen/Qwen3-VL-4B-Instruct Qwen/Qwen3-VL-4B-Instruct Qwen/Qwen3-VL-4B-Instruct
qwen3vl-8b Qwen/Qwen3-VL-8B-Instruct Qwen/Qwen3-VL-8B-Instruct Qwen/Qwen3-VL-8B-Instruct
qwen3vl-32b Qwen/Qwen3-VL-32B-Instruct Qwen/Qwen3-VL-32B-Instruct Qwen/Qwen3-VL-32B-Instruct
deepseek-ocr deepseek-ai/DeepSeek-OCR deepseek-ai/DeepSeek-OCR deepseek-ai/DeepSeek-OCR
hunyuan-ocr Tencent-Hunyuan/HunyuanOCR Tencent-Hunyuan/HunyuanOCR Tencent-Hunyuan/HunyuanOCR
paddleocr-vl PaddlePaddle/PaddleOCR-VL PaddlePaddle/PaddleOCR-VL PaddlePaddle/PaddleOCR-VL
rmbg2.0 AI-ModelScope/RMBG-2.0 AI-ModelScope/RMBG-2.0 AI-ModelScope/RMBG-2.0
voxcpm OpenBMB/VoxCPM-0.5B voxcpm OpenBMB/VoxCPM-0.5B
voxcpm1.5 OpenBMB/VoxCPM1.5 OpenBMB/VoxCPM1.5 OpenBMB/VoxCPM1.5
glm-asr-nano-2512 ZhipuAI/GLM-ASR-Nano-2512 ZhipuAI/GLM-ASR-Nano-2512 ZhipuAI/GLM-ASR-Nano-2512
fun-asr-nano-2512 FunAudioLLM/Fun-ASR-Nano-2512 FunAudioLLM/Fun-ASR-Nano-2512 FunAudioLLM/Fun-ASR-Nano-2512
``` ```
@@ -397,18 +397,18 @@ fun-asr-nano-2512 FunAudioLLM/Fun-ASR-Nano-2512
| 模型 | 大小 | RAM 使用 | | 模型 | 大小 | RAM 使用 |
|------|------|----------| |------|------|----------|
| qwen3-0.6b | ~1.2 GB | ~2 GB | | Qwen/Qwen3-0.6B | ~1.2 GB | ~2 GB |
| qwen3vl-2b | ~4 GB | ~6 GB | | Qwen/Qwen3-VL-2B-Instruct | ~4 GB | ~6 GB |
| qwen3vl-8b | ~16 GB | ~20 GB | | Qwen/Qwen3-VL-8B-Instruct | ~16 GB | ~20 GB |
| qwen3vl-32b | ~64 GB | ~70 GB | | Qwen/Qwen3-VL-32B-Instruct | ~64 GB | ~70 GB |
## 后续步骤 ## 后续步骤
成功安装后： 成功安装后：
1. 阅读[快速入门指南](./getting-started.zh-CN.md) 1. 阅读[快速入门指南](./getting-started.zh-CN.md)
2. 下载您的第一个模型：`aha download -m qwen3-0.6b` 2. 下载您的第一个模型：`aha download -m Qwen/Qwen3-0.6B`
3. 启动服务：`aha cli -m qwen3-0.6b` 3. 启动服务：`aha cli -m Qwen/Qwen3-0.6B`
4. 探索 [API 参考](./api.zh-CN.md) 4. 探索 [API 参考](./api.zh-CN.md)
## 另见 ## 另见
docs/supported-models.md
+31 -38
View File
@@ -4,60 +4,53 @@ aha supports a growing collection of state-of-the-art AI models across multiple
## Language Model ## Language Model
| Model | Parameters | Description | Use Case | License | | Model | Parameters | Model Id | License |
|-------|-----------|-------------|----------|---------| |-------|-----------|--------|---------|
| **Qwen3-0.6B** | 0.6B | Latest generation | Advanced reasoning | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Qwen3-0.6B** | 0.6B | Qwen/Qwen3-0.6B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **MiniCPM4-0.5B** | 0.5B | Efficient lightweight | Edge deployment | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **MiniCPM4-0.5B** | 0.5B | OpenBMB/MiniCPM4-0.5B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **LFM2-1.2B** | 1.2B | Efficient lightweight | Edge deployment | [lfm1.0](https://huggingface.co/LiquidAI/LFM2-1.2B/blob/main/LICENSE) | | **LFM2-1.2B** | 1.2B | LiquidAI/LFM2-1.2B | [lfm1.0](https://huggingface.co/LiquidAI/LFM2-1.2B/blob/main/LICENSE) |
| **LFM2.5-1.2B-Instruct** | 1.2B | Efficient lightweight | Edge deployment | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) | | **LFM2.5-1.2B-Instruct** | 1.2B | LiquidAI/LFM2.5-1.2B-Instruct | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) |
## Vision & Multimodal ## Vision & Multimodal
| Model | Parameters | Description | License | | Model | Parameters | Model Id | License |
|-------|-----------|-------------|---------| |-------|-----------|----------|---------|
| **Qwen2.5-VL-3B** | 3B | Image understanding | [Qwen Research License](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE) | | **Qwen2.5-VL** | 3B <br> 7B | Qwen/Qwen2.5-VL-3B-Instruct <br> Qwen/Qwen2.5-VL-7B-Instruct | [Qwen 研究许可协议](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE) <br> [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen2.5-VL-7B** | 7B | Image understanding | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Qwen3-VL** | 2B <br> 4B <br> 8B <br> 32B | Qwen/Qwen3-VL-2B-Instruct <br> Qwen/Qwen3-VL-4B-Instruct <br> Qwen/Qwen3-VL-8B-Instruct <br> Qwen/Qwen3-VL-32B-Instruct | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-VL-2B** | 2B | Enhanced multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Qwen3.5** | 0.8B <br> 2B <br> 4B <br> 9B | Qwen/Qwen3.5-0.8B <br> Qwen/Qwen3.5-2B <br> Qwen/Qwen3.5-4B <br> Qwen/Qwen3.5-9B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-VL-4B** | 4B | Enhanced multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **LFM2.5-VL-1.6B** | 1.6B | LiquidAI/LFM2.5-VL-1.6B | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) |
| **Qwen3-VL-8B** | 8B | Enhanced multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **LFM2-VL-1.6B** | 1.6B | LiquidAI/LFM2-VL-1.6B | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) |
| **Qwen3-VL-32B** | 32B | Enhanced multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-0.8B** | 0.8B | Native Multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-2B** | 2B | Native Multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-4B** | 4B | Native Multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B** | 9B | Native Multimodal | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## OCR ## OCR
| Model | Languages | Type | Strength | License | | Model | Languages | Model Id | License |
|-------|-----------|------|----------|---------| |-------|-----------|--------|---------|
| **PaddleOCR-VL** | Multi | Lightweight | General documents | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **PaddleOCR-VL** | Multi | PaddlePaddle/PaddleOCR-VL <br> PaddlePaddle/PaddleOCR-VL-1.5 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **PaddleOCR-VL1.5** | Multi | Lightweight | General documents | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Hunyuan-OCR** | Chinese | Tencent-Hunyuan/HunyuanOCR | [Tencent Hunyuan Community License](https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE) |
| **Hunyuan-OCR** | Chinese | Deep learning | Complex layouts | [Tencent Hunyuan Community License](https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE) | | **DeepSeek-OCR** | Multi | deepseek-ai/DeepSeek-OCR <br> deepseek-ai/DeepSeek-OCR-2 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
| **DeepSeek-OCR** | Multi | Scene text | Natural images | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) | | **GLM-OCR** | 8 | ZhipuAI/GLM-OCR | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
| **GLM-OCR** | 8 | Scene text | complex document | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
## Speech Recognition (ASR) ## Speech Recognition (ASR)
| Model | Parameters | Language | Real-time | Speed | License | | Model | Parameters | Language | Model Id | License |
|-------|-----------|----------|-----------|-------|---------| |-------|-----------|----------|----------|---------|
| **Fun-ASR-Nano-2512** | 2G | Chinese/English | Yes | Fast | Not Specified | | **Fun-ASR-Nano-2512** | 2G | Chinese/English | FunAudioLLM/Fun-ASR-Nano-2512 | Not Specified |
| **GLM-ASR-Nano-2512** | 4.5G | Chinese/English | Yes | Fast | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) | | **GLM-ASR-Nano-2512** | 4.5G | Chinese/English | ZhipuAI/GLM-ASR-Nano-2512 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
| **Qwen3-ASR-0.6B** | 0.6B | Chinese/English | Yes | Fast | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Qwen3-ASR** | 0.6B <br> 1.7B | Chinese/English | Qwen/Qwen3-ASR-0.6B <br> Qwen/Qwen3-ASR-1.7B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-ASR-1.7B** | 1.7B | Chinese/English | Yes | Fast | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Audio Generation ## Audio Generation
| Model | Parameters | Type | Description | License | | Model | version | Model Id | License |
|-------|-----------|------|-------------|---------| |-------|-----------|---------|---------|
| **VoxCPM-0.5B** | 0.5B | Voice Codec | Neural audio codec | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **VoxCPM** | 1<br>1.5 | OpenBMB/VoxCPM-0.5B <br> OpenBMB/VoxCPM1.5 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **VoxCPM1.5** | - | Voice Codec | Enhanced voice generation | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## Image Processing ## Image Processing
| Model | Type | Description | License | | Model | Type | Model Id | License |
|-------|------|-------------|---------| |-------|------|-------------|---------|
| **RMBG-2.0** | Background Removal | Remove image backgrounds | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/deed.en) | | **RMBG-2.0** | Background Removal | AI-ModelScope/RMBG-2.0 | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/deed.en) |
## Model Sources ## Model Sources
docs/supported-models.zh-CN.md
+30 -38
View File
@@ -4,60 +4,52 @@ aha 支持多个领域的最先进 AI 模型集合。
## 文本生成 ## 文本生成
| 模型 | 参数量 | 描述 | 使用场景 | 开源协议 | | 模型 | 参数量 | 模型id | 开源协议 |
|------|--------|------|----------|---------| |------|--------|------|---------|
| **Qwen3-0.6B** | 0.6B | 最新一代 | 高级推理 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Qwen3-0.6B** | 0.6B | Qwen/Qwen3-0.6B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **MiniCPM4-0.5B** | 0.5B | 高效轻量级 | 边缘部署 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **MiniCPM4-0.5B** | 0.5B | OpenBMB/MiniCPM4-0.5B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **LFM2-1.2B** | 1.2B | 高效轻量级 | 边缘部署 | [lfm1.0](https://huggingface.co/LiquidAI/LFM2-1.2B/blob/main/LICENSE) | | **LFM2-1.2B** | 1.2B | LiquidAI/LFM2-1.2B | [lfm1.0](https://huggingface.co/LiquidAI/LFM2-1.2B/blob/main/LICENSE) |
| **LFM2.5-1.2B-Instruct** | 1.2B | 高效轻量级 | 边缘部署 | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) | | **LFM2.5-1.2B-Instruct** | 1.2B | LiquidAI/LFM2.5-1.2B-Instruct | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) |
## 视觉与多模态 ## 视觉与多模态
| 模型 | 参数量 | 描述 | 开源协议 | | 模型 | 参数量 | 模型id | 开源协议 |
|------|--------|------|--------|---------| |------|--------|------|---------|
| **Qwen2.5-VL-3B** | 3B | 图像理解 | [Qwen 研究许可协议](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE) | | **Qwen2.5-VL** | 3B <br> 7B | Qwen/Qwen2.5-VL-3B-Instruct <br> Qwen/Qwen2.5-VL-7B-Instruct | [Qwen 研究许可协议](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE) <br> [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen2.5-VL-7B** | 7B | 图像理解 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Qwen3-VL** | 2B <br> 4B <br> 8B <br> 32B | Qwen/Qwen3-VL-2B-Instruct <br> Qwen/Qwen3-VL-4B-Instruct <br> Qwen/Qwen3-VL-8B-Instruct <br> Qwen/Qwen3-VL-32B-Instruct | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-VL-2B** | 2B | 增强多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Qwen3.5** | 0.8B <br> 2B <br> 4B <br> 9B | Qwen/Qwen3.5-0.8B <br> Qwen/Qwen3.5-2B <br> Qwen/Qwen3.5-4B <br> Qwen/Qwen3.5-9B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-VL-4B** | 4B | 增强多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **LFM2.5-VL-1.6B** | 1.6B | LiquidAI/LFM2.5-VL-1.6B | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) |
| **Qwen3-VL-8B** | 8B | 增强多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **LFM2-VL-1.6B** | 1.6B | LiquidAI/LFM2-VL-1.6B | [lfm1.0](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE) |
| **Qwen3-VL-32B** | 32B | 增强多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-0.8B** | 0.8B | 原生多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-2B** | 2B | 原生多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-4B** | 4B | 原生多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3.5-9B** | 9B | 原生多模态 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## OCR ## OCR
| 模型 | 语言 | 类型 | 优势 | 开源协议 | | 模型 | 语言 | 模型id | 开源协议 |
|------|------|------|------|---------| |------|------|------|---------|
| **PaddleOCR-VL** | 多语言 | 轻量级 | 通用文档 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **PaddleOCR-VL** | 多语言 | PaddlePaddle/PaddleOCR-VL <br> PaddlePaddle/PaddleOCR-VL-1.5 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **PaddleOCR-VL1.5** | 多语言 | 轻量级 | 通用文档 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Hunyuan-OCR** | 中文 | Tencent-Hunyuan/HunyuanOCR | [腾讯混元社区许可协议](https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE) |
| **Hunyuan-OCR** | 中文 | 深度学习 | 复杂布局 | [腾讯混元社区许可协议](https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE) | | **DeepSeek-OCR** | 多语言 | deepseek-ai/DeepSeek-OCR <br> deepseek-ai/DeepSeek-OCR-2 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
| **DeepSeek-OCR** | 多语言 | 场景文字 | 自然图像 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) | | **GLM-OCR** | 8 | ZhipuAI/GLM-OCR | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
| **GLM-OCR** | 8 | 场景文字 | 复杂文档 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
## 语音识别 (ASR) ## 语音识别 (ASR)
| 模型 | 参数量 | 语言 | 实时 | 速度 | 开源协议 | | 模型 | 参数量 | 语言 | 模型id | 开源协议 |
|------|--------|------|------|------|---------| |------|--------|------|-----|---------|
| **Fun-ASR-Nano-2512** | 2512M | 中/英 | 是 | 快速 | 未标明 | | **Fun-ASR-Nano-2512** | - | 中/英 | FunAudioLLM/Fun-ASR-Nano-2512 | 未标明 |
| **GLM-ASR-Nano-2512** | 2512M | 中/英 | 是 | 快速 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) | | **GLM-ASR-Nano-2512** | - | 中/英 | ZhipuAI/GLM-ASR-Nano-2512 | [MIT](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) |
| **Qwen3-ASR-0.6B** | 0.6B | 中/英 | 是 | 快速 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **Qwen3-ASR** | 0.6B <br> 1.7B | 中/英 | Qwen/Qwen3-ASR-0.6B <br> Qwen/Qwen3-ASR-1.7B | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **Qwen3-ASR-1.7B** | 1.7B | 中/英 | 是 | 快速 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## 语音生成 ## 语音生成
| 模型 | 参数量 | 类型 | 描述 | 开源协议 | | 模型 | 版本 | 模型id | 开源协议 |
|------|--------|------|------|---------| |------|--------|------|------|
| **VoxCPM-0.5B** | 0.5B | 语音编解码器 | 神经音频编解码 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) | | **VoxCPM** | 1<br>1.5 | OpenBMB/VoxCPM-0.5B <br> OpenBMB/VoxCPM1.5 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
| **VoxCPM1.5** | - | 语音编解码器 | 增强语音生成 | [Apache 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md) |
## 图像处理 ## 图像处理
| 模型 | 类型 | 描述 | 开源协议 | | 模型 | 类型 | 模型id | 开源协议 |
|------|------|------|---------| |------|------|-----|---------|
| **RMBG-2.0** | 背景移除 | 移除图像背景 | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/deed.zh-hans) | | **RMBG-2.0** | 背景移除 | AI-ModelScope/RMBG-2.0 | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/deed.zh-hans) |
## 模型来源 ## 模型来源
scripts/download_and_run.sh
+35 -35
View File
@@ -11,27 +11,27 @@ show_help() {
echo "Usage: $0 [model_alias]" echo "Usage: $0 [model_alias]"
echo "" echo ""
echo "Arguments:" echo "Arguments:"
echo " model_alias The model alias to download (e.g., voxcpm, qwen2.5vl-3b)" echo " model_alias The model alias to download (e.g., voxcpm, Qwen/Qwen2.5-VL-3B-Instruct)"
echo "" echo ""
echo "Available models:" echo "Available models:"
echo " minicpm4-0.5b" echo " OpenBMB/MiniCPM4-0.5B"
echo " qwen2.5vl-3b" echo " Qwen/Qwen2.5-VL-3B-Instruct"
echo " qwen2.5vl-7b" echo " Qwen/Qwen2.5-VL-7B-Instruct"
echo " qwen3-0.6b" echo " Qwen/Qwen3-0.6B"
echo " qwen3asr-0.6b" echo " Qwen/Qwen3-ASR-0.6B"
echo " qwen3asr-1.7b" echo " Qwen/Qwen3-ASR-1.7B"
echo " qwen3vl-2b" echo " Qwen/Qwen3-VL-2B-Instruct"
echo " qwen3vl-4b" echo " Qwen/Qwen3-VL-4B-Instruct"
echo " qwen3vl-8b" echo " Qwen/Qwen3-VL-8B-Instruct"
echo " qwen3vl-32b" echo " Qwen/Qwen3-VL-32B-Instruct"
echo " deepseek-ocr" echo " deepseek-ai/DeepSeek-OCR"
echo " hunyuan-ocr" echo " Tencent-Hunyuan/HunyuanOCR"
echo " paddleocr-vl" echo " PaddlePaddle/PaddleOCR-VL"
echo " rmbg2.0" echo " AI-ModelScope/RMBG-2.0"
echo " voxcpm" echo " voxcpm"
echo " voxcpm1.5" echo " OpenBMB/VoxCPM1.5"
echo " glm-asr-nano-2512" echo " ZhipuAI/GLM-ASR-Nano-2512"
echo " fun-asr-nano-2512" echo " FunAudioLLM/Fun-ASR-Nano-2512"
echo "" echo ""
exit 1 exit 1
} }
@@ -46,58 +46,58 @@ MODEL_ALIAS=$1
# Map alias to Repo ID # Map alias to Repo ID
MODEL_ID="" MODEL_ID=""
case $MODEL_ALIAS in case $MODEL_ALIAS in
"minicpm4-0.5b") "OpenBMB/MiniCPM4-0.5B")
MODEL_ID="OpenBMB/MiniCPM4-0.5B" MODEL_ID="OpenBMB/MiniCPM4-0.5B"
;; ;;
"qwen2.5vl-3b") "Qwen/Qwen2.5-VL-3B-Instruct")
MODEL_ID="Qwen/Qwen2.5-VL-3B-Instruct" MODEL_ID="Qwen/Qwen2.5-VL-3B-Instruct"
;; ;;
"qwen2.5vl-7b") "Qwen/Qwen2.5-VL-7B-Instruct")
MODEL_ID="Qwen/Qwen2.5-VL-7B-Instruct" MODEL_ID="Qwen/Qwen2.5-VL-7B-Instruct"
;; ;;
"qwen3-0.6b") "Qwen/Qwen3-0.6B")
MODEL_ID="Qwen/Qwen3-0.6B" MODEL_ID="Qwen/Qwen3-0.6B"
;; ;;
"qwen3asr-0.6b") "Qwen/Qwen3-ASR-0.6B")
MODEL_ID="Qwen/Qwen3-ASR-0.6B" MODEL_ID="Qwen/Qwen3-ASR-0.6B"
;; ;;
"qwen3asr-1.7b") "Qwen/Qwen3-ASR-1.7B")
MODEL_ID="Qwen/Qwen3-ASR-1.7B" MODEL_ID="Qwen/Qwen3-ASR-1.7B"
;; ;;
"qwen3vl-2b") "Qwen/Qwen3-VL-2B-Instruct")
MODEL_ID="Qwen/Qwen3-VL-2B-Instruct" MODEL_ID="Qwen/Qwen3-VL-2B-Instruct"
;; ;;
"qwen3vl-4b") "Qwen/Qwen3-VL-4B-Instruct")
MODEL_ID="Qwen/Qwen3-VL-4B-Instruct" MODEL_ID="Qwen/Qwen3-VL-4B-Instruct"
;; ;;
"qwen3vl-8b") "Qwen/Qwen3-VL-8B-Instruct")
MODEL_ID="Qwen/Qwen3-VL-8B-Instruct" MODEL_ID="Qwen/Qwen3-VL-8B-Instruct"
;; ;;
"qwen3vl-32b") "Qwen/Qwen3-VL-32B-Instruct")
MODEL_ID="Qwen/Qwen3-VL-32B-Instruct" MODEL_ID="Qwen/Qwen3-VL-32B-Instruct"
;; ;;
"deepseek-ocr") "deepseek-ai/DeepSeek-OCR")
MODEL_ID="deepseek-ai/DeepSeek-OCR" MODEL_ID="deepseek-ai/DeepSeek-OCR"
;; ;;
"hunyuan-ocr") "Tencent-Hunyuan/HunyuanOCR")
MODEL_ID="Tencent-Hunyuan/HunyuanOCR" MODEL_ID="Tencent-Hunyuan/HunyuanOCR"
;; ;;
"paddleocr-vl") "PaddlePaddle/PaddleOCR-VL")
MODEL_ID="PaddlePaddle/PaddleOCR-VL" MODEL_ID="PaddlePaddle/PaddleOCR-VL"
;; ;;
"rmbg2.0") "AI-ModelScope/RMBG-2.0")
MODEL_ID="briaai/RMBG-2.0" MODEL_ID="briaai/RMBG-2.0"
;; ;;
"voxcpm") "voxcpm")
MODEL_ID="openbmb/VoxCPM-0.5B" MODEL_ID="openbmb/VoxCPM-0.5B"
;; ;;
"voxcpm1.5") "OpenBMB/VoxCPM1.5")
MODEL_ID="openbmb/VoxCPM1.5" MODEL_ID="openbmb/VoxCPM1.5"
;; ;;
"glm-asr-nano-2512") "ZhipuAI/GLM-ASR-Nano-2512")
MODEL_ID="zai-org/GLM-ASR-Nano-2512" MODEL_ID="zai-org/GLM-ASR-Nano-2512"
;; ;;
"fun-asr-nano-2512") "FunAudioLLM/Fun-ASR-Nano-2512")
MODEL_ID="FunAudioLLM/Fun-ASR-Nano-2512" MODEL_ID="FunAudioLLM/Fun-ASR-Nano-2512"
;; ;;
*) *)
src/api/mod.rs
+31 -102
View File
@@ -2,7 +2,7 @@ use std::pin::pin;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, OnceLock}; use std::sync::{Arc, OnceLock};
use aha::models::{GenerateModel, ModelInstance, WhichModel, load_model}; use aha::models::{GenerateModel, ModelInstance, common::model_mapping::WhichModel, load_model};
use aha::params::chat::ChatCompletionParameters; use aha::params::chat::ChatCompletionParameters;
use aha::process::cleanup_pid_file; use aha::process::cleanup_pid_file;
use aha::utils::string_to_static_str; use aha::utils::string_to_static_str;
@@ -243,70 +243,34 @@ struct ErrorResponse {
error: String, error: String,
} }
/// Convert WhichModel to a display-friendly model ID (kebab-case) // /// Get the owner/organization name for a model
fn which_model_to_id(which_model: WhichModel) -> &'static str { // fn which_model_to_owner(which_model: WhichModel) -> &'static str {
match which_model { // match which_model {
WhichModel::MiniCPM4_0_5B => "minicpm4-0.5b", // WhichModel::MiniCPM4_0_5B => "OpenBMB",
WhichModel::LFM2_1_2B => "lfm2-1.2b", // WhichModel::Qwen2_5VL3B | WhichModel::Qwen2_5VL7B => "Qwen",
WhichModel::LFM2_5_1_2BInstruct => "lfm2.5-1.2b-instruct", // WhichModel::Qwen3_0_6B | WhichModel::Qwen3ASR0_6B | WhichModel::Qwen3ASR1_7B => "Qwen",
WhichModel::LFM2_5VL1_6B => "lfm2.5-vl-1.6b", // WhichModel::Qwen3VL2B
WhichModel::LFM2VL1_6B => "lfm2-vl-1.6b", // | WhichModel::Qwen3VL4B
WhichModel::Qwen2_5VL3B => "qwen2.5vl-3b", // | WhichModel::Qwen3VL8B
WhichModel::Qwen2_5VL7B => "qwen2.5vl-7b", // | WhichModel::Qwen3VL32B
WhichModel::Qwen3_0_6B => "qwen3-0.6b", // | WhichModel::Qwen3_5Gguf => "Qwen",
WhichModel::Qwen3_5_0_8B => "qwen3.5-0.8b", // WhichModel::Qwen3_5_0_8B
WhichModel::Qwen3_5_2B => "qwen3.5-2b", // | WhichModel::Qwen3_5_2B
WhichModel::Qwen3_5_4B => "qwen3.5-4b", // | WhichModel::Qwen3_5_4B
WhichModel::Qwen3_5_9B => "qwen3.5-9b", // | WhichModel::Qwen3_5_9B => "Qwen",
WhichModel::Qwen3_5Gguf => "qwen3.5-gguf", // WhichModel::DeepSeekOCR | WhichModel::DeepSeekOCR2 => "deepseek-ai",
WhichModel::Qwen3ASR0_6B => "qwen3asr-0.6b", // WhichModel::HunyuanOCR => "Tencent-Hunyuan",
WhichModel::Qwen3ASR1_7B => "qwen3asr-1.7b", // WhichModel::PaddleOCRVL | WhichModel::PaddleOCRVL1_5 => "PaddlePaddle",
WhichModel::Qwen3VL2B => "qwen3vl-2b", // WhichModel::RMBG2_0 => "AI-ModelScope",
WhichModel::Qwen3VL4B => "qwen3vl-4b", // WhichModel::VoxCPM | WhichModel::VoxCPM1_5 => "OpenBMB",
WhichModel::Qwen3VL8B => "qwen3vl-8b", // WhichModel::GlmASRNano2512 | WhichModel::GlmOCR => "ZhipuAI",
WhichModel::Qwen3VL32B => "qwen3vl-32b", // WhichModel::FunASRNano2512 => "FunAudioLLM",
WhichModel::DeepSeekOCR => "deepseek-ocr", // WhichModel::LFM2_1_2B
WhichModel::DeepSeekOCR2 => "deepseek-ocr2", // | WhichModel::LFM2_5_1_2BInstruct
WhichModel::HunyuanOCR => "hunyuan-ocr", // | WhichModel::LFM2_5VL1_6B
WhichModel::PaddleOCRVL => "paddleocr-vl", // | WhichModel::LFM2VL1_6B => "LiquidAI",
WhichModel::PaddleOCRVL1_5 => "paddleocr-vl1.5", // }
WhichModel::RMBG2_0 => "rmbg2.0", // }
WhichModel::VoxCPM => "voxcpm",
WhichModel::VoxCPM1_5 => "voxcpm1.5",
WhichModel::GlmASRNano2512 => "glm-asr-nano-2512",
WhichModel::FunASRNano2512 => "fun-asr-nano-2512",
WhichModel::GlmOCR => "glm-ocr",
}
}
/// Get the owner/organization name for a model
fn which_model_to_owner(which_model: WhichModel) -> &'static str {
match which_model {
WhichModel::MiniCPM4_0_5B => "OpenBMB",
WhichModel::Qwen2_5VL3B | WhichModel::Qwen2_5VL7B => "Qwen",
WhichModel::Qwen3_0_6B | WhichModel::Qwen3ASR0_6B | WhichModel::Qwen3ASR1_7B => "Qwen",
WhichModel::Qwen3VL2B
| WhichModel::Qwen3VL4B
| WhichModel::Qwen3VL8B
| WhichModel::Qwen3VL32B
| WhichModel::Qwen3_5Gguf => "Qwen",
WhichModel::Qwen3_5_0_8B
| WhichModel::Qwen3_5_2B
| WhichModel::Qwen3_5_4B
| WhichModel::Qwen3_5_9B => "Qwen",
WhichModel::DeepSeekOCR | WhichModel::DeepSeekOCR2 => "deepseek-ai",
WhichModel::HunyuanOCR => "Tencent-Hunyuan",
WhichModel::PaddleOCRVL | WhichModel::PaddleOCRVL1_5 => "PaddlePaddle",
WhichModel::RMBG2_0 => "AI-ModelScope",
WhichModel::VoxCPM | WhichModel::VoxCPM1_5 => "OpenBMB",
WhichModel::GlmASRNano2512 | WhichModel::GlmOCR => "ZhipuAI",
WhichModel::FunASRNano2512 => "FunAudioLLM",
WhichModel::LFM2_1_2B
| WhichModel::LFM2_5_1_2BInstruct
| WhichModel::LFM2_5VL1_6B
| WhichModel::LFM2VL1_6B => "LiquidAI",
}
}
#[get("/models")] #[get("/models")]
pub(crate) async fn models() -> (Status, (ContentType, Json<serde_json::Value>)) { pub(crate) async fn models() -> (Status, (ContentType, Json<serde_json::Value>)) {
@@ -315,10 +279,10 @@ pub(crate) async fn models() -> (Status, (ContentType, Json<serde_json::Value>))
let which_model = guard.which_model; let which_model = guard.which_model;
let model_obj = ModelObject { let model_obj = ModelObject {
id: which_model_to_id(which_model).to_string(), id: which_model.as_string(),
object: "model".to_string(), object: "model".to_string(),
created: None, // We don't track creation time created: None, // We don't track creation time
owned_by: which_model_to_owner(which_model).to_string(), owned_by: which_model.model_owner(),
}; };
drop(guard); drop(guard);
@@ -419,41 +383,6 @@ mod tests {
assert_eq!(WhichModel::VoxCPM.model_type(), "image"); assert_eq!(WhichModel::VoxCPM.model_type(), "image");
assert_eq!(WhichModel::VoxCPM1_5.model_type(), "image"); assert_eq!(WhichModel::VoxCPM1_5.model_type(), "image");
} }
// Test model_id retrieval
#[test]
fn test_get_model_id() {
assert_eq!(WhichModel::Qwen3_0_6B.model_id(), "Qwen/Qwen3-0.6B");
assert_eq!(
WhichModel::DeepSeekOCR.model_id(),
"deepseek-ai/DeepSeek-OCR"
);
assert_eq!(WhichModel::VoxCPM1_5.model_id(), "OpenBMB/VoxCPM1.5");
}
// Test OpenAI-compatible model ID conversion
#[test]
fn test_which_model_to_id() {
assert_eq!(which_model_to_id(WhichModel::Qwen3_0_6B), "qwen3-0.6b");
assert_eq!(which_model_to_id(WhichModel::DeepSeekOCR), "deepseek-ocr");
assert_eq!(which_model_to_id(WhichModel::VoxCPM1_5), "voxcpm1.5");
assert_eq!(
which_model_to_id(WhichModel::MiniCPM4_0_5B),
"minicpm4-0.5b"
);
}
// Test owner/organization mapping
#[test]
fn test_which_model_to_owner() {
assert_eq!(which_model_to_owner(WhichModel::Qwen3_0_6B), "Qwen");
assert_eq!(which_model_to_owner(WhichModel::DeepSeekOCR), "deepseek-ai");
assert_eq!(which_model_to_owner(WhichModel::VoxCPM1_5), "OpenBMB");
assert_eq!(
which_model_to_owner(WhichModel::HunyuanOCR),
"Tencent-Hunyuan"
);
}
} }
// Shutdown endpoint // Shutdown endpoint
src/exec/deepseek_ocr.rs
+1 -1
View File
@@ -28,7 +28,7 @@ impl ExecModel for DeepSeekORExec {
let message = format!( let message = format!(
r#"{{ r#"{{
"model": "deepseek-ocr", "model": "deepseek-ai/DeepSeek-OCR",
"messages": [ "messages": [
{{ {{
"role": "user", "role": "user",
src/exec/glm_ocr.rs
+1 -1
View File
@@ -27,7 +27,7 @@ impl ExecModel for GlmOcrExec {
let message = format!( let message = format!(
r#"{{ r#"{{
"model": "glm-ocr", "model": "ZhipuAI/GLM-OCR",
"messages": [ "messages": [
{{ {{
"role": "user", "role": "user",
src/exec/hunyuan_ocr.rs
+1 -1
View File
@@ -28,7 +28,7 @@ impl ExecModel for HunyuanORExec {
let message = format!( let message = format!(
r#"{{ r#"{{
"model": "hunyuan-ocr", "model": "Tencent-Hunyuan/HunyuanOCR",
"messages": [ "messages": [
{{ {{
"role": "user", "role": "user",
src/exec/paddleocr_vl.rs
+1 -1
View File
@@ -28,7 +28,7 @@ impl ExecModel for PaddleOVLExec {
let message = format!( let message = format!(
r#"{{ r#"{{
"model": "paddleocr-vl", "model": "PaddlePaddle/PaddleOCR-VL",
"messages": [ "messages": [
{{ {{
"role": "user", "role": "user",
src/exec/rmbg2_0.rs
+1 -1
View File
@@ -29,7 +29,7 @@ impl ExecModel for RMBG2_0Exec {
// Create ChatCompletionParameters for image background removal // Create ChatCompletionParameters for image background removal
let message = format!( let message = format!(
r#"{{ r#"{{
"model": "rmbg2.0", "model": "AI-ModelScope/RMBG-2.0",
"messages": [ "messages": [
{{ {{
"role": "user", "role": "user",
src/exec/voxcpm.rs
+1 -1
View File
@@ -31,7 +31,7 @@ impl ExecModel for VoxCPMExec {
Some("啥子小师叔，打狗还要看主人，你再要继续，我就是你的对手".to_string()), // todo args Some("啥子小师叔，打狗还要看主人，你再要继续，我就是你的对手".to_string()), // todo args
Some("file://./assets/audio/voice_01.wav".to_string()), // todo args Some("file://./assets/audio/voice_01.wav".to_string()), // todo args
2, 2,
100, // max_len (voxcpm uses 100 vs voxcpm1.5's 4096) 100, // max_len (voxcpm uses 100 vs OpenBMB/VoxCPM1.5's 4096)
10, 10,
2.0, 2.0,
6.0, 6.0,
src/main.rs
+27 -55
View File
@@ -2,12 +2,12 @@ use std::sync::atomic::{AtomicBool, Ordering};
use std::{net::IpAddr, str::FromStr, sync::Arc}; use std::{net::IpAddr, str::FromStr, sync::Arc};
use aha::{ use aha::{
models::WhichModel, models::common::model_mapping::WhichModel,
process::{cleanup_pid_file, create_pid_file}, process::{cleanup_pid_file, create_pid_file},
utils::{download_model, get_default_save_dir}, utils::{download_model, get_default_save_dir},
}; };
use anyhow::anyhow; use anyhow::anyhow;
use clap::{Args, Parser, Subcommand, ValueEnum}; use clap::{Args, Parser, Subcommand};
use rocket::{ use rocket::{
Config, Config,
data::{ByteUnit, Limits}, data::{ByteUnit, Limits},
@@ -213,7 +213,7 @@ struct ListArgs {
/// Get the default weight path for a given model /// Get the default weight path for a given model
/// Returns ~/.aha/{model_id} e.g., ~/.aha/OpenBMB/VoxCPM1.5 /// Returns ~/.aha/{model_id} e.g., ~/.aha/OpenBMB/VoxCPM1.5
fn get_default_weight_path(model: WhichModel) -> String { fn get_default_weight_path(model: WhichModel) -> String {
let model_id = model.model_id(); let model_id = model.as_string();
let save_dir = get_default_save_dir().expect("Failed to get home directory"); let save_dir = get_default_save_dir().expect("Failed to get home directory");
format!("{}/{}", save_dir, model_id) format!("{}/{}", save_dir, model_id)
} }
@@ -221,7 +221,7 @@ fn get_default_weight_path(model: WhichModel) -> String {
/// Check if a model is downloaded by verifying the model directory exists /// Check if a model is downloaded by verifying the model directory exists
/// Returns true if ~/.aha/{model_id} directory exists, false otherwise /// Returns true if ~/.aha/{model_id} directory exists, false otherwise
fn is_model_downloaded(model: WhichModel) -> bool { fn is_model_downloaded(model: WhichModel) -> bool {
let model_id = model.model_id(); let model_id = model.as_string();
let save_dir = match get_default_save_dir() { let save_dir = match get_default_save_dir() {
Some(dir) => dir, Some(dir) => dir,
None => return false, None => return false,
@@ -233,8 +233,8 @@ fn is_model_downloaded(model: WhichModel) -> bool {
/// Model information for JSON output /// Model information for JSON output
#[derive(Serialize)] #[derive(Serialize)]
struct ModelInfo { struct ModelInfo {
name: String,
model_id: String, model_id: String,
owner: String,
#[serde(rename = "type")] #[serde(rename = "type")]
model_type: String, model_type: String,
downloaded: bool, downloaded: bool,
@@ -242,48 +242,17 @@ struct ModelInfo {
/// List all supported models /// List all supported models
fn run_list(args: ListArgs) -> anyhow::Result<()> { fn run_list(args: ListArgs) -> anyhow::Result<()> {
let models = [ let models = WhichModel::model_list();
WhichModel::MiniCPM4_0_5B,
WhichModel::LFM2_1_2B,
WhichModel::LFM2_5_1_2BInstruct,
WhichModel::Qwen2_5VL3B,
WhichModel::Qwen2_5VL7B,
WhichModel::Qwen3_0_6B,
WhichModel::Qwen3_5_0_8B,
WhichModel::Qwen3_5_2B,
WhichModel::Qwen3_5_4B,
WhichModel::Qwen3_5_9B,
WhichModel::Qwen3ASR0_6B,
WhichModel::Qwen3ASR1_7B,
WhichModel::Qwen3VL2B,
WhichModel::Qwen3VL4B,
WhichModel::Qwen3VL8B,
WhichModel::Qwen3VL32B,
WhichModel::DeepSeekOCR,
WhichModel::DeepSeekOCR2,
WhichModel::HunyuanOCR,
WhichModel::PaddleOCRVL,
WhichModel::PaddleOCRVL1_5,
WhichModel::RMBG2_0,
WhichModel::VoxCPM,
WhichModel::VoxCPM1_5,
WhichModel::GlmASRNano2512,
WhichModel::FunASRNano2512,
WhichModel::GlmOCR,
];
if args.json { if args.json {
// JSON output // JSON output
let model_infos: Vec<ModelInfo> = models let model_infos: Vec<ModelInfo> = models
.iter() .iter()
.map(|model| { .map(|model| ModelInfo {
let possible_value = model.to_possible_value().unwrap(); model_id: model.as_string(),
ModelInfo { owner: model.model_owner(),
name: possible_value.get_name().to_string(),
model_id: model.model_id().to_string(),
model_type: model.model_type().to_string(), model_type: model.model_type().to_string(),
downloaded: is_model_downloaded(*model), downloaded: is_model_downloaded(*model),
}
}) })
.collect(); .collect();
println!("{}", serde_json::to_string_pretty(&model_infos)?); println!("{}", serde_json::to_string_pretty(&model_infos)?);
@@ -292,20 +261,23 @@ fn run_list(args: ListArgs) -> anyhow::Result<()> {
println!("Available models:"); println!("Available models:");
println!(); println!();
println!( println!(
"{:<30} {:<40} {:<10}", "{:<40} {:<20} {:<10} {:<10}",
"Model Name", "ModelScope ID", "Download" "Model ID", "Owner", "type", "Download"
); );
println!("{}", "-".repeat(80)); println!("{}", "-".repeat(80));
for model in models { for model in models {
let possible_value = model.to_possible_value().unwrap(); let model_id = model.as_string();
let name = possible_value.get_name(); let owner = model.model_owner();
let id = model.model_id(); let model_type = model.model_type();
let download_status = if is_model_downloaded(model) { let download_status = if is_model_downloaded(model) {
" ✔" " ✔"
} else { } else {
"" ""
}; };
println!("{:<30} {:<40} {:<10}", name, id, download_status); println!(
"{:<40} {:<20} {:<10} {:<10}",
model_id, owner, model_type, download_status
);
} }
} }
@@ -322,9 +294,9 @@ async fn run_cli(args: CliArgs) -> anyhow::Result<()> {
gguf_path, gguf_path,
mmproj_path, mmproj_path,
} = args; } = args;
let model_id = common.model.model_id(); let model_id = common.model.as_string();
let (model_path, gguf, mmproj) = if model_id.eq("GGUF") { let (model_path, gguf, mmproj) = if model_id.contains("gguf") {
if gguf_path.is_none() { if gguf_path.is_none() {
return Err(anyhow!("gguf model path is required")); return Err(anyhow!("gguf model path is required"));
} }
@@ -338,8 +310,8 @@ async fn run_cli(args: CliArgs) -> anyhow::Result<()> {
None => get_default_save_dir().expect("Failed to get home directory"), None => get_default_save_dir().expect("Failed to get home directory"),
}; };
let max_retries = download_retries.unwrap_or(3); let max_retries = download_retries.unwrap_or(3);
download_model(model_id, &save_dir, max_retries).await?; download_model(&model_id, &save_dir, max_retries).await?;
save_dir + "/" + model_id save_dir + "/" + &model_id
} }
}; };
(model_path, None, None) (model_path, None, None)
@@ -359,8 +331,8 @@ async fn run_serv(args: ServArgs) -> anyhow::Result<()> {
gguf_path, gguf_path,
mmproj_path, mmproj_path,
} = args; } = args;
let model_id = common.model.model_id(); let model_id = common.model.as_string();
let (model_path, gguf, mmproj) = if model_id.eq("GGUF") { let (model_path, gguf, mmproj) = if model_id.contains("gguf") {
if gguf_path.is_none() { if gguf_path.is_none() {
return Err(anyhow!("gguf model path is required")); return Err(anyhow!("gguf model path is required"));
} }
@@ -427,7 +399,7 @@ async fn run_download(args: DownloadArgs) -> anyhow::Result<()> {
save_dir, save_dir,
download_retries, download_retries,
} = args; } = args;
let model_id = model.model_id(); let model_id = model.as_string();
let save_dir = match save_dir { let save_dir = match save_dir {
Some(dir) => dir, Some(dir) => dir,
@@ -435,7 +407,7 @@ async fn run_download(args: DownloadArgs) -> anyhow::Result<()> {
}; };
let max_retries = download_retries.unwrap_or(3); let max_retries = download_retries.unwrap_or(3);
download_model(model_id, &save_dir, max_retries).await?; download_model(&model_id, &save_dir, max_retries).await?;
Ok(()) Ok(())
} }
@@ -587,7 +559,7 @@ fn run_run(args: RunArgs) -> anyhow::Result<()> {
/// Run the 'delete' subcommand: delete model from default location /// Run the 'delete' subcommand: delete model from default location
fn run_delete(args: DeleteArgs) -> anyhow::Result<()> { fn run_delete(args: DeleteArgs) -> anyhow::Result<()> {
let DeleteArgs { model } = args; let DeleteArgs { model } = args;
let model_id = model.model_id(); let model_id = model.as_string();
let save_dir = get_default_save_dir().expect("Failed to get home directory"); let save_dir = get_default_save_dir().expect("Failed to get home directory");
let model_path = format!("{}/{}", save_dir, model_id); let model_path = format!("{}/{}", save_dir, model_id);
src/models/common/mod.rs
+1
View File
@@ -8,6 +8,7 @@ use candle_nn::{
}; };
pub mod gguf; pub mod gguf;
pub mod model_mapping;
use crate::{ use crate::{
position_embed::rope::{RoPE, apply_rotary_pos_emb, apply_rotary_pos_emb_roformer}, position_embed::rope::{RoPE, apply_rotary_pos_emb, apply_rotary_pos_emb_roformer},
src/models/common/model_mapping.rs
+128
View File
@@ -0,0 +1,128 @@
use clap::ValueEnum;
#[derive(Debug, Clone, Copy, PartialEq, Eq, clap::ValueEnum)]
pub enum WhichModel {
#[value(name = "LiquidAI/LFM2-1.2B")]
LFM2_1_2B,
#[value(name = "LiquidAI/LFM2.5-1.2B-Instruct")]
LFM2_5_1_2BInstruct,
#[value(name = "LiquidAI/LFM2.5-VL-1.6B")]
LFM2_5VL1_6B,
#[value(name = "LiquidAI/LFM2-VL-1.6B")]
LFM2VL1_6B,
#[value(name = "OpenBMB/MiniCPM4-0.5B")]
MiniCPM4_0_5B,
#[value(name = "Qwen/Qwen2.5-VL-3B-Instruct")]
Qwen2_5VL3B,
#[value(name = "Qwen/Qwen2.5-VL-7B-Instruct")]
Qwen2_5VL7B,
#[value(name = "Qwen/Qwen3-0.6B")]
Qwen3_0_6B,
#[value(name = "Qwen/Qwen3.5-0.8B")]
Qwen3_5_0_8B,
#[value(name = "Qwen/Qwen3.5-2B")]
Qwen3_5_2B,
#[value(name = "Qwen/Qwen3.5-4B")]
Qwen3_5_4B,
#[value(name = "Qwen/Qwen3.5-9B")]
Qwen3_5_9B,
#[value(name = "qwen3.5-gguf")] // todo
Qwen3_5Gguf,
#[value(name = "Qwen/Qwen3-ASR-0.6B")]
Qwen3ASR0_6B,
#[value(name = "Qwen/Qwen3-ASR-1.7B")]
Qwen3ASR1_7B,
#[value(name = "Qwen/Qwen3-VL-2B-Instruct")]
Qwen3VL2B,
#[value(name = "Qwen/Qwen3-VL-4B-Instruct")]
Qwen3VL4B,
#[value(name = "Qwen/Qwen3-VL-8B-Instruct")]
Qwen3VL8B,
#[value(name = "Qwen/Qwen3-VL-32B-Instruct")]
Qwen3VL32B,
#[value(name = "deepseek-ai/DeepSeek-OCR")]
DeepSeekOCR,
#[value(name = "deepseek-ai/DeepSeek-OCR-2")]
DeepSeekOCR2,
#[value(name = "Tencent-Hunyuan/HunyuanOCR")]
HunyuanOCR,
#[value(name = "PaddlePaddle/PaddleOCR-VL")]
PaddleOCRVL,
#[value(name = "PaddlePaddle/PaddleOCR-VL-1.5")]
PaddleOCRVL1_5,
#[value(name = "AI-ModelScope/RMBG-2.0")]
RMBG2_0,
#[value(name = "OpenBMB/VoxCPM-0.5B")]
VoxCPM,
#[value(name = "OpenBMB/VoxCPM1.5")]
VoxCPM1_5,
#[value(name = "ZhipuAI/GLM-ASR-Nano-2512")]
GlmASRNano2512,
#[value(name = "FunAudioLLM/Fun-ASR-Nano-2512")]
FunASRNano2512,
#[value(name = "ZhipuAI/GLM-OCR")]
GlmOCR,
}
impl WhichModel {
/// Get the ModelScope model ID for this model variant
pub fn as_string(&self) -> String {
self.to_possible_value()
.expect("not exists")
.get_name()
.to_string()
}
/// Get the WhichModel enum list
pub fn model_list() -> Vec<Self> {
WhichModel::value_variants().to_vec()
}
pub fn model_owner(&self) -> String {
let name = self.as_string();
let names: Vec<&str> = name.split("/").collect();
if names.len() < 2 {
"none".to_string()
} else {
names.first().map_or("none", |&s| s).to_string()
}
}
/// Get the model type category for this model variant
pub fn model_type(self) -> &'static str {
match self {
// LLM models
WhichModel::MiniCPM4_0_5B
| WhichModel::Qwen3_0_6B
| WhichModel::LFM2_1_2B
| WhichModel::LFM2_5_1_2BInstruct => "llm",
WhichModel::Qwen2_5VL3B
| WhichModel::Qwen2_5VL7B
| WhichModel::Qwen3VL2B
| WhichModel::Qwen3VL4B
| WhichModel::Qwen3VL8B
| WhichModel::Qwen3VL32B
| WhichModel::Qwen3_5_0_8B
| WhichModel::Qwen3_5_2B
| WhichModel::Qwen3_5_4B
| WhichModel::Qwen3_5_9B
| WhichModel::Qwen3_5Gguf
| WhichModel::LFM2_5VL1_6B
| WhichModel::LFM2VL1_6B => "vlm",
// OCR models
WhichModel::DeepSeekOCR
| WhichModel::DeepSeekOCR2
| WhichModel::HunyuanOCR
| WhichModel::GlmOCR
| WhichModel::PaddleOCRVL
| WhichModel::PaddleOCRVL1_5 => "ocr",
// ASR models
WhichModel::Qwen3ASR0_6B
| WhichModel::Qwen3ASR1_7B
| WhichModel::GlmASRNano2512
| WhichModel::FunASRNano2512 => "asr",
// Image models
WhichModel::RMBG2_0 => "image",
WhichModel::VoxCPM | WhichModel::VoxCPM1_5 => "tts",
}
}
}
src/models/deepseek_ocr/generate.rs
+1 -1
View File
@@ -44,7 +44,7 @@ impl DeepseekOCRGenerateModel {
let model_name = std::path::Path::new(path) let model_name = std::path::Path::new(path)
.file_name() .file_name()
.and_then(|s| s.to_str()) .and_then(|s| s.to_str())
.unwrap_or("deepseek-ocr") .unwrap_or("deepseek-ai/DeepSeek-OCR")
.to_string(); .to_string();
let version = if model_name.contains("2") || cfg.vision_config.width.qwen2_0_5b.is_some() { let version = if model_name.contains("2") || cfg.vision_config.width.qwen2_0_5b.is_some() {
2usize 2usize
src/models/glm_ocr/generate.rs
+1 -1
View File
@@ -60,7 +60,7 @@ impl GlmOcrGenerateModel {
let model_name = std::path::Path::new(path) let model_name = std::path::Path::new(path)
.file_name() .file_name()
.and_then(|s| s.to_str()) .and_then(|s| s.to_str())
.unwrap_or("glm-ocr") .unwrap_or("ZhipuAI/GLM-OCR")
.to_string(); .to_string();
Ok(Self { Ok(Self {
// chat_template, // chat_template,
src/models/mod.rs
+3 -143
View File
@@ -22,8 +22,9 @@ pub mod rmbg2_0;
pub mod voxcpm; pub mod voxcpm;
pub mod w2v_bert_2_0; pub mod w2v_bert_2_0;
use crate::params::chat::{ use crate::{
ChatCompletionChunkResponse, ChatCompletionParameters, ChatCompletionResponse, models::common::model_mapping::WhichModel,
params::chat::{ChatCompletionChunkResponse, ChatCompletionParameters, ChatCompletionResponse},
}; };
use anyhow::{Result, anyhow}; use anyhow::{Result, anyhow};
use rocket::futures::Stream; use rocket::futures::Stream;
@@ -40,147 +41,6 @@ use crate::models::{
rmbg2_0::generate::RMBG2_0Model, voxcpm::generate::VoxCPMGenerate, rmbg2_0::generate::RMBG2_0Model, voxcpm::generate::VoxCPMGenerate,
}; };
#[derive(Debug, Clone, Copy, PartialEq, Eq, clap::ValueEnum)]
pub enum WhichModel {
#[value(name = "lfm2-1.2b")]
LFM2_1_2B,
#[value(name = "lfm2.5-1.2b-instruct")]
LFM2_5_1_2BInstruct,
#[value(name = "lfm2.5-vl-1.6b")]
LFM2_5VL1_6B,
#[value(name = "lfm2-vl-1.6b")]
LFM2VL1_6B,
#[value(name = "minicpm4-0.5b")]
MiniCPM4_0_5B,
#[value(name = "qwen2.5vl-3b")]
Qwen2_5VL3B,
#[value(name = "qwen2.5vl-7b")]
Qwen2_5VL7B,
#[value(name = "qwen3-0.6b")]
Qwen3_0_6B,
#[value(name = "qwen3.5-0.8b")]
Qwen3_5_0_8B,
#[value(name = "qwen3.5-2b")]
Qwen3_5_2B,
#[value(name = "qwen3.5-4b")]
Qwen3_5_4B,
#[value(name = "qwen3.5-9b")]
Qwen3_5_9B,
#[value(name = "qwen3.5-gguf")]
Qwen3_5Gguf,
#[value(name = "qwen3asr-0.6b")]
Qwen3ASR0_6B,
#[value(name = "qwen3asr-1.7b")]
Qwen3ASR1_7B,
#[value(name = "qwen3vl-2b")]
Qwen3VL2B,
#[value(name = "qwen3vl-4b")]
Qwen3VL4B,
#[value(name = "qwen3vl-8b")]
Qwen3VL8B,
#[value(name = "qwen3vl-32b")]
Qwen3VL32B,
#[value(name = "deepseek-ocr")]
DeepSeekOCR,
#[value(name = "deepseek-ocr2")]
DeepSeekOCR2,
#[value(name = "hunyuan-ocr")]
HunyuanOCR,
#[value(name = "paddleocr-vl")]
PaddleOCRVL,
#[value(name = "paddleocr-vl1.5")]
PaddleOCRVL1_5,
#[value(name = "rmbg2.0")]
RMBG2_0,
#[value(name = "voxcpm")]
VoxCPM,
#[value(name = "voxcpm1.5")]
VoxCPM1_5,
#[value(name = "glm-asr-nano-2512")]
GlmASRNano2512,
#[value(name = "fun-asr-nano-2512")]
FunASRNano2512,
#[value(name = "glm-ocr")]
GlmOCR,
}
impl WhichModel {
/// Get the ModelScope model ID for this model variant
pub fn model_id(self) -> &'static str {
match self {
WhichModel::LFM2_1_2B => "LiquidAI/LFM2-1.2B",
WhichModel::LFM2_5_1_2BInstruct => "LiquidAI/LFM2.5-1.2B-Instruct",
WhichModel::LFM2_5VL1_6B => "LiquidAI/LFM2.5-VL-1.6B",
WhichModel::LFM2VL1_6B => "LiquidAI/LFM2-VL-1.6B",
WhichModel::MiniCPM4_0_5B => "OpenBMB/MiniCPM4-0.5B",
WhichModel::Qwen2_5VL3B => "Qwen/Qwen2.5-VL-3B-Instruct",
WhichModel::Qwen2_5VL7B => "Qwen/Qwen2.5-VL-7B-Instruct",
WhichModel::Qwen3_0_6B => "Qwen/Qwen3-0.6B",
WhichModel::Qwen3_5_0_8B => "Qwen/Qwen3.5-0.8B",
WhichModel::Qwen3_5_2B => "Qwen/Qwen3.5-2B",
WhichModel::Qwen3_5_4B => "Qwen/Qwen3.5-4B",
WhichModel::Qwen3_5_9B => "Qwen/Qwen3.5-9B",
WhichModel::Qwen3_5Gguf => "GGUF",
WhichModel::Qwen3ASR0_6B => "Qwen/Qwen3-ASR-0.6B",
WhichModel::Qwen3ASR1_7B => "Qwen/Qwen3-ASR-1.7B",
WhichModel::Qwen3VL2B => "Qwen/Qwen3-VL-2B-Instruct",
WhichModel::Qwen3VL4B => "Qwen/Qwen3-VL-4B-Instruct",
WhichModel::Qwen3VL8B => "Qwen/Qwen3-VL-8B-Instruct",
WhichModel::Qwen3VL32B => "Qwen/Qwen3-VL-32B-Instruct",
WhichModel::DeepSeekOCR => "deepseek-ai/DeepSeek-OCR",
WhichModel::DeepSeekOCR2 => "deepseek-ai/DeepSeek-OCR-2",
WhichModel::HunyuanOCR => "Tencent-Hunyuan/HunyuanOCR",
WhichModel::PaddleOCRVL => "PaddlePaddle/PaddleOCR-VL",
WhichModel::PaddleOCRVL1_5 => "PaddlePaddle/PaddleOCR-VL-1.5",
WhichModel::RMBG2_0 => "AI-ModelScope/RMBG-2.0",
WhichModel::VoxCPM => "OpenBMB/VoxCPM-0.5B",
WhichModel::VoxCPM1_5 => "OpenBMB/VoxCPM1.5",
WhichModel::GlmASRNano2512 => "ZhipuAI/GLM-ASR-Nano-2512",
WhichModel::FunASRNano2512 => "FunAudioLLM/Fun-ASR-Nano-2512",
WhichModel::GlmOCR => "ZhipuAI/GLM-OCR",
}
}
/// Get the model type category for this model variant
pub fn model_type(self) -> &'static str {
match self {
// LLM models
WhichModel::MiniCPM4_0_5B
| WhichModel::Qwen3_0_6B
| WhichModel::LFM2_1_2B
| WhichModel::LFM2_5_1_2BInstruct => "llm",
WhichModel::Qwen2_5VL3B
| WhichModel::Qwen2_5VL7B
| WhichModel::Qwen3VL2B
| WhichModel::Qwen3VL4B
| WhichModel::Qwen3VL8B
| WhichModel::Qwen3VL32B
| WhichModel::Qwen3_5_0_8B
| WhichModel::Qwen3_5_2B
| WhichModel::Qwen3_5_4B
| WhichModel::Qwen3_5_9B
| WhichModel::Qwen3_5Gguf
| WhichModel::LFM2_5VL1_6B
| WhichModel::LFM2VL1_6B => "vlm",
// OCR models
WhichModel::DeepSeekOCR
| WhichModel::DeepSeekOCR2
| WhichModel::HunyuanOCR
| WhichModel::GlmOCR
| WhichModel::PaddleOCRVL
| WhichModel::PaddleOCRVL1_5 => "ocr",
// ASR models
WhichModel::Qwen3ASR0_6B
| WhichModel::Qwen3ASR1_7B
| WhichModel::GlmASRNano2512
| WhichModel::FunASRNano2512 => "asr",
// Image models
WhichModel::RMBG2_0 => "image",
WhichModel::VoxCPM | WhichModel::VoxCPM1_5 => "tts",
}
}
}
pub trait GenerateModel { pub trait GenerateModel {
fn generate(&mut self, mes: ChatCompletionParameters) -> Result<ChatCompletionResponse>; fn generate(&mut self, mes: ChatCompletionParameters) -> Result<ChatCompletionResponse>;
fn generate_stream( fn generate_stream(
src/models/rmbg2_0/generate.rs
+1 -1
View File
@@ -44,7 +44,7 @@ impl RMBG2_0Model {
let model_name = std::path::Path::new(path) let model_name = std::path::Path::new(path)
.file_name() .file_name()
.and_then(|s| s.to_str()) .and_then(|s| s.to_str())
.unwrap_or("rmbg2.0") .unwrap_or("AI-ModelScope/RMBG-2.0")
.to_string(); .to_string();
Ok(Self { Ok(Self {
model, model,
src/models/voxcpm/generate.rs
+1 -1
View File
@@ -86,7 +86,7 @@ impl VoxCPMGenerate {
let m_dtype = get_dtype(dtype, cfg_dtype); let m_dtype = get_dtype(dtype, cfg_dtype);
let model_list = find_type_files(path, "bin")?; let model_list = find_type_files(path, "bin")?;
// voxcpm0.5B模型文件是.bin类型， voxcpm1.5模型文件是.safetensors类型 // voxcpm0.5B模型文件是.bin类型， OpenBMB/VoxCPM1.5模型文件是.safetensors类型
let vb_voxcpm = if model_list.is_empty() { let vb_voxcpm = if model_list.is_empty() {
let model_list = find_type_files(path, "safetensors")?; let model_list = find_type_files(path, "safetensors")?;
unsafe { VarBuilder::from_mmaped_safetensors(&model_list, m_dtype, device)? } unsafe { VarBuilder::from_mmaped_safetensors(&model_list, m_dtype, device)? }
src/params/shared.rs
+9
View File
@@ -5,9 +5,18 @@ pub struct Usage {
/// Number of tokens in the prompt. /// Number of tokens in the prompt.
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub prompt_tokens: Option<u32>, pub prompt_tokens: Option<u32>,
/// Number of tokens in the prompt.
#[serde(skip_serializing_if = "Option::is_none")]
pub prompt_ms: Option<f64>,
/// Number of tokens in the completion. /// Number of tokens in the completion.
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub completion_tokens: Option<u32>, pub completion_tokens: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub completion_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub completion_per_token_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub completion_tps: Option<f64>,
/// Number of tokens in the entire response. /// Number of tokens in the entire response.
pub total_tokens: u32, pub total_tokens: u32,
/// Breakdown of tokens used in the prompt. /// Breakdown of tokens used in the prompt.
src/process.rs
+2 -2
View File
@@ -258,7 +258,7 @@ mod tests {
let service = ServiceInfo { let service = ServiceInfo {
service_id: "12345@10100".to_string(), service_id: "12345@10100".to_string(),
pid: 12345, pid: 12345,
model: Some("qwen3-0.6b".to_string()), model: Some("Qwen/Qwen3-0.6B".to_string()),
port: 10100, port: 10100,
address: "127.0.0.1".to_string(), address: "127.0.0.1".to_string(),
status: ServiceStatus::Running, status: ServiceStatus::Running,
@@ -266,7 +266,7 @@ mod tests {
let service_clone = service.clone(); let service_clone = service.clone();
assert_eq!(service_clone.service_id, "12345@10100"); assert_eq!(service_clone.service_id, "12345@10100");
assert_eq!(service_clone.pid, 12345); assert_eq!(service_clone.pid, 12345);
assert_eq!(service_clone.model, Some("qwen3-0.6b".to_string())); assert_eq!(service_clone.model, Some("Qwen/Qwen3-0.6B".to_string()));
assert_eq!(service_clone.port, 10100); assert_eq!(service_clone.port, 10100);
} }
src/utils/mod.rs
+62 -18
View File
@@ -494,25 +494,8 @@ pub fn build_audio_completion_response(
response response
} }
pub fn build_completion_response( fn build_response(res: String, model_name: &str, usage: Option<Usage>) -> ChatCompletionResponse {
res: String,
model_name: &str,
completion_tokens: Option<u32>,
prompt_tokens: Option<u32>,
) -> ChatCompletionResponse {
let id = uuid::Uuid::new_v4().to_string(); let id = uuid::Uuid::new_v4().to_string();
let usage = if prompt_tokens.is_none() && completion_tokens.is_none() {
None
} else {
Some(Usage {
prompt_tokens,
completion_tokens,
total_tokens: prompt_tokens.unwrap_or(0) + completion_tokens.unwrap_or(0),
prompt_tokens_details: None,
completion_tokens_details: None,
})
};
let mut response = ChatCompletionResponse { let mut response = ChatCompletionResponse {
id: Some(id), id: Some(id),
choices: vec![], choices: vec![],
@@ -588,6 +571,67 @@ pub fn build_completion_response(
response response
} }
pub fn build_completion_response(
res: String,
model_name: &str,
completion_tokens: Option<u32>,
prompt_tokens: Option<u32>,
) -> ChatCompletionResponse {
let usage = if prompt_tokens.is_none() && completion_tokens.is_none() {
None
} else {
Some(Usage {
prompt_tokens,
prompt_ms: None,
completion_tokens,
completion_ms: None,
completion_per_token_ms: None,
completion_tps: None,
total_tokens: prompt_tokens.unwrap_or(0) + completion_tokens.unwrap_or(0),
prompt_tokens_details: None,
completion_tokens_details: None,
})
};
build_response(res, model_name, usage)
}
pub fn build_completion_response_with_time(
res: String,
model_name: &str,
completion_tokens: Option<u32>,
completion_ms: Option<f64>,
prompt_tokens: Option<u32>,
prompt_ms: Option<f64>,
) -> ChatCompletionResponse {
let usage = if prompt_tokens.is_none() && completion_tokens.is_none() {
None
} else {
let (completion_per_token_ms, completion_tps) = if let Some(prompt_tokens) = prompt_tokens
&& let Some(prompt_ms) = prompt_ms
{
let per_token_ms = prompt_ms / prompt_tokens as f64;
let tps = prompt_tokens as f64 / (prompt_ms / 1000.0);
(Some(per_token_ms), Some(tps))
} else {
(None, None)
};
Some(Usage {
prompt_tokens,
prompt_ms,
completion_tokens,
completion_ms,
completion_per_token_ms,
completion_tps,
total_tokens: prompt_tokens.unwrap_or(0) + completion_tokens.unwrap_or(0),
prompt_tokens_details: None,
completion_tokens_details: None,
})
};
build_response(res, model_name, usage)
}
pub fn build_completion_chunk_response( pub fn build_completion_chunk_response(
res: String, res: String,
model_name: &str, model_name: &str,
tests/messy_test.rs
+10 -3
View File
@@ -5,7 +5,10 @@
// use std::io::{Read, Seek}; // use std::io::{Read, Seek};
// use std::{io::Cursor, time::Instant}; // use std::{io::Cursor, time::Instant};
use aha::utils::{timestamp, timestamp_millis}; use aha::{
models::common::model_mapping::WhichModel,
utils::{timestamp, timestamp_millis},
};
// use aha::utils::tensor_utils::repeat_interleave; // use aha::utils::tensor_utils::repeat_interleave;
// use crate::params::chat::ChatCompletionParameters; // use crate::params::chat::ChatCompletionParameters;
use anyhow::Result; use anyhow::Result;
@@ -36,8 +39,12 @@ async fn download_test() -> Result<()> {
#[test] #[test]
fn messy_test() -> Result<()> { fn messy_test() -> Result<()> {
// RUST_BACKTRACE=1 cargo test -F cuda --test messy_test messy_test -r -- --nocapture // RUST_BACKTRACE=1 cargo test -F cuda --test messy_test messy_test -r -- --nocapture
println!("当前秒级时间戳: {}", timestamp()); let model = WhichModel::LFM2_1_2B;
println!("当前毫秒级时间戳: {}", timestamp_millis()); println!("model: {:?}, model_id: {}", model, model.as_string());
let model_list = WhichModel::model_list();
println!("model_list: {:#?}", model_list);
// println!("当前秒级时间戳: {}", timestamp());
// println!("当前毫秒级时间戳: {}", timestamp_millis());
// let t1 = Tensor::randn(0.0, 1.0, (1, 2, 6), device)?; // let t1 = Tensor::randn(0.0, 1.0, (1, 2, 6), device)?;
// println!(" t1: {}", t1); // println!(" t1: {}", t1);
tests/test_deepseek_ocr.rs
+3 -3
View File
@@ -10,7 +10,7 @@ fn deepseek_ocr2_generate() -> Result<()> {
// RUST_BACKTRACE=1 cargo test -F cuda --test test_deepseek_ocr deepseek_ocr2_generate -r -- --nocapture // RUST_BACKTRACE=1 cargo test -F cuda --test test_deepseek_ocr deepseek_ocr2_generate -r -- --nocapture
let message = r#" let message = r#"
{ {
"model": "deepseek-ocr2", "model": "deepseek-ai/DeepSeek-OCR2",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -59,7 +59,7 @@ fn deepseek_ocr_generate() -> Result<()> {
// RUST_BACKTRACE=1 cargo test -F cuda --test test_deepseek_ocr deepseek_ocr_generate -r -- --nocapture // RUST_BACKTRACE=1 cargo test -F cuda --test test_deepseek_ocr deepseek_ocr_generate -r -- --nocapture
let message = r#" let message = r#"
{ {
"model": "deepseek-ocr", "model": "deepseek-ai/DeepSeek-OCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -109,7 +109,7 @@ async fn deepseek_ocr_stream() -> Result<()> {
let message = r#" let message = r#"
{ {
"model": "deepseek-ocr", "model": "deepseek-ai/DeepSeek-OCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
tests/test_glm_ocr.rs
+2 -2
View File
@@ -10,7 +10,7 @@ fn glm_ocr_generate() -> Result<()> {
// RUST_BACKTRACE=1 cargo test -F cuda --test test_glm_ocr glm_ocr_generate -r -- --nocapture // RUST_BACKTRACE=1 cargo test -F cuda --test test_glm_ocr glm_ocr_generate -r -- --nocapture
let message = r#" let message = r#"
{ {
"model": "glm-ocr", "model": "ZhipuAI/GLM-OCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -60,7 +60,7 @@ async fn glm_ocr_stream() -> Result<()> {
let message = r#" let message = r#"
{ {
"model": "glm-ocr", "model": "ZhipuAI/GLM-OCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
tests/test_health_models.rs
+1 -1
View File
@@ -1,4 +1,4 @@
use aha::models::WhichModel; use aha::models::common::model_mapping::WhichModel;
// Import helper functions from api module - these will need to be made public // Import helper functions from api module - these will need to be made public
// or tested through integration testing // or tested through integration testing
tests/test_hunyuan_ocr.rs
+2 -2
View File
@@ -10,7 +10,7 @@ fn hunyuan_ocr_generate() -> Result<()> {
// RUST_BACKTRACE=1 cargo test -F cuda --test test_hunyuan_ocr hunyuan_ocr_generate -r -- --nocapture // RUST_BACKTRACE=1 cargo test -F cuda --test test_hunyuan_ocr hunyuan_ocr_generate -r -- --nocapture
let message = r#" let message = r#"
{ {
"model": "hunyuan-ocr", "model": "Tencent-Hunyuan/HunyuanOCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
@@ -60,7 +60,7 @@ async fn hunyuan_ocr_stream() -> Result<()> {
let message = r#" let message = r#"
{ {
"model": "hunyuan-ocr", "model": "Tencent-Hunyuan/HunyuanOCR",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
tests/test_rmbg2_0.rs
+1 -1
View File
@@ -14,7 +14,7 @@ fn rmbg2_0_generate() -> Result<()> {
let message = r#" let message = r#"
{ {
"model": "rmbg2.0", "model": "AI-ModelScope/RMBG-2.0",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",
tests/test_voxcpm1_5.rs
+1 -1
View File
@@ -18,7 +18,7 @@ fn voxcpm1_5_use_message_generate() -> Result<()> {
let model_path = format!("{}/OpenBMB/VoxCPM1.5/", save_dir); let model_path = format!("{}/OpenBMB/VoxCPM1.5/", save_dir);
let message = r#" let message = r#"
{ {
"model": "voxcpm1.5", "model": "OpenBMB/VoxCPM1.5",
"messages": [ "messages": [
{ {
"role": "user", "role": "user",