tckm.yan/aha
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update docs

Browse Source
This commit is contained in:
jhqxxx
2026-02-07 13:28:13 +08:00
parent ca16051065
commit fcbe736e1a
13 changed files with 180 additions and 175 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+4 -6
View File
@@ -68,13 +68,10 @@ cargo build --release --features cuda
cargo build --release --features metal cargo build --release --features metal
# Flash Attention (faster inference) # Flash Attention (faster inference)
cargo build --release --features flash-attn cargo build --release --features cuda,flash-attn
# FFmpeg (multimedia processing) # FFmpeg (multimedia processing)
cargo build --release --features ffmpeg cargo build --release --features ffmpeg
# Combine multiple features
cargo build --release --features "cuda,flash-attn"
``` ```
### CLI Quick Reference ### CLI Quick Reference
@@ -111,7 +108,8 @@ curl http://localhost:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "qwen3-0.6b",
"messages": [{"role": "user", "content": "Hello!"}] "messages": [{"role": "user", "content": "Hello!"}],
"stream": false
} }
' '
``` ```
@@ -154,7 +152,7 @@ curl http://localhost:10100/chat/completions \
### Using aha as a Library ### Using aha as a Library
> cargo add aha > cargo add aha
```bash ```rust
# VoxCPM example # VoxCPM example
use aha::models::voxcpm::generate::VoxCPMGenerate; use aha::models::voxcpm::generate::VoxCPMGenerate;
use aha::utils::audio_utils::save_wav; use aha::utils::audio_utils::save_wav;
README.zh-CN.md
+5 -7
View File
@@ -22,7 +22,7 @@
**轻量 AI 推理引擎 —— 文本、视觉、语音与 OCR 一站式解决方案** **轻量 AI 推理引擎 —— 文本、视觉、语音与 OCR 一站式解决方案**
aha 是一款基于 Rust 和 Candle 框架构建的高性能跨平台 AI 推理引擎。将最先进的 AI 模型带到您的本地机器——无需 API 密钥，无需云依赖，纯粹、快速的 AI 直接在您的硬件上运行。 aha 是一款基于 Rust 和 Candle 框架构建的高性能跨平台 AI 推理引擎。将最先进的 AI 模型带到您的本地机器——无需 API 密钥，无需云依赖，纯粹、快速的 AI，直接在您的硬件上运行。
## 更新日志 ## 更新日志
@@ -68,13 +68,10 @@ cargo build --release --features cuda
cargo build --release --features metal cargo build --release --features metal
# Flash Attention (更快推理) # Flash Attention (更快推理)
cargo build --release --features flash-attn cargo build --release --features cuda,flash-attn
# FFmpeg (多媒体处理) # FFmpeg (多媒体处理)
cargo build --release --features ffmpeg cargo build --release --features ffmpeg
# 组合多个特性
cargo build --release --features "cuda,flash-attn"
``` ```
### CLI 快速参考 ### CLI 快速参考
@@ -111,7 +108,8 @@ curl http://localhost:10100/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "qwen3-0.6b", "model": "qwen3-0.6b",
"messages": [{"role": "user", "content": "你好！"}] "messages": [{"role": "user", "content": "你好！"}],
"stream": false
}' }'
``` ```
@@ -154,7 +152,7 @@ curl http://localhost:10100/chat/completions \
### aha 作为库使用 ### aha 作为库使用
> cargo add aha > cargo add aha
```bash ```rust
# VoxCPM示例 # VoxCPM示例
use aha::models::voxcpm::generate::VoxCPMGenerate; use aha::models::voxcpm::generate::VoxCPMGenerate;
use aha::utils::audio_utils::save_wav; use aha::utils::audio_utils::save_wav;
docs/api.md
+39 -26
View File
@@ -39,8 +39,6 @@ Success responses follow this structure:
"data": { ... }, "data": { ... },
"model": "model-name", "model": "model-name",
"usage": { "usage": {
"prompt_tokens": 10,
"completion_tokens": 20,
"total_tokens": 30 "total_tokens": 30
} }
} }
@@ -95,15 +93,15 @@ For vision/audio models, content can be an array:
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "Describe this image"}, {"type": "text", "text": "Describe this image"},
{"type": "image_url", "image_url": {"url": "file:///path/to/image.jpg"}} {"type": "image", "image_url": {"url": "file:///path/to/image.jpg"}}
] ]
} }
``` ```
Supported content types: Supported content types:
- `text` - Text content - `text` - Text content
- `image_url` - Image file (file://, base64://, or http://) - `image_url` - Image file (file://, base64://, https:// or http://)
- `audio_url` - Audio file (file:// or base64://) - `audio_url` - Audio file (file://, base64://, https:// or http://)
#### Examples #### Examples
@@ -148,7 +146,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "What is in this image?"}, {"type": "text", "text": "What is in this image?"},
{"type": "image_url", "image_url": {"url": "file:///path/to/image.jpg"}} {"type": "image", "image_url": {"url": "file:///path/to/image.jpg"}}
] ]
} }
] ]
@@ -167,7 +165,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "Extract all text"}, {"type": "text", "text": "Extract all text"},
{"type": "image_url", "image_url": {"url": "file:///path/to/document.png"}} {"type": "image", "image_url": {"url": "file:///path/to/document.png"}}
] ]
} }
] ]
@@ -186,7 +184,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "Transcribe this audio"}, {"type": "text", "text": "Transcribe this audio"},
{"type": "audio_url", "audio_url": {"url": "file:///path/to/audio.wav"}} {"type": "audio", "audio_url": {"url": "file:///path/to/audio.wav"}}
] ]
} }
] ]
@@ -237,8 +235,6 @@ data: [DONE]
} }
], ],
"usage": { "usage": {
"prompt_tokens": 10,
"completion_tokens": 9,
"total_tokens": 19 "total_tokens": 19
} }
} }
@@ -265,8 +261,7 @@ POST /audio/speech
| Parameter | Type | Required | Description | | Parameter | Type | Required | Description |
|-----------|------|----------|-------------| |-----------|------|----------|-------------|
| `model` | string | Yes | Model identifier (e.g., "voxcpm1.5") | | `model` | string | Yes | Model identifier (e.g., "voxcpm1.5") |
| `input` | string | Yes | Text to convert to speech | | `messages` | array | Yes | Array of message objects |
| `voice` | string | No | Voice selection (default: "default") |
#### Example #### Example
@@ -275,15 +270,21 @@ curl http://127.0.0.1:10100/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "voxcpm1.5", "model": "voxcpm1.5",
"input": "你好，世界！", "messages": [
"voice": "default" {
}' \ "role": "user",
--output speech.wav "content": [
{"type": "text", "text": "Hello, this is AHA speaking."},
{"type": "audio", "audio_url": {"url": "https://package-release.coderbox.cn/aiway/test/other/%E5%93%AA%E5%90%92.wav"}}
]
}
]
}'
``` ```
#### Response #### Response
Returns audio data in WAV format. Returns audio data in base64 WAV format.
#### Supported Models #### Supported Models
@@ -303,7 +304,7 @@ POST /images/remove_background
| Parameter | Type | Required | Description | | Parameter | Type | Required | Description |
|-----------|------|----------|-------------| |-----------|------|----------|-------------|
| `model` | string | Yes | Model identifier (e.g., "rmbg2.0") | | `model` | string | Yes | Model identifier (e.g., "rmbg2.0") |
| `image` | string | Yes | Image file path (file://) or base64 data | | `messages` | array | Yes | Array of message objects |
#### Example #### Example
@@ -314,9 +315,15 @@ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "rmbg2.0",
"image": "file:///path/to/photo.png" "messages": [
}' \ {
--output no-background.png "role": "user",
"content": [
{"type": "image", "image_url": {"url": "file:///path/to/document.jpg"}}
]
}
]
}'
``` ```
**From Base64:** **From Base64:**
@@ -326,14 +333,20 @@ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "rmbg2.0",
"image": "base64://$(base64 -w 0 photo.png)" "messages": [
}' \ {
--output no-background.png "role": "user",
"content": [
{"type": "image", "image_url": {"url": "base64://$(base64 -w 0 photo.png)"}}
]
}
]
}'
``` ```
#### Response #### Response
Returns the processed image in PNG format. Returns the processed image in base64 PNG format.
#### Supported Models #### Supported Models
@@ -373,7 +386,7 @@ Currently, AHA does not implement rate limiting. The server can handle concurren
## OpenAI Compatibility ## OpenAI Compatibility
AHA's API is designed to be compatible with OpenAI's API format. This means you can use existing OpenAI client libraries with minimal changes: AHA's text generation API is designed to be compatible with OpenAI's API format. Multimodal APIs are derived from the text generation API with minimal changes:
### Python Example ### Python Example
docs/api.zh-CN.md
+40 -27
View File
@@ -39,8 +39,6 @@ Content-Type: application/json
"data": { ... }, "data": { ... },
"model": "model-name", "model": "model-name",
"usage": { "usage": {
"prompt_tokens": 10,
"completion_tokens": 20,
"total_tokens": 30 "total_tokens": 30
} }
} }
@@ -77,7 +75,7 @@ POST /chat/completions
| `temperature` | number | 否 | 采样温度（0-2，默认：1） | | `temperature` | number | 否 | 采样温度（0-2，默认：1） |
| `top_p` | number | 否 | 核采样（0-1，默认：1） | | `top_p` | number | 否 | 核采样（0-1，默认：1） |
| `max_tokens` | integer | 否 | 要生成的最大令牌数 | | `max_tokens` | integer | 否 | 要生成的最大令牌数 |
| `stream` | boolean | 否 | 启用流式传输（默认：false） | | `stream` | boolean | 否 | 启用流式传输（默认：true） |
#### 消息对象 #### 消息对象
@@ -95,15 +93,15 @@ POST /chat/completions
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "描述这张图片"}, {"type": "text", "text": "描述这张图片"},
{"type": "image_url", "image_url": {"url": "file:///path/to/image.jpg"}} {"type": "image", "image_url": {"url": "file:///path/to/image.jpg"}}
] ]
} }
``` ```
支持的内容类型： 支持的内容类型：
- `text` - 文本内容 - `text` - 文本内容
- `image_url` - 图像文件（file://、base64:// 或 http://） - `image` - 图像文件（file://,base64://, https:// 或 http://）
- `audio_url` - 音频文件（file:// 或 base64://） - `audio` - 音频文件（file://,base64://, https:// 或 http://）
#### 示例 #### 示例
@@ -148,7 +146,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "这张图片里有什么？"}, {"type": "text", "text": "这张图片里有什么？"},
{"type": "image_url", "image_url": {"url": "file:///path/to/image.jpg"}} {"type": "image", "image_url": {"url": "file:///path/to/image.jpg"}}
] ]
} }
] ]
@@ -167,7 +165,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "提取所有文本"}, {"type": "text", "text": "提取所有文本"},
{"type": "image_url", "image_url": {"url": "file:///path/to/document.png"}} {"type": "image", "image_url": {"url": "file:///path/to/document.png"}}
] ]
} }
] ]
@@ -186,7 +184,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "转写这段音频"}, {"type": "text", "text": "转写这段音频"},
{"type": "audio_url", "audio_url": {"url": "file:///path/to/audio.wav"}} {"type": "audio", "audio_url": {"url": "file:///path/to/audio.wav"}}
] ]
} }
] ]
@@ -237,8 +235,6 @@ data: [DONE]
} }
], ],
"usage": { "usage": {
"prompt_tokens": 10,
"completion_tokens": 9,
"total_tokens": 19 "total_tokens": 19
} }
} }
@@ -265,8 +261,7 @@ POST /audio/speech
| 参数 | 类型 | 必需 | 描述 | | 参数 | 类型 | 必需 | 描述 |
|------|------|------|------| |------|------|------|------|
| `model` | string | 是 | 模型标识符（如 "voxcpm1.5"） | | `model` | string | 是 | 模型标识符（如 "voxcpm1.5"） |
| `input` | string | 是 | 要转换为语音的文本 | | `messages` | array | 是 | 消息对象数组 |
| `voice` | string | 否 | 语音选择（默认："default"） |
#### 示例 #### 示例
@@ -275,15 +270,21 @@ curl http://127.0.0.1:10100/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "voxcpm1.5", "model": "voxcpm1.5",
"input": "你好，世界！", "messages": [
"voice": "default" {
}' \ "role": "user",
--output speech.wav "content": [
{"type": "text", "text": "你好，这是 AHA 在说话。"},
{"type": "audio", "audio_url": {"url": "https://package-release.coderbox.cn/aiway/test/other/%E5%93%AA%E5%90%92.wav"}}
]
}
]
}'
``` ```
#### 响应 #### 响应
以 WAV 格式返回音频数据。 以 base64 WAV 格式返回音频数据。
#### 支持的模型 #### 支持的模型
@@ -303,7 +304,7 @@ POST /images/remove_background
| 参数 | 类型 | 必需 | 描述 | | 参数 | 类型 | 必需 | 描述 |
|------|------|------|------| |------|------|------|------|
| `model` | string | 是 | 模型标识符（如 "rmbg2.0"） | | `model` | string | 是 | 模型标识符（如 "rmbg2.0"） |
| `image` | string | 是 | 图像文件路径（file://）或 base64 数据 | | `messages` | array | 是 | 消息对象数组 |
#### 示例 #### 示例
@@ -314,9 +315,15 @@ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "rmbg2.0",
"image": "file:///path/to/photo.png" "messages": [
}' \ {
--output no-background.png "role": "user",
"content": [
{"type": "image", "image_url": {"url": "file:///path/to/document.jpg"}}
]
}
]
}'
``` ```
**从 Base64：** **从 Base64：**
@@ -326,14 +333,20 @@ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "rmbg2.0",
"image": "base64://$(base64 -w 0 photo.png)" "messages": [
}' \ {
--output no-background.png "role": "user",
"content": [
{"type": "image", "image_url": {"url": "base64://$(base64 -w 0 photo.png)"}}
]
}
]
}'
``` ```
#### 响应 #### 响应
以 PNG 格式返回处理后的图像。 以base64 PNG 格式返回处理后的图像。
#### 支持的模型 #### 支持的模型
@@ -373,7 +386,7 @@ curl http://127.0.0.1:10100/images/remove_background \
## OpenAI 兼容性 ## OpenAI 兼容性
AHA 的 API 设计为与 OpenAI 的 API 格式兼容。这意味着您可以使用现有的 OpenAI 客户端库，只需最少的更改： AHA 的 文本生成 API 设计与 OpenAI 的 API 格式兼容。多模态 API 源自文本生成 API，仅做了少量改动：
### Python 示例 ### Python 示例
docs/concepts.md
+5 -7
View File
@@ -126,9 +126,7 @@ All models implement the `GenerateModel` trait:
```rust ```rust
pub trait GenerateModel { pub trait GenerateModel {
// Initialize model with weights
fn init(model_path: &str, config: Option<GenerationConfig>) -> Result<Self>;
// Generate response // Generate response
fn generate(&mut self, prompt: &str, params: GenerationParams) -> Result<String>; fn generate(&mut self, prompt: &str, params: GenerationParams) -> Result<String>;
@@ -167,8 +165,8 @@ This provides:
- `audio_utils.rs` - Audio processing (WAV, MP3) - `audio_utils.rs` - Audio processing (WAV, MP3)
- `image_utils.rs` - Image processing (resize, encode/decode) - `image_utils.rs` - Image processing (resize, encode/decode)
- `download.rs` - Model downloading with retry logic - `tensor_utils.rs` - Tensor utility methods
- `common.rs` - Common utilities and constants - `mod.rs` - Common utilities and constants
## Design Patterns ## Design Patterns
@@ -269,7 +267,7 @@ Flash Attention optimizes long-sequence processing:
Enable with: Enable with:
```bash ```bash
cargo build --features "cuda,flash-attn" cargo build --features cuda,flash-attn
``` ```
### Memory-Mapped Tensors ### Memory-Mapped Tensors
@@ -294,7 +292,7 @@ Dynamic precision selection based on hardware:
- No external API calls after model download - No external API calls after model download
- No telemetry or data collection - No telemetry or data collection
- Full control over your data - Data remains entirely on the local system
### Memory Safety ### Memory Safety
docs/concepts.zh-CN.md
+4 -6
View File
@@ -126,8 +126,6 @@ HTTP API 层提供 REST 端点：
```rust ```rust
pub trait GenerateModel { pub trait GenerateModel {
// 使用权重初始化模型
fn init(model_path: &str, config: Option<GenerationConfig>) -> Result<Self>;
// 生成响应 // 生成响应
fn generate(&mut self, prompt: &str, params: GenerationParams) -> Result<String>; fn generate(&mut self, prompt: &str, params: GenerationParams) -> Result<String>;
@@ -167,8 +165,8 @@ pub trait GenerateModel {
- `audio_utils.rs` - 音频处理（WAV、MP3） - `audio_utils.rs` - 音频处理（WAV、MP3）
- `image_utils.rs` - 图像处理（调整大小、编码/解码） - `image_utils.rs` - 图像处理（调整大小、编码/解码）
- `download.rs` - 带重试逻辑的模型下载 - `tensor_utils.rs` - Tensor常用方法
- `common.rs` - 通用工具和常量 - `mod.rs` - 通用工具和常量
## 设计模式 ## 设计模式
@@ -269,7 +267,7 @@ Flash Attention 优化长序列处理：
启用方式： 启用方式：
```bash ```bash
cargo build --features "cuda,flash-attn" cargo build --features cuda,flash-attn
``` ```
### 内存映射张量 ### 内存映射张量
@@ -294,7 +292,7 @@ cargo build --features "cuda,flash-attn"
- 模型下载后无外部 API 调用 - 模型下载后无外部 API 调用
- 无遥测或数据收集 - 无遥测或数据收集
- 完全控制您的数据 - 数据完全保留在本地
### 内存安全 ### 内存安全
docs/development.md
+5 -25
View File
@@ -34,9 +34,9 @@ cargo build
cargo build --release cargo build --release
# Build with features # Build with features
cargo build --features "cuda,flash-attn" cargo build --features cuda,flash-attn
cargo build --features "metal" cargo build --features metal
cargo build --features "ffmpeg" cargo build --features ffmpeg
``` ```
### Development Workflow ### Development Workflow
@@ -98,13 +98,12 @@ aha/
│ └── utils/ # Utility functions │ └── utils/ # Utility functions
│ ├── audio_utils.rs │ ├── audio_utils.rs
│ ├── image_utils.rs │ ├── image_utils.rs
│ ├── download.rs │ ├── tensor_utils.rs
│ └── common.rs │ └── mod.rs
├── tests/ # Integration tests ├── tests/ # Integration tests
│ ├── test_qwen2_5vl.rs │ ├── test_qwen2_5vl.rs
│ ├── test_qwen3vl.rs │ ├── test_qwen3vl.rs
│ └── ... │ └── ...
├── examples/ # Example code
└── docs/ # Documentation └── docs/ # Documentation
``` ```
@@ -140,16 +139,6 @@ pub struct NewModelConfig {
// ... other config fields // ... other config fields
} }
impl Default for NewModelConfig {
fn default() -> Self {
Self {
vocab_size: 32000,
hidden_size: 2048,
num_hidden_layers: 24,
num_attention_heads: 32,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GenerationConfig { pub struct GenerationConfig {
@@ -158,15 +147,6 @@ pub struct GenerationConfig {
pub max_tokens: usize, pub max_tokens: usize,
} }
impl Default for GenerationConfig {
fn default() -> Self {
Self {
temperature: 1.0,
top_p: 1.0,
max_tokens: 100,
}
}
}
``` ```
#### model.rs #### model.rs
docs/development.zh-CN.md
+5 -25
View File
@@ -34,9 +34,9 @@ cargo build
cargo build --release cargo build --release
# 使用功能构建 # 使用功能构建
cargo build --features "cuda,flash-attn" cargo build --features cuda,flash-attn
cargo build --features "metal" cargo build --features metal
cargo build --features "ffmpeg" cargo build --features ffmpeg
``` ```
### 开发工作流程 ### 开发工作流程
@@ -98,13 +98,12 @@ aha/
│ └── utils/ # 工具函数 │ └── utils/ # 工具函数
│ ├── audio_utils.rs │ ├── audio_utils.rs
│ ├── image_utils.rs │ ├── image_utils.rs
│ ├── download.rs │ ├── tensor_utils.rs
│ └── common.rs │ └── mod.rs
├── tests/ # 集成测试 ├── tests/ # 集成测试
│ ├── test_qwen2_5vl.rs │ ├── test_qwen2_5vl.rs
│ ├── test_qwen3vl.rs │ ├── test_qwen3vl.rs
│ └── ... │ └── ...
├── examples/ # 示例代码
└── docs/ # 文档 └── docs/ # 文档
``` ```
@@ -140,16 +139,6 @@ pub struct NewModelConfig {
// ... 其他配置字段 // ... 其他配置字段
} }
impl Default for NewModelConfig {
fn default() -> Self {
Self {
vocab_size: 32000,
hidden_size: 2048,
num_hidden_layers: 24,
num_attention_heads: 32,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GenerationConfig { pub struct GenerationConfig {
@@ -158,15 +147,6 @@ pub struct GenerationConfig {
pub max_tokens: usize, pub max_tokens: usize,
} }
impl Default for GenerationConfig {
fn default() -> Self {
Self {
temperature: 1.0,
top_p: 1.0,
max_tokens: 100,
}
}
}
``` ```
#### model.rs #### model.rs
docs/getting-started.md
+29 -16
View File
@@ -110,10 +110,11 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "Describe this image in detail."}, {"type": "text", "text": "Describe this image in detail."},
{"type": "image_url", "image_url": {"url": "file:///path/to/image.jpg"}} {"type": "image", "image_url": {"url": "file:///path/to/image.jpg"}}
] ]
} }
] ],
"stream": false
}' }'
``` ```
@@ -133,7 +134,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "Extract all text from this image."}, {"type": "text", "text": "Extract all text from this image."},
{"type": "image_url", "image_url": {"url": "file:///path/to/document.jpg"}} {"type": "image", "image_url": {"url": "file:///path/to/document.jpg"}}
] ]
} }
] ]
@@ -156,7 +157,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "Transcribe this audio."}, {"type": "text", "text": "Transcribe this audio."},
{"type": "audio_url", "audio_url": {"url": "file:///path/to/audio.wav"}} {"type": "audio", "audio_url": {"url": "file:///path/to/audio.wav"}}
] ]
} }
] ]
@@ -174,10 +175,16 @@ curl http://127.0.0.1:10100/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "voxcpm1.5", "model": "voxcpm1.5",
"input": "Hello, this is AHA speaking.", "messages": [
"voice": "default" {
}' \ "role": "user",
--output speech.wav "content": [
{"type": "text", "text": "Hello, this is AHA speaking."},
{"type": "audio", "audio_url": {"url": "https://package-release.coderbox.cn/aiway/test/other/%E5%93%AA%E5%90%92.wav"}}
]
}
]
}'
``` ```
### Background Removal ### Background Removal
@@ -191,9 +198,15 @@ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "rmbg2.0",
"image": "file:///path/to/photo.png" "messages": [
}' \ {
--output no-background.png "role": "user",
"content": [
{"type": "image", "image_url": {"url": "file:///path/to/document.jpg"}}
]
}
]
}'
``` ```
### Direct Inference (Without Server) ### Direct Inference (Without Server)
@@ -202,7 +215,7 @@ curl http://127.0.0.1:10100/images/remove_background \
# Run inference directly without starting HTTP server # Run inference directly without starting HTTP server
aha run -m qwen3-0.6b \ aha run -m qwen3-0.6b \
-i "Write a haiku about AI" \ -i "Write a haiku about AI" \
--weight-path ~/.aha/qwen3-0.6b --weight-path ~/.aha/Qwen/Qwen3-0.6B
``` ```
## Configuration Options ## Configuration Options
@@ -239,7 +252,7 @@ aha download -m qwen3vl-2b -s /data/models
## Streaming Responses ## Streaming Responses
For real-time responses, use streaming: For chat/completions, using no "stream" field or "stream": true enables streaming, while "stream": false is for non-streaming responses:
```bash ```bash
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
@@ -249,7 +262,7 @@ curl http://127.0.0.1:10100/chat/completions \
"messages": [ "messages": [
{"role": "user", "content": "Tell me a story"} {"role": "user", "content": "Tell me a story"}
], ],
"stream": true "stream": false
}' }'
``` ```
@@ -308,7 +321,7 @@ aha download -m qwen3vl-2b
Then use them later without internet: Then use them later without internet:
```bash ```bash
aha serv -m qwen3vl-2b --weight-path ~/.aha/qwen3vl-2b aha serv -m qwen3vl-2b --weight-path ~/.aha/Qwen/Qwen3-VL-2B-Instruct
``` ```
### 4. Manage Disk Space ### 4. Manage Disk Space
@@ -366,7 +379,7 @@ aha cli -m qwen3-0.6b
## Examples Repository ## Examples Repository
For more examples, check out the [examples](../examples/) directory in the repository. For more examples, check out the [tests](../tests/) directory in the repository.
## See Also ## See Also
docs/getting-started.zh-CN.md
+33 -19
View File
@@ -13,7 +13,7 @@ aha list
### 2. 下载第一个模型 ### 2. 下载第一个模型
```bash ```bash
# 下载一个小型文本模型开始 # 从下载一个小型文本模型开始
aha download -m qwen3-0.6b aha download -m qwen3-0.6b
``` ```
@@ -37,7 +37,8 @@ curl http://127.0.0.1:10100/chat/completions \
"model": "qwen3-0.6b", "model": "qwen3-0.6b",
"messages": [ "messages": [
{"role": "user", "content": "你好，AHA！"} {"role": "user", "content": "你好，AHA！"}
] ],
"stream": false
}' }'
``` ```
@@ -110,10 +111,11 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "详细描述这张图片。"}, {"type": "text", "text": "详细描述这张图片。"},
{"type": "image_url", "image_url": {"url": "file:///path/to/image.jpg"}} {"type": "image", "image_url": {"url": "file:///path/to/image.jpg"}}
] ]
} }
] ],
"stream": false
}' }'
``` ```
@@ -133,7 +135,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "提取这张图片中的所有文本。"}, {"type": "text", "text": "提取这张图片中的所有文本。"},
{"type": "image_url", "image_url": {"url": "file:///path/to/document.jpg"}} {"type": "image", "image_url": {"url": "file:///path/to/document.jpg"}}
] ]
} }
] ]
@@ -156,7 +158,7 @@ curl http://127.0.0.1:10100/chat/completions \
"role": "user", "role": "user",
"content": [ "content": [
{"type": "text", "text": "转写这段音频。"}, {"type": "text", "text": "转写这段音频。"},
{"type": "audio_url", "audio_url": {"url": "file:///path/to/audio.wav"}} {"type": "audio", "audio_url": {"url": "file:///path/to/audio.wav"}}
] ]
} }
] ]
@@ -174,10 +176,16 @@ curl http://127.0.0.1:10100/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "voxcpm1.5", "model": "voxcpm1.5",
"input": "你好，这是 AHA 在说话。", "messages": [
"voice": "default" {
}' \ "role": "user",
--output speech.wav "content": [
{"type": "text", "text": "你好，这是 AHA 在说话。"},
{"type": "audio", "audio_url": {"url": "https://package-release.coderbox.cn/aiway/test/other/%E5%93%AA%E5%90%92.wav"}}
]
}
]
}'
``` ```
### 背景移除 ### 背景移除
@@ -191,9 +199,15 @@ curl http://127.0.0.1:10100/images/remove_background \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"model": "rmbg2.0", "model": "rmbg2.0",
"image": "file:///path/to/photo.png" "messages": [
}' \ {
--output no-background.png "role": "user",
"content": [
{"type": "image", "image_url": {"url": "file:///path/to/document.jpg"}}
]
}
]
}'
``` ```
### 直接推理（无需服务器） ### 直接推理（无需服务器）
@@ -201,8 +215,8 @@ curl http://127.0.0.1:10100/images/remove_background \
```bash ```bash
# 直接运行推理，无需启动 HTTP 服务器 # 直接运行推理，无需启动 HTTP 服务器
aha run -m qwen3-0.6b \ aha run -m qwen3-0.6b \
-i "写一首关于 AI 的俳句" \ -i "写一首关于AI的俳句" \
--weight-path ~/.aha/qwen3-0.6b --weight-path ~/.aha/Qwen/Qwen3-0.6B
``` ```
## 配置选项 ## 配置选项
@@ -239,7 +253,7 @@ aha download -m qwen3vl-2b -s /data/models
## 流式响应 ## 流式响应
对于实时响应，使用流式传输： 对于chat/completions，无"stream"字段或"stream": true时使用流式传输，"stream": false为非流式：
```bash ```bash
curl http://127.0.0.1:10100/chat/completions \ curl http://127.0.0.1:10100/chat/completions \
@@ -249,7 +263,7 @@ curl http://127.0.0.1:10100/chat/completions \
"messages": [ "messages": [
{"role": "user", "content": "给我讲个故事"} {"role": "user", "content": "给我讲个故事"}
], ],
"stream": true "stream": false
}' }'
``` ```
@@ -308,7 +322,7 @@ aha download -m qwen3vl-2b
稍后在没有网络的情况下使用： 稍后在没有网络的情况下使用：
```bash ```bash
aha serv -m qwen3vl-2b --weight-path ~/.aha/qwen3vl-2b aha serv -m qwen3vl-2b --weight-path ~/.aha/Qwen/Qwen3-VL-2B-Instruct
``` ```
### 4. 管理磁盘空间 ### 4. 管理磁盘空间
@@ -366,7 +380,7 @@ aha cli -m qwen3-0.6b
## 示例仓库 ## 示例仓库
更多示例，请查看仓库中的 [examples](../examples/) 目录。 更多示例，请查看仓库中的 [tests](../tests/) 目录。
## 另见 ## 另见
docs/installation.md
+5 -5
View File
@@ -70,7 +70,7 @@ cargo build --release --features cuda
cargo build --release --features metal cargo build --release --features metal
# With Flash Attention # With Flash Attention
cargo build --release --features "cuda,flash-attn" cargo build --release --features cuda,flash-attn
# With FFmpeg support # With FFmpeg support
cargo build --release --features ffmpeg cargo build --release --features ffmpeg
@@ -213,7 +213,7 @@ cargo build --release --features metal
Enables Flash Attention for optimized long-sequence processing. Enables Flash Attention for optimized long-sequence processing.
```bash ```bash
cargo build --release --features "cuda,flash-attn" cargo build --release --features cuda,flash-attn
``` ```
**Requirements**: **Requirements**:
@@ -250,13 +250,13 @@ You can combine multiple features:
```bash ```bash
# Maximum performance on NVIDIA GPU # Maximum performance on NVIDIA GPU
cargo build --release --features "cuda,flash-attn" cargo build --release --features cuda,flash-attn
# Apple Silicon with audio support # Apple Silicon with audio support
cargo build --release --features "metal,ffmpeg" cargo build --release --features metal,ffmpeg
# Everything enabled # Everything enabled
cargo build --release --features "cuda,flash-attn,ffmpeg" cargo build --release --features cuda,flash-attn,ffmpeg
``` ```
## Verification ## Verification
docs/installation.zh-CN.md
+5 -5
View File
@@ -70,7 +70,7 @@ cargo build --release --features cuda
cargo build --release --features metal cargo build --release --features metal
# 启用 Flash Attention # 启用 Flash Attention
cargo build --release --features "cuda,flash-attn" cargo build --release --features cuda,flash-attn
# 启用 FFmpeg 支持 # 启用 FFmpeg 支持
cargo build --release --features ffmpeg cargo build --release --features ffmpeg
@@ -212,7 +212,7 @@ cargo build --release --features metal
启用 Flash Attention 以优化长序列处理。 启用 Flash Attention 以优化长序列处理。
```bash ```bash
cargo build --release --features "cuda,flash-attn" cargo build --release --features cuda,flash-attn
``` ```
**要求**： **要求**：
@@ -249,13 +249,13 @@ cargo build --release --features ffmpeg
```bash ```bash
# NVIDIA GPU 上的最佳性能 # NVIDIA GPU 上的最佳性能
cargo build --release --features "cuda,flash-attn" cargo build --release --features cuda,flash-attn
# 带音频支持的 Apple Silicon # 带音频支持的 Apple Silicon
cargo build --release --features "metal,ffmpeg" cargo build --release --features metal,ffmpeg
# 启用所有功能 # 启用所有功能
cargo build --release --features "cuda,flash-attn,ffmpeg" cargo build --release --features cuda,flash-attn,ffmpeg
``` ```
## 验证安装 ## 验证安装
src/models/mod.rs
+1 -1
View File
@@ -48,7 +48,7 @@ pub enum WhichModel {
Qwen3ASR0_6B, Qwen3ASR0_6B,
#[value(name = "qwen3asr-1.7b", hide = true)] #[value(name = "qwen3asr-1.7b", hide = true)]
Qwen3ASR1_7B, Qwen3ASR1_7B,
#[value(name = "qwen3vl-4b", hide = true)] #[value(name = "qwen3vl-2b", hide = true)]
Qwen3vl2B, Qwen3vl2B,
#[value(name = "qwen3vl-4b", hide = true)] #[value(name = "qwen3vl-4b", hide = true)]
Qwen3vl4B, Qwen3vl4B,