feat: CLI Subcommand Support
- **CLI Subcommand Support**: Added three new subcommands for better command organization: - `aha cli` - Download model and start HTTP service (default, backward compatible) - `aha serv` - Start HTTP service only (requires `--weight-path`) - `aha download` - Download model only (no service start)
This commit is contained in:
XiaoYang
+109
@@ -0,0 +1,109 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||||
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **CLI Subcommand Support**: Added three new subcommands for better command organization:
|
||||||
|
- `aha cli` - Download model and start HTTP service (default, backward compatible)
|
||||||
|
- `aha serv` - Start HTTP service only (requires `--weight-path`)
|
||||||
|
- `aha download` - Download model only (no service start)
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **CLI Structure**: Refactored CLI to use clap's Subcommand feature while maintaining backward compatibility
|
||||||
|
- **Backward Compatibility**: Commands without subcommand now default to `cli` subcommand:
|
||||||
|
- `aha -m qwen3vl-2b` is equivalent to `aha cli -m qwen3vl-2b`
|
||||||
|
- All existing parameter options and defaults remain unchanged
|
||||||
|
|
||||||
|
### Technical Details
|
||||||
|
|
||||||
|
**Subcommand Parameters:**
|
||||||
|
|
||||||
|
`aha cli`:
|
||||||
|
- `-a, --address <ADDRESS>` - Server address (default: 127.0.0.1)
|
||||||
|
- `-p, --port <PORT>` - Server port (default: 10100)
|
||||||
|
- `-m, --model <MODEL>` - Model to use (required)
|
||||||
|
- `--weight-path <WEIGHT_PATH>` - Local model weight path (optional)
|
||||||
|
- `--save-dir <SAVE_DIR>` - Directory to save downloaded model (optional)
|
||||||
|
- `--download-retries <DOWNLOAD_RETRIES>` - Download retry attempts (default: 3)
|
||||||
|
|
||||||
|
`aha serv`:
|
||||||
|
- `-a, --address <ADDRESS>` - Server address (default: 127.0.0.1)
|
||||||
|
- `-p, --port <PORT>` - Server port (default: 10100)
|
||||||
|
- `-m, --model <MODEL>` - Model to use (required)
|
||||||
|
- `--weight-path <WEIGHT_PATH>` - Local model weight path (required)
|
||||||
|
|
||||||
|
`aha download`:
|
||||||
|
- `-m, --model <MODEL>` - Model to download (required)
|
||||||
|
- `-s, --save-dir <SAVE_DIR>` - Directory to save downloaded model (optional)
|
||||||
|
- `--download-retries <DOWNLOAD_RETRIES>` - Download retry attempts (default: 3)
|
||||||
|
|
||||||
|
**Code Changes:**
|
||||||
|
- Modified `src/main.rs` only
|
||||||
|
- Extracted common functions: `get_model_id()`, `start_http_server()`
|
||||||
|
- Reused existing `download_model()` and `init()` functions
|
||||||
|
- No changes to other modules or dependencies
|
||||||
|
|
||||||
|
## [0.1.8] - 2025-01-20
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Support for Fun-ASR-Nano-2512 model
|
||||||
|
- Support for Qwen3-0.6B model
|
||||||
|
|
||||||
|
## [0.1.7] - 2024-XX-XX
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Support for GLM-ASR-Nano-2512 model
|
||||||
|
|
||||||
|
## [0.1.6] - 2024-XX-XX
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Support for RMBG-2.0 model (background removal)
|
||||||
|
|
||||||
|
## [0.1.5] - 2024-XX-XX
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Support for VoxCPM1.5 model
|
||||||
|
|
||||||
|
## [0.1.4] - 2024-XX-XX
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Support for PaddleOCR-VL model
|
||||||
|
|
||||||
|
## [0.1.3] - 2024-XX-XX
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Support for Hunyuan-OCR model
|
||||||
|
|
||||||
|
## [0.1.2] - 2024-XX-XX
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Support for DeepSeek-OCR model
|
||||||
|
|
||||||
|
## [0.1.1] - 2024-XX-XX
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Support for Qwen3VL model family (2B, 4B, 8B, 32B)
|
||||||
|
|
||||||
|
## [0.1.0] - 2024-XX-XX
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Initial release
|
||||||
|
- Support for Qwen2.5VL models (3B, 7B)
|
||||||
|
- Support for MiniCPM4-0.5B model
|
||||||
|
- Support for VoxCPM-0.5B model
|
||||||
@@ -0,0 +1,242 @@
|
|||||||
|
# AHA 命令行使用说明
|
||||||
|
|
||||||
|
## 概述
|
||||||
|
|
||||||
|
AHA 是一个基于 Candle 框架的高性能模型推理库，支持多种多模态模型，包括视觉、语言和语音模型。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
aha [COMMAND] [OPTIONS]
|
||||||
|
```
|
||||||
|
|
||||||
|
## 全局选项
|
||||||
|
|
||||||
|
| 选项 | 说明 | 默认值 |
|
||||||
|
|------|------|--------|
|
||||||
|
| `-a, --address <ADDRESS>` | 服务监听地址 | 127.0.0.1 |
|
||||||
|
| `-p, --port <PORT>` | 服务监听端口 | 10100 |
|
||||||
|
| `-m, --model <MODEL>` | 模型类型（必选） | - |
|
||||||
|
| `--weight-path <WEIGHT_PATH>` | 本地模型权重路径 | - |
|
||||||
|
| `--save-dir <SAVE_DIR>` | 模型下载保存目录 | ~/.aha/ |
|
||||||
|
| `--download-retries <DOWNLOAD_RETRIES>` | 下载重试次数 | 3 |
|
||||||
|
| `-h, --help` | 显示帮助信息 | - |
|
||||||
|
| `-V, --version` | 显示版本号 | - |
|
||||||
|
|
||||||
|
## 子命令
|
||||||
|
|
||||||
|
### cli - 下载模型并启动服务（默认）
|
||||||
|
|
||||||
|
下载指定的模型并启动 HTTP 服务。当不指定子命令时，默认使用此命令。
|
||||||
|
|
||||||
|
**语法：**
|
||||||
|
```bash
|
||||||
|
aha cli [OPTIONS] --model <MODEL>
|
||||||
|
```
|
||||||
|
|
||||||
|
**选项：**
|
||||||
|
|
||||||
|
| 选项 | 说明 | 默认值 |
|
||||||
|
|------|------|--------|
|
||||||
|
| `-a, --address <ADDRESS>` | 服务监听地址 | 127.0.0.1 |
|
||||||
|
| `-p, --port <PORT>` | 服务监听端口 | 10100 |
|
||||||
|
| `-m, --model <MODEL>` | 模型类型（必选） | - |
|
||||||
|
| `--weight-path <WEIGHT_PATH>` | 本地模型权重路径（如指定则跳过下载） | - |
|
||||||
|
| `--save-dir <SAVE_DIR>` | 模型下载保存目录 | ~/.aha/ |
|
||||||
|
| `--download-retries <DOWNLOAD_RETRIES>` | 下载重试次数 | 3 |
|
||||||
|
|
||||||
|
**示例：**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 下载模型并启动服务（默认端口 10100）
|
||||||
|
aha cli -m qwen3vl-2b
|
||||||
|
|
||||||
|
# 指定端口和保存目录
|
||||||
|
aha cli -m qwen3vl-2b -p 8080 --save-dir /data/models
|
||||||
|
|
||||||
|
# 使用本地模型（不下载）
|
||||||
|
aha cli -m qwen3vl-2b --weight-path /path/to/model
|
||||||
|
|
||||||
|
# 向后兼容方式（等同于 cli 子命令）
|
||||||
|
aha -m qwen3vl-2b
|
||||||
|
```
|
||||||
|
|
||||||
|
### serv - 启动服务
|
||||||
|
|
||||||
|
仅启动 HTTP 服务，不下载模型。必须通过 `--weight-path` 指定本地模型路径。
|
||||||
|
|
||||||
|
**语法：**
|
||||||
|
```bash
|
||||||
|
aha serv [OPTIONS] --model <MODEL> --weight-path <WEIGHT_PATH>
|
||||||
|
```
|
||||||
|
|
||||||
|
**选项：**
|
||||||
|
|
||||||
|
| 选项 | 说明 | 默认值 |
|
||||||
|
|------|------|--------|
|
||||||
|
| `-a, --address <ADDRESS>` | 服务监听地址 | 127.0.0.1 |
|
||||||
|
| `-p, --port <PORT>` | 服务监听端口 | 10100 |
|
||||||
|
| `-m, --model <MODEL>` | 模型类型（必选） | - |
|
||||||
|
| `--weight-path <WEIGHT_PATH>` | 本地模型权重路径（必选） | - |
|
||||||
|
|
||||||
|
**示例：**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 使用本地模型启动服务
|
||||||
|
aha serv -m qwen3vl-2b --weight-path /path/to/model
|
||||||
|
|
||||||
|
# 指定端口启动
|
||||||
|
aha serv -m qwen3vl-2b --weight-path /path/to/model -p 8080
|
||||||
|
|
||||||
|
# 指定监听地址
|
||||||
|
aha serv -m qwen3vl-2b --weight-path /path/to/model -a 0.0.0.0
|
||||||
|
```
|
||||||
|
|
||||||
|
### download - 下载模型
|
||||||
|
|
||||||
|
仅下载指定模型，不启动服务。
|
||||||
|
|
||||||
|
**语法：**
|
||||||
|
```bash
|
||||||
|
aha download [OPTIONS] --model <MODEL>
|
||||||
|
```
|
||||||
|
|
||||||
|
**选项：**
|
||||||
|
|
||||||
|
| 选项 | 说明 | 默认值 |
|
||||||
|
|------|------|--------|
|
||||||
|
| `-m, --model <MODEL>` | 模型类型（必选） | - |
|
||||||
|
| `-s, --save-dir <SAVE_DIR>` | 模型下载保存目录 | ~/.aha/ |
|
||||||
|
| `--download-retries <DOWNLOAD_RETRIES>` | 下载重试次数 | 3 |
|
||||||
|
|
||||||
|
**示例：**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 下载模型到默认目录
|
||||||
|
aha download -m qwen3vl-2b
|
||||||
|
|
||||||
|
# 指定保存目录
|
||||||
|
aha download -m qwen3vl-2b -s /data/models
|
||||||
|
|
||||||
|
# 指定下载重试次数
|
||||||
|
aha download -m qwen3vl-2b --download-retries 5
|
||||||
|
|
||||||
|
# 下载 MiniCPM4-0.5B 模型
|
||||||
|
aha download -m minicpm4-0.5b -s models
|
||||||
|
```
|
||||||
|
|
||||||
|
## 支持的模型
|
||||||
|
|
||||||
|
| 模型标识 | 模型名称 | 说明 |
|
||||||
|
|---------|---------|------|
|
||||||
|
| `minicpm4-0.5b` | OpenBMB/MiniCPM4-0.5B | 面壁智能 MiniCPM4 0.5B 模型 |
|
||||||
|
| `qwen2.5vl-3b` | Qwen/Qwen2.5-VL-3B-Instruct | 通义千问 2.5 VL 3B 模型 |
|
||||||
|
| `qwen2.5vl-7b` | Qwen/Qwen2.5-VL-7B-Instruct | 通义千问 2.5 VL 7B 模型 |
|
||||||
|
| `qwen3-0.6b` | Qwen/Qwen3-0.6B | 通义千问 3 0.6B 模型 |
|
||||||
|
| `qwen3vl-2b` | Qwen/Qwen3-VL-2B-Instruct | 通义千问 3 VL 2B 模型 |
|
||||||
|
| `qwen3vl-4b` | Qwen/Qwen3-VL-4B-Instruct | 通义千问 3 VL 4B 模型 |
|
||||||
|
| `qwen3vl-8b` | Qwen/Qwen3-VL-8B-Instruct | 通义千问 3 VL 8B 模型 |
|
||||||
|
| `qwen3vl-32b` | Qwen/Qwen3-VL-32B-Instruct | 通义千问 3 VL 32B 模型 |
|
||||||
|
| `deepseek-ocr` | deepseek-ai/DeepSeek-OCR | DeepSeek OCR 模型 |
|
||||||
|
| `hunyuan-ocr` | Tencent-Hunyuan/HunyuanOCR | 腾讯混元 OCR 模型 |
|
||||||
|
| `paddleocr-vl` | PaddlePaddle/PaddleOCR-VL | 百度飞桨 OCR VL 模型 |
|
||||||
|
| `RMBG2.0` | AI-ModelScope/RMBG-2.0 | RMBG 2.0 背景移除模型 |
|
||||||
|
| `voxcpm` | OpenBMB/VoxCPM-0.5B | 面壁智能 VoxCPM 0.5B 语音生成模型 |
|
||||||
|
| `voxcpm1.5` | OpenBMB/VoxCPM1.5 | 面壁智能 VoxCPM 1.5 语音生成模型 |
|
||||||
|
| `glm-asr-nano-2512` | ZhipuAI/GLM-ASR-Nano-2512 | 智谱 AI ASR Nano 2512 语音识别模型 |
|
||||||
|
| `fun-asr-nano-2512` | FunAudioLLM/Fun-ASR-Nano-2512 | 通义百聆 ASR Nano 2512 语音识别模型 |
|
||||||
|
|
||||||
|
## 常见使用场景
|
||||||
|
|
||||||
|
### 场景 1：快速启动推理服务
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 一条命令下载并启动服务
|
||||||
|
aha -m qwen3vl-2b
|
||||||
|
```
|
||||||
|
|
||||||
|
### 场景 2：使用已有模型启动服务
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 假设模型已下载到 /data/models/Qwen/Qwen3-VL-2B-Instruct
|
||||||
|
aha serv -m qwen3vl-2b --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct
|
||||||
|
```
|
||||||
|
|
||||||
|
### 场景 3：预先下载模型
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 下载模型到指定目录，稍后使用
|
||||||
|
aha download -m qwen3vl-2b -s /data/models
|
||||||
|
|
||||||
|
# 后续启动时直接使用
|
||||||
|
aha serv -m qwen3vl-2b --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct
|
||||||
|
```
|
||||||
|
|
||||||
|
### 场景 4：自定义服务端口和地址
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 在 0.0.0.0:8080 启动服务，允许外部访问
|
||||||
|
aha -m qwen3vl-2b -a 0.0.0.0 -p 8080
|
||||||
|
```
|
||||||
|
|
||||||
|
## API 接口
|
||||||
|
|
||||||
|
服务启动后，提供以下 API 接口：
|
||||||
|
|
||||||
|
### 对话接口
|
||||||
|
- **端点**: `POST /chat/completions`
|
||||||
|
- **功能**: 多模态对话和文本生成
|
||||||
|
- **支持模型**: Qwen2.5VL, Qwen3, Qwen3VL, DeepSeekOCR, GLM-ASR-Nano-2512, Fun-ASR-Nano-2512 等
|
||||||
|
- **格式**: OpenAI Chat Completion 格式
|
||||||
|
- **流式支持**: 支持
|
||||||
|
|
||||||
|
### 图像处理接口
|
||||||
|
- **端点**: `POST /images/remove_background`
|
||||||
|
- **功能**: 图像背景移除
|
||||||
|
- **支持模型**: RMBG-2.0
|
||||||
|
- **格式**: OpenAI Chat Completion 格式
|
||||||
|
- **流式支持**: 不支持
|
||||||
|
|
||||||
|
### 语音生成接口
|
||||||
|
- **端点**: `POST /audio/speech`
|
||||||
|
- **功能**: 语音合成和生成
|
||||||
|
- **支持模型**: VoxCPM, VoxCPM1.5
|
||||||
|
- **格式**: OpenAI Chat Completion 格式
|
||||||
|
- **流式支持**: 不支持
|
||||||
|
|
||||||
|
## 向后兼容性
|
||||||
|
|
||||||
|
为了保持与旧版本的兼容性，以下两种使用方式是等效的：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 新方式（推荐）
|
||||||
|
aha cli -m qwen3vl-2b
|
||||||
|
|
||||||
|
# 旧方式（向后兼容）
|
||||||
|
aha -m qwen3vl-2b
|
||||||
|
```
|
||||||
|
|
||||||
|
## 注意事项
|
||||||
|
|
||||||
|
1. **serv 子命令必须指定 `--weight-path`**：由于 `serv` 子命令不下载模型，必须通过 `--weight-path` 指定已下载的模型路径。
|
||||||
|
|
||||||
|
2. **下载重试机制**：默认重试 3 次，每次失败后等待 2 秒再重试。可通过 `--download-retries` 调整重试次数。
|
||||||
|
|
||||||
|
3. **默认保存目录**：模型默认保存到 `~/.aha/` 目录下，可通过 `--save-dir` 或 `-d` 参数自定义。
|
||||||
|
|
||||||
|
4. **端口占用**：启动服务前确保指定的端口未被占用，默认端口为 10100。
|
||||||
|
|
||||||
|
5. **权限问题**：如果保存到系统目录（如 `/data/models`），确保有相应的写入权限。
|
||||||
|
|
||||||
|
## 获取帮助
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 查看主帮助
|
||||||
|
aha --help
|
||||||
|
|
||||||
|
# 查看子命令帮助
|
||||||
|
aha cli --help
|
||||||
|
aha serv --help
|
||||||
|
aha download --help
|
||||||
|
|
||||||
|
# 查看版本信息
|
||||||
|
aha --version
|
||||||
|
```
|
||||||
+164
-27
@@ -1,7 +1,7 @@
|
|||||||
use std::{net::IpAddr, str::FromStr, time::Duration};
|
use std::{net::IpAddr, str::FromStr, time::Duration};
|
||||||
|
|
||||||
use aha::{models::WhichModel, utils::get_default_save_dir};
|
use aha::{models::WhichModel, utils::get_default_save_dir};
|
||||||
use clap::Parser;
|
use clap::{Args, Parser, Subcommand};
|
||||||
use modelscope::ModelScope;
|
use modelscope::ModelScope;
|
||||||
use rocket::{
|
use rocket::{
|
||||||
Config,
|
Config,
|
||||||
@@ -14,26 +14,109 @@ use crate::api::init;
|
|||||||
mod api;
|
mod api;
|
||||||
|
|
||||||
#[derive(Parser, Debug)]
|
#[derive(Parser, Debug)]
|
||||||
|
#[command(name = "aha")]
|
||||||
#[command(version, about, long_about = None)]
|
#[command(version, about, long_about = None)]
|
||||||
struct Args {
|
struct Cli {
|
||||||
|
/// Service listen address
|
||||||
#[arg(short, long, default_value = "127.0.0.1")]
|
#[arg(short, long, default_value = "127.0.0.1")]
|
||||||
address: String,
|
address: Option<String>,
|
||||||
|
|
||||||
#[arg(short, long, default_value_t = 10100)]
|
|
||||||
port: u16,
|
|
||||||
|
|
||||||
|
/// Service listen port
|
||||||
#[arg(short, long)]
|
#[arg(short, long)]
|
||||||
model: WhichModel,
|
port: Option<u16>,
|
||||||
|
|
||||||
|
/// Model type (required for backward compatibility)
|
||||||
|
#[arg(short, long)]
|
||||||
|
model: Option<WhichModel>,
|
||||||
|
|
||||||
|
/// Local model weight path
|
||||||
#[arg(long)]
|
#[arg(long)]
|
||||||
weight_path: Option<String>,
|
weight_path: Option<String>,
|
||||||
|
|
||||||
|
/// Model download save directory
|
||||||
#[arg(long)]
|
#[arg(long)]
|
||||||
save_dir: Option<String>,
|
save_dir: Option<String>,
|
||||||
|
|
||||||
|
/// Download retry count
|
||||||
|
#[arg(long)]
|
||||||
|
download_retries: Option<u32>,
|
||||||
|
|
||||||
|
#[command(subcommand)]
|
||||||
|
command: Option<Commands>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Subcommand, Debug)]
|
||||||
|
enum Commands {
|
||||||
|
/// Download model and start service (default)
|
||||||
|
Cli(CliArgs),
|
||||||
|
/// Start service only (requires --weight-path)
|
||||||
|
Serv(ServArgs),
|
||||||
|
/// Download model only
|
||||||
|
Download(DownloadArgs),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Common/shared arguments for server operations
|
||||||
|
#[derive(Args, Debug)]
|
||||||
|
struct CommonArgs {
|
||||||
|
/// Service listen address
|
||||||
|
#[arg(short, long, default_value = "127.0.0.1")]
|
||||||
|
address: String,
|
||||||
|
|
||||||
|
/// Service listen port
|
||||||
|
#[arg(short, long, default_value_t = 10100)]
|
||||||
|
port: u16,
|
||||||
|
|
||||||
|
/// Model type (required)
|
||||||
|
#[arg(short, long)]
|
||||||
|
model: WhichModel,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Arguments for the 'cli' subcommand (download + serve)
|
||||||
|
#[derive(Args, Debug)]
|
||||||
|
struct CliArgs {
|
||||||
|
#[command(flatten)]
|
||||||
|
common: CommonArgs,
|
||||||
|
|
||||||
|
/// Local model weight path (skip download if provided)
|
||||||
|
#[arg(long)]
|
||||||
|
weight_path: Option<String>,
|
||||||
|
|
||||||
|
/// Model download save directory
|
||||||
|
#[arg(long)]
|
||||||
|
save_dir: Option<String>,
|
||||||
|
|
||||||
|
/// Download retry count
|
||||||
#[arg(long)]
|
#[arg(long)]
|
||||||
download_retries: Option<u32>,
|
download_retries: Option<u32>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Arguments for the 'serv' subcommand (serve only)
|
||||||
|
#[derive(Args, Debug)]
|
||||||
|
struct ServArgs {
|
||||||
|
#[command(flatten)]
|
||||||
|
common: CommonArgs,
|
||||||
|
|
||||||
|
/// Local model weight path (required)
|
||||||
|
#[arg(long, required = true)]
|
||||||
|
weight_path: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Arguments for the 'download' subcommand (download only)
|
||||||
|
#[derive(Args, Debug)]
|
||||||
|
struct DownloadArgs {
|
||||||
|
/// Model type (required)
|
||||||
|
#[arg(short, long)]
|
||||||
|
model: WhichModel,
|
||||||
|
|
||||||
|
/// Model download save directory
|
||||||
|
#[arg(short, long)]
|
||||||
|
save_dir: Option<String>,
|
||||||
|
|
||||||
|
/// Download retry count
|
||||||
|
#[arg(long)]
|
||||||
|
download_retries: Option<u32>,
|
||||||
|
}
|
||||||
|
|
||||||
async fn download_model(model_id: &str, save_dir: &str, max_retries: u32) -> anyhow::Result<()> {
|
async fn download_model(model_id: &str, save_dir: &str, max_retries: u32) -> anyhow::Result<()> {
|
||||||
let mut attempts = 0u32;
|
let mut attempts = 0u32;
|
||||||
loop {
|
loop {
|
||||||
@@ -67,10 +150,9 @@ async fn download_model(model_id: &str, save_dir: &str, max_retries: u32) -> any
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::main]
|
/// Get the ModelScope model ID for a given WhichModel variant
|
||||||
async fn main() -> anyhow::Result<()> {
|
fn get_model_id(model: WhichModel) -> &'static str {
|
||||||
let args = Args::parse();
|
match model {
|
||||||
let model_id = match &args.model {
|
|
||||||
WhichModel::MiniCPM4_0_5B => "OpenBMB/MiniCPM4-0.5B",
|
WhichModel::MiniCPM4_0_5B => "OpenBMB/MiniCPM4-0.5B",
|
||||||
WhichModel::Qwen2_5vl3B => "Qwen/Qwen2.5-VL-3B-Instruct",
|
WhichModel::Qwen2_5vl3B => "Qwen/Qwen2.5-VL-3B-Instruct",
|
||||||
WhichModel::Qwen2_5vl7B => "Qwen/Qwen2.5-VL-7B-Instruct",
|
WhichModel::Qwen2_5vl7B => "Qwen/Qwen2.5-VL-7B-Instruct",
|
||||||
@@ -87,30 +169,89 @@ async fn main() -> anyhow::Result<()> {
|
|||||||
WhichModel::VoxCPM1_5 => "OpenBMB/VoxCPM1.5",
|
WhichModel::VoxCPM1_5 => "OpenBMB/VoxCPM1.5",
|
||||||
WhichModel::GlmASRNano2512 => "ZhipuAI/GLM-ASR-Nano-2512",
|
WhichModel::GlmASRNano2512 => "ZhipuAI/GLM-ASR-Nano-2512",
|
||||||
WhichModel::FunASRNano2512 => "FunAudioLLM/Fun-ASR-Nano-2512",
|
WhichModel::FunASRNano2512 => "FunAudioLLM/Fun-ASR-Nano-2512",
|
||||||
};
|
}
|
||||||
let model_path = match &args.weight_path {
|
}
|
||||||
Some(path) => path.clone(),
|
|
||||||
|
/// Run the 'cli' subcommand: download model (if needed) and start service
|
||||||
|
async fn run_cli(args: CliArgs) -> anyhow::Result<()> {
|
||||||
|
let CliArgs { common, weight_path, save_dir, download_retries } = args;
|
||||||
|
let model_id = get_model_id(common.model);
|
||||||
|
|
||||||
|
let model_path = match weight_path {
|
||||||
|
Some(path) => path,
|
||||||
None => {
|
None => {
|
||||||
let save_dir = match &args.save_dir {
|
let save_dir = match save_dir {
|
||||||
Some(dir) => dir.clone(),
|
Some(dir) => dir,
|
||||||
None => get_default_save_dir().expect("Failed to get home directory"),
|
None => get_default_save_dir().expect("Failed to get home directory"),
|
||||||
};
|
};
|
||||||
let max_retries = args.download_retries.unwrap_or(3);
|
let max_retries = download_retries.unwrap_or(3);
|
||||||
download_model(model_id, &save_dir, max_retries).await?;
|
download_model(model_id, &save_dir, max_retries).await?;
|
||||||
save_dir + "/" + model_id
|
save_dir + "/" + model_id
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
// println!("-------------------download path: {}", model_path);
|
|
||||||
init(args.model, model_path)?;
|
init(common.model, model_path)?;
|
||||||
start_http_server(&args).await?;
|
start_http_server(common.address, common.port).await?;
|
||||||
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
pub(crate) async fn start_http_server(args: &Args) -> anyhow::Result<()> {
|
/// Run the 'serv' subcommand: start service only (no download)
|
||||||
|
async fn run_serv(args: ServArgs) -> anyhow::Result<()> {
|
||||||
|
let ServArgs { common, weight_path } = args;
|
||||||
|
|
||||||
|
init(common.model, weight_path)?;
|
||||||
|
start_http_server(common.address, common.port).await?;
|
||||||
|
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Run the 'download' subcommand: download model only (no server)
|
||||||
|
async fn run_download(args: DownloadArgs) -> anyhow::Result<()> {
|
||||||
|
let DownloadArgs { model, save_dir, download_retries } = args;
|
||||||
|
let model_id = get_model_id(model);
|
||||||
|
|
||||||
|
let save_dir = match save_dir {
|
||||||
|
Some(dir) => dir,
|
||||||
|
None => get_default_save_dir().expect("Failed to get home directory"),
|
||||||
|
};
|
||||||
|
let max_retries = download_retries.unwrap_or(3);
|
||||||
|
|
||||||
|
download_model(model_id, &save_dir, max_retries).await?;
|
||||||
|
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::main]
|
||||||
|
async fn main() -> anyhow::Result<()> {
|
||||||
|
let cli = Cli::parse();
|
||||||
|
|
||||||
|
match cli.command {
|
||||||
|
Some(Commands::Cli(args)) => run_cli(args).await,
|
||||||
|
Some(Commands::Serv(args)) => run_serv(args).await,
|
||||||
|
Some(Commands::Download(args)) => run_download(args).await,
|
||||||
|
None => {
|
||||||
|
// Backward compatibility: when no subcommand is provided, use 'cli' behavior
|
||||||
|
let model = cli.model.expect("Model is required (use -m or --model)");
|
||||||
|
let args = CliArgs {
|
||||||
|
common: CommonArgs {
|
||||||
|
address: cli.address.unwrap_or_else(|| "127.0.0.1".to_string()),
|
||||||
|
port: cli.port.unwrap_or(10100),
|
||||||
|
model,
|
||||||
|
},
|
||||||
|
weight_path: cli.weight_path,
|
||||||
|
save_dir: cli.save_dir,
|
||||||
|
download_retries: cli.download_retries,
|
||||||
|
};
|
||||||
|
run_cli(args).await
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) async fn start_http_server(address: String, port: u16) -> anyhow::Result<()> {
|
||||||
let mut builder = rocket::build().configure(Config {
|
let mut builder = rocket::build().configure(Config {
|
||||||
address: IpAddr::from_str(&args.address)?,
|
address: IpAddr::from_str(&address)?,
|
||||||
port: args.port,
|
port,
|
||||||
limits: Limits::default()
|
limits: Limits::default()
|
||||||
.limit("string", ByteUnit::Mebibyte(5))
|
.limit("string", ByteUnit::Mebibyte(5))
|
||||||
.limit("json", ByteUnit::Mebibyte(5))
|
.limit("json", ByteUnit::Mebibyte(5))
|
||||||
@@ -128,7 +269,3 @@ pub(crate) async fn start_http_server(args: &Args) -> anyhow::Result<()> {
|
|||||||
builder.launch().await?;
|
builder.launch().await?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
// fn main() {
|
|
||||||
// println!("Hello, world!");
|
|
||||||
// }
|
|
||||||
|
|||||||
Reference in New Issue
Block a user