diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..8e4ba4f --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,109 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added + +- **CLI Subcommand Support**: Added three new subcommands for better command organization: + - `aha cli` - Download model and start HTTP service (default, backward compatible) + - `aha serv` - Start HTTP service only (requires `--weight-path`) + - `aha download` - Download model only (no service start) + +### Changed + +- **CLI Structure**: Refactored CLI to use clap's Subcommand feature while maintaining backward compatibility +- **Backward Compatibility**: Commands without subcommand now default to `cli` subcommand: + - `aha -m qwen3vl-2b` is equivalent to `aha cli -m qwen3vl-2b` + - All existing parameter options and defaults remain unchanged + +### Technical Details + +**Subcommand Parameters:** + +`aha cli`: +- `-a, --address
` - Server address (default: 127.0.0.1) +- `-p, --port ` - Server port (default: 10100) +- `-m, --model ` - Model to use (required) +- `--weight-path ` - Local model weight path (optional) +- `--save-dir ` - Directory to save downloaded model (optional) +- `--download-retries ` - Download retry attempts (default: 3) + +`aha serv`: +- `-a, --address
` - Server address (default: 127.0.0.1) +- `-p, --port ` - Server port (default: 10100) +- `-m, --model ` - Model to use (required) +- `--weight-path ` - Local model weight path (required) + +`aha download`: +- `-m, --model ` - Model to download (required) +- `-s, --save-dir ` - Directory to save downloaded model (optional) +- `--download-retries ` - Download retry attempts (default: 3) + +**Code Changes:** +- Modified `src/main.rs` only +- Extracted common functions: `get_model_id()`, `start_http_server()` +- Reused existing `download_model()` and `init()` functions +- No changes to other modules or dependencies + +## [0.1.8] - 2025-01-20 + +### Added + +- Support for Fun-ASR-Nano-2512 model +- Support for Qwen3-0.6B model + +## [0.1.7] - 2024-XX-XX + +### Added + +- Support for GLM-ASR-Nano-2512 model + +## [0.1.6] - 2024-XX-XX + +### Added + +- Support for RMBG-2.0 model (background removal) + +## [0.1.5] - 2024-XX-XX + +### Added + +- Support for VoxCPM1.5 model + +## [0.1.4] - 2024-XX-XX + +### Added + +- Support for PaddleOCR-VL model + +## [0.1.3] - 2024-XX-XX + +### Added + +- Support for Hunyuan-OCR model + +## [0.1.2] - 2024-XX-XX + +### Added + +- Support for DeepSeek-OCR model + +## [0.1.1] - 2024-XX-XX + +### Added + +- Support for Qwen3VL model family (2B, 4B, 8B, 32B) + +## [0.1.0] - 2024-XX-XX + +### Added + +- Initial release +- Support for Qwen2.5VL models (3B, 7B) +- Support for MiniCPM4-0.5B model +- Support for VoxCPM-0.5B model diff --git a/docs/CLI_USAGE.md b/docs/CLI_USAGE.md new file mode 100644 index 0000000..85e54d8 --- /dev/null +++ b/docs/CLI_USAGE.md @@ -0,0 +1,242 @@ +# AHA 命令行使用说明 + +## 概述 + +AHA 是一个基于 Candle 框架的高性能模型推理库，支持多种多模态模型，包括视觉、语言和语音模型。 + +```bash +aha [COMMAND] [OPTIONS] +``` + +## 全局选项 + +| 选项 | 说明 | 默认值 | +|------|------|--------| +| `-a, --address
` | 服务监听地址 | 127.0.0.1 | +| `-p, --port ` | 服务监听端口 | 10100 | +| `-m, --model ` | 模型类型（必选） | - | +| `--weight-path ` | 本地模型权重路径 | - | +| `--save-dir ` | 模型下载保存目录 | ~/.aha/ | +| `--download-retries ` | 下载重试次数 | 3 | +| `-h, --help` | 显示帮助信息 | - | +| `-V, --version` | 显示版本号 | - | + +## 子命令 + +### cli - 下载模型并启动服务（默认） + +下载指定的模型并启动 HTTP 服务。当不指定子命令时，默认使用此命令。 + +**语法：** +```bash +aha cli [OPTIONS] --model +``` + +**选项：** + +| 选项 | 说明 | 默认值 | +|------|------|--------| +| `-a, --address
` | 服务监听地址 | 127.0.0.1 | +| `-p, --port ` | 服务监听端口 | 10100 | +| `-m, --model ` | 模型类型（必选） | - | +| `--weight-path ` | 本地模型权重路径（如指定则跳过下载） | - | +| `--save-dir ` | 模型下载保存目录 | ~/.aha/ | +| `--download-retries ` | 下载重试次数 | 3 | + +**示例：** + +```bash +# 下载模型并启动服务（默认端口 10100） +aha cli -m qwen3vl-2b + +# 指定端口和保存目录 +aha cli -m qwen3vl-2b -p 8080 --save-dir /data/models + +# 使用本地模型（不下载） +aha cli -m qwen3vl-2b --weight-path /path/to/model + +# 向后兼容方式（等同于 cli 子命令） +aha -m qwen3vl-2b +``` + +### serv - 启动服务 + +仅启动 HTTP 服务，不下载模型。必须通过 `--weight-path` 指定本地模型路径。 + +**语法：** +```bash +aha serv [OPTIONS] --model --weight-path +``` + +**选项：** + +| 选项 | 说明 | 默认值 | +|------|------|--------| +| `-a, --address
` | 服务监听地址 | 127.0.0.1 | +| `-p, --port ` | 服务监听端口 | 10100 | +| `-m, --model ` | 模型类型（必选） | - | +| `--weight-path ` | 本地模型权重路径（必选） | - | + +**示例：** + +```bash +# 使用本地模型启动服务 +aha serv -m qwen3vl-2b --weight-path /path/to/model + +# 指定端口启动 +aha serv -m qwen3vl-2b --weight-path /path/to/model -p 8080 + +# 指定监听地址 +aha serv -m qwen3vl-2b --weight-path /path/to/model -a 0.0.0.0 +``` + +### download - 下载模型 + +仅下载指定模型，不启动服务。 + +**语法：** +```bash +aha download [OPTIONS] --model +``` + +**选项：** + +| 选项 | 说明 | 默认值 | +|------|------|--------| +| `-m, --model ` | 模型类型（必选） | - | +| `-s, --save-dir ` | 模型下载保存目录 | ~/.aha/ | +| `--download-retries ` | 下载重试次数 | 3 | + +**示例：** + +```bash +# 下载模型到默认目录 +aha download -m qwen3vl-2b + +# 指定保存目录 +aha download -m qwen3vl-2b -s /data/models + +# 指定下载重试次数 +aha download -m qwen3vl-2b --download-retries 5 + +# 下载 MiniCPM4-0.5B 模型 +aha download -m minicpm4-0.5b -s models +``` + +## 支持的模型 + +| 模型标识 | 模型名称 | 说明 | +|---------|---------|------| +| `minicpm4-0.5b` | OpenBMB/MiniCPM4-0.5B | 面壁智能 MiniCPM4 0.5B 模型 | +| `qwen2.5vl-3b` | Qwen/Qwen2.5-VL-3B-Instruct | 通义千问 2.5 VL 3B 模型 | +| `qwen2.5vl-7b` | Qwen/Qwen2.5-VL-7B-Instruct | 通义千问 2.5 VL 7B 模型 | +| `qwen3-0.6b` | Qwen/Qwen3-0.6B | 通义千问 3 0.6B 模型 | +| `qwen3vl-2b` | Qwen/Qwen3-VL-2B-Instruct | 通义千问 3 VL 2B 模型 | +| `qwen3vl-4b` | Qwen/Qwen3-VL-4B-Instruct | 通义千问 3 VL 4B 模型 | +| `qwen3vl-8b` | Qwen/Qwen3-VL-8B-Instruct | 通义千问 3 VL 8B 模型 | +| `qwen3vl-32b` | Qwen/Qwen3-VL-32B-Instruct | 通义千问 3 VL 32B 模型 | +| `deepseek-ocr` | deepseek-ai/DeepSeek-OCR | DeepSeek OCR 模型 | +| `hunyuan-ocr` | Tencent-Hunyuan/HunyuanOCR | 腾讯混元 OCR 模型 | +| `paddleocr-vl` | PaddlePaddle/PaddleOCR-VL | 百度飞桨 OCR VL 模型 | +| `RMBG2.0` | AI-ModelScope/RMBG-2.0 | RMBG 2.0 背景移除模型 | +| `voxcpm` | OpenBMB/VoxCPM-0.5B | 面壁智能 VoxCPM 0.5B 语音生成模型 | +| `voxcpm1.5` | OpenBMB/VoxCPM1.5 | 面壁智能 VoxCPM 1.5 语音生成模型 | +| `glm-asr-nano-2512` | ZhipuAI/GLM-ASR-Nano-2512 | 智谱 AI ASR Nano 2512 语音识别模型 | +| `fun-asr-nano-2512` | FunAudioLLM/Fun-ASR-Nano-2512 | 通义百聆 ASR Nano 2512 语音识别模型 | + +## 常见使用场景 + +### 场景 1：快速启动推理服务 + +```bash +# 一条命令下载并启动服务 +aha -m qwen3vl-2b +``` + +### 场景 2：使用已有模型启动服务 + +```bash +# 假设模型已下载到 /data/models/Qwen/Qwen3-VL-2B-Instruct +aha serv -m qwen3vl-2b --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct +``` + +### 场景 3：预先下载模型 + +```bash +# 下载模型到指定目录，稍后使用 +aha download -m qwen3vl-2b -s /data/models + +# 后续启动时直接使用 +aha serv -m qwen3vl-2b --weight-path /data/models/Qwen/Qwen3-VL-2B-Instruct +``` + +### 场景 4：自定义服务端口和地址 + +```bash +# 在 0.0.0.0:8080 启动服务，允许外部访问 +aha -m qwen3vl-2b -a 0.0.0.0 -p 8080 +``` + +## API 接口 + +服务启动后，提供以下 API 接口： + +### 对话接口 +- **端点**: `POST /chat/completions` +- **功能**: 多模态对话和文本生成 +- **支持模型**: Qwen2.5VL, Qwen3, Qwen3VL, DeepSeekOCR, GLM-ASR-Nano-2512, Fun-ASR-Nano-2512 等 +- **格式**: OpenAI Chat Completion 格式 +- **流式支持**: 支持 + +### 图像处理接口 +- **端点**: `POST /images/remove_background` +- **功能**: 图像背景移除 +- **支持模型**: RMBG-2.0 +- **格式**: OpenAI Chat Completion 格式 +- **流式支持**: 不支持 + +### 语音生成接口 +- **端点**: `POST /audio/speech` +- **功能**: 语音合成和生成 +- **支持模型**: VoxCPM, VoxCPM1.5 +- **格式**: OpenAI Chat Completion 格式 +- **流式支持**: 不支持 + +## 向后兼容性 + +为了保持与旧版本的兼容性，以下两种使用方式是等效的： + +```bash +# 新方式（推荐） +aha cli -m qwen3vl-2b + +# 旧方式（向后兼容） +aha -m qwen3vl-2b +``` + +## 注意事项 + +1. **serv 子命令必须指定 `--weight-path`**：由于 `serv` 子命令不下载模型，必须通过 `--weight-path` 指定已下载的模型路径。 + +2. **下载重试机制**：默认重试 3 次，每次失败后等待 2 秒再重试。可通过 `--download-retries` 调整重试次数。 + +3. **默认保存目录**：模型默认保存到 `~/.aha/` 目录下，可通过 `--save-dir` 或 `-d` 参数自定义。 + +4. **端口占用**：启动服务前确保指定的端口未被占用，默认端口为 10100。 + +5. **权限问题**：如果保存到系统目录（如 `/data/models`），确保有相应的写入权限。 + +## 获取帮助 + +```bash +# 查看主帮助 +aha --help + +# 查看子命令帮助 +aha cli --help +aha serv --help +aha download --help + +# 查看版本信息 +aha --version +``` diff --git a/src/main.rs b/src/main.rs index be1118e..43fc48a 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,7 +1,7 @@ use std::{net::IpAddr, str::FromStr, time::Duration}; use aha::{models::WhichModel, utils::get_default_save_dir}; -use clap::Parser; +use clap::{Args, Parser, Subcommand}; use modelscope::ModelScope; use rocket::{ Config, @@ -14,26 +14,109 @@ use crate::api::init; mod api; #[derive(Parser, Debug)] +#[command(name = "aha")] #[command(version, about, long_about = None)] -struct Args { +struct Cli { + /// Service listen address #[arg(short, long, default_value = "127.0.0.1")] - address: String, - - #[arg(short, long, default_value_t = 10100)] - port: u16, + address: Option, + /// Service listen port #[arg(short, long)] - model: WhichModel, + port: Option, + /// Model type (required for backward compatibility) + #[arg(short, long)] + model: Option, + + /// Local model weight path #[arg(long)] weight_path: Option, + /// Model download save directory #[arg(long)] save_dir: Option, + /// Download retry count + #[arg(long)] + download_retries: Option, + + #[command(subcommand)] + command: Option, +} + +#[derive(Subcommand, Debug)] +enum Commands { + /// Download model and start service (default) + Cli(CliArgs), + /// Start service only (requires --weight-path) + Serv(ServArgs), + /// Download model only + Download(DownloadArgs), +} + +/// Common/shared arguments for server operations +#[derive(Args, Debug)] +struct CommonArgs { + /// Service listen address + #[arg(short, long, default_value = "127.0.0.1")] + address: String, + + /// Service listen port + #[arg(short, long, default_value_t = 10100)] + port: u16, + + /// Model type (required) + #[arg(short, long)] + model: WhichModel, +} + +/// Arguments for the 'cli' subcommand (download + serve) +#[derive(Args, Debug)] +struct CliArgs { + #[command(flatten)] + common: CommonArgs, + + /// Local model weight path (skip download if provided) + #[arg(long)] + weight_path: Option, + + /// Model download save directory + #[arg(long)] + save_dir: Option, + + /// Download retry count #[arg(long)] download_retries: Option, } + +/// Arguments for the 'serv' subcommand (serve only) +#[derive(Args, Debug)] +struct ServArgs { + #[command(flatten)] + common: CommonArgs, + + /// Local model weight path (required) + #[arg(long, required = true)] + weight_path: String, +} + +/// Arguments for the 'download' subcommand (download only) +#[derive(Args, Debug)] +struct DownloadArgs { + /// Model type (required) + #[arg(short, long)] + model: WhichModel, + + /// Model download save directory + #[arg(short, long)] + save_dir: Option, + + /// Download retry count + #[arg(long)] + download_retries: Option, +} + async fn download_model(model_id: &str, save_dir: &str, max_retries: u32) -> anyhow::Result<()> { let mut attempts = 0u32; loop { @@ -67,10 +150,9 @@ async fn download_model(model_id: &str, save_dir: &str, max_retries: u32) -> any } } -#[tokio::main] -async fn main() -> anyhow::Result<()> { - let args = Args::parse(); - let model_id = match &args.model { +/// Get the ModelScope model ID for a given WhichModel variant +fn get_model_id(model: WhichModel) -> &'static str { + match model { WhichModel::MiniCPM4_0_5B => "OpenBMB/MiniCPM4-0.5B", WhichModel::Qwen2_5vl3B => "Qwen/Qwen2.5-VL-3B-Instruct", WhichModel::Qwen2_5vl7B => "Qwen/Qwen2.5-VL-7B-Instruct", @@ -87,30 +169,89 @@ async fn main() -> anyhow::Result<()> { WhichModel::VoxCPM1_5 => "OpenBMB/VoxCPM1.5", WhichModel::GlmASRNano2512 => "ZhipuAI/GLM-ASR-Nano-2512", WhichModel::FunASRNano2512 => "FunAudioLLM/Fun-ASR-Nano-2512", - }; - let model_path = match &args.weight_path { - Some(path) => path.clone(), + } +} + +/// Run the 'cli' subcommand: download model (if needed) and start service +async fn run_cli(args: CliArgs) -> anyhow::Result<()> { + let CliArgs { common, weight_path, save_dir, download_retries } = args; + let model_id = get_model_id(common.model); + + let model_path = match weight_path { + Some(path) => path, None => { - let save_dir = match &args.save_dir { - Some(dir) => dir.clone(), + let save_dir = match save_dir { + Some(dir) => dir, None => get_default_save_dir().expect("Failed to get home directory"), }; - let max_retries = args.download_retries.unwrap_or(3); + let max_retries = download_retries.unwrap_or(3); download_model(model_id, &save_dir, max_retries).await?; save_dir + "/" + model_id } }; - // println!("-------------------download path: {}", model_path); - init(args.model, model_path)?; - start_http_server(&args).await?; + + init(common.model, model_path)?; + start_http_server(common.address, common.port).await?; Ok(()) } -pub(crate) async fn start_http_server(args: &Args) -> anyhow::Result<()> { +/// Run the 'serv' subcommand: start service only (no download) +async fn run_serv(args: ServArgs) -> anyhow::Result<()> { + let ServArgs { common, weight_path } = args; + + init(common.model, weight_path)?; + start_http_server(common.address, common.port).await?; + + Ok(()) +} + +/// Run the 'download' subcommand: download model only (no server) +async fn run_download(args: DownloadArgs) -> anyhow::Result<()> { + let DownloadArgs { model, save_dir, download_retries } = args; + let model_id = get_model_id(model); + + let save_dir = match save_dir { + Some(dir) => dir, + None => get_default_save_dir().expect("Failed to get home directory"), + }; + let max_retries = download_retries.unwrap_or(3); + + download_model(model_id, &save_dir, max_retries).await?; + + Ok(()) +} + +#[tokio::main] +async fn main() -> anyhow::Result<()> { + let cli = Cli::parse(); + + match cli.command { + Some(Commands::Cli(args)) => run_cli(args).await, + Some(Commands::Serv(args)) => run_serv(args).await, + Some(Commands::Download(args)) => run_download(args).await, + None => { + // Backward compatibility: when no subcommand is provided, use 'cli' behavior + let model = cli.model.expect("Model is required (use -m or --model)"); + let args = CliArgs { + common: CommonArgs { + address: cli.address.unwrap_or_else(|| "127.0.0.1".to_string()), + port: cli.port.unwrap_or(10100), + model, + }, + weight_path: cli.weight_path, + save_dir: cli.save_dir, + download_retries: cli.download_retries, + }; + run_cli(args).await + } + } +} + +pub(crate) async fn start_http_server(address: String, port: u16) -> anyhow::Result<()> { let mut builder = rocket::build().configure(Config { - address: IpAddr::from_str(&args.address)?, - port: args.port, + address: IpAddr::from_str(&address)?, + port, limits: Limits::default() .limit("string", ByteUnit::Mebibyte(5)) .limit("json", ByteUnit::Mebibyte(5)) @@ -127,8 +268,4 @@ pub(crate) async fn start_http_server(args: &Args) -> anyhow::Result<()> { builder.launch().await?; Ok(()) -} - -// fn main() { -// println!("Hello, world!"); -// } +} \ No newline at end of file