tckm.yan/aha
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add PaddleOCR-VL1.5 support

Browse Source
This commit is contained in:
jhqxxx
2026-03-17 21:30:50 +08:00
parent fb4a50f745
commit 55a5007c07
6 changed files with 191 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/model-card.md
+63
View File
@@ -0,0 +1,63 @@
## PaddleOCR-VL
#### Options prompt:
"OCR:" | "Table Recognition:" | "Formula Recognition:" | "Chart Recognition:"
## PaddleOCR-VL1.5
#### Options prompt:
"OCR:" | "Table Recognition:" | "Formula Recognition:" | "Chart Recognition:" | "Spotting:" | "Seal Recognition:"
## DeepSeekOCR
#### Metadata
* base_size: 512 | 640 | 1024 | 1280
* image_size: 512 | 640 | 1024 | 1280
* crop_mode: false | true
##### Example:
```json
{"base_size": "640", "image_size": "640", "crop_mode": "false"}
```
#### Processing Types
| Type | base_size | image_size | crop_mode |
|------|-----------|------------|-----------|
| Tiny | 512 | 512 | false |
| Small | 640 | 640 | false |
| Base | 1024 | 1024 | false |
| Large | 1280 | 1280 | false |
| Gundam | 1024 | 640 | true |
## GLM-OCR
#### Prompt Limited
##### Document Parsing prompt：
"Text Recognition:" | "Formula Recognition:" | "Table Recognition:"
###### Information Extraction:
extract structured information from documents. Prompts must follow a strict JSON schema, For example, to extract personal ID information:
```json
Please output the information in the image in the following JSON format:
{
"id_number": "",
"last_name": "",
"first_name": "",
"date_of_birth": "",
"address": {
"street": "",
"city": "",
"state": "",
"zip_code": ""
},
"dates": {
"issue_date": "",
"expiration_date": ""
},
"sex": ""
}
```
## HunyuanOCR
Application-oriented Prompts
| Task | English | Chinese |
|------|---------|---------|
| Spotting | Detect and recognize text in the image, and output the text coordinates in a formatted manner. | 检测并识别图片中的文字，将文本坐标格式化输出。|
| Parsing | 1. Identify the formula in the image and represent it using LaTeX format. <br> 2.Parse the table in the image into HTML. <br> 3. Parse the chart in the image; use Mermaid format for flowcharts and Markdown for other charts. <br> 4.Extract all information from the main body of the document image and represent it in markdown format, ignoring headers and footers. Tables should be expressed in HTML format, formulas in the document should be represented using LaTeX format, and the parsing should be organized according to the reading order. | 1. 识别图片中的公式，用 LaTeX 格式表示。 <br> 2. 把图中的表格解析为 HTML。 <br> 3. 解析图中的图表，对于流程图使用 Mermaid 格式表示，其他图表使用 Markdown 格式表示。 <br> 4. 提取文档图片中正文的所有信息用 markdown 格式表示，其中页眉、页脚部分忽略，表格用 html 格式表达，文档中公式用 latex 格式表示，按照阅读顺序组织进行解析。 |
| Information Extraction | 1. Output the value of Key. <br> 2. Extract the content of the fields: ['key1','key2', ...] from the image and return it in JSON format. <br> 3. Extract the subtitles from the image. | 1. 输出 Key 的值。 <br> 2. 提取图片中的: ['key1','key2', ...] 的字段内容，并按照 JSON 格式返回。 <br> 3. 提取图片中的字幕。 |
| Translation | First extract the text, then translate the text content into English. If it is a document, ignore the header and footer. Formulas should be represented in LaTeX format, and tables should be represented in HTML format. | 先提取文字，再将文字内容翻译为英文。若是文档，则其中页眉、页脚忽略。公式用latex格式表示，表格用html格式表示。 |
docs/model-card.zh-CN.md
+63
View File
@@ -0,0 +1,63 @@
## PaddleOCR-VL
#### 可选提示词:
"OCR:" | "Table Recognition:" | "Formula Recognition:" | "Chart Recognition:"
## PaddleOCR-VL1.5
#### 可选提示词:
"OCR:" | "Table Recognition:" | "Formula Recognition:" | "Chart Recognition:" | "Spotting:" | "Seal Recognition:"
## DeepSeekOCR
#### Metadata
* base_size: 512 | 640 | 1024 | 1280
* image_size: 512 | 640 | 1024 | 1280
* crop_mode: false | true
##### 示例:
```json
{"base_size": "640", "image_size": "640", "crop_mode": "false"}
```
#### 处理模式
| Type | base_size | image_size | crop_mode |
|------|-----------|------------|-----------|
| Tiny | 512 | 512 | false |
| Small | 640 | 640 | false |
| Base | 1024 | 1024 | false |
| Large | 1280 | 1280 | false |
| Gundam | 1024 | 640 | true |
## GLM-OCR
#### 限定提示词
##### 文档解析：
"Text Recognition:" | "Formula Recognition:" | "Table Recognition:"
###### 信息提取:
extract structured information from documents. Prompts must follow a strict JSON schema, For example, to extract personal ID information:
```json
请以以下 JSON 格式输出图像中的信息:
{
"id_number": "",
"last_name": "",
"first_name": "",
"date_of_birth": "",
"address": {
"street": "",
"city": "",
"state": "",
"zip_code": ""
},
"dates": {
"issue_date": "",
"expiration_date": ""
},
"sex": ""
}
```
## HunyuanOCR
场景化提示词
| 任务 | 英文 | 中文 |
|------|---------|---------|
| 检测 | Detect and recognize text in the image, and output the text coordinates in a formatted manner. | 检测并识别图片中的文字，将文本坐标格式化输出。|
| 解析 | 1. Identify the formula in the image and represent it using LaTeX format. <br> 2.Parse the table in the image into HTML. <br> 3. Parse the chart in the image; use Mermaid format for flowcharts and Markdown for other charts. <br> 4.Extract all information from the main body of the document image and represent it in markdown format, ignoring headers and footers. Tables should be expressed in HTML format, formulas in the document should be represented using LaTeX format, and the parsing should be organized according to the reading order. | 1. 识别图片中的公式，用 LaTeX 格式表示。 <br> 2. 把图中的表格解析为 HTML。 <br> 3. 解析图中的图表，对于流程图使用 Mermaid 格式表示，其他图表使用 Markdown 格式表示。 <br> 4. 提取文档图片中正文的所有信息用 markdown 格式表示，其中页眉、页脚部分忽略，表格用 html 格式表达，文档中公式用 latex 格式表示，按照阅读顺序组织进行解析。 |
| 信息提取 | 1. Output the value of Key. <br> 2. Extract the content of the fields: ['key1','key2', ...] from the image and return it in JSON format. <br> 3. Extract the subtitles from the image. | 1. 输出 Key 的值。 <br> 2. 提取图片中的: ['key1','key2', ...] 的字段内容，并按照 JSON 格式返回。 <br> 3. 提取图片中的字幕。 |
| 翻译 | First extract the text, then translate the text content into English. If it is a document, ignore the header and footer. Formulas should be represented in LaTeX format, and tables should be represented in HTML format. | 先提取文字，再将文字内容翻译为英文。若是文档，则其中页眉、页脚忽略。公式用latex格式表示，表格用html格式表示。 |
src/api/mod.rs
+2 -1
View File
@@ -264,6 +264,7 @@ fn which_model_to_id(which_model: WhichModel) -> &'static str {
WhichModel::DeepSeekOCR => "deepseek-ocr",
WhichModel::HunyuanOCR => "hunyuan-ocr",
WhichModel::PaddleOCRVL => "paddleocr-vl",
WhichModel::PaddleOCRVL1_5 => "paddleocr-vl1.5",
WhichModel::RMBG2_0 => "rmbg2.0",
WhichModel::VoxCPM => "voxcpm",
WhichModel::VoxCPM1_5 => "voxcpm1.5",
@@ -290,7 +291,7 @@ fn which_model_to_owner(which_model: WhichModel) -> &'static str {
| WhichModel::Qwen3_5_9B => "Qwen",
WhichModel::DeepSeekOCR => "deepseek-ai",
WhichModel::HunyuanOCR => "Tencent-Hunyuan",
WhichModel::PaddleOCRVL => "PaddlePaddle",
WhichModel::PaddleOCRVL | WhichModel::PaddleOCRVL1_5 => "PaddlePaddle",
WhichModel::RMBG2_0 => "AI-ModelScope",
WhichModel::VoxCPM | WhichModel::VoxCPM1_5 => "OpenBMB",
WhichModel::GlmASRNano2512 | WhichModel::GlmOCR => "ZhipuAI",
src/main.rs
+5
View File
@@ -260,6 +260,7 @@ fn run_list(args: ListArgs) -> anyhow::Result<()> {
WhichModel::DeepSeekOCR,
WhichModel::HunyuanOCR,
WhichModel::PaddleOCRVL,
WhichModel::PaddleOCRVL1_5,
WhichModel::RMBG2_0,
WhichModel::VoxCPM,
WhichModel::VoxCPM1_5,
@@ -527,6 +528,10 @@ fn run_run(args: RunArgs) -> anyhow::Result<()> {
use aha::exec::paddleocr_vl::PaddleOVLExec;
PaddleOVLExec::run(&input, output.as_deref(), &weight_path)?;
}
WhichModel::PaddleOCRVL1_5 => {
use aha::exec::paddleocr_vl::PaddleOVLExec;
PaddleOVLExec::run(&input, output.as_deref(), &weight_path)?;
}
WhichModel::RMBG2_0 => {
use aha::exec::rmbg2_0::RMBG2_0Exec;
RMBG2_0Exec::run(&input, output.as_deref(), &weight_path)?;
src/models/mod.rs
+9 -1
View File
@@ -74,6 +74,8 @@ pub enum WhichModel {
HunyuanOCR,
#[value(name = "paddleocr-vl", hide = true)]
PaddleOCRVL,
#[value(name = "paddleocr-vl1.5", hide = true)]
PaddleOCRVL1_5,
#[value(name = "rmbg2.0")]
RMBG2_0,
#[value(name = "voxcpm", hide = true)]
@@ -110,6 +112,7 @@ impl WhichModel {
WhichModel::DeepSeekOCR => "deepseek-ai/DeepSeek-OCR",
WhichModel::HunyuanOCR => "Tencent-Hunyuan/HunyuanOCR",
WhichModel::PaddleOCRVL => "PaddlePaddle/PaddleOCR-VL",
WhichModel::PaddleOCRVL1_5 => "PaddlePaddle/PaddleOCR-VL-1.5",
WhichModel::RMBG2_0 => "AI-ModelScope/RMBG-2.0",
WhichModel::VoxCPM => "OpenBMB/VoxCPM-0.5B",
WhichModel::VoxCPM1_5 => "OpenBMB/VoxCPM1.5",
@@ -139,7 +142,8 @@ impl WhichModel {
WhichModel::DeepSeekOCR
| WhichModel::HunyuanOCR
| WhichModel::GlmOCR
| WhichModel::PaddleOCRVL => "ocr",
| WhichModel::PaddleOCRVL
| WhichModel::PaddleOCRVL1_5 => "ocr",
// ASR models
WhichModel::Qwen3ASR0_6B
| WhichModel::Qwen3ASR1_7B
@@ -316,6 +320,10 @@ pub fn load_model<'a>(
let model = PaddleOCRVLGenerateModel::init(path, None, None)?;
ModelInstance::PaddleOCRVL(Box::new(model))
}
WhichModel::PaddleOCRVL1_5 => {
let model = PaddleOCRVLGenerateModel::init(path, None, None)?;
ModelInstance::PaddleOCRVL(Box::new(model))
}
WhichModel::RMBG2_0 => {
let model = RMBG2_0Model::init(path, None, None)?;
ModelInstance::RMBG2_0(Box::new(model))
tests/test_paddleocr_vl.rs
+49
View File
@@ -5,6 +5,55 @@ use aha_openai_dive::v1::resources::chat::ChatCompletionParameters;
use anyhow::Result;
use rocket::futures::StreamExt;
#[test]
fn paddleocr_vl_1_5_generate() -> Result<()> {
// RUST_BACKTRACE=1 cargo test -F cuda --test test_paddleocr_vl paddleocr_vl_1_5_generate -r -- --nocapture
let message = r#"
{
"model": "paddleocr_vl1.5",
"messages": [
{
"role": "user",
"content": [
{
"type": "image",
"image_url":
{
"url": "file://./assets/img/ocr_test1.png"
}
},
{
"type": "text",
"text": "Spotting:"
}
]
}
],
"stream": false
}
"#;
let save_dir =
aha::utils::get_default_save_dir().ok_or(anyhow::anyhow!("Failed to get save dir"))?;
let model_path = format!("{}/PaddlePaddle/PaddleOCR-VL-1.5/", save_dir);
let mes: ChatCompletionParameters = serde_json::from_str(message)?;
let i_start = Instant::now();
let mut model = PaddleOCRVLGenerateModel::init(&model_path, None, None)?;
let i_duration = i_start.elapsed();
println!("Time elapsed in load model is: {:?}", i_duration);
let i_start = Instant::now();
let res = model.generate(mes)?;
let i_duration = i_start.elapsed();
println!("generate: \n {:?}", res);
if let Some(usage) = &res.usage {
let num_token = usage.total_tokens;
let duration_secs = i_duration.as_secs_f64();
let tps = num_token as f64 / duration_secs;
println!("Tokens per second (TPS): {:.2}", tps);
}
println!("Time elapsed in generate is: {:?}", i_duration);
Ok(())
}
#[test]
fn paddleocr_vl_generate() -> Result<()> {
// RUST_BACKTRACE=1 cargo test -F cuda --test test_paddleocr_vl paddleocr_vl_generate -r -- --nocapture