feat(pdf): refactor MinerU parsing to the official file_parse API - #3953
Open
zonas0574 wants to merge 1 commit into
Open
feat(pdf): refactor MinerU parsing to the official file_parse API#3953zonas0574 wants to merge 1 commit into
zonas0574 wants to merge 1 commit into
Conversation
Collaborator
|
thanks, our team will review this |
Tsan1024
reviewed
Aug 13, 2026
| "pdf": { | ||
| "strategy": "auto", | ||
| "mineru_endpoint": "http://127.0.0.1:8000", | ||
| "mineru_api_key": "your-api-key", |
Contributor
There was a problem hiding this comment.
这里方便提供下使用的是哪个版本的开源镜像服务吗?MinerU 3.4.4 提到的鉴权是MinerU -> vLLM 的下游鉴权,这里的api-key 是vllm部分的输入?
Author
There was a problem hiding this comment.
对是3.4.4,但这个其实目前没有用,我看之前在我也没删就保留,要不我先删了 = =
Contributor
There was a problem hiding this comment.
对是3.4.4,但这个其实目前没有用,我看之前在我也没删就保留,要不我先删了 = =
之前的应该是有开发者支持了一版调用mineru官方api的方式,里面会有apikey存在。可以看看如何兼容官方api和自运营api方式哈,比如加个字段区分?
Contributor
There was a problem hiding this comment.
当 PDF 解析策略为 mineru 时,MinerU 是必需依赖;当策略为 auto 且配置了
mineru_endpoint 时,它也承担本地解析失败后的回退职责。目前在实际解析 PDF 时才
调用 /file_parse,因此 endpoint 配置错误、服务未启动或协议不兼容可能会延后到运
行期才发现。
这里是否可以考虑在 OpenViking 初始化完成前增加一个 MinerU 启动预检?例如:
- 对 {mineru_endpoint}/health 做一次短时轮询。
- 收到 HTTP 200、合法 JSON 且包含 protocol_version 时视为就绪。
- strategy="mineru",或 strategy="auto" 且配置了 mineru_endpoint 时执行。
- strategy="local",以及未配置 endpoint 的 auto 模式可跳过。
- 若预检超时,启动时给出包含 endpoint、最后错误和修复建议的提示。
这样可以更早发现依赖配置问题,也避免首个 PDF 导入任务才暴露错误
示例
import asyncio
import time
import httpx
async def wait_for_mineru_ready(endpoint: str, timeout: float = 5.0) -> None:
health_url = f"{endpoint.rstrip('/')}/health"
deadline = time.monotonic() + timeout
last_error = "no response"
async with httpx.AsyncClient(timeout=0.5) as client:
while time.monotonic() < deadline:
try:
response = await client.get(health_url)
payload = response.json()
if (
response.status_code == 200
and isinstance(payload, dict)
and payload.get("protocol_version")
):
return
last_error = (
f"unexpected response: HTTP {response.status_code}, "
f"protocol_version={payload.get('protocol_version')!r}"
)
except (httpx.HTTPError, ValueError) as exc:
last_error = str(exc)
await asyncio.sleep(0.2)
raise RuntimeError(
f"MinerU startup preflight failed for {health_url} after {timeout}s. "
f"Last error: {last_error}. "
"Start a compatible mineru-api service or correct
pdf.mineru_endpoint."
)
在服务初始化流程中、设置 _initialized = True 前调用:
pdf_config = self._config.pdf
should_preflight_mineru = (
pdf_config.strategy == "mineru"
or (
pdf_config.strategy == "auto"
and pdf_config.mineru_endpoint is not None
)
)
if should_preflight_mineru and pdf_config.mineru_endpoint:
await wait_for_mineru_ready(pdf_config.mineru_endpoint)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
支持最新本地mineru部署的解析API
Human Involvement
Related Issue
#3910
Type of Change
Changes Made
Testing
Checklist
Screenshots (if applicable)
Additional Notes
不支持官方云平台https://mineru.net/apiManage/docs (只有异步接口),仅支持开源本地部署版本