Skip to content

Add oi_parser parse provider and pipeline - #81

Open
thiagosantos1 wants to merge 1 commit into
run-llama:mainfrom
thiagosantos1:feature/oi-parser-provider
Open

Add oi_parser parse provider and pipeline#81
thiagosantos1 wants to merge 1 commit into
run-llama:mainfrom
thiagosantos1:feature/oi-parser-provider

Conversation

@thiagosantos1

@thiagosantos1 thiagosantos1 commented Jul 16, 2026

Copy link
Copy Markdown

This resubmits #77, which was closed because oi-parser API keys required manual admin approval (not self-serve). That's now fixed — details below. Opening a new PR since closed PRs can't be reopened.


Adds an oi_parser parse provider + pipeline for the hosted oi-parser document-parsing API (POST /v1/extract, X-API-Key auth, returns markdown + layout). Also includes a layout cross-eval adapter + label mapper so the pipeline can be scored on the layout-detection dataset.

Getting an API key: sign up at https://oi-parser.ai/ — keys are now issued instantly and active on signup, with no admin review or "Pending approval" step. One active key per user, valid 14 days, and a new key can be self-issued anytime — also with no approval. Set it via OI_PARSER_API_KEY (optionally override the endpoint with OI_PARSER_BASE_URL).

Recommended run flags:

  • --max_concurrent 20 — the per-API-key concurrency cap on the default key; exceeding it only wastes retries. Raise it if your key is provisioned with a higher limit.
  • --per_file_timeout 900 — the service allows up to ~900s per document; the default 600s can cut off the slowest documents.

Example:

parse-bench inference run oi_parser --input_dir <dir> --max_concurrent 20 --per_file_timeout 900
parse-bench evaluation run <output>

@thiagosantos1
thiagosantos1 force-pushed the feature/oi-parser-provider branch from 427ce0f to 8365d0e Compare July 20, 2026 06:16
@thiagosantos1

Copy link
Copy Markdown
Author

@boyang-zhang1 Please let me know if there's anything missing from our side to move the PR forward

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant