Convert a PDF to clean Markdown, with formulas as LaTeX and proper Markdown tables. Built on OpenDataLoader hybrid mode (a fast local parser + a local AI backend), with an optional second pass where an AI assistant (Claude / Codex) reviews and fixes the formulas and tables.
中文说明见 README.zh-CN.md
Plain PDF-to-text tools mangle display equations into unreadable strings
(α = m 0 mt mf (1)) and flatten tables. PDF2MD:
- keeps body text from the embedded text layer (no OCR garbage from figures),
- detects whether OCR is needed automatically,
- turns display equations into LaTeX and tables into real Markdown,
- in precise mode, has an AI verify formulas/tables against the page images while leaving the body prose byte-for-byte untouched (to minimise cost).
- Python 3.9+
- A local Java 11+ runtime (OpenDataLoader runs as a Java JAR)
pip install -r requirements.txt
# or:
pip install "opendataloader-pdf[hybrid]" pymupdf| Mode | What it does | Speed |
|---|---|---|
| fast | OpenDataLoader only — one complete Markdown file | ~20 s |
| precise | After extraction, renders every page that holds a formula or table so an AI fixes only formulas, tables, and garbled formatting (body prose untouched), then merges the fixes back in | ~20 s extract + AI review |
The recommended workflow: an assistant invoking this skill asks the user to pick fast or precise first.
# Fast: one-shot Markdown
python3 PDF2MD.skill convert paper.pdf out/ --mode fast
# Precise: extract, then let the AI review formulas/tables
python3 PDF2MD.skill convert paper.pdf out/ --mode precise
# ...the AI writes replacements.json, then:
python3 PDF2MD.skill repair out/paper_work/paper_manifest.json \
out/paper_work/paper_replacements.jsonText inside figures/charts is never transcribed — only body text, formulas, and tables are captured.
- Text-layer PDF (most published papers) → backend runs
--no-ocr: reads embedded text, no OCR garbage from images. - Scanned / image-only PDF → backend runs
--force-ocr: the page is an image, so OCR is unavoidable.
Decided automatically from extractable character counts; override with
--ocr {auto,off,force}.
| Option | Meaning |
|---|---|
--mode fast|precise |
Required. fast = ODL only; precise = ODL + AI review |
--ocr auto|off|force |
OCR policy (default auto) |
--port N |
Backend port, default 5002 (auto-switches if busy) |
--dpi N |
DPI of rendered review images (precise), default 200 |
--enrich-formula |
fast mode only: run the local image→LaTeX model (faithful but slow) |
out/
├── paper.md # final complete Markdown
├── paper.json # ODL structured output
├── paper_images/ # extracted images
└── paper_work/ # intermediate artifacts (draft, manifest, review_pages, log)
A page is reviewed only if it shows a real display formula or a real table.
High-precision signals avoid false positives from inline text (Fe3O4,
kJ/mol, 70%) while never missing a numbered display equation — each one
carries a $$…$$ wrapper, a strong math operator (∫ ∑ √ ∏ ∂ ∇ ∮), or a
right-hand equation number (1). Real tables are detected by their |---|
separator row.
The AI then emits minimal find → replace edits (copying the broken text
verbatim from the draft), so body prose is never rewritten. See the docstring
at the top of PDF2MD.skill for the exact replacements.json
format.
- This project is released under the MIT License (see LICENSE).
- It wraps OpenDataLoader (Apache-2.0).
- It depends on PyMuPDF, which is licensed under AGPL-3.0. PyMuPDF is installed separately by the user; if you redistribute this tool as a hosted network service, review the AGPL terms.