读 PDF 脚本的维护记录:引擎实测与吸收的两个 DSH 插件
这是 阅读 PDF 背后的维护记录:引擎为什么这样选、从哪些上游插件拿了什么。用脚本不用看,脚本出问题或者要改脚本时再看。
PDF 里没有现成的"文字层"和"图片层":每页是一串绘图指令,包括画字形、画矢量线条和贴位图。文字靠解析文字指令提取,图片靠把整页渲染出来。这两种结果都是推导出来的,所以文字不够用时就看渲染图。
引擎怎么选:实测对比
测试材料是一份 12 页的中文日报:headless Chrome 打印生成,中英文混排,末页有表格。2026-09-29 在本机实测:
| 做法 | 语序 | 换行 | 表格 | "人工"两个字 | 扫描件 |
|---|---|---|---|---|---|
pdfium get_text_bounded | 对 | 保留 | 一行一格,用空格分隔 | 正常 | 返回空,由脚本提示改用 render |
pdf-inspector extract_pages_markdown | 错:中文段落里的英文和数字被挪到行首 | 段落合并 | 重建成 GFM 表格 | 正常 | 标出需要 OCR 的页 |
| Claude Code Read | 对 | 保留 | 文字里一行一格,另有渲染图 | 文字层是康熙部首"⼈⼯",渲染图正常 | 靠渲染图 |
pdf-inspector 在中文段落上的实际输出(原文是"GPT-6 Astra 发布次日引爆社区:鹈鹕骑自行车 SVG 测试成为标志性评测方式,466 赞的楼主发现……"):
GPT-6 Astra SVG 466 发布次日引爆社区:鹈鹕骑自行车 测试成为标志性评测方式, " " 4o 赞的楼主发现 极高 推理等级画出的鹈鹕惨不忍睹
结论:正文交给 pdfium,pdf-inspector 只用于分类和表格。它判断"扫描件 / 需要 OCR 的页"很准:用两页纯图片合成的 PDF 测试,结果是 scanned,置信度 0.95,两页都被标出。但它的语序问题不能让模型直接吃进去。
康熙部首问题出在这份 PDF 字体的 ToUnicode 映射表上,Claude Code 的文字层原样照搬了。pdfium 和 pdf-inspector 取出的都是正常汉字,脚本另外做了一层映射兜底。
吸收的两个 DSH 插件
DeepSeek Harness 官方的 read 不读 PDF(tool-fs README 写的是 "PDF, audio, and video remain deferred"),社区在 Discussions1 里做了几个插件。这里存档的是其中两个的核心文件,只取和读 PDF 有关的几份。两者都是只提取文字,都不给模型看页面图片。
| 插件 | 钉的 commit | 许可证 | 存档 |
|---|---|---|---|
| jiaoqsh/dsh-document2(帖子 #27463) | 393aad9(2026-08-16) | MIT | src/tool.ts、pdf-inspector.ts、pdf-worker.ts、pages.ts |
| vaxilicaihouxian/dsh-pdf-reader4(帖子 #64495) | 0758700(2026-09-12) | MIT,只在 package.json 里声明,上游没有 LICENSE 文件 | src/pdf-reader.ts |
抓取日期都是 2026-09-29。上游 README、Office 转换、浏览器阅读视图和测试都没有收,要看原貌请点上面钉了 commit 的链接。
dsh-document:pdf-inspector 放进 Worker 线程
新增一个 read_document 工具,参数设计和内置的 read 一致:file_path + offset / limit 按行分页,PDF 另外多一个 pages。转换关掉了图片输出:
includePageMarkers: true, includeImages: false
- 每次转换新开一个 Worker 线程,取消就直接 terminate,大文件不会卡住 DSH 主进程。
- 页码写法
"1-3,7"、<!-- Page N -->页标记、开头给出<warning>列出缺文字的页,这三点本脚本都照搬了。 - 引擎是 pdf-inspector,所以中文语序的问题它也有。插件作者自己也写了 "layout-heavy PDFs may collapse paragraphs into long lines"。
dsh-document src/pdf-worker.ts(Worker 里调 pdf-inspector)
/**
* Worker-thread entry for PDF conversion. Runs `@firecrawl/pdf-inspector`'s
* WASM build, whose `processPdf` is synchronous, off the main event loop;
* the parent terminates the worker to cancel. This module is executed by
* Node directly (never bundled into the plugin entry), so it stays free of
* TypeScript-only runtime syntax and imports nothing from the plugin.
* @module @jiaoqsh/dsh-document/pdf-worker
*/
import { readFileSync } from 'node:fs'
import { createRequire } from 'node:module'
import { parentPort, workerData } from 'node:worker_threads'
import { initSync, processPdf } from '@firecrawl/pdf-inspector-wasm'
import type { MarkdownProfile } from '@firecrawl/pdf-inspector-wasm'
/** What the parent posts as `workerData`. */
export interface PdfWorkerRequest {
bytes: Uint8Array
/** 1-based pages to convert; absent means all. */
pages?: number[]
profile: MarkdownProfile
}
/** The engine facts the parent needs; everything else stays in the worker. */
export interface PdfWorkerResult {
markdown: string
pdfType: 'TextBased' | 'Scanned' | 'ImageBased' | 'Mixed'
pageCount: number
/** 1-based. */
pagesNeedingOcr: number[]
title?: string
}
/** Worker → parent message. */
export type PdfWorkerReply =
| { ok: true; result: PdfWorkerResult }
| { ok: false; message: string }
const require = createRequire(import.meta.url)
function run(request: PdfWorkerRequest): PdfWorkerResult {
initSync({
module: readFileSync(require.resolve('@firecrawl/pdf-inspector-wasm/pdf_inspector_wasm_bg.wasm')),
})
const result = processPdf(new Uint8Array(request.bytes), {
profile: request.profile,
includePageMarkers: true,
includeImages: false,
...(request.pages === undefined ? {} : { pages: request.pages }),
})
return {
markdown: result.markdown ?? '',
pdfType: result.pdfType,
pageCount: result.pageCount,
pagesNeedingOcr: result.pagesNeedingOcr,
...(result.title === undefined ? {} : { title: result.title }),
}
}
if (parentPort !== null) {
let reply: PdfWorkerReply
try {
reply = { ok: true, result: run(workerData as PdfWorkerRequest) }
} catch (error: unknown) {
reply = { ok: false, message: error instanceof Error ? error.message : String(error) }
}
parentPort.postMessage(reply)
}
dsh-document src/pdf-inspector.ts(分类、页码越界、扫描件报错)
/**
* The pdf-inspector engine: PDF → Markdown with page selection, page markers,
* and scanned-page facts. Each conversion runs in a fresh worker thread so a
* large document never blocks the harness event loop, and cancellation
* terminates the worker.
* @module @jiaoqsh/dsh-document/pdf-inspector
*/
import { Worker } from 'node:worker_threads'
import type { MarkdownProfile } from '@firecrawl/pdf-inspector-wasm'
import {
DocumentConversionError,
type ConvertRequest,
type ConvertResult,
type DocumentConverter,
type PdfKind,
} from './converter.ts'
import type { PdfWorkerReply, PdfWorkerRequest, PdfWorkerResult } from './pdf-worker.ts'
/** Deployment choices for the PDF engine. */
export interface PdfInspectorOptions {
/** `fidelity` keeps source structure; `compact` spends fewer tokens. */
profile: MarkdownProfile
}
const KIND: Record<PdfWorkerResult['pdfType'], PdfKind> = {
TextBased: 'text',
Scanned: 'scanned',
ImageBased: 'image',
Mixed: 'mixed',
}
// The worker file sits beside this module in both the source tree and the
// built lib/. A computed name keeps bundlers from rewriting the URL as an
// asset. Source mode exists only for tests: Node then strips the worker's
// types itself, and the flag makes that hold on every supported Node (it is
// the default from 22.18 and merely redundant there).
const WORKER_IS_SOURCE = import.meta.url.endsWith('.ts')
const WORKER_URL = new URL(WORKER_IS_SOURCE ? 'pdf-worker.ts' : 'pdf-worker.js', import.meta.url)
const WORKER_EXEC_ARGV = WORKER_IS_SOURCE
? [...process.execArgv, '--experimental-strip-types', '--no-warnings']
: undefined
/** Classify an engine failure message; the engine reports plain strings, not codes. */
function engineError(message: string): DocumentConversionError {
const text = message.replace(/^process PDF: /u, '')
const lower = text.toLowerCase()
if (lower.includes('not a pdf') || lower.includes('malformed') || lower.includes('invalid')) {
return new DocumentConversionError(text, 'malformed')
}
if (lower.includes('encrypt') || lower.includes('password')) {
return new DocumentConversionError(text, 'encrypted')
}
return new DocumentConversionError(text, 'unknown')
}
/**
* Run one conversion in a worker and settle with its reply.
* @param request - bytes, pages, and profile for the worker.
* @param signal - terminates the worker when it fires.
* @returns the worker's facts and Markdown.
*/
export function runPdfWorker(request: PdfWorkerRequest, signal: AbortSignal): Promise<PdfWorkerResult> {
return new Promise<PdfWorkerResult>((resolve, reject) => {
const worker = new Worker(WORKER_URL, {
workerData: request,
...(WORKER_EXEC_ARGV === undefined ? {} : { execArgv: WORKER_EXEC_ARGV }),
})
let settled = false
const settle = (outcome: () => void): void => {
if (settled) return
settled = true
signal.removeEventListener('abort', onAbort)
// Terminate is idempotent; after a normal reply it only reclaims the thread.
void worker.terminate()
outcome()
}
const onAbort = (): void => settle(() => reject(new DocumentConversionError('the conversion was cancelled', 'aborted')))
signal.addEventListener('abort', onAbort, { once: true })
worker.once('message', (reply: PdfWorkerReply) => settle(() => {
if (reply.ok) resolve(reply.result)
else reject(engineError(reply.message))
}))
worker.once('error', (error: Error) => settle(() => reject(new DocumentConversionError(error.message, 'unknown'))))
worker.once('exit', (code: number) => settle(() => reject(
new DocumentConversionError(`the PDF worker exited with code ${code} before replying`, 'unknown'),
)))
})
}
function formatPages(pages: readonly number[]): string {
const sorted = [...pages].sort((a, b) => a - b)
const parts: string[] = []
for (let i = 0; i < sorted.length;) {
let j = i
while (j + 1 < sorted.length && sorted[j + 1] === sorted[j]! + 1) j += 1
parts.push(j === i ? `${sorted[i]}` : `${sorted[i]}-${sorted[j]}`)
i = j + 1
}
return parts.join(', ')
}
/**
* Build the PDF engine.
* @param options - deployment choices.
* @returns a converter that handles only `pdf`.
*/
export function pdfInspectorConverter(options: PdfInspectorOptions): DocumentConverter {
return {
name: 'pdf-inspector',
formats: ['pdf'],
async convert({ bytes, pages, signal }: ConvertRequest): Promise<ConvertResult> {
if (signal.aborted) throw new DocumentConversionError('the conversion was cancelled', 'aborted')
const raw = await runPdfWorker({
bytes,
profile: options.profile,
...(pages === undefined ? {} : { pages: [...pages] }),
}, signal)
if (pages !== undefined) {
const beyond = pages.filter(page => page > raw.pageCount)
if (beyond.length > 0) {
throw new DocumentConversionError(
`the document has ${raw.pageCount} page${raw.pageCount === 1 ? '' : 's'}; requested page${beyond.length === 1 ? '' : 's'} ${formatPages(beyond)} ${beyond.length === 1 ? 'does' : 'do'} not exist`,
'pageRange',
)
}
}
const kind = KIND[raw.pdfType]
if (raw.markdown.trim().length === 0) {
const plural = pages === undefined ? raw.pageCount !== 1 : pages.length !== 1
const scope = pages === undefined
? `all ${raw.pageCount} page${plural ? 's' : ''}`
: `the requested page${plural ? 's' : ''} (${formatPages(pages)})`
const verb = plural ? 'contain' : 'contains'
if (kind === 'scanned' || kind === 'image') {
throw new DocumentConversionError(
`${scope} ${verb} no extractable text: this is ${kind === 'image' ? 'an image-only' : 'a scanned'} PDF and needs OCR`,
'scanned',
)
}
if (raw.pagesNeedingOcr.length > 0) {
throw new DocumentConversionError(
`${scope} ${verb} no extractable text (pages needing OCR: ${formatPages(raw.pagesNeedingOcr)})`,
'scanned',
)
}
throw new DocumentConversionError('the document contains no extractable text', 'empty')
}
return {
markdown: raw.markdown,
pdf: {
pageCount: raw.pageCount,
kind,
pagesNeedingOcr: raw.pagesNeedingOcr,
...(pages === undefined ? {} : { pages: [...pages] }),
...(raw.title === undefined ? {} : { title: raw.title }),
},
}
},
}
}
export { formatPages }
dsh-document src/tool.ts(给模型的工具定义和输出格式)
/**
* The model-facing `read_document` tool: resolve a path through `ctx.fs`,
* read the bytes under the input cap, convert them to Markdown through the
* routed engine, and return one line-numbered window. PDFs additionally
* accept a page selection and report page facts.
* @module @jiaoqsh/dsh-document/tool
*/
import type { Context } from '@deepseek-ai/cordis'
import type {} from '@deepseek-ai/dsh-fs'
import type {} from '@deepseek-ai/dsh-system-prompt'
import { defineTool } from '@deepseek-ai/dsh-tools'
import type { GenericCallView, ToolExecution } from '@deepseek-ai/dsh-tools'
import {
DOCUMENT_EXTENSIONS,
DOCUMENT_FORMATS,
DocumentConversionError,
formatOf,
type DocumentConverter,
type DocumentFormat,
type PdfFacts,
} from './converter.ts'
import { parsePages } from './pages.ts'
import { formatPages } from './pdf-inspector.ts'
import { windowLines, type WindowLine } from './window.ts'
/** Deployment bounds after defaulting (see `Config` in index.ts). */
export interface ReadDocumentCaps {
/** Inclusive byte cap on the source file; larger files are refused before conversion. */
maxInputBytes: number
/** Default and maximum number of lines returned by one call. */
readLimit: number
/** Maximum characters returned for a single line. */
maxLineLength: number
/** Maximum bytes of line text returned by one call. */
maxOutputBytes: number
/** Maximum distinct PDF pages one `pages` selection may name. */
pdfMaxPages: number
}
/** Canonical value of one successful call. */
export interface ReadDocumentOutcome {
path: string
format: DocumentFormat
offset: number
lines: WindowLine[]
totalLines: number
truncatedByBytes: boolean
/** PDF facts; absent for other formats. */
pdf?: PdfFacts
}
/** Validated arguments after defaulting. */
interface ReadInput {
filePath: string
offset: number
limit: number
pages?: number[]
}
const EXTENSION_LIST = DOCUMENT_EXTENSIONS.map(ext => `.${ext}`).join(', ')
const PDF_KINDS = ['text', 'scanned', 'image', 'mixed'] as const
function parsePositiveInteger(value: number, name: string): number {
if (!Number.isFinite(value) || !Number.isInteger(value) || value < 1) {
throw new Error(`${name} must be a positive integer`)
}
return value
}
/**
* Validate the constraints the schema DSL does not express.
* @param args - schema-validated raw arguments.
* @param caps - the deployment line cap and PDF page cap.
* @returns validated input with `offset` defaulted to 1 and `limit` to the cap.
*/
export function parseReadArgs(
args: { file_path: string; offset?: number; limit?: number; pages?: string },
caps: Pick<ReadDocumentCaps, 'readLimit' | 'pdfMaxPages'>,
): ReadInput {
if (args.file_path.trim().length === 0) throw new Error('file_path must be a non-empty string')
const offset = args.offset === undefined ? 1 : parsePositiveInteger(args.offset, 'offset')
const limit = args.limit === undefined ? caps.readLimit : parsePositiveInteger(args.limit, 'limit')
if (limit > caps.readLimit) throw new Error(`limit must be less than or equal to ${caps.readLimit}`)
const pages = args.pages === undefined ? undefined : parsePages(args.pages, caps.pdfMaxPages)
return { filePath: args.file_path, offset, limit, ...(pages === undefined ? {} : { pages }) }
}
const KIND_LABEL: Record<PdfFacts['kind'], string> = {
text: 'text-based',
scanned: 'scanned',
image: 'image-only',
mixed: 'mixed text and scanned pages',
}
/**
* Model-facing text for one outcome: an envelope naming the path and format,
* PDF facts when present, numbered lines, and a footer that says how to
* continue.
* @param outcome - the canonical value.
* @returns the rendered text.
*/
export function formatReadOutput(outcome: ReadDocumentOutcome): string {
const endLine = outcome.lines.at(-1)?.number ?? Math.max(0, outcome.offset - 1)
let footer: string
if (outcome.truncatedByBytes) {
footer = `(Output capped. Showing lines ${outcome.offset}-${endLine}. Use offset=${endLine + 1} to continue.)`
} else if (endLine < outcome.totalLines) {
footer = `(Showing lines ${outcome.offset}-${endLine} of ${outcome.totalLines}. Use offset=${endLine + 1} to continue.)`
} else {
footer = `(End of document - total ${outcome.totalLines} lines)`
}
const body = outcome.lines.length > 0
? `${outcome.lines.map(line => `${line.number}: ${line.text}`).join('\n')}\n\n${footer}`
: footer
const head = [`<path>${outcome.path}</path>`, `<format>${outcome.format}</format>`]
if (outcome.pdf !== undefined) {
const { pageCount, kind, pages, pagesNeedingOcr, title } = outcome.pdf
const shown = pages === undefined ? 'all pages' : `showing page${pages.length === 1 ? '' : 's'} ${formatPages(pages)}`
head.push(`<pdf>${pageCount} page${pageCount === 1 ? '' : 's'}, ${KIND_LABEL[kind]}; ${shown}${title === undefined ? '' : `; title: ${title}`}</pdf>`)
if (pagesNeedingOcr.length > 0) {
head.push(`<warning>Page${pagesNeedingOcr.length === 1 ? '' : 's'} ${formatPages(pagesNeedingOcr)} contain${pagesNeedingOcr.length === 1 ? 's' : ''} no extractable text (scanned or image content); ${pagesNeedingOcr.length === 1 ? 'its' : 'their'} content is missing below and would need OCR.</warning>`)
}
}
return `${head.join('\n')}\n<content>\n${body}\n</content>`
}
/** Model-facing message for one conversion failure. */
function describeFailure(displayPath: string, error: DocumentConversionError): string {
switch (error.code) {
case 'encrypted':
return `cannot read "${displayPath}": the document is encrypted; supply a decrypted copy`
case 'unsupported':
return `cannot read "${displayPath}": ${error.message}`
case 'empty':
return `cannot read "${displayPath}": ${error.message}`
case 'scanned':
return `cannot read "${displayPath}": ${error.message}, which this tool does not perform`
case 'pageRange':
return `cannot read "${displayPath}": ${error.message}`
case 'malformed':
case 'missingPart':
return `cannot read "${displayPath}": the file is damaged or incomplete (${error.message})`
case 'resourceLimit':
return `cannot read "${displayPath}": the document exceeds the converter's internal safety limits (${error.message})`
case 'aborted':
return `cannot read "${displayPath}": ${error.message}`
case 'io':
case 'unknown':
return `cannot read "${displayPath}": ${error.message}`
}
}
/**
* Register the `read_document` tool and its system-prompt guidance.
* @param ctx - the plugin context; registrations are effects scoped to it.
* @param converterFor - the format router over the mounted engines.
* @param caps - resolved deployment bounds.
*/
export function applyReadDocumentTool(
ctx: Context,
converterFor: (format: DocumentFormat) => DocumentConverter,
caps: ReadDocumentCaps,
): void {
ctx.systemPrompt.section({
name: 'tool:read_document',
order: 100,
text: `Use the read_document tool — not read or shell commands — to inspect PDF, Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, and CSV files. It returns the document converted to line-numbered Markdown; use offset and limit to continue reading long documents. For a PDF, pass pages (for example "1-3,7") to read only those pages; page markers like <!-- Page 4 --> show where each page starts.`,
})
ctx.tools.register(defineTool({
name: 'read_document',
description: `Read a document file as Markdown: PDF, Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, and CSV files are converted to line-numbered Markdown text. Supported: ${EXTENSION_LIST}. Returns text only and writes no files. PDFs report page count, whether pages are scanned, and accept a pages selection; scanned or image-only pages are not OCRed. For plain text or source files use read; for images use read_image.`,
parameters: {
file_path: { type: 'string', required: true, description: 'Path to the document, resolved by the filesystem backend; relative paths resolve against the session workspace.' },
offset: { type: 'number', description: '1-based first line of the converted Markdown to return. Defaults to 1.' },
limit: { type: 'number', description: `Maximum number of lines to return. Defaults to ${caps.readLimit}.` },
pages: { type: 'string', description: `PDF only: 1-based pages to convert, as numbers and ranges like "1-3,7" (at most ${caps.pdfMaxPages} pages). Defaults to every page.` },
},
output: {
schema: {
type: 'object',
additionalProperties: false,
properties: {
path: { type: 'string', required: true },
format: { type: 'string', enum: [...DOCUMENT_FORMATS], required: true },
offset: { type: 'integer', required: true },
lines: {
type: 'array',
required: true,
items: {
type: 'object',
additionalProperties: false,
properties: {
number: { type: 'integer', required: true },
text: { type: 'string', required: true },
},
},
},
totalLines: { type: 'integer', required: true },
truncatedByBytes: { type: 'boolean', required: true },
pdf: {
type: 'object',
additionalProperties: false,
properties: {
pageCount: { type: 'integer', required: true },
kind: { type: 'string', enum: [...PDF_KINDS], required: true },
pagesNeedingOcr: { type: 'array', required: true, items: { type: 'integer' } },
pages: { type: 'array', items: { type: 'integer' } },
title: { type: 'string' },
},
},
},
},
render: (_args, value) => [{ type: 'text', text: formatReadOutput(value as ReadDocumentOutcome) }],
},
// Read-only: safe beside other tool calls in the same step.
isConcurrencySafe: () => true,
async execute(args, exec: ToolExecution): Promise<ReadDocumentOutcome> {
const input = parseReadArgs(args, caps)
const format = formatOf(input.filePath)
if (format === undefined) {
throw new Error(
`cannot read "${input.filePath}": unsupported extension. Supported: ${EXTENSION_LIST}. For plain text or source files use the read tool.`,
)
}
if (input.pages !== undefined && format !== 'pdf') {
throw new Error(`pages applies to PDF files only; "${input.filePath}" is ${format}`)
}
// Per-session workspace, like the shipped filesystem tools; a non-agent
// caller falls back to the provider default.
const cwd = exec.agent?.session.header.cwd
const target = await ctx.fs.resolve(input.filePath, {
...cwd === undefined ? {} : { cwd },
signal: exec.signal,
})
const info = await ctx.fs.stat(target, exec.signal)
if (info === undefined) throw new Error(`cannot read "${target.displayPath}": not found`)
if (info.type !== 'file') throw new Error(`cannot read "${target.displayPath}": not a regular file`)
// The provider enforces the input cap, so an oversized file never buffers.
const bytes = await ctx.fs.readBytes(target, exec.signal, caps.maxInputBytes)
let converted
try {
converted = await converterFor(format).convert({
bytes,
format,
signal: exec.signal,
...(input.pages === undefined ? {} : { pages: input.pages }),
})
} catch (error: unknown) {
if (error instanceof DocumentConversionError) throw new Error(describeFailure(target.displayPath, error))
throw error
}
const window = windowLines(converted.markdown, {
offset: input.offset,
limit: input.limit,
maxLineLength: caps.maxLineLength,
maxBytes: caps.maxOutputBytes,
})
return {
path: target.displayPath,
format,
offset: input.offset,
lines: window.lines,
totalLines: window.totalLines,
truncatedByBytes: window.truncatedByBytes,
...(converted.pdf === undefined ? {} : { pdf: converted.pdf }),
}
},
// Pure display: a generic read-kind card titled by the file, with the
// requested pages or line window when the raw args carry one, and a
// follow-along location.
presentCall(args): GenericCallView {
const { offset, limit, pages } = args
const window = pages !== undefined && pages.length > 0
? ` (pages ${pages})`
: limit !== undefined && limit > 0
? ` (lines ${offset ?? 1} - ${(offset ?? 1) + limit - 1})`
: offset !== undefined ? ` (from line ${offset})` : ''
return {
card: 'generic',
title: `Read ${args.file_path} as Markdown${window}`,
kind: 'read',
locations: [{ path: args.file_path }],
}
},
}))
}
dsh-document src/pages.ts(页码写法解析)
/**
* The `pages` argument grammar: comma-separated 1-based page numbers and
* inclusive ranges, e.g. `"1-3,7,10-12"`.
* @module @jiaoqsh/dsh-document/pages
*/
const TOKEN = /^(\d+)(?:-(\d+))?$/
/**
* Parse a page selection into a sorted, de-duplicated list.
* @param spec - the model-supplied text.
* @param maxPages - the largest number of distinct pages one call may select.
* @returns ascending 1-based page numbers.
* @throws Error naming the offending token, or the count when it exceeds `maxPages`.
*/
export function parsePages(spec: string, maxPages: number): number[] {
const pages = new Set<number>()
for (const rawToken of spec.split(',')) {
const token = rawToken.trim()
const match = TOKEN.exec(token)
if (match === null) {
throw new Error(`pages must be comma-separated page numbers or ranges like "1-3,7", got "${token}"`)
}
const first = Number(match[1])
const last = match[2] === undefined ? first : Number(match[2])
if (first < 1 || last < first) throw new Error(`pages contains an invalid range "${token}"`)
if (last - first + 1 > maxPages || pages.size + (last - first + 1) > maxPages) {
throw new Error(`pages selects more than ${maxPages} pages`)
}
for (let page = first; page <= last; page += 1) pages.add(page)
}
if (pages.size === 0) throw new Error('pages must select at least one page')
return [...pages].sort((a, b) => a - b)
}
dsh-pdf-reader:pdf.js 拼接文字片段
pdf_open 一次性解析全部页面的文字,缓存在内存里,pdf_extract 再按页码范围取。插件的重点是阅读流程:阅读位置写进会话日志、笔记另存为 paper.pdf.dsh-notes.md、侧栏有阅读视图。取文字只有一行:
content.items.map(item => 'str' in item ? item.str : '').join(' ').replace(/\s+/g, ' ').trim()
getTextContent()返回的是带坐标的文字片段,用空格拼接后再压缩空白,一页变成一整行,段落和表格结构都丢了。这一点是读代码得出的,没有实际跑过。- 没有文字的页返回空字符串,也不提示原因;本脚本改成给出
<warning>,并提示改用render。 - 依赖里有
@napi-rs/canvas,但 Host 端源码没有用到。只有浏览器阅读视图在 canvas 上画页面,那是给人看的。
dsh-pdf-reader src/pdf-reader.ts
import { Service, type Context } from '@deepseek-ai/cordis'
import type { FileSystem } from '@deepseek-ai/dsh-fs'
import { getDocument } from 'pdfjs-dist/legacy/build/pdf.mjs'
import type { PdfDocument, PdfDocumentId, PdfPageText } from './types.js'
import { PdfDocumentId as makePdfDocumentId } from './types.js'
declare module '@deepseek-ai/cordis' {
interface Context {
/** Mounted PDF reader capability. */
pdfReader: PdfReader
}
}
/** The replaceable PDF parsing service consumed by the tool plugin. */
export abstract class PdfReader extends Service {
constructor(ctx: Context) {
super(ctx, 'pdfReader')
}
/** Open a PDF and return stable metadata. */
abstract open(path: string, signal?: AbortSignal): Promise<PdfDocument>
/** Extract one inclusive page range. */
abstract extract(document: PdfDocumentId, fromPage: number, toPage: number, signal?: AbortSignal): Promise<readonly PdfPageText[]>
/** Resolve a document id to metadata. */
abstract get(document: PdfDocumentId): PdfDocument | undefined
}
interface LoadedDocument {
readonly metadata: PdfDocument
readonly pages: readonly string[]
}
/** Local provider using the harness filesystem seam and pdfjs-dist. */
export class PdfReaderLocal extends PdfReader {
static inject = ['fs']
private readonly fs: FileSystem
private readonly maxPdfBytes: number
private readonly documents = new Map<PdfDocumentId, LoadedDocument>()
constructor(ctx: Context, config: { maxPdfBytes?: number } = {}) {
super(ctx)
this.fs = ctx.fs
this.maxPdfBytes = config.maxPdfBytes ?? 128 * 1024 * 1024
if (!Number.isSafeInteger(this.maxPdfBytes) || this.maxPdfBytes < 1) {
throw new Error('pdf-reader: maxPdfBytes must be a positive safe integer')
}
}
async open(path: string, signal?: AbortSignal): Promise<PdfDocument> {
const target = await this.fs.resolve(path, { signal })
const info = await this.fs.stat(target, signal)
if (info?.type !== 'file') throw new Error(`PDF file is not a regular file: ${path}`)
const bytes = await this.fs.readBytes(target, signal, this.maxPdfBytes)
signal?.throwIfAborted()
const loadingTask = getDocument({ data: bytes, useSystemFonts: true })
const pdf = await loadingTask.promise
const pages: string[] = []
for (let pageNumber = 1; pageNumber <= pdf.numPages; pageNumber++) {
signal?.throwIfAborted()
const page = await pdf.getPage(pageNumber)
const content = await page.getTextContent()
pages.push(content.items.map(item => 'str' in item ? item.str : '').join(' ').replace(/\s+/g, ' ').trim())
page.cleanup()
}
const pageCount = pdf.numPages
pdf.cleanup()
const id = makePdfDocumentId(`pdf-${stableId(target.displayPath)}`)
const metadata: PdfDocument = {
id,
path: target.displayPath,
title: basename(target.displayPath),
pageCount,
sizeBytes: bytes.byteLength,
}
this.documents.set(id, { metadata, pages })
return metadata
}
async extract(document: PdfDocumentId, fromPage: number, toPage: number, signal?: AbortSignal): Promise<readonly PdfPageText[]> {
const loaded = this.documents.get(document)
if (loaded === undefined) throw new Error(`PDF document is not open: ${document}`)
if (!Number.isSafeInteger(fromPage) || !Number.isSafeInteger(toPage) || fromPage < 1 || toPage < fromPage || toPage > loaded.pages.length) {
throw new Error(`PDF page range must be within 1-${loaded.pages.length}`)
}
const result: PdfPageText[] = []
for (let page = fromPage; page <= toPage; page++) {
signal?.throwIfAborted()
result.push({ page, text: loaded.pages[page - 1] ?? '' })
}
return result
}
get(document: PdfDocumentId): PdfDocument | undefined {
return this.documents.get(document)?.metadata
}
}
function basename(path: string): string {
const name = path.split(/[\\/]/).pop() ?? path
return name.toLowerCase().endsWith('.pdf') ? name.slice(0, -4) : name
}
function stableId(path: string): string {
let hash = 2166136261
for (const char of path) {
hash ^= char.charCodeAt(0)
hash = Math.imul(hash, 16777619)
}
return Math.abs(hash >>> 0).toString(36)
}
export default PdfReader
本脚本从中拿了什么、补了什么
- 拿来:dsh-document 的页码写法、页标记、缺文字页的
<warning>,以及按页分段、输出设上限的思路;dsh-pdf-reader 的按页码范围读取。 - 补上:pdfium 按阅读顺序取文字,替代有语序问题的引擎;
render渲染页面图片,处理扫描件和图表;info给出下一步建议;康熙部首映射。 - 形态不同:插件是 DSH 里的 tool,有 schema 校验和沙箱;这里是脚本,任何有 shell 的 Agent 都能跑。代价是没有沙箱,输出上限要靠脚本自己的
--max-chars控制。
没做的
- OCR:pdf-inspector 有
process_pdf_with_ocr,但第一次调用要下载 OCR 模型,这里没有接入,也没有测。现在扫描件只能靠render加上能看图的模型。 - 页面里的嵌入图片单独导出:没有做。
render出的是整页。 - 超大 PDF(几百 MB)的内存占用没有测过。
- 可以再砍:去掉
markdown命令和 pdf-inspector 依赖。它现在只做两件事——判断扫描件(pdfium 取不到文字的页就是扫描页)和重建表格(中文语序有问题,本来就要打折扣用)。去掉后脚本剩三个命令、两个依赖。没做,要改得重新测。
完整脚本 scripts/pdf_read.py
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "pypdfium2==5.13.0",
# "pdf-inspector==1.25.2",
# "pillow==12.3.0",
# ]
# ///
"""Read a PDF for an AI agent: facts first, then text, then page images where text is not enough.
Run with `uv run pdf_read.py <command> <file.pdf> ...` — uv installs the pinned dependencies
from the header above into a cached environment, no manual setup.
Commands:
info page count, text-based vs scanned, pages without text, pages with tables/columns
text plain text in reading order (pdfium), with <!-- Page N --> markers
markdown structured Markdown (pdf-inspector): headings and tables, but see the CJK caveat
render page images (PNG) for pages whose text layer is missing or not enough
Pages are 1-based and written like "1-3,7" or "4-" (page 4 to the end).
Output is capped by --max-chars and always stops on a page boundary with a continuation hint,
so one call never floods the model context.
Engines:
pypdfium2 (Apache-2.0 / BSD-3) Chrome's PDFium: reading-order text and page rendering
pdf-inspector (MIT, firecrawl) classification and Markdown; the engine of the dsh-document plugin
"""
from __future__ import annotations
import argparse
import sys
import unicodedata
from pathlib import Path
import pdf_inspector
import pypdfium2 as pdfium
DEFAULT_MAX_CHARS = 30_000
DEFAULT_MAX_RENDER = 10
class UsageError(Exception):
"""A problem the caller can fix by changing arguments; printed without a traceback."""
# ---------------------------------------------------------------- helpers
def parse_pages(spec: str | None, page_count: int) -> list[int]:
"""Parse "1-3,7" / "4-" into sorted unique 1-based pages; None means every page."""
if spec is None:
return list(range(1, page_count + 1))
pages: set[int] = set()
for part in spec.replace(" ", "").split(","):
if not part:
continue
try:
if "-" in part:
start_text, end_text = part.split("-", 1)
start = int(start_text)
end = int(end_text) if end_text else page_count
else:
start = end = int(part)
except ValueError:
raise UsageError(f'bad page spec "{part}": use numbers and ranges like "1-3,7" or "4-"') from None
if start < 1 or end < start:
raise UsageError(f'bad page range "{part}": pages start at 1 and ranges go low-high')
if end > page_count:
raise UsageError(f'page {end} does not exist: the document has {page_count} pages')
pages.update(range(start, end + 1))
if not pages:
raise UsageError("empty page spec")
return sorted(pages)
def format_pages(pages: list[int]) -> str:
"""Compress [1,2,3,7] to "1-3, 7"."""
parts: list[str] = []
i = 0
while i < len(pages):
j = i
while j + 1 < len(pages) and pages[j + 1] == pages[j] + 1:
j += 1
parts.append(str(pages[i]) if i == j else f"{pages[i]}-{pages[j]}")
i = j + 1
return ", ".join(parts) if parts else "none"
def normalize(text: str) -> str:
"""Map Kangxi radicals (U+2F00-U+2FDF) to ordinary CJK characters.
Some PDF producers map glyphs such as 人 to the radical ⼈ in their ToUnicode table:
the page looks right but the extracted text does not match searches. NFKC fixes exactly
this range; applying NFKC to the whole text would also rewrite full-width punctuation.
"""
if not any(0x2F00 <= ord(ch) <= 0x2FDF for ch in text):
return text
return "".join(unicodedata.normalize("NFKC", ch) if 0x2F00 <= ord(ch) <= 0x2FDF else ch for ch in text)
def open_pdf(path: Path, password: str | None) -> pdfium.PdfDocument:
if not path.is_file():
raise UsageError(f"not a file: {path}")
try:
return pdfium.PdfDocument(str(path), password=password)
except pdfium.PdfiumError as error:
message = str(error)
if "password" in message.lower():
raise UsageError(f"{path} is encrypted: pass --password") from None
raise UsageError(f"cannot open {path} as PDF: {message}") from None
def page_text(document: pdfium.PdfDocument, page_number: int) -> str:
page = document[page_number - 1]
try:
textpage = page.get_textpage()
try:
text = textpage.get_text_bounded()
finally:
textpage.close()
finally:
page.close()
return normalize(text.replace("\r\n", "\n").replace("\r", "\n")).strip()
def emit_capped(chunks: list[tuple[int, str]], max_chars: int, total_pages: int) -> None:
"""Print page chunks until max_chars; stop on a page boundary and say how to continue."""
used = 0
shown: list[int] = []
for index, (page_number, chunk) in enumerate(chunks):
if shown and used + len(chunk) > max_chars:
rest = [number for number, _ in chunks[index:]]
print(
f"\n(Output capped at {max_chars} chars. Showing pages {format_pages(shown)} of {total_pages}. "
f"Continue with --pages {format_pages(rest).replace(' ', '')}.)"
)
return
if not shown and len(chunk) > max_chars:
print(chunk[:max_chars])
print(
f"\n(Page {page_number} alone exceeds {max_chars} chars and was truncated. "
f"Raise --max-chars or render the page.)"
)
shown.append(page_number)
used = max_chars
continue
print(chunk)
shown.append(page_number)
used += len(chunk)
print(f"\n(Showing pages {format_pages(shown)} of {total_pages}.)")
# ---------------------------------------------------------------- commands
def cmd_info(args: argparse.Namespace) -> None:
document = open_pdf(args.file, args.password)
page_count = len(document)
metadata = {key: value for key, value in document.get_metadata_dict().items() if value}
document.close()
print(f"file: {args.file}")
print(f"pages: {page_count}")
for key in ("Title", "Author", "Subject", "Creator", "Producer", "CreationDate"):
if key in metadata:
print(f"{key.lower()}: {metadata[key]}")
if args.password is not None:
print("classification: skipped (encrypted PDF; pdf-inspector cannot take a password here)")
return
classification = pdf_inspector.classify_pdf(str(args.file))
layout = pdf_inspector.extract_pages_markdown(str(args.file))
no_text = sorted(page + 1 for page in classification.pages_needing_ocr)
tables = sorted(page + 1 for page in layout.pages_with_tables)
columns = sorted(page + 1 for page in layout.pages_with_columns)
print(f"type: {classification.pdf_type} (confidence {classification.confidence:.2f})")
print(f"pages without usable text: {format_pages(no_text)}")
print(f"pages with tables: {format_pages(tables)}")
print(f"pages with columns: {format_pages(columns)}")
hints = []
if no_text:
hints.append(f"render {format_pages(no_text).replace(' ', '')} and look at the images (no text layer)")
if tables:
hints.append(f"use markdown or render for {format_pages(tables).replace(' ', '')} (tables)")
if len(no_text) < page_count:
hints.append("use text for everything else" if hints else "use text")
print("next: " + "; ".join(hints))
def cmd_text(args: argparse.Namespace) -> None:
document = open_pdf(args.file, args.password)
page_count = len(document)
pages = parse_pages(args.pages, page_count)
chunks: list[tuple[int, str]] = []
empty: list[int] = []
for page_number in pages:
text = page_text(document, page_number)
if text:
chunks.append((page_number, f"<!-- Page {page_number} -->\n{text}\n"))
else:
empty.append(page_number)
chunks.append((page_number, f"<!-- Page {page_number}: no text layer, use render -->\n"))
document.close()
if empty:
print(
f"<warning>Pages {format_pages(empty)} have no extractable text (scanned or image-only). "
f"Their content is missing below; render them to read.</warning>\n"
)
emit_capped(chunks, args.max_chars, page_count)
def cmd_markdown(args: argparse.Namespace) -> None:
document = open_pdf(args.file, args.password)
page_count = len(document)
document.close()
if args.password is not None:
raise UsageError("markdown does not support encrypted PDFs; use text or render")
pages = parse_pages(args.pages, page_count)
result = pdf_inspector.extract_pages_markdown(str(args.file), [page - 1 for page in pages])
chunks: list[tuple[int, str]] = []
empty: list[int] = []
for page in result.pages:
page_number = page.page + 1
markdown = normalize(page.markdown).strip()
if not markdown or page.needs_ocr:
empty.append(page_number)
chunks.append((page_number, f"<!-- Page {page_number} -->\n{markdown}\n"))
print(
"<note>pdf-inspector rebuilds headings and tables, but in Chinese/Japanese paragraphs it can move "
"Latin words and numbers to the start of a line. Check prose against the text command.</note>\n"
)
if empty:
print(f"<warning>Pages {format_pages(empty)} need OCR; render them to read.</warning>\n")
emit_capped(chunks, args.max_chars, page_count)
def cmd_render(args: argparse.Namespace) -> None:
document = open_pdf(args.file, args.password)
page_count = len(document)
if args.pages is None:
raise UsageError('render needs --pages, e.g. --pages 3 or --pages "1-2,5"')
pages = parse_pages(args.pages, page_count)
if len(pages) > args.max_pages:
raise UsageError(
f"{len(pages)} pages requested, limit is {args.max_pages}: render fewer pages or raise --max-pages"
)
out = args.out or args.file.with_name(f"{args.file.stem}.pages")
out.mkdir(parents=True, exist_ok=True)
for page_number in pages:
page = document[page_number - 1]
try:
image = page.render(scale=args.dpi / 72).to_pil()
finally:
page.close()
target = out / f"page-{page_number:03d}.png"
image.save(target, optimize=True)
print(f"{target} ({image.width}x{image.height})")
document.close()
print(f"\n(Rendered {len(pages)} of {page_count} pages at {args.dpi} dpi. Open the PNGs with your image tool.)")
# ---------------------------------------------------------------- entry
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="pdf_read.py", description=__doc__.split("\n\n")[0])
sub = parser.add_subparsers(dest="command", required=True)
def add(name: str, help_text: str) -> argparse.ArgumentParser:
command = sub.add_parser(name, help=help_text)
command.add_argument("file", type=Path, help="path to the PDF")
command.add_argument("--password", help="password for an encrypted PDF")
return command
add("info", "page count, type, pages without text, tables, columns")
for name, help_text in (("text", "reading-order text"), ("markdown", "structured Markdown")):
command = add(name, help_text)
command.add_argument("--pages", help='1-based pages like "1-3,7" or "4-"; default all')
command.add_argument("--max-chars", type=int, default=DEFAULT_MAX_CHARS, help="output cap per call")
render = add("render", "page images as PNG")
render.add_argument("--pages", help='1-based pages like "3" or "1-2,5"; required')
render.add_argument("--dpi", type=int, default=150, help="resolution; 150 reads body text, 200+ for small print")
render.add_argument("--out", type=Path, help="output directory; default <file>.pages/ next to the PDF")
render.add_argument("--max-pages", type=int, default=DEFAULT_MAX_RENDER, help="refuse larger batches")
return parser
def main() -> int:
args = build_parser().parse_args()
handlers = {"info": cmd_info, "text": cmd_text, "markdown": cmd_markdown, "render": cmd_render}
try:
handlers[args.command](args)
except UsageError as error:
print(f"error: {error}", file=sys.stderr)
return 2
return 0
if __name__ == "__main__":
sys.exit(main())
-
Discussions,2026-10-02 读。 ↩
-
jiaoqsh/dsh-document,2026-10-02 读。 ↩
-
vaxilicaihouxian/dsh-pdf-reader,2026-10-02 读。 ↩