feat(skills): 新增 office-text-extract + markdown-to-docx 技能(从 Hermes 迁移,含脚本,路径改相对)
This commit is contained in:
parent
31c05c8ceb
commit
972e9307c8
95
skills_library/all/markdown-to-docx/SKILL.md
Normal file
95
skills_library/all/markdown-to-docx/SKILL.md
Normal file
@ -0,0 +1,95 @@
|
||||
---
|
||||
name: markdown-to-docx
|
||||
description: "Convert Markdown to .docx with CJK fonts, no pandoc."
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [docx, word, markdown, cjk, chinese, conversion, docx-js, node]
|
||||
category: productivity
|
||||
related_skills: [docx, pdf, officecli]
|
||||
---
|
||||
|
||||
# Markdown → DOCX (no pandoc / LibreOffice)
|
||||
|
||||
## When to use
|
||||
|
||||
Trigger on: "转成 Word / 导出 docx / Word 版本 / 白皮书 Word 版" — any request to turn a Markdown document (whitepaper, report, spec, README) into a `.docx`, especially on hosts where `pandoc`, `LibreOffice`/`soffice`, or `python-docx` are NOT installed.
|
||||
|
||||
The bundled `docx` skill covers creating `.docx` by hand and editing existing files, but assumes pandoc/soffice for reading/verification. When those binaries are missing, this skill is the fallback: parse Markdown with `marked` and emit `.docx` with `docx-js`, fully controlling CJK fonts.
|
||||
|
||||
## Prerequisites (no sudo)
|
||||
|
||||
```bash
|
||||
# node + npm are usually already present; registry reachable via `npm ping`
|
||||
mkdir -p /tmp/md2docx && cd /tmp/md2docx && npm init -y >/dev/null
|
||||
npm install docx marked adm-zip
|
||||
```
|
||||
|
||||
`docx` (docx-js) and `marked` generate/parse; `adm-zip` is for verification only.
|
||||
|
||||
## The approach
|
||||
|
||||
`marked.lexer(md)` returns a flat token stream. Walk it once and map each `type` to a docx-js element:
|
||||
|
||||
| marked token | docx-js output |
|
||||
|--------------|----------------|
|
||||
| `heading` | `Paragraph({ heading: HeadingLevel.HEADING_<depth> })` |
|
||||
| `paragraph` | `Paragraph` from inline runs |
|
||||
| `code` | single `Paragraph` — one `TextRun` per line joined with `new TextRun({ break: 1 })`, gray `shading` + box `border` |
|
||||
| `table` | `Table` — `columnWidths` + per-cell `width` (both DXA), header row shaded + bold |
|
||||
| `list` | bullet (`•`) / numbered (`N.`) paragraphs with `indent` + `hanging` |
|
||||
| `blockquote` | indented paragraph with left border, italic gray |
|
||||
| `hr` | empty paragraph with bottom border (NOT a table) |
|
||||
|
||||
A complete, tested converter is shipped at `scripts/convert_md_to_docx.js`. Run:
|
||||
|
||||
```bash
|
||||
node scripts/convert_md_to_docx.js input.md output.docx [title]
|
||||
```
|
||||
|
||||
Optional 4th arg sets the Word `title` property (defaults to "Converted from Markdown" — pass the real doc title so the `.docx` metadata isn't a stale leftover). The converter auto-inserts a `PageBreak` before every H1 (after the first) so each `#` chapter starts on a fresh page — right for multi-chapter whitepapers and 投标文件 (bid documents); for continuous-flow docs, delete the `firstH1Seen` block in the main loop.
|
||||
|
||||
Inline runs are built recursively from `tok.tokens`, mapping `strong`/`em`/`codespan`/`link` to run formatting. `codespan` (`` `...` ``) renders as red Consolas — critical for documents full of `${param}$`, `sqlExe`, `load_xxx()`.
|
||||
|
||||
## CJK font handling (the main gotcha)
|
||||
|
||||
Word does NOT reliably auto-fallback Chinese glyphs. Set the **`eastAsia`** attribute on every run's `font`, alongside `ascii`/`hAnsi`:
|
||||
|
||||
```js
|
||||
const FONT_BODY = { ascii: "Times New Roman", hAnsi: "Times New Roman", eastAsia: "宋体" };
|
||||
const FONT_HEAD = { ascii: "Arial", hAnsi: "Arial", eastAsia: "微软雅黑" };
|
||||
const FONT_CODE = { ascii: "Consolas", hAnsi: "Consolas", eastAsia: "宋体" };
|
||||
// ...new TextRun({ text, font: FONT_BODY })
|
||||
```
|
||||
|
||||
Convention for Chinese technical docs: 标题 `微软雅黑` navy, 正文 `宋体`, 代码 `Consolas`. Omit `eastAsia` and the whole document renders in a Latin fallback font.
|
||||
|
||||
## docx-js footguns (reinforced from the bundled `docx` skill)
|
||||
|
||||
- **Tables need dual widths**: `columnWidths` on the `Table` AND `width` on every `TableCell`, both `WidthType.DXA` (PERCENTAGE breaks in Google Docs). Sum to the usable page width (A4 11906 DXA − 1440×2 margins = **9026 DXA**).
|
||||
- **Shading type is `ShadingType.CLEAR`, never `SOLID`** (SOLID renders black).
|
||||
- **No `\n` in a `TextRun`** — multi-line code blocks use `new TextRun({ break: 1 })` between line runs.
|
||||
- **`PageBreak` must live inside a `Paragraph`.**
|
||||
- Lists: either use a `numbering` config or prefix the marker manually (`• ` / `N. `); never assume a literal bullet is styled as a list marker.
|
||||
|
||||
## Verification without soffice
|
||||
|
||||
No LibreOffice available to render → verify by reading the OOXML directly with `adm-zip`:
|
||||
|
||||
```js
|
||||
const AdmZip = require("adm-zip");
|
||||
const z = new AdmZip("out.docx");
|
||||
const xml = z.readAsText("word/document.xml");
|
||||
(xml.match(/<w:tbl>/g) || []).length // table count
|
||||
(xml.match(/w:fill="F5F5F5"/g) || []).length // code blocks
|
||||
(xml.match(/w:fill="D9E2F3"/g) || []).length // header cells
|
||||
xml.includes("eastAsia") // CJK fonts present
|
||||
```
|
||||
|
||||
Spot-check key strings (proper nouns, code identifiers) with `xml.includes(...)` to catch content loss. The docx-js `Packer.toBuffer` output is well-formed OOXML — adm-zip reading `word/document.xml` is sufficient structural proof of validity in the absence of a renderer.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **`require('docx/package.json')` throws** `ERR_PACKAGE_PATH_NOT_EXPORTED` — the `exports` map blocks it. Verify install with `typeof require('docx').Document === 'function'` instead.
|
||||
- **Don't reach for pandoc/soffice installs** on sudo-less hosts as a first move — npm + docx-js is self-contained and faster to stand up.
|
||||
- Keep the `.md` source alongside the `.docx`; regenerate by rerunning the script after any content edit (single source of truth).
|
||||
@ -0,0 +1,244 @@
|
||||
// convert_md_to_docx.js — Markdown → .docx with CJK font support.
|
||||
// No pandoc / LibreOffice / python-docx required.
|
||||
//
|
||||
// Usage: node convert_md_to_docx.js input.md output.docx
|
||||
// Deps: npm install docx marked
|
||||
//
|
||||
// Verified 2026-08-17 on a sudo-less host (A4, 42 headings, 6 tables,
|
||||
// 15 code blocks, CJK fonts via eastAsia). Uses marked.lexer() token walk.
|
||||
|
||||
const fs = require('fs');
|
||||
const {
|
||||
Document, Packer, Paragraph, TextRun, HeadingLevel, Table, TableRow, TableCell,
|
||||
WidthType, AlignmentType, BorderStyle, ShadingType, PageBreak,
|
||||
} = require('docx');
|
||||
const { marked } = require('marked');
|
||||
|
||||
const SRC = process.argv[2];
|
||||
const OUT = process.argv[3];
|
||||
const TITLE = process.argv[4] || 'Converted from Markdown';
|
||||
if (!SRC || !OUT) {
|
||||
console.error('Usage: node convert_md_to_docx.js input.md output.docx [title]');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// ---- fonts (CJK via eastAsia) ----
|
||||
const FONT_BODY = { ascii: "Times New Roman", hAnsi: "Times New Roman", eastAsia: "宋体" };
|
||||
const FONT_HEAD = { ascii: "Arial", hAnsi: "Arial", eastAsia: "微软雅黑" };
|
||||
const FONT_CODE = { ascii: "Consolas", hAnsi: "Consolas", eastAsia: "宋体" };
|
||||
const NAVY = "1F3864";
|
||||
const GRAY = "595959";
|
||||
const CODE_BG = "F5F5F5";
|
||||
const CODE_BORDER = "CCCCCC";
|
||||
|
||||
const md = fs.readFileSync(SRC, 'utf8');
|
||||
const tokens = marked.lexer(md);
|
||||
|
||||
// A4 usable width in DXA (page 11906 - margins 1440*2 = 9026)
|
||||
const PAGE_W = 9026;
|
||||
|
||||
function cellWidths(n) {
|
||||
const w = Math.floor(PAGE_W / n);
|
||||
const arr = new Array(n).fill(w);
|
||||
arr[n - 1] = PAGE_W - w * (n - 1); // last cell absorbs remainder
|
||||
return arr;
|
||||
}
|
||||
|
||||
// ---- inline run builder ----
|
||||
function inlineRuns(toks, opts = {}) {
|
||||
const out = [];
|
||||
const font = opts.font || FONT_BODY;
|
||||
const size = opts.size || 21; // half-points (10.5pt body)
|
||||
const color = opts.color;
|
||||
(toks || []).forEach(t => {
|
||||
switch (t.type) {
|
||||
case 'text':
|
||||
if (t.text) out.push(new TextRun({ text: t.text, font, size, color }));
|
||||
break;
|
||||
case 'strong':
|
||||
out.push(...inlineRuns(t.tokens, { ...opts, bold: true }));
|
||||
break;
|
||||
case 'em':
|
||||
out.push(...inlineRuns(t.tokens, { ...opts, italics: true }));
|
||||
break;
|
||||
case 'codespan':
|
||||
out.push(new TextRun({ text: t.text, font: FONT_CODE, size: size - 1, color: "C7254E" }));
|
||||
break;
|
||||
case 'del':
|
||||
out.push(...inlineRuns(t.tokens, { ...opts, strike: true }));
|
||||
break;
|
||||
case 'link':
|
||||
out.push(...inlineRuns(t.tokens, { ...opts, color: "0563C1", underline: {} }));
|
||||
break;
|
||||
case 'html':
|
||||
case 'br':
|
||||
break;
|
||||
default:
|
||||
if (t.tokens) out.push(...inlineRuns(t.tokens, opts));
|
||||
else if (t.text) out.push(new TextRun({ text: t.text, font, size, color }));
|
||||
}
|
||||
});
|
||||
if (opts.bold || opts.italics || opts.strike) {
|
||||
out.forEach(r => {
|
||||
if (opts.bold) r.bold = () => true;
|
||||
if (opts.italics) r.italics = () => true;
|
||||
if (opts.strike) r.strike = () => true;
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function plainText(toks) {
|
||||
let s = '';
|
||||
(toks || []).forEach(t => {
|
||||
if (t.type === 'text' || t.type === 'codespan') s += t.text || '';
|
||||
else if (t.tokens) s += plainText(t.tokens);
|
||||
});
|
||||
return s;
|
||||
}
|
||||
|
||||
// ---- block handlers ----
|
||||
function headingPara(tok) {
|
||||
const lvl = tok.depth;
|
||||
const isTitle = lvl === 1;
|
||||
const size = isTitle ? 36 : lvl === 2 ? 28 : 24;
|
||||
const map = { 1: HeadingLevel.HEADING_1, 2: HeadingLevel.HEADING_2, 3: HeadingLevel.HEADING_3,
|
||||
4: HeadingLevel.HEADING_4, 5: HeadingLevel.HEADING_5, 6: HeadingLevel.HEADING_6 };
|
||||
return new Paragraph({
|
||||
heading: map[lvl],
|
||||
alignment: isTitle ? AlignmentType.CENTER : AlignmentType.LEFT,
|
||||
spacing: { before: isTitle ? 0 : 240, after: isTitle ? 240 : 120 },
|
||||
children: inlineRuns(tok.tokens, { font: FONT_HEAD, size, color: NAVY, bold: true }),
|
||||
});
|
||||
}
|
||||
|
||||
function paragraphPara(tok) {
|
||||
const children = inlineRuns(tok.tokens || []);
|
||||
if (!children.length) return new Paragraph({ children: [] });
|
||||
return new Paragraph({ spacing: { after: 120, line: 360 }, children });
|
||||
}
|
||||
|
||||
function codePara(tok) {
|
||||
const lines = (tok.text || '').replace(/\n$/, '').split('\n');
|
||||
const children = [];
|
||||
lines.forEach((line, i) => {
|
||||
if (i > 0) children.push(new TextRun({ break: 1 }));
|
||||
children.push(new TextRun({ text: line, font: FONT_CODE, size: 18 }));
|
||||
});
|
||||
const border = { style: BorderStyle.SINGLE, size: 4, color: CODE_BORDER };
|
||||
return new Paragraph({
|
||||
spacing: { before: 80, after: 120 },
|
||||
shading: { type: ShadingType.CLEAR, fill: CODE_BG },
|
||||
border: { top: border, bottom: border, left: border, right: border },
|
||||
children,
|
||||
});
|
||||
}
|
||||
|
||||
function tableBlock(tok) {
|
||||
const headerCells = tok.header || [];
|
||||
const ncols = headerCells.length || 1;
|
||||
const widths = cellWidths(ncols);
|
||||
const rows = [];
|
||||
|
||||
const buildRow = (cells, isHeader) => new TableRow({
|
||||
tableHeader: isHeader,
|
||||
children: cells.map((c, i) => new TableCell({
|
||||
width: { size: widths[i], type: WidthType.DXA },
|
||||
shading: isHeader ? { type: ShadingType.CLEAR, fill: "D9E2F3" } : undefined,
|
||||
children: [new Paragraph({
|
||||
spacing: { after: 0 },
|
||||
children: [new TextRun({
|
||||
text: plainText(c.tokens || []), font: FONT_BODY, size: 20,
|
||||
bold: isHeader, color: isHeader ? NAVY : undefined,
|
||||
})],
|
||||
})],
|
||||
})),
|
||||
});
|
||||
|
||||
rows.push(buildRow(headerCells, true));
|
||||
(tok.rows || []).forEach(r => rows.push(buildRow(r, false)));
|
||||
|
||||
const b = { style: BorderStyle.SINGLE, size: 4, color: "BFBFBF" };
|
||||
return new Table({
|
||||
width: { size: PAGE_W, type: WidthType.DXA },
|
||||
columnWidths: widths,
|
||||
borders: { top: b, bottom: b, left: b, right: b,
|
||||
insideHorizontal: b, insideVertical: b },
|
||||
rows,
|
||||
});
|
||||
}
|
||||
|
||||
function listBlock(tok, ordered) {
|
||||
const paras = [];
|
||||
const walk = (items, depth) => {
|
||||
items.forEach((item, idx) => {
|
||||
const marker = ordered ? `${idx + 1}. ` : '• ';
|
||||
const indentLeft = 360 + depth * 360;
|
||||
item.tokens.forEach(bt => {
|
||||
if (bt.type === 'list') { walk(bt.items, depth + 1); return; }
|
||||
if (bt.type === 'text' || bt.type === 'paragraph' || bt.type === 'space') {
|
||||
const children = inlineRuns(bt.tokens || []);
|
||||
children.unshift(new TextRun({ text: marker, font: FONT_BODY, size: 21, bold: ordered }));
|
||||
paras.push(new Paragraph({
|
||||
spacing: { after: 40, line: 340 },
|
||||
indent: { left: indentLeft, hanging: 240 },
|
||||
children,
|
||||
}));
|
||||
}
|
||||
});
|
||||
});
|
||||
};
|
||||
walk(tok.items, 0);
|
||||
return paras;
|
||||
}
|
||||
|
||||
function blockquotePara(tok) {
|
||||
return new Paragraph({
|
||||
spacing: { after: 120 },
|
||||
indent: { left: 360 },
|
||||
border: { left: { style: BorderStyle.SINGLE, size: 12, color: NAVY } },
|
||||
children: [new TextRun({ text: plainText(tok.tokens || []), font: FONT_BODY, size: 21, italics: true, color: GRAY })],
|
||||
});
|
||||
}
|
||||
|
||||
function hrPara() {
|
||||
return new Paragraph({
|
||||
spacing: { before: 120, after: 120 },
|
||||
border: { bottom: { style: BorderStyle.SINGLE, size: 6, color: "BFBFBF" } },
|
||||
children: [],
|
||||
});
|
||||
}
|
||||
|
||||
// ---- main loop ----
|
||||
const children = [];
|
||||
let firstH1Seen = false;
|
||||
for (const tok of tokens) {
|
||||
switch (tok.type) {
|
||||
case 'heading':
|
||||
if (tok.depth === 1 && firstH1Seen) {
|
||||
children.push(new Paragraph({ children: [new PageBreak()] }));
|
||||
}
|
||||
if (tok.depth === 1) firstH1Seen = true;
|
||||
children.push(headingPara(tok));
|
||||
break;
|
||||
case 'paragraph': children.push(paragraphPara(tok)); break;
|
||||
case 'code': children.push(codePara(tok)); break;
|
||||
case 'table': children.push(tableBlock(tok)); break;
|
||||
case 'list': children.push(...listBlock(tok, !!tok.ordered)); break;
|
||||
case 'blockquote':children.push(blockquotePara(tok)); break;
|
||||
case 'hr': children.push(hrPara()); break;
|
||||
case 'space': break;
|
||||
default: break;
|
||||
}
|
||||
}
|
||||
|
||||
const doc = new Document({
|
||||
creator: 'Hermes Agent',
|
||||
title: TITLE,
|
||||
sections: [{ properties: {}, children }],
|
||||
});
|
||||
|
||||
Packer.toBuffer(doc).then(buf => {
|
||||
fs.writeFileSync(OUT, buf);
|
||||
console.log('WROTE', OUT, buf.length, 'bytes');
|
||||
});
|
||||
87
skills_library/all/office-text-extract/SKILL.md
Normal file
87
skills_library/all/office-text-extract/SKILL.md
Normal file
@ -0,0 +1,87 @@
|
||||
---
|
||||
name: office-text-extract
|
||||
description: "Extract text from .docx/.pptx/.xlsx/.pdf without pandoc."
|
||||
version: 2.0.0
|
||||
author: Hermes Agent
|
||||
license: Apache-2.0 (docx/pptx/xlsx engine from genspark-ai/genoffice)
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [office, docx, pptx, xlsx, pdf, text-extraction, headless, genoffice, pymupdf]
|
||||
related_skills: [docx, xlsx, powerpoint, pdf, markdown-to-docx, ocr-and-documents]
|
||||
---
|
||||
|
||||
# Office Text Extraction (headless)
|
||||
|
||||
从 `.docx` / `.pptx` / `.xlsx` / `.pdf` 提取纯文本,**无需 pandoc / LibreOffice**。统一入口一个命令覆盖四种格式:
|
||||
|
||||
- `.docx` / `.pptx` / `.xlsx` → GenOffice 引擎(github.com/genspark-ai/genoffice,Apache-2.0)的 `file-parse` + `docx-engine`,esbuild 打成单文件 Node CLI
|
||||
- `.pdf`(文本型)→ pymupdf(Python)
|
||||
- `.pdf`(扫描件/图片型,需 OCR)→ marker-pdf,见 `ocr-and-documents` 技能
|
||||
|
||||
## 何时用
|
||||
|
||||
- 需要**读取**已有 Office/PDF 文档内容(本机无 pandoc/LibreOffice 时尤其适用)
|
||||
- 给 AI 附加 Office/PDF 附件前先转文本
|
||||
- 快速抽取正文做检索、摘要、diff
|
||||
|
||||
## 工具位置
|
||||
|
||||
```
|
||||
scripts/office-extract # 统一入口(bash,按扩展名分发)★ 主入口
|
||||
scripts/genoffice-extract.mjs # docx/pptx/xlsx 引擎(Node,单文件,~1MB)
|
||||
scripts/extract_pdf.py # pdf 文本提取(pymupdf)
|
||||
scripts/build.sh # 可复现重建 Node 引擎(从 genoffice 源重新打包)
|
||||
```
|
||||
|
||||
要求:Node 18+(docx/pptx/xlsx)+ `pymupdf`(pdf)。
|
||||
|
||||
## 用法
|
||||
|
||||
```bash
|
||||
BIN=scripts/office-extract
|
||||
|
||||
# 统一入口,四种格式都行(纯文本到 stdout)
|
||||
"$BIN" /path/to/file.docx
|
||||
"$BIN" /path/to/file.pdf
|
||||
|
||||
# JSON 模式({ok,kind,ext,text|error}),便于程序化处理
|
||||
"$BIN" --json /path/to/file.xlsx
|
||||
```
|
||||
|
||||
也可以直接调底层:`node scripts/genoffice-extract.mjs <docx|pptx|xlsx>`、`python3 scripts/extract_pdf.py <pdf>`。
|
||||
|
||||
## 输出格式
|
||||
|
||||
- **docx**:标题转 `#`/`##`(按级别),列表项转 `-`,有序列表转 `1. 2. 3.`,表格行转 `cell1 | cell2 | ...`,正文段落原样。接近 Markdown。
|
||||
- **pptx**:每页一段 `## Slide N`,每段文字一行,按 slide 编号升序。
|
||||
- **xlsx**:每个工作表一段 `# SheetName`,行内单元格用 ` | ` 连接(空单元格占位对齐),inlineStr/sharedStrings/布尔/数字均正确处理。
|
||||
- **pdf**:每页文本用 `\n\n` 分隔,中文/多页正常。
|
||||
- 中文、ASCII 图、多列表格均无损(docx/pdf 实测通过)。
|
||||
|
||||
## 坑
|
||||
|
||||
1. **PDF 分两类**:
|
||||
- 文本型 PDF(有嵌入文本层)→ `extract_pdf.py`(pymupdf),即时返回。
|
||||
- 扫描件/图片型 PDF(无文本层)→ pymupdf 提取为空,脚本会提示改用 marker-pdf OCR(`ocr-and-documents` 技能的 `extract_marker.py`)。marker-pdf 需 ~5GB(PyTorch+模型)且首次运行下载模型。
|
||||
2. **只有提取,没有生成/编辑**:本工具只读文本。生成 docx 用 `docx` 技能(docx-js)、md→docx 用 `markdown-to-docx` 技能;编辑已有 docx 的批注/修订/图表需 `docx-engine` 的 patch 能力(未封装)。
|
||||
3. **Node 版本**:bundle target 是 node18,老环境可能不兼容。
|
||||
4. **大文件**:docx 解析会展开整个 OOXML;超大文件(>50MB)内存占用高。
|
||||
|
||||
## 重建 Node 引擎(genoffice 上游更新时)
|
||||
|
||||
```bash
|
||||
bash scripts/build.sh
|
||||
```
|
||||
|
||||
`build.sh` 从 genoffice main 分支重新下载 `file-parse` + `docx-engine` 源码、装依赖、esbuild 打包,覆盖 `scripts/genoffice-extract.mjs`。需网络可达 raw.githubusercontent.com(若被墙,先建 SOCKS5 隧道并设 `PROXY=socks5h://127.0.0.1:1080`)。
|
||||
|
||||
## 迁移到其他机器 / pipeline-app
|
||||
|
||||
- **docx/pptx/xlsx**:`genoffice-extract.mjs` 是纯 Node 单文件工具,**只需 Node 18+**,零依赖。拷过去 `node <path> <file>` 即用。
|
||||
- **pdf**:需要目标机 `pip install pymupdf`(~25MB,一条命令)。
|
||||
- **pdf OCR**:需要 `pip install marker-pdf` + ~5GB 磁盘(首次运行下载模型)。
|
||||
|
||||
三者迁移成本递增:docx/pptx/xlsx(零依赖)< pdf 文本(pip 装 pymupdf)< pdf OCR(5GB)。
|
||||
|
||||
- 若 pipeline-app 服务器有 Node:拷 `genoffice-extract.mjs` 即可在服务器 shell 用。
|
||||
- 若要 pipeline-app 的 DSPY/前端调用:另写一个薄 Sage 模块,用 `subprocess`/`child_process` 调对应 CLI 取 stdout(后续集成决策,本 skill 不涉及)。
|
||||
146
skills_library/all/office-text-extract/scripts/build.sh
Normal file
146
skills_library/all/office-text-extract/scripts/build.sh
Normal file
@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env bash
|
||||
# 从 genspark-ai/genoffice 重建 genoffice-extract.mjs
|
||||
# 用法: [PROXY=socks5h://127.0.0.1:1080] bash build.sh
|
||||
# 产物覆盖本目录下的 genoffice-extract.mjs
|
||||
set -euo pipefail
|
||||
|
||||
BASE="https://raw.githubusercontent.com/genspark-ai/genoffice/main"
|
||||
PROXY="${PROXY:-}"
|
||||
CURL=(curl -s --fail --connect-timeout 30)
|
||||
[ -n "$PROXY" ] && CURL+=(-x "$PROXY")
|
||||
|
||||
SKILL_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
# 仅需 file-parse + docx-engine 两个包的源码(PDF 未打包,pdf.ts 不会被 cli.ts 引入)
|
||||
FILES=(
|
||||
packages/docx-engine/package.json
|
||||
packages/docx-engine/src/blank.ts
|
||||
packages/docx-engine/src/chart.ts
|
||||
packages/docx-engine/src/generate.ts
|
||||
packages/docx-engine/src/index.ts
|
||||
packages/docx-engine/src/ink.ts
|
||||
packages/docx-engine/src/list-markers.ts
|
||||
packages/docx-engine/src/math.ts
|
||||
packages/docx-engine/src/metafile.ts
|
||||
packages/docx-engine/src/notes.ts
|
||||
packages/docx-engine/src/parse.ts
|
||||
packages/docx-engine/src/patch.ts
|
||||
packages/docx-engine/src/protection.ts
|
||||
packages/docx-engine/src/scan.ts
|
||||
packages/docx-engine/src/section.ts
|
||||
packages/docx-engine/src/sources.ts
|
||||
packages/docx-engine/src/symbol-fonts.ts
|
||||
packages/docx-engine/src/text-patch.ts
|
||||
packages/docx-engine/src/theme.ts
|
||||
packages/docx-engine/src/tiff.ts
|
||||
packages/docx-engine/src/types.ts
|
||||
packages/docx-engine/src/watermark.ts
|
||||
packages/docx-engine/src/xml-utils.ts
|
||||
packages/docx-engine/src/zip-load.ts
|
||||
packages/docx-engine/src/vendor/emf-converter/index.mjs
|
||||
packages/file-parse/package.json
|
||||
packages/file-parse/src/docx.ts
|
||||
packages/file-parse/src/index.ts
|
||||
packages/file-parse/src/parse.ts
|
||||
packages/file-parse/src/pdf.ts
|
||||
packages/file-parse/src/pptx.ts
|
||||
packages/file-parse/src/xlsx.ts
|
||||
)
|
||||
|
||||
echo "[1/4] 下载源码 ($((${#FILES[@]})) 个文件)..."
|
||||
for f in "${FILES[@]}"; do
|
||||
mkdir -p "$WORK/$(dirname "$f")"
|
||||
"${CURL[@]}" "$BASE/$f" -o "$WORK/$f"
|
||||
done
|
||||
|
||||
echo "[2/4] 写入口 cli.ts..."
|
||||
cat > "$WORK/cli.ts" <<'CLIEOF'
|
||||
#!/usr/bin/env node
|
||||
import { readFile } from 'node:fs/promises'
|
||||
import { extname } from 'node:path'
|
||||
import { docxToText } from './packages/file-parse/src/docx.ts'
|
||||
import { pptxToText } from './packages/file-parse/src/pptx.ts'
|
||||
import { xlsxToText } from './packages/file-parse/src/xlsx.ts'
|
||||
|
||||
const SUPPORTED = new Set(['docx', 'pptx', 'xlsx', 'pdf'])
|
||||
|
||||
function usage() {
|
||||
console.error(
|
||||
[
|
||||
'genoffice-extract — 提取 Office 文档纯文本(无 pandoc/LibreOffice 依赖)',
|
||||
'',
|
||||
'用法:',
|
||||
' genoffice-extract <file.docx|pptx|xlsx> 输出纯文本到 stdout',
|
||||
' genoffice-extract --json <file> 输出 {ok,kind,ext,text|error} JSON',
|
||||
'',
|
||||
'支持: .docx .pptx .xlsx (.pdf 未打包,请用 pdf/pymupdf 技能)',
|
||||
].join('\n'),
|
||||
)
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2)
|
||||
const json = args.includes('--json')
|
||||
const files = args.filter((a) => !a.startsWith('--'))
|
||||
if (files.length === 0) {
|
||||
usage()
|
||||
process.exit(2)
|
||||
}
|
||||
const file = files[0]
|
||||
const ext = extname(file).slice(1).toLowerCase()
|
||||
|
||||
if (!SUPPORTED.has(ext)) {
|
||||
const err = `Unsupported file type: .${ext || 'unknown'} (支持 docx/pptx/xlsx)`
|
||||
if (json) console.log(JSON.stringify({ ok: false, kind: 'unsupported', ext, error: err }))
|
||||
else console.error(err)
|
||||
process.exit(1)
|
||||
}
|
||||
if (ext === 'pdf') {
|
||||
const err = 'PDF 提取未打包(pdfjs-dist 体积/worker 复杂);请用 pdf 或 ocr-and-documents 技能'
|
||||
if (json) console.log(JSON.stringify({ ok: false, kind: 'unsupported', ext, error: err }))
|
||||
else console.error(err)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
let bytes
|
||||
try {
|
||||
bytes = await readFile(file)
|
||||
} catch (e) {
|
||||
const err = `无法读取文件: ${e instanceof Error ? e.message : String(e)}`
|
||||
if (json) console.log(JSON.stringify({ ok: false, kind: 'unsupported', ext, error: err }))
|
||||
else console.error(err)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
try {
|
||||
let text = ''
|
||||
if (ext === 'docx') text = await docxToText(bytes)
|
||||
else if (ext === 'pptx') text = await pptxToText(bytes)
|
||||
else if (ext === 'xlsx') text = await xlsxToText(bytes)
|
||||
if (json) console.log(JSON.stringify({ ok: true, kind: 'text', ext, text }))
|
||||
else console.log(text)
|
||||
} catch (e) {
|
||||
const err = e instanceof Error ? e.message : String(e)
|
||||
if (json) console.log(JSON.stringify({ ok: false, kind: 'text', ext, error: err }))
|
||||
else {
|
||||
console.error('提取失败: ' + err)
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main()
|
||||
CLIEOF
|
||||
|
||||
echo "[3/4] 安装依赖并 esbuild 打包..."
|
||||
cd "$WORK"
|
||||
npm init -y >/dev/null 2>&1
|
||||
npm install jszip fast-xml-parser utif2 esbuild >/dev/null 2>&1
|
||||
npx esbuild cli.ts --bundle --platform=node --format=esm --target=node18 \
|
||||
--outfile="$SKILL_DIR/genoffice-extract.mjs" \
|
||||
--alias:@genoffice/docx-engine="$WORK/packages/docx-engine/src/index.ts" 2>&1 | grep -v 'tsconfig.base.json' || true
|
||||
|
||||
chmod +x "$SKILL_DIR/genoffice-extract.mjs"
|
||||
echo "[4/4] 完成: $SKILL_DIR/genoffice-extract.mjs ($(du -h "$SKILL_DIR/genoffice-extract.mjs" | cut -f1))"
|
||||
67
skills_library/all/office-text-extract/scripts/extract_pdf.py
Executable file
67
skills_library/all/office-text-extract/scripts/extract_pdf.py
Executable file
@ -0,0 +1,67 @@
|
||||
#!/usr/bin/env python3
|
||||
"""从 PDF 提取纯文本(pymupdf)。输出格式与 genoffice-extract.mjs 对齐。
|
||||
|
||||
用法:
|
||||
extract_pdf.py <file.pdf> 纯文本到 stdout
|
||||
extract_pdf.py --json <file.pdf> {ok,kind,ext,text|error} JSON
|
||||
|
||||
注意: pymupdf 只提取文本型 PDF(有嵌入文本层)。扫描件/图片型 PDF 无文本层,
|
||||
提取为空 —— 此时提示改用 marker-pdf OCR。
|
||||
"""
|
||||
import sys
|
||||
import json
|
||||
import pymupdf
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = sys.argv[1:]
|
||||
as_json = '--json' in args
|
||||
files = [a for a in args if not a.startswith('--')]
|
||||
if not files:
|
||||
print('usage: extract_pdf.py [--json] <file.pdf>', file=sys.stderr)
|
||||
return 2
|
||||
|
||||
path = files[0]
|
||||
ext = 'pdf'
|
||||
|
||||
def emit(ok: bool, kind: str, text: str = '', error: str = '') -> int:
|
||||
if as_json:
|
||||
payload = {'ok': ok, 'kind': kind, 'ext': ext}
|
||||
if ok:
|
||||
payload['text'] = text
|
||||
else:
|
||||
payload['error'] = error
|
||||
print(json.dumps(payload, ensure_ascii=False))
|
||||
else:
|
||||
if ok:
|
||||
sys.stdout.write(text)
|
||||
else:
|
||||
print(error, file=sys.stderr)
|
||||
return 0 if ok else 1
|
||||
|
||||
try:
|
||||
doc = pymupdf.open(path)
|
||||
except Exception as e: # noqa: BLE001
|
||||
return emit(False, 'unsupported', error='无法打开 PDF: %s' % (e,))
|
||||
|
||||
try:
|
||||
pages = [page.get_text() for page in doc]
|
||||
text = '\n\n'.join(pages).strip()
|
||||
doc.close()
|
||||
if not text:
|
||||
return emit(
|
||||
False, 'text',
|
||||
error=('该 PDF 无文本层(可能是扫描件/图片型)。'
|
||||
'pymupdf 无法 OCR,请改用 marker-pdf OCR(见 ocr-and-documents 技能)。'),
|
||||
)
|
||||
return emit(True, 'text', text=text)
|
||||
except Exception as e: # noqa: BLE001
|
||||
try:
|
||||
doc.close()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
return emit(False, 'text', error='提取失败: %s' % (e,))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
29239
skills_library/all/office-text-extract/scripts/genoffice-extract.mjs
Executable file
29239
skills_library/all/office-text-extract/scripts/genoffice-extract.mjs
Executable file
File diff suppressed because one or more lines are too long
39
skills_library/all/office-text-extract/scripts/office-extract
Executable file
39
skills_library/all/office-text-extract/scripts/office-extract
Executable file
@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env bash
|
||||
# office-extract — 统一 Office/PDF 文本提取入口
|
||||
# 按扩展名分发: .docx/.pptx/.xlsx → genoffice-extract.mjs (Node)
|
||||
# .pdf → extract_pdf.py (pymupdf)
|
||||
# 用法: office-extract [--json] <file.docx|pptx|xlsx|pdf>
|
||||
set -euo pipefail
|
||||
|
||||
DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
if [ $# -eq 0 ]; then
|
||||
echo "office-extract — 统一 Office/PDF 文本提取" >&2
|
||||
echo "用法: office-extract [--json] <file.docx|pptx|xlsx|pdf>" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# 找最后一个非 -- 开头的参数作为文件路径
|
||||
file=""
|
||||
for a in "$@"; do
|
||||
case "$a" in
|
||||
--*) ;;
|
||||
*) file="$a" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
ext="${file##*.}"
|
||||
ext="$(printf '%s' "$ext" | tr '[:upper:]' '[:lower:]')"
|
||||
|
||||
case "$ext" in
|
||||
docx|pptx|xlsx)
|
||||
exec node "$DIR/genoffice-extract.mjs" "$@"
|
||||
;;
|
||||
pdf)
|
||||
exec python3 "$DIR/extract_pdf.py" "$@"
|
||||
;;
|
||||
*)
|
||||
echo "不支持的格式: .${ext:-未知}(支持 docx/pptx/xlsx/pdf)" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
Loading…
x
Reference in New Issue
Block a user