@xpert-ai/plugin-markitdown 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +152 -0
- package/dist/index.d.ts +4 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +34 -0
- package/dist/lib/markitdown-bootstrap.service.d.ts +25 -0
- package/dist/lib/markitdown-bootstrap.service.d.ts.map +1 -0
- package/dist/lib/markitdown-bootstrap.service.js +150 -0
- package/dist/lib/markitdown.middleware.d.ts +11 -0
- package/dist/lib/markitdown.middleware.d.ts.map +1 -0
- package/dist/lib/markitdown.middleware.js +94 -0
- package/dist/lib/markitdown.module.d.ts +7 -0
- package/dist/lib/markitdown.module.d.ts.map +1 -0
- package/dist/lib/markitdown.module.js +33 -0
- package/dist/lib/markitdown.types.d.ts +23 -0
- package/dist/lib/markitdown.types.d.ts.map +1 -0
- package/dist/lib/markitdown.types.js +55 -0
- package/dist/lib/markitdown.validator.d.ts +9 -0
- package/dist/lib/markitdown.validator.d.ts.map +1 -0
- package/dist/lib/markitdown.validator.js +64 -0
- package/dist/lib/skills/SKILL.md +120 -0
- package/dist/lib/skills/index.d.ts +13 -0
- package/dist/lib/skills/index.d.ts.map +1 -0
- package/dist/lib/skills/index.js +25 -0
- package/dist/lib/skills/references/supported-formats.md +149 -0
- package/dist/lib/types.d.ts +3 -0
- package/dist/lib/types.d.ts.map +1 -0
- package/dist/lib/types.js +16 -0
- package/package.json +41 -0
package/README.md
ADDED
|
@@ -0,0 +1,152 @@
|
|
|
1
|
+
# Xpert Plugin: MarkItDown Middleware
|
|
2
|
+
|
|
3
|
+
`@xpert-ai/plugin-markitdown` adds document-to-Markdown conversion support to Xpert agents by bootstrapping Microsoft's [MarkItDown](https://github.com/microsoft/markitdown) inside the agent sandbox and teaching the agent how to use it through `sandbox_shell`. The middleware prepares the sandbox runtime, injects MarkItDown usage guidance into the system prompt, and keeps the tool ready for on-demand file conversion.
|
|
4
|
+
|
|
5
|
+
## Key Features
|
|
6
|
+
|
|
7
|
+
- Bootstraps `markitdown` via pip inside the sandbox on first use.
|
|
8
|
+
- Supports a wide range of file formats: **PDF, DOCX, PPTX, XLSX, HTML, CSV, JSON, XML, ZIP, images (JPEG/PNG), audio (MP3/WAV), EPUB, RSS**, and more.
|
|
9
|
+
- Writes embedded skill assets (`SKILL.md` plus reference docs) into the sandbox for agent self-guidance.
|
|
10
|
+
- Appends a MarkItDown-specific system prompt so the agent uses `sandbox_shell` with `markitdown` for file conversion.
|
|
11
|
+
- Re-checks bootstrap state before MarkItDown shell commands and re-installs automatically if the sandbox container was reset.
|
|
12
|
+
- Validates agent drafts and warns when sandbox support or `SandboxShell` is missing.
|
|
13
|
+
- Configurable pip extras (`all`, `ocr`, `az-doc-intel`) to control which optional dependencies are installed.
|
|
14
|
+
|
|
15
|
+
## Quick Start
|
|
16
|
+
|
|
17
|
+
1. **Register the Plugin**
|
|
18
|
+
Start Xpert with the package in your plugin list:
|
|
19
|
+
```sh
|
|
20
|
+
PLUGINS=@xpert-ai/plugin-markitdown
|
|
21
|
+
```
|
|
22
|
+
The plugin registers the global `MarkItDownPluginModule`.
|
|
23
|
+
|
|
24
|
+
2. **Enable Sandbox Support**
|
|
25
|
+
Turn on the agent sandbox feature for the team/agent that will perform file conversion.
|
|
26
|
+
|
|
27
|
+
3. **Add `SandboxShell` on the Same Agent**
|
|
28
|
+
This middleware relies on the `sandbox_shell` tool exposed by the `SandboxShell` middleware.
|
|
29
|
+
|
|
30
|
+
4. **Add the MarkItDown Middleware**
|
|
31
|
+
In the Xpert console (or agent definition), add a middleware entry with strategy `MarkItDownSkill`.
|
|
32
|
+
|
|
33
|
+
5. **Optionally Configure the Bootstrap Behavior**
|
|
34
|
+
Example middleware block:
|
|
35
|
+
```json
|
|
36
|
+
{
|
|
37
|
+
"type": "MarkItDownSkill",
|
|
38
|
+
"options": {
|
|
39
|
+
"version": "latest",
|
|
40
|
+
"extras": "all",
|
|
41
|
+
"skillsDir": "/workspace/.xpert/skills/markitdown"
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
6. **Ask the Agent to Convert Files**
|
|
47
|
+
Once configured, the agent can convert documents to Markdown:
|
|
48
|
+
```
|
|
49
|
+
User: Please convert the uploaded report.pdf to Markdown.
|
|
50
|
+
Agent: (runs `markitdown report.pdf > report.md` via sandbox_shell)
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Configuration
|
|
54
|
+
|
|
55
|
+
| Field | Type | Description | Default |
|
|
56
|
+
| ----- | ---- | ----------- | ------- |
|
|
57
|
+
| `version` | string | Version of `markitdown` to install via pip in the sandbox. | `"latest"` |
|
|
58
|
+
| `extras` | string | Python extras to install (e.g. `"all"`, `"ocr"`, `"az-doc-intel"`). Use `"all"` for full functionality. | `"all"` |
|
|
59
|
+
| `skillsDir` | string | Path inside the sandbox where `SKILL.md` and reference files are written. | `"/workspace/.xpert/skills/markitdown"` |
|
|
60
|
+
|
|
61
|
+
## Supported File Formats
|
|
62
|
+
|
|
63
|
+
| Format | Extensions | Notes |
|
|
64
|
+
|--------|-----------|-------|
|
|
65
|
+
| PDF | `.pdf` | Text extraction, layout preservation |
|
|
66
|
+
| Word | `.docx` | Full structure: headings, tables, lists, images |
|
|
67
|
+
| PowerPoint | `.pptx` | Slides, speaker notes, embedded content |
|
|
68
|
+
| Excel | `.xlsx` | Sheets as Markdown tables |
|
|
69
|
+
| HTML | `.html`, `.htm` | Web page content extraction |
|
|
70
|
+
| CSV | `.csv` | Converted to Markdown tables |
|
|
71
|
+
| JSON | `.json` | Structured representation |
|
|
72
|
+
| XML | `.xml` | Structured representation |
|
|
73
|
+
| Images | `.jpg`, `.jpeg`, `.png` | EXIF metadata, OCR (with extras) |
|
|
74
|
+
| Audio | `.mp3`, `.wav` | Speech-to-text transcription |
|
|
75
|
+
| ZIP | `.zip` | Recursively converts contained files |
|
|
76
|
+
| EPUB | `.epub` | Book content extraction |
|
|
77
|
+
| RSS/Atom | feeds | Feed entry extraction |
|
|
78
|
+
|
|
79
|
+
## Runtime Behavior
|
|
80
|
+
|
|
81
|
+
- On first use, the middleware checks `/workspace/.xpert/.markitdown-bootstrap.json` to determine whether the sandbox is already bootstrapped.
|
|
82
|
+
- If bootstrap is missing or outdated, it verifies Python/pip availability, installs `markitdown[<extras>]` via pip, writes skill assets, and refreshes the stamp file.
|
|
83
|
+
- The middleware appends a system prompt that tells the agent to:
|
|
84
|
+
- use `markitdown` for file-to-Markdown conversion
|
|
85
|
+
- read the sandbox skill file before first use
|
|
86
|
+
- redirect output to a file with `>` for saving results
|
|
87
|
+
- inspect output carefully and summarize results
|
|
88
|
+
- When the agent calls `sandbox_shell` with a `markitdown` command, the middleware ensures bootstrap has completed before the command runs.
|
|
89
|
+
- Non-MarkItDown `sandbox_shell` commands are passed through unchanged.
|
|
90
|
+
|
|
91
|
+
## Validation Rules
|
|
92
|
+
|
|
93
|
+
The plugin contributes draft validation warnings in Xpert when:
|
|
94
|
+
|
|
95
|
+
- the agent uses `MarkItDownSkill` but sandbox support is disabled
|
|
96
|
+
- the agent uses `MarkItDownSkill` without `SandboxShell` on the same agent
|
|
97
|
+
|
|
98
|
+
## Sandbox Assets
|
|
99
|
+
|
|
100
|
+
During bootstrap, the plugin writes the following assets into the sandbox:
|
|
101
|
+
|
|
102
|
+
- `SKILL.md` — comprehensive command reference with quick start, CLI usage, batch patterns, and advanced features
|
|
103
|
+
- `references/supported-formats.md` — detailed per-format conversion guidance
|
|
104
|
+
- a bootstrap stamp file used to avoid unnecessary reinstallation
|
|
105
|
+
|
|
106
|
+
## Configuration Precedence
|
|
107
|
+
|
|
108
|
+
Configuration is resolved in this order, from lowest to highest priority:
|
|
109
|
+
|
|
110
|
+
1. Built-in defaults
|
|
111
|
+
2. Environment variables:
|
|
112
|
+
- `MARKITDOWN_VERSION`
|
|
113
|
+
- `MARKITDOWN_SKILLS_DIR`
|
|
114
|
+
- `MARKITDOWN_EXTRAS`
|
|
115
|
+
3. Plugin-level config resolved by the host plugin config resolver
|
|
116
|
+
4. Middleware `options`
|
|
117
|
+
|
|
118
|
+
## Advanced Features
|
|
119
|
+
|
|
120
|
+
### LLM-based Image Description
|
|
121
|
+
|
|
122
|
+
If `OPENAI_API_KEY` is set in the sandbox environment, MarkItDown can use an LLM to generate descriptions for images embedded in documents:
|
|
123
|
+
|
|
124
|
+
```bash
|
|
125
|
+
export OPENAI_API_KEY="your-key"
|
|
126
|
+
markitdown photo.jpg
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
### Azure Document Intelligence
|
|
130
|
+
|
|
131
|
+
For enhanced PDF/image processing with Azure Document Intelligence:
|
|
132
|
+
|
|
133
|
+
```bash
|
|
134
|
+
export AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT="https://your-endpoint.cognitiveservices.azure.com/"
|
|
135
|
+
export AZURE_DOCUMENT_INTELLIGENCE_KEY="your-key"
|
|
136
|
+
markitdown --use-docintel complex-document.pdf
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
To use this feature, set `extras` to `"az-doc-intel"` or `"all"` in the middleware configuration.
|
|
140
|
+
|
|
141
|
+
## Development & Testing
|
|
142
|
+
|
|
143
|
+
```bash
|
|
144
|
+
pnpm nx build @xpert-ai/plugin-markitdown
|
|
145
|
+
pnpm nx test @xpert-ai/plugin-markitdown
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
TypeScript output is emitted to `middlewares/markitdown/dist`.
|
|
149
|
+
|
|
150
|
+
## License
|
|
151
|
+
|
|
152
|
+
This project follows the [AGPL-3.0 License](../../../LICENSE) located at the repository root.
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,sBAAsB,CAAA;AAWvD,QAAA,MAAM,MAAM,EAAE,WAyBb,CAAA;AAED,eAAe,MAAM,CAAA"}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
import { readFileSync } from 'node:fs';
|
|
2
|
+
import { dirname, join } from 'node:path';
|
|
3
|
+
import { fileURLToPath } from 'node:url';
|
|
4
|
+
import { MarkItDownPluginModule } from './lib/markitdown.module.js';
|
|
5
|
+
import { MarkItDownIcon } from './lib/types.js';
|
|
6
|
+
const moduleDir = dirname(fileURLToPath(import.meta.url));
|
|
7
|
+
const packageJson = JSON.parse(readFileSync(join(moduleDir, '../package.json'), 'utf8'));
|
|
8
|
+
const plugin = {
|
|
9
|
+
meta: {
|
|
10
|
+
name: packageJson.name,
|
|
11
|
+
version: packageJson.version,
|
|
12
|
+
level: 'organization',
|
|
13
|
+
category: 'middleware',
|
|
14
|
+
icon: {
|
|
15
|
+
type: 'svg',
|
|
16
|
+
value: MarkItDownIcon
|
|
17
|
+
},
|
|
18
|
+
displayName: 'MarkItDown',
|
|
19
|
+
description: 'Installs Microsoft MarkItDown in the sandbox and provides skills for converting files (PDF, DOCX, PPTX, images, audio, etc.) to Markdown.',
|
|
20
|
+
keywords: ['markitdown', 'markdown', 'pdf', 'docx', 'conversion', 'document'],
|
|
21
|
+
author: 'XpertAI Team'
|
|
22
|
+
},
|
|
23
|
+
register(ctx) {
|
|
24
|
+
ctx.logger.log('register markitdown plugin');
|
|
25
|
+
return { module: MarkItDownPluginModule, global: true };
|
|
26
|
+
},
|
|
27
|
+
async onStart(ctx) {
|
|
28
|
+
ctx.logger.log('markitdown plugin started');
|
|
29
|
+
},
|
|
30
|
+
async onStop(ctx) {
|
|
31
|
+
ctx.logger.log('markitdown plugin stopped');
|
|
32
|
+
}
|
|
33
|
+
};
|
|
34
|
+
export default plugin;
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
import { BaseSandbox, type IPluginConfigResolver } from '@xpert-ai/plugin-sdk';
|
|
2
|
+
import { MarkItDownConfig } from './markitdown.types.js';
|
|
3
|
+
type MarkItDownBootstrapBackend = Pick<BaseSandbox, 'execute' | 'uploadFiles'>;
|
|
4
|
+
export declare class MarkItDownBootstrapService {
|
|
5
|
+
private readonly pluginConfigResolver?;
|
|
6
|
+
constructor(pluginConfigResolver?: IPluginConfigResolver);
|
|
7
|
+
resolveConfig(config?: Partial<MarkItDownConfig>): MarkItDownConfig;
|
|
8
|
+
getStampPath(): string;
|
|
9
|
+
buildSystemPrompt(config?: {
|
|
10
|
+
version?: string;
|
|
11
|
+
skillsDir?: string;
|
|
12
|
+
extras?: string;
|
|
13
|
+
}): string;
|
|
14
|
+
isMarkItDownCommand(command: string): boolean;
|
|
15
|
+
ensureBootstrap(backend: MarkItDownBootstrapBackend, config?: {
|
|
16
|
+
version?: string;
|
|
17
|
+
skillsDir?: string;
|
|
18
|
+
extras?: string;
|
|
19
|
+
}): Promise<import("@xpert-ai/plugin-sdk").ExecuteResponse>;
|
|
20
|
+
private getBootstrapAssets;
|
|
21
|
+
private writeStamp;
|
|
22
|
+
private writeAssets;
|
|
23
|
+
}
|
|
24
|
+
export {};
|
|
25
|
+
//# sourceMappingURL=markitdown-bootstrap.service.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"markitdown-bootstrap.service.d.ts","sourceRoot":"","sources":["../../src/lib/markitdown-bootstrap.service.ts"],"names":[],"mappings":"AAEA,OAAO,EACL,WAAW,EACX,KAAK,qBAAqB,EAE3B,MAAM,sBAAsB,CAAA;AAE7B,OAAO,EAKL,gBAAgB,EAEjB,MAAM,uBAAuB,CAAA;AAI9B,KAAK,0BAA0B,GAAG,IAAI,CAAC,WAAW,EAAE,SAAS,GAAG,aAAa,CAAC,CAAA;AAQ9E,qBACa,0BAA0B;IAInC,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAC;gBAArB,oBAAoB,CAAC,EAAE,qBAAqB;IAG/D,aAAa,CAAC,MAAM,CAAC,EAAE,OAAO,CAAC,gBAAgB,CAAC,GAAG,gBAAgB;IAmBnE,YAAY,IAAI,MAAM;IAItB,iBAAiB,CAAC,MAAM;;;;KAAuB,GAAG,MAAM;IAwBxD,mBAAmB,CAAC,OAAO,EAAE,MAAM,GAAG,OAAO;IAOvC,eAAe,CAAC,OAAO,EAAE,0BAA0B,EAAE,MAAM;;;;KAAuB;IA2DxF,OAAO,CAAC,kBAAkB;YAIZ,UAAU;YAaV,WAAW;CA4B1B"}
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
import { __decorate, __metadata, __param } from "tslib";
|
|
2
|
+
import { Buffer } from 'node:buffer';
|
|
3
|
+
import { posix as path } from 'node:path';
|
|
4
|
+
import { PLUGIN_CONFIG_RESOLVER_TOKEN } from '@xpert-ai/plugin-sdk';
|
|
5
|
+
import { Inject, Injectable, Optional } from '@nestjs/common';
|
|
6
|
+
import { DEFAULT_MARKITDOWN_VERSION, DEFAULT_MARKITDOWN_SKILLS_DIR, DEFAULT_MARKITDOWN_STAMP_PATH, MARKITDOWN_BOOTSTRAP_SCHEMA_VERSION, MarkItDownConfigSchema } from './markitdown.types.js';
|
|
7
|
+
import { MarkItDownPluginName } from './types.js';
|
|
8
|
+
import { getSkillAssets } from './skills/index.js';
|
|
9
|
+
let MarkItDownBootstrapService = class MarkItDownBootstrapService {
|
|
10
|
+
constructor(pluginConfigResolver) {
|
|
11
|
+
this.pluginConfigResolver = pluginConfigResolver;
|
|
12
|
+
}
|
|
13
|
+
resolveConfig(config) {
|
|
14
|
+
const defaults = {
|
|
15
|
+
version: process.env['MARKITDOWN_VERSION'] || DEFAULT_MARKITDOWN_VERSION,
|
|
16
|
+
skillsDir: process.env['MARKITDOWN_SKILLS_DIR'] || DEFAULT_MARKITDOWN_SKILLS_DIR,
|
|
17
|
+
extras: process.env['MARKITDOWN_EXTRAS'] || 'all'
|
|
18
|
+
};
|
|
19
|
+
const pluginConfig = this.pluginConfigResolver?.resolve(MarkItDownPluginName, {
|
|
20
|
+
defaults
|
|
21
|
+
}) ?? defaults;
|
|
22
|
+
const middlewareConfig = MarkItDownConfigSchema.partial().parse(config ?? {});
|
|
23
|
+
return MarkItDownConfigSchema.parse({
|
|
24
|
+
...defaults,
|
|
25
|
+
...pluginConfig,
|
|
26
|
+
...middlewareConfig
|
|
27
|
+
});
|
|
28
|
+
}
|
|
29
|
+
getStampPath() {
|
|
30
|
+
return DEFAULT_MARKITDOWN_STAMP_PATH;
|
|
31
|
+
}
|
|
32
|
+
buildSystemPrompt(config = this.resolveConfig()) {
|
|
33
|
+
return [
|
|
34
|
+
'The `markitdown` command (from Microsoft markitdown) is installed in the sandbox via pip.',
|
|
35
|
+
'IMPORTANT: Use the `markitdown` command to convert files to Markdown. It supports PDF, DOCX, PPTX, XLSX, HTML, CSV, JSON, XML, ZIP, images (JPEG/PNG with EXIF and OCR), audio (MP3/WAV with speech-to-text), RSS feeds, and more.',
|
|
36
|
+
'Always run file conversion via the `sandbox_shell` tool using the `markitdown` command.',
|
|
37
|
+
`Before your first use, read the skill file at \`${config.skillsDir}/SKILL.md\` with \`cat ${config.skillsDir}/SKILL.md\` to learn all available options and usage patterns.`,
|
|
38
|
+
'',
|
|
39
|
+
'QUICK USAGE:',
|
|
40
|
+
'- Convert a file: `markitdown path/to/file.pdf`',
|
|
41
|
+
'- Convert a URL: `markitdown https://example.com/page.html`',
|
|
42
|
+
'- Save output: `markitdown path/to/file.docx > output.md`',
|
|
43
|
+
'- Piped input: `cat file.html | markitdown`',
|
|
44
|
+
'',
|
|
45
|
+
'GUIDELINES:',
|
|
46
|
+
'- Output goes to stdout by default. Redirect with `>` to save to a file.',
|
|
47
|
+
'- For large files, the conversion may take a moment. The default timeout should be sufficient for most files.',
|
|
48
|
+
'- For images, markitdown extracts EXIF metadata and can do OCR if configured.',
|
|
49
|
+
'- For audio files, markitdown performs speech-to-text transcription.',
|
|
50
|
+
'- Inspect the output carefully and summarize results back to the user.',
|
|
51
|
+
'',
|
|
52
|
+
`For detailed format-specific guidance, read \`${config.skillsDir}/references/supported-formats.md\`.`
|
|
53
|
+
].join('\n');
|
|
54
|
+
}
|
|
55
|
+
isMarkItDownCommand(command) {
|
|
56
|
+
if (!command) {
|
|
57
|
+
return false;
|
|
58
|
+
}
|
|
59
|
+
return /\bmarkitdown\b/.test(command);
|
|
60
|
+
}
|
|
61
|
+
async ensureBootstrap(backend, config = this.resolveConfig()) {
|
|
62
|
+
if (!backend || typeof backend.execute !== 'function') {
|
|
63
|
+
throw new Error('Sandbox backend is not available for MarkItDown bootstrap.');
|
|
64
|
+
}
|
|
65
|
+
const stampPath = this.getStampPath();
|
|
66
|
+
const bootstrapAssets = this.getBootstrapAssets(config);
|
|
67
|
+
// Check stamp to see if already bootstrapped with same version
|
|
68
|
+
const stampCheck = await backend.execute(`cat ${shellQuote(stampPath)} 2>/dev/null || echo ''`);
|
|
69
|
+
const stampContent = stampCheck?.output?.trim() ?? '';
|
|
70
|
+
if (stampContent) {
|
|
71
|
+
try {
|
|
72
|
+
const stamp = JSON.parse(stampContent);
|
|
73
|
+
if (stamp.version === config.version) {
|
|
74
|
+
// Stamp matches, but verify the markitdown binary actually exists
|
|
75
|
+
const whichResult = await backend.execute('which markitdown 2>/dev/null');
|
|
76
|
+
if (whichResult?.exitCode === 0 && whichResult?.output?.trim()) {
|
|
77
|
+
if (stamp.bootstrapVersion !== MARKITDOWN_BOOTSTRAP_SCHEMA_VERSION) {
|
|
78
|
+
await this.writeAssets(backend, bootstrapAssets);
|
|
79
|
+
await this.writeStamp(backend, config.version);
|
|
80
|
+
}
|
|
81
|
+
return { output: 'already bootstrapped', exitCode: 0, truncated: false };
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
}
|
|
85
|
+
catch {
|
|
86
|
+
// stamp is corrupted, re-bootstrap
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
// 1. Check Python/pip availability
|
|
90
|
+
const pipCheck = await backend.execute('which pip3 2>/dev/null || which pip 2>/dev/null');
|
|
91
|
+
if (pipCheck?.exitCode !== 0 || !pipCheck?.output?.trim()) {
|
|
92
|
+
throw new Error('Python pip is not available in the sandbox. MarkItDown requires Python with pip to be pre-installed.');
|
|
93
|
+
}
|
|
94
|
+
const pipCmd = pipCheck.output.trim().split('\n')[0];
|
|
95
|
+
// 2. Install markitdown via pip
|
|
96
|
+
const versionSpec = config.version === 'latest' ? '' : `==${config.version}`;
|
|
97
|
+
const extrasSpec = config.extras ? `[${config.extras}]` : '';
|
|
98
|
+
const installCmd = `${pipCmd} install "markitdown${extrasSpec}${versionSpec}"`;
|
|
99
|
+
const installResult = await backend.execute(installCmd);
|
|
100
|
+
if (installResult?.exitCode !== 0) {
|
|
101
|
+
throw new Error(`MarkItDown install failed: ${installResult?.output || 'Unknown error'}`);
|
|
102
|
+
}
|
|
103
|
+
// 3. Upload skill files to sandbox
|
|
104
|
+
await this.writeAssets(backend, bootstrapAssets);
|
|
105
|
+
// 4. Write stamp file
|
|
106
|
+
await this.writeStamp(backend, config.version);
|
|
107
|
+
return installResult;
|
|
108
|
+
}
|
|
109
|
+
getBootstrapAssets(config) {
|
|
110
|
+
return getSkillAssets(config.skillsDir);
|
|
111
|
+
}
|
|
112
|
+
async writeStamp(backend, version) {
|
|
113
|
+
const stampPath = this.getStampPath();
|
|
114
|
+
const stampData = JSON.stringify({
|
|
115
|
+
tool: 'markitdown',
|
|
116
|
+
version,
|
|
117
|
+
bootstrapVersion: MARKITDOWN_BOOTSTRAP_SCHEMA_VERSION,
|
|
118
|
+
installedAt: new Date().toISOString()
|
|
119
|
+
});
|
|
120
|
+
await backend.execute(`mkdir -p ${shellQuote(path.dirname(stampPath))} && echo ${shellQuote(stampData)} > ${shellQuote(stampPath)}`);
|
|
121
|
+
}
|
|
122
|
+
async writeAssets(backend, assets) {
|
|
123
|
+
const canUploadDirectly = typeof backend.uploadFiles === 'function' && assets.every((asset) => !path.isAbsolute(asset.path));
|
|
124
|
+
if (canUploadDirectly) {
|
|
125
|
+
const results = await backend.uploadFiles(assets.map(({ path, content }) => [path, Buffer.from(content, 'utf8')]));
|
|
126
|
+
const failed = results?.filter((result) => result.error);
|
|
127
|
+
if (failed?.length) {
|
|
128
|
+
throw new Error(`Failed to write MarkItDown skill assets: ${failed.map((item) => item.path).join(', ')}`);
|
|
129
|
+
}
|
|
130
|
+
return;
|
|
131
|
+
}
|
|
132
|
+
for (const asset of assets) {
|
|
133
|
+
const dir = path.dirname(asset.path);
|
|
134
|
+
const result = await backend.execute(`mkdir -p ${shellQuote(dir)} && cat <<'__XPERT_MARKITDOWN_EOF__' > ${shellQuote(asset.path)}\n${asset.content}\n__XPERT_MARKITDOWN_EOF__`);
|
|
135
|
+
if (result?.exitCode !== 0) {
|
|
136
|
+
throw new Error(`Failed to write MarkItDown skill asset ${asset.path}: ${result?.output || 'Unknown error'}`);
|
|
137
|
+
}
|
|
138
|
+
}
|
|
139
|
+
}
|
|
140
|
+
};
|
|
141
|
+
MarkItDownBootstrapService = __decorate([
|
|
142
|
+
Injectable(),
|
|
143
|
+
__param(0, Optional()),
|
|
144
|
+
__param(0, Inject(PLUGIN_CONFIG_RESOLVER_TOKEN)),
|
|
145
|
+
__metadata("design:paramtypes", [Object])
|
|
146
|
+
], MarkItDownBootstrapService);
|
|
147
|
+
export { MarkItDownBootstrapService };
|
|
148
|
+
function shellQuote(value) {
|
|
149
|
+
return `'${value.replace(/'/g, `'\"'\"'`)}'`;
|
|
150
|
+
}
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
import { TAgentMiddlewareMeta } from '@metad/contracts';
|
|
2
|
+
import { AgentMiddleware, IAgentMiddlewareContext, IAgentMiddlewareStrategy } from '@xpert-ai/plugin-sdk';
|
|
3
|
+
import { MarkItDownBootstrapService } from './markitdown-bootstrap.service.js';
|
|
4
|
+
import { MarkItDownConfig } from './markitdown.types.js';
|
|
5
|
+
export declare class MarkItDownSkillMiddleware implements IAgentMiddlewareStrategy<Partial<MarkItDownConfig>> {
|
|
6
|
+
private readonly markitdownBootstrapService;
|
|
7
|
+
constructor(markitdownBootstrapService: MarkItDownBootstrapService);
|
|
8
|
+
meta: TAgentMiddlewareMeta;
|
|
9
|
+
createMiddleware(options: Partial<MarkItDownConfig>, _context: IAgentMiddlewareContext): AgentMiddleware;
|
|
10
|
+
}
|
|
11
|
+
//# sourceMappingURL=markitdown.middleware.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"markitdown.middleware.d.ts","sourceRoot":"","sources":["../../src/lib/markitdown.middleware.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAA;AAEvD,OAAO,EACL,eAAe,EAIf,uBAAuB,EACvB,wBAAwB,EAGzB,MAAM,sBAAsB,CAAA;AAC7B,OAAO,EAAE,0BAA0B,EAAE,MAAM,mCAAmC,CAAA;AAC9E,OAAO,EAEL,gBAAgB,EAEjB,MAAM,uBAAuB,CAAA;AAK9B,qBAEa,yBAA0B,YAAW,wBAAwB,CAAC,OAAO,CAAC,gBAAgB,CAAC,CAAC;IACvF,OAAO,CAAC,QAAQ,CAAC,0BAA0B;gBAA1B,0BAA0B,EAAE,0BAA0B;IAEnF,IAAI,EAAE,oBAAoB,CAezB;IAED,gBAAgB,CAAC,OAAO,EAAE,OAAO,CAAC,gBAAgB,CAAC,EAAE,QAAQ,EAAE,uBAAuB,GAAG,eAAe;CAkDzG"}
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
import { __decorate, __metadata } from "tslib";
|
|
2
|
+
import { SystemMessage } from '@langchain/core/messages';
|
|
3
|
+
import { Injectable } from '@nestjs/common';
|
|
4
|
+
import { AgentMiddlewareStrategy } from '@xpert-ai/plugin-sdk';
|
|
5
|
+
import { MarkItDownBootstrapService } from './markitdown-bootstrap.service.js';
|
|
6
|
+
import { MARKITDOWN_SKILL_MIDDLEWARE_NAME, MarkItDownConfigFormSchema } from './markitdown.types.js';
|
|
7
|
+
import { MarkItDownIcon } from './types.js';
|
|
8
|
+
const SANDBOX_SHELL_TOOL_NAME = 'sandbox_shell';
|
|
9
|
+
let MarkItDownSkillMiddleware = class MarkItDownSkillMiddleware {
|
|
10
|
+
constructor(markitdownBootstrapService) {
|
|
11
|
+
this.markitdownBootstrapService = markitdownBootstrapService;
|
|
12
|
+
this.meta = {
|
|
13
|
+
name: MARKITDOWN_SKILL_MIDDLEWARE_NAME,
|
|
14
|
+
label: {
|
|
15
|
+
en_US: 'MarkItDown Skill',
|
|
16
|
+
zh_Hans: 'MarkItDown 技能'
|
|
17
|
+
},
|
|
18
|
+
description: {
|
|
19
|
+
en_US: 'Installs Microsoft MarkItDown into the sandbox and teaches the agent to convert files (PDF, DOCX, PPTX, images, audio, etc.) to Markdown via sandbox_shell.',
|
|
20
|
+
zh_Hans: '将 Microsoft MarkItDown 安装到 sandbox 中,并教会智能体通过 sandbox_shell 将文件(PDF、DOCX、PPTX、图片、音频等)转换为 Markdown。'
|
|
21
|
+
},
|
|
22
|
+
icon: {
|
|
23
|
+
type: 'svg',
|
|
24
|
+
value: MarkItDownIcon
|
|
25
|
+
},
|
|
26
|
+
configSchema: MarkItDownConfigFormSchema
|
|
27
|
+
};
|
|
28
|
+
}
|
|
29
|
+
createMiddleware(options, _context) {
|
|
30
|
+
const config = this.markitdownBootstrapService.resolveConfig(options);
|
|
31
|
+
return {
|
|
32
|
+
name: MARKITDOWN_SKILL_MIDDLEWARE_NAME,
|
|
33
|
+
tools: [],
|
|
34
|
+
beforeAgent: async (_state, runtime) => {
|
|
35
|
+
const backend = getSandboxBackend(runtime);
|
|
36
|
+
await this.markitdownBootstrapService.ensureBootstrap(backend, config);
|
|
37
|
+
},
|
|
38
|
+
wrapModelCall: async (request, handler) => {
|
|
39
|
+
const backend = getSandboxBackend(request.runtime);
|
|
40
|
+
if (!backend) {
|
|
41
|
+
return handler(request);
|
|
42
|
+
}
|
|
43
|
+
const prompt = this.markitdownBootstrapService.buildSystemPrompt(config);
|
|
44
|
+
const baseContent = `${request.systemMessage?.content ?? ''}`.trim();
|
|
45
|
+
const content = [baseContent, prompt].filter(Boolean).join('\n\n');
|
|
46
|
+
return handler({
|
|
47
|
+
...request,
|
|
48
|
+
systemMessage: new SystemMessage({
|
|
49
|
+
content
|
|
50
|
+
})
|
|
51
|
+
});
|
|
52
|
+
},
|
|
53
|
+
wrapToolCall: async (request, handler) => {
|
|
54
|
+
if (!isSandboxShellTool(request.tool)) {
|
|
55
|
+
return handler(request);
|
|
56
|
+
}
|
|
57
|
+
const command = getSandboxShellCommand(request);
|
|
58
|
+
if (!this.markitdownBootstrapService.isMarkItDownCommand(command)) {
|
|
59
|
+
return handler(request);
|
|
60
|
+
}
|
|
61
|
+
// Ensure markitdown is installed before running the command
|
|
62
|
+
const backend = getSandboxBackend(request.runtime);
|
|
63
|
+
if (backend) {
|
|
64
|
+
await this.markitdownBootstrapService.ensureBootstrap(backend, config);
|
|
65
|
+
}
|
|
66
|
+
return handler(request);
|
|
67
|
+
}
|
|
68
|
+
};
|
|
69
|
+
}
|
|
70
|
+
};
|
|
71
|
+
MarkItDownSkillMiddleware = __decorate([
|
|
72
|
+
Injectable(),
|
|
73
|
+
AgentMiddlewareStrategy(MARKITDOWN_SKILL_MIDDLEWARE_NAME),
|
|
74
|
+
__metadata("design:paramtypes", [MarkItDownBootstrapService])
|
|
75
|
+
], MarkItDownSkillMiddleware);
|
|
76
|
+
export { MarkItDownSkillMiddleware };
|
|
77
|
+
function getSandboxBackend(runtime) {
|
|
78
|
+
const backend = runtime?.configurable?.sandbox?.backend;
|
|
79
|
+
if (backend && typeof backend.execute === 'function') {
|
|
80
|
+
return backend;
|
|
81
|
+
}
|
|
82
|
+
return null;
|
|
83
|
+
}
|
|
84
|
+
function isSandboxShellTool(tool) {
|
|
85
|
+
return tool?.name === SANDBOX_SHELL_TOOL_NAME;
|
|
86
|
+
}
|
|
87
|
+
function getSandboxShellCommand(request) {
|
|
88
|
+
const args = request.toolCall?.args;
|
|
89
|
+
if (!args || typeof args !== 'object') {
|
|
90
|
+
return '';
|
|
91
|
+
}
|
|
92
|
+
const command = args['command'];
|
|
93
|
+
return typeof command === 'string' ? command : '';
|
|
94
|
+
}
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
import { IOnPluginBootstrap, IOnPluginDestroy } from '@xpert-ai/plugin-sdk';
|
|
2
|
+
export declare class MarkItDownPluginModule implements IOnPluginBootstrap, IOnPluginDestroy {
|
|
3
|
+
private logEnabled;
|
|
4
|
+
onPluginBootstrap(): void | Promise<void>;
|
|
5
|
+
onPluginDestroy(): void | Promise<void>;
|
|
6
|
+
}
|
|
7
|
+
//# sourceMappingURL=markitdown.module.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"markitdown.module.d.ts","sourceRoot":"","sources":["../../src/lib/markitdown.module.ts"],"names":[],"mappings":"AAAA,OAAO,EAAqB,kBAAkB,EAAE,gBAAgB,EAAE,MAAM,sBAAsB,CAAA;AAM9F,qBAQa,sBAAuB,YAAW,kBAAkB,EAAE,gBAAgB;IACjF,OAAO,CAAC,UAAU,CAAO;IAEzB,iBAAiB,IAAI,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC;IAMzC,eAAe,IAAI,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC;CAKxC"}
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
var MarkItDownPluginModule_1;
|
|
2
|
+
import { __decorate } from "tslib";
|
|
3
|
+
import { XpertServerPlugin } from '@xpert-ai/plugin-sdk';
|
|
4
|
+
import chalk from 'chalk';
|
|
5
|
+
import { MarkItDownBootstrapService } from './markitdown-bootstrap.service.js';
|
|
6
|
+
import { MarkItDownSkillMiddleware } from './markitdown.middleware.js';
|
|
7
|
+
import { MarkItDownSkillValidator } from './markitdown.validator.js';
|
|
8
|
+
let MarkItDownPluginModule = MarkItDownPluginModule_1 = class MarkItDownPluginModule {
|
|
9
|
+
constructor() {
|
|
10
|
+
this.logEnabled = true;
|
|
11
|
+
}
|
|
12
|
+
onPluginBootstrap() {
|
|
13
|
+
if (this.logEnabled) {
|
|
14
|
+
console.log(chalk.green(`${MarkItDownPluginModule_1.name} is being bootstrapped...`));
|
|
15
|
+
}
|
|
16
|
+
}
|
|
17
|
+
onPluginDestroy() {
|
|
18
|
+
if (this.logEnabled) {
|
|
19
|
+
console.log(chalk.green(`${MarkItDownPluginModule_1.name} is being destroyed...`));
|
|
20
|
+
}
|
|
21
|
+
}
|
|
22
|
+
};
|
|
23
|
+
MarkItDownPluginModule = MarkItDownPluginModule_1 = __decorate([
|
|
24
|
+
XpertServerPlugin({
|
|
25
|
+
imports: [],
|
|
26
|
+
providers: [
|
|
27
|
+
MarkItDownBootstrapService,
|
|
28
|
+
MarkItDownSkillMiddleware,
|
|
29
|
+
MarkItDownSkillValidator
|
|
30
|
+
]
|
|
31
|
+
})
|
|
32
|
+
], MarkItDownPluginModule);
|
|
33
|
+
export { MarkItDownPluginModule };
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
import { JsonSchemaObjectType } from '@metad/contracts';
|
|
2
|
+
import { z } from 'zod';
|
|
3
|
+
export declare const MARKITDOWN_SKILL_MIDDLEWARE_NAME = "MarkItDownSkill";
|
|
4
|
+
export declare const DEFAULT_MARKITDOWN_VERSION = "latest";
|
|
5
|
+
export declare const DEFAULT_MARKITDOWN_SKILLS_DIR = "/workspace/.xpert/skills/markitdown";
|
|
6
|
+
export declare const DEFAULT_MARKITDOWN_STAMP_PATH = "/workspace/.xpert/.markitdown-bootstrap.json";
|
|
7
|
+
export declare const MARKITDOWN_BOOTSTRAP_SCHEMA_VERSION = 1;
|
|
8
|
+
export declare const MarkItDownConfigSchema: z.ZodObject<{
|
|
9
|
+
version: z.ZodDefault<z.ZodString>;
|
|
10
|
+
skillsDir: z.ZodDefault<z.ZodString>;
|
|
11
|
+
extras: z.ZodDefault<z.ZodString>;
|
|
12
|
+
}, "strip", z.ZodTypeAny, {
|
|
13
|
+
version?: string;
|
|
14
|
+
skillsDir?: string;
|
|
15
|
+
extras?: string;
|
|
16
|
+
}, {
|
|
17
|
+
version?: string;
|
|
18
|
+
skillsDir?: string;
|
|
19
|
+
extras?: string;
|
|
20
|
+
}>;
|
|
21
|
+
export type MarkItDownConfig = z.infer<typeof MarkItDownConfigSchema>;
|
|
22
|
+
export declare const MarkItDownConfigFormSchema: JsonSchemaObjectType;
|
|
23
|
+
//# sourceMappingURL=markitdown.types.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"markitdown.types.d.ts","sourceRoot":"","sources":["../../src/lib/markitdown.types.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAA;AACvD,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAA;AAEvB,eAAO,MAAM,gCAAgC,oBAAoB,CAAA;AACjE,eAAO,MAAM,0BAA0B,WAAW,CAAA;AAClD,eAAO,MAAM,6BAA6B,wCAAwC,CAAA;AAClF,eAAO,MAAM,6BAA6B,iDAAiD,CAAA;AAC3F,eAAO,MAAM,mCAAmC,IAAI,CAAA;AAEpD,eAAO,MAAM,sBAAsB;;;;;;;;;;;;EAIjC,CAAA;AAEF,MAAM,MAAM,gBAAgB,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,sBAAsB,CAAC,CAAA;AAErE,eAAO,MAAM,0BAA0B,EAAE,oBA2CxC,CAAA"}
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
import { z } from 'zod';
|
|
2
|
+
export const MARKITDOWN_SKILL_MIDDLEWARE_NAME = 'MarkItDownSkill';
|
|
3
|
+
export const DEFAULT_MARKITDOWN_VERSION = 'latest';
|
|
4
|
+
export const DEFAULT_MARKITDOWN_SKILLS_DIR = '/workspace/.xpert/skills/markitdown';
|
|
5
|
+
export const DEFAULT_MARKITDOWN_STAMP_PATH = '/workspace/.xpert/.markitdown-bootstrap.json';
|
|
6
|
+
export const MARKITDOWN_BOOTSTRAP_SCHEMA_VERSION = 1;
|
|
7
|
+
export const MarkItDownConfigSchema = z.object({
|
|
8
|
+
version: z.string().min(1).default(DEFAULT_MARKITDOWN_VERSION),
|
|
9
|
+
skillsDir: z.string().min(1).default(DEFAULT_MARKITDOWN_SKILLS_DIR),
|
|
10
|
+
extras: z.string().min(1).default('all')
|
|
11
|
+
});
|
|
12
|
+
export const MarkItDownConfigFormSchema = {
|
|
13
|
+
type: 'object',
|
|
14
|
+
properties: {
|
|
15
|
+
version: {
|
|
16
|
+
type: 'string',
|
|
17
|
+
title: {
|
|
18
|
+
en_US: 'MarkItDown Version',
|
|
19
|
+
zh_Hans: 'MarkItDown 版本'
|
|
20
|
+
},
|
|
21
|
+
description: {
|
|
22
|
+
en_US: 'The markitdown version to install via pip in the sandbox (e.g. "latest" or "0.1.1").',
|
|
23
|
+
zh_Hans: '在 sandbox 中通过 pip 安装的 markitdown 版本(如 "latest" 或 "0.1.1")。'
|
|
24
|
+
},
|
|
25
|
+
default: DEFAULT_MARKITDOWN_VERSION
|
|
26
|
+
},
|
|
27
|
+
extras: {
|
|
28
|
+
type: 'string',
|
|
29
|
+
title: {
|
|
30
|
+
en_US: 'Pip Extras',
|
|
31
|
+
zh_Hans: 'Pip Extras'
|
|
32
|
+
},
|
|
33
|
+
description: {
|
|
34
|
+
en_US: 'Python extras to install (e.g. "all", "ocr", "az-doc-intel"). Use "all" for full functionality.',
|
|
35
|
+
zh_Hans: '要安装的 Python extras(如 "all"、"ocr"、"az-doc-intel")。使用 "all" 获得完整功能。'
|
|
36
|
+
},
|
|
37
|
+
default: 'all'
|
|
38
|
+
},
|
|
39
|
+
skillsDir: {
|
|
40
|
+
type: 'string',
|
|
41
|
+
title: {
|
|
42
|
+
en_US: 'Skills Directory',
|
|
43
|
+
zh_Hans: 'Skills 目录'
|
|
44
|
+
},
|
|
45
|
+
description: {
|
|
46
|
+
en_US: 'Path inside the sandbox where skill files (SKILL.md and references) are written.',
|
|
47
|
+
zh_Hans: 'sandbox 中写入 skill 文件(SKILL.md 和 references)的目录路径。'
|
|
48
|
+
},
|
|
49
|
+
default: DEFAULT_MARKITDOWN_SKILLS_DIR,
|
|
50
|
+
'x-ui': {
|
|
51
|
+
span: 2
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
};
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
import { ChecklistItem, TXpertTeamDraft } from '@metad/contracts';
|
|
2
|
+
type XpertDraftValidateEvent = {
|
|
3
|
+
draft: TXpertTeamDraft;
|
|
4
|
+
};
|
|
5
|
+
export declare class MarkItDownSkillValidator {
|
|
6
|
+
handle(event: XpertDraftValidateEvent): ChecklistItem[];
|
|
7
|
+
}
|
|
8
|
+
export {};
|
|
9
|
+
//# sourceMappingURL=markitdown.validator.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"markitdown.validator.d.ts","sourceRoot":"","sources":["../../src/lib/markitdown.validator.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,aAAa,EAGb,eAAe,EAEhB,MAAM,kBAAkB,CAAA;AAQzB,KAAK,uBAAuB,GAAG;IAC7B,KAAK,EAAE,eAAe,CAAA;CACvB,CAAA;AAED,qBACa,wBAAwB;IAEnC,MAAM,CAAC,KAAK,EAAE,uBAAuB;CAmDtC"}
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
import { __decorate, __metadata } from "tslib";
|
|
2
|
+
import { getAgentMiddlewareNodes } from '@metad/contracts';
|
|
3
|
+
import { Injectable } from '@nestjs/common';
|
|
4
|
+
import { OnEvent } from '@nestjs/event-emitter';
|
|
5
|
+
import { MARKITDOWN_SKILL_MIDDLEWARE_NAME } from './markitdown.types.js';
|
|
6
|
+
const SANDBOX_SHELL_MIDDLEWARE_NAME = 'SandboxShell';
|
|
7
|
+
const EventNameXpertValidate = 'xpert.validate';
|
|
8
|
+
let MarkItDownSkillValidator = class MarkItDownSkillValidator {
|
|
9
|
+
handle(event) {
|
|
10
|
+
const draft = event.draft;
|
|
11
|
+
const sandboxEnabled = !!draft.team?.features?.sandbox?.enabled;
|
|
12
|
+
const agentNodes = draft.nodes.filter((node) => node.type === 'agent');
|
|
13
|
+
const items = [];
|
|
14
|
+
agentNodes.forEach((agentNode) => {
|
|
15
|
+
const middlewares = getAgentMiddlewareNodes(draft, agentNode.key).map((node) => node);
|
|
16
|
+
const middlewareEntities = middlewares.map((node) => ({
|
|
17
|
+
node,
|
|
18
|
+
entity: node.entity
|
|
19
|
+
}));
|
|
20
|
+
const hasSandboxShell = middlewareEntities.some(({ entity }) => entity.provider === SANDBOX_SHELL_MIDDLEWARE_NAME);
|
|
21
|
+
middlewareEntities
|
|
22
|
+
.filter(({ entity }) => entity.provider === MARKITDOWN_SKILL_MIDDLEWARE_NAME)
|
|
23
|
+
.forEach(({ node, entity }) => {
|
|
24
|
+
if (!sandboxEnabled) {
|
|
25
|
+
items.push({
|
|
26
|
+
node: node.key,
|
|
27
|
+
ruleCode: 'MARKITDOWN_SKILL_SANDBOX_DISABLED',
|
|
28
|
+
field: 'provider',
|
|
29
|
+
value: entity.provider,
|
|
30
|
+
level: 'warning',
|
|
31
|
+
message: {
|
|
32
|
+
en_US: 'MarkItDownSkill requires the agent sandbox feature to be enabled.',
|
|
33
|
+
zh_Hans: 'MarkItDownSkill 需要先启用智能体的 sandbox 功能。'
|
|
34
|
+
}
|
|
35
|
+
});
|
|
36
|
+
}
|
|
37
|
+
if (!hasSandboxShell) {
|
|
38
|
+
items.push({
|
|
39
|
+
node: node.key,
|
|
40
|
+
ruleCode: 'MARKITDOWN_SKILL_SANDBOX_SHELL_MISSING',
|
|
41
|
+
field: 'provider',
|
|
42
|
+
value: entity.provider,
|
|
43
|
+
level: 'warning',
|
|
44
|
+
message: {
|
|
45
|
+
en_US: 'MarkItDownSkill should be paired with the SandboxShell middleware on the same agent.',
|
|
46
|
+
zh_Hans: 'MarkItDownSkill 需要与同一智能体上的 SandboxShell 中间件配合使用。'
|
|
47
|
+
}
|
|
48
|
+
});
|
|
49
|
+
}
|
|
50
|
+
});
|
|
51
|
+
});
|
|
52
|
+
return items;
|
|
53
|
+
}
|
|
54
|
+
};
|
|
55
|
+
__decorate([
|
|
56
|
+
OnEvent(EventNameXpertValidate),
|
|
57
|
+
__metadata("design:type", Function),
|
|
58
|
+
__metadata("design:paramtypes", [Object]),
|
|
59
|
+
__metadata("design:returntype", void 0)
|
|
60
|
+
], MarkItDownSkillValidator.prototype, "handle", null);
|
|
61
|
+
MarkItDownSkillValidator = __decorate([
|
|
62
|
+
Injectable()
|
|
63
|
+
], MarkItDownSkillValidator);
|
|
64
|
+
export { MarkItDownSkillValidator };
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: markitdown
|
|
3
|
+
description: Converts files and URLs to Markdown. Use when the user needs to extract text from PDF, DOCX, PPTX, XLSX, HTML, images, audio, or other documents and convert them to clean Markdown format.
|
|
4
|
+
allowed-tools: Bash(markitdown:*)
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# File-to-Markdown Conversion with markitdown
|
|
8
|
+
|
|
9
|
+
## Quick start
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
# Convert a local file to Markdown (output to stdout)
|
|
13
|
+
markitdown document.pdf
|
|
14
|
+
|
|
15
|
+
# Save output to a file
|
|
16
|
+
markitdown document.pdf > document.md
|
|
17
|
+
|
|
18
|
+
# Convert from a URL
|
|
19
|
+
markitdown https://example.com/page.html
|
|
20
|
+
|
|
21
|
+
# Pipe input
|
|
22
|
+
cat report.docx | markitdown
|
|
23
|
+
|
|
24
|
+
# Convert with explicit output flag
|
|
25
|
+
markitdown document.pptx -o output.md
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Supported File Formats
|
|
29
|
+
|
|
30
|
+
| Format | Extensions | Notes |
|
|
31
|
+
|--------|-----------|-------|
|
|
32
|
+
| PDF | `.pdf` | Text extraction, layout preservation |
|
|
33
|
+
| Word | `.docx` | Full document structure, tables, images |
|
|
34
|
+
| PowerPoint | `.pptx` | Slides, speaker notes, embedded content |
|
|
35
|
+
| Excel | `.xlsx` | Sheets as Markdown tables |
|
|
36
|
+
| HTML | `.html`, `.htm` | Web page content extraction |
|
|
37
|
+
| CSV | `.csv` | Converted to Markdown tables |
|
|
38
|
+
| JSON | `.json` | Structured representation |
|
|
39
|
+
| XML | `.xml` | Structured representation |
|
|
40
|
+
| Plain Text | `.txt`, `.md`, `.rst` | Pass-through with minimal processing |
|
|
41
|
+
| Images | `.jpg`, `.jpeg`, `.png` | EXIF metadata extraction, OCR if available |
|
|
42
|
+
| Audio | `.mp3`, `.wav` | Speech-to-text transcription |
|
|
43
|
+
| ZIP | `.zip` | Recursively converts contained files |
|
|
44
|
+
| RSS/Atom | `.xml` (feeds) | Feed entry extraction |
|
|
45
|
+
| EPUB | `.epub` | Book content extraction |
|
|
46
|
+
|
|
47
|
+
## CLI Usage
|
|
48
|
+
|
|
49
|
+
### Basic conversion
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
# Convert any supported file
|
|
53
|
+
markitdown <input-file>
|
|
54
|
+
|
|
55
|
+
# Convert from URL
|
|
56
|
+
markitdown <url>
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### Output options
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
# Redirect to file
|
|
63
|
+
markitdown input.pdf > output.md
|
|
64
|
+
|
|
65
|
+
# Use -o flag
|
|
66
|
+
markitdown input.pdf -o output.md
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Piped input
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
# Pipe from another command
|
|
73
|
+
cat document.html | markitdown
|
|
74
|
+
curl -s https://example.com | markitdown
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### Batch conversion
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
# Convert all PDFs in a directory
|
|
81
|
+
for f in *.pdf; do markitdown "$f" > "${f%.pdf}.md"; done
|
|
82
|
+
|
|
83
|
+
# Convert multiple specific files
|
|
84
|
+
for f in report.docx slides.pptx data.xlsx; do
|
|
85
|
+
markitdown "$f" > "${f%.*}.md"
|
|
86
|
+
done
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
## Advanced Features
|
|
90
|
+
|
|
91
|
+
### Image description with LLM
|
|
92
|
+
|
|
93
|
+
If `OPENAI_API_KEY` is set in the environment, markitdown can use an LLM to generate descriptions for images:
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
export OPENAI_API_KEY="your-key"
|
|
97
|
+
markitdown photo.jpg
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### Document Intelligence (Azure)
|
|
101
|
+
|
|
102
|
+
For enhanced PDF/image processing with Azure Document Intelligence:
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
export AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT="https://your-endpoint.cognitiveservices.azure.com/"
|
|
106
|
+
export AZURE_DOCUMENT_INTELLIGENCE_KEY="your-key"
|
|
107
|
+
markitdown --use-docintel complex-document.pdf
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
## Tips
|
|
111
|
+
|
|
112
|
+
- **Large files**: For very large documents, conversion may take longer. Be patient and let it complete.
|
|
113
|
+
- **Tables**: Excel files and HTML tables are converted to proper Markdown table syntax.
|
|
114
|
+
- **Images in documents**: Images embedded in DOCX/PPTX are described if LLM support is configured.
|
|
115
|
+
- **Error handling**: If a file format is not supported, markitdown will output an informative error message.
|
|
116
|
+
- **Chaining**: Combine markitdown output with other tools for further processing:
|
|
117
|
+
```bash
|
|
118
|
+
markitdown report.pdf | grep "revenue"
|
|
119
|
+
markitdown data.xlsx | head -50
|
|
120
|
+
```
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
export type MarkItDownSkillAsset = {
|
|
2
|
+
path: string;
|
|
3
|
+
content: string;
|
|
4
|
+
};
|
|
5
|
+
/**
|
|
6
|
+
* Returns all markitdown skill files as assets ready to be uploaded
|
|
7
|
+
* into the sandbox container.
|
|
8
|
+
*
|
|
9
|
+
* @param skillsDir - The target directory inside the sandbox where
|
|
10
|
+
* skill files will be written, e.g. `/workspace/.xpert/skills/markitdown`.
|
|
11
|
+
*/
|
|
12
|
+
export declare function getSkillAssets(skillsDir: string): MarkItDownSkillAsset[];
|
|
13
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/lib/skills/index.ts"],"names":[],"mappings":"AAOA,MAAM,MAAM,oBAAoB,GAAG;IACjC,IAAI,EAAE,MAAM,CAAA;IACZ,OAAO,EAAE,MAAM,CAAA;CAChB,CAAA;AAWD;;;;;;GAMG;AACH,wBAAgB,cAAc,CAAC,SAAS,EAAE,MAAM,GAAG,oBAAoB,EAAE,CAKxE"}
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
import { readFileSync } from 'node:fs';
|
|
2
|
+
import { dirname, join } from 'node:path';
|
|
3
|
+
import { posix as path } from 'node:path';
|
|
4
|
+
import { fileURLToPath } from 'node:url';
|
|
5
|
+
const moduleDir = dirname(fileURLToPath(import.meta.url));
|
|
6
|
+
function readSkill(relativePath) {
|
|
7
|
+
return readFileSync(join(moduleDir, relativePath), 'utf8');
|
|
8
|
+
}
|
|
9
|
+
const SKILL_FILES = [
|
|
10
|
+
{ name: 'SKILL.md', src: 'SKILL.md' },
|
|
11
|
+
{ name: 'references/supported-formats.md', src: 'references/supported-formats.md' }
|
|
12
|
+
];
|
|
13
|
+
/**
|
|
14
|
+
* Returns all markitdown skill files as assets ready to be uploaded
|
|
15
|
+
* into the sandbox container.
|
|
16
|
+
*
|
|
17
|
+
* @param skillsDir - The target directory inside the sandbox where
|
|
18
|
+
* skill files will be written, e.g. `/workspace/.xpert/skills/markitdown`.
|
|
19
|
+
*/
|
|
20
|
+
export function getSkillAssets(skillsDir) {
|
|
21
|
+
return SKILL_FILES.map(({ name, src }) => ({
|
|
22
|
+
path: path.join(skillsDir, name),
|
|
23
|
+
content: readSkill(src)
|
|
24
|
+
}));
|
|
25
|
+
}
|
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
# Supported Formats — Detailed Guide
|
|
2
|
+
|
|
3
|
+
## PDF (.pdf)
|
|
4
|
+
|
|
5
|
+
MarkItDown extracts text from PDF files while preserving the document structure.
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
markitdown report.pdf
|
|
9
|
+
markitdown report.pdf > report.md
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
**Notes:**
|
|
13
|
+
- Scanned PDFs (image-based) require OCR support. Install with `markitdown[ocr]` extras or use Azure Document Intelligence with `--use-docintel`.
|
|
14
|
+
- Multi-column layouts are processed left-to-right, top-to-bottom.
|
|
15
|
+
- Tables in PDFs are extracted as Markdown tables when possible.
|
|
16
|
+
- Headers and footers are typically included in the output.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Word Documents (.docx)
|
|
21
|
+
|
|
22
|
+
Full support for Microsoft Word documents including:
|
|
23
|
+
- Headings (H1–H6 mapping)
|
|
24
|
+
- Paragraphs with formatting (bold, italic)
|
|
25
|
+
- Tables (converted to Markdown tables)
|
|
26
|
+
- Bulleted and numbered lists
|
|
27
|
+
- Embedded images (described with LLM if `OPENAI_API_KEY` is set)
|
|
28
|
+
- Hyperlinks
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
markitdown contract.docx
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## PowerPoint (.pptx)
|
|
37
|
+
|
|
38
|
+
Converts presentations to Markdown with one section per slide:
|
|
39
|
+
- Slide titles become headings
|
|
40
|
+
- Bullet points preserved as lists
|
|
41
|
+
- Speaker notes included (when present)
|
|
42
|
+
- Embedded images described (with LLM support)
|
|
43
|
+
- Tables converted to Markdown tables
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
markitdown presentation.pptx
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Excel (.xlsx)
|
|
52
|
+
|
|
53
|
+
Each worksheet is converted to a Markdown table:
|
|
54
|
+
- Sheet names become section headings
|
|
55
|
+
- Cell values preserved
|
|
56
|
+
- Formulas are evaluated (showing results, not formulas)
|
|
57
|
+
- Multiple sheets are separated by headings
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
markitdown financials.xlsx
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## HTML (.html, .htm)
|
|
66
|
+
|
|
67
|
+
Converts web pages to clean Markdown:
|
|
68
|
+
- Strips scripts, styles, and navigation elements
|
|
69
|
+
- Preserves semantic structure (headings, lists, tables, links)
|
|
70
|
+
- Inline images referenced
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
markitdown page.html
|
|
74
|
+
markitdown https://example.com/article
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## CSV (.csv)
|
|
80
|
+
|
|
81
|
+
Converts CSV data directly to Markdown tables:
|
|
82
|
+
- First row treated as table header
|
|
83
|
+
- All subsequent rows as table body
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
markitdown data.csv
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
## Images (.jpg, .jpeg, .png)
|
|
92
|
+
|
|
93
|
+
For images, markitdown can:
|
|
94
|
+
1. **Extract EXIF metadata** (camera model, date, GPS coordinates, etc.)
|
|
95
|
+
2. **OCR** (when `markitdown[ocr]` extras are installed)
|
|
96
|
+
3. **LLM description** (when `OPENAI_API_KEY` is set)
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
markitdown photo.jpg
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## Audio (.mp3, .wav)
|
|
105
|
+
|
|
106
|
+
Speech-to-text transcription using available backends:
|
|
107
|
+
- Produces a Markdown document with the transcribed text
|
|
108
|
+
- Requires audio processing dependencies (`markitdown[all]`)
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
markitdown recording.mp3
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## ZIP Archives (.zip)
|
|
117
|
+
|
|
118
|
+
Recursively extracts and converts all supported files within the archive:
|
|
119
|
+
- Each file gets its own section
|
|
120
|
+
- Nested directories are preserved in the heading structure
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
markitdown documents.zip
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
---
|
|
127
|
+
|
|
128
|
+
## RSS/Atom Feeds
|
|
129
|
+
|
|
130
|
+
Extracts feed entries as a structured Markdown document:
|
|
131
|
+
- Feed title as main heading
|
|
132
|
+
- Each entry with title, date, summary/content
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
markitdown https://example.com/feed.xml
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## EPUB (.epub)
|
|
141
|
+
|
|
142
|
+
Converts e-book content to Markdown:
|
|
143
|
+
- Chapters become sections
|
|
144
|
+
- Rich formatting preserved
|
|
145
|
+
- Images referenced
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
markitdown book.epub
|
|
149
|
+
```
|
|
@@ -0,0 +1,3 @@
|
|
|
1
|
+
export declare const MarkItDownPluginName = "markitdown";
|
|
2
|
+
export declare const MarkItDownIcon = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<svg version=\"1.1\" id=\"Layer_1\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" x=\"0px\" y=\"0px\"\n\t viewBox=\"0 0 129 129\" style=\"enable-background:new 0 0 129 129;\" xml:space=\"preserve\">\n<style type=\"text/css\">\n\t.st0{fill:#F25022;}\n\t.st1{fill:#7FBA00;}\n\t.st2{fill:#00A4EF;}\n\t.st3{fill:#FFB900;}\n</style>\n<path class=\"st0\" d=\"M0,0h61.3v61.3H0V0z\"/>\n<path class=\"st1\" d=\"M67.7,0H129v61.3H67.7V0z\"/>\n<path class=\"st2\" d=\"M0,67.7h61.3V129H0V67.7z\"/>\n<path class=\"st3\" d=\"M67.7,67.7H129V129H67.7V67.7z\"/>\n</svg>";
|
|
3
|
+
//# sourceMappingURL=types.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../../src/lib/types.ts"],"names":[],"mappings":"AAAA,eAAO,MAAM,oBAAoB,eAAe,CAAA;AAGhD,eAAO,MAAM,cAAc,4nBAapB,CAAA"}
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
export const MarkItDownPluginName = 'markitdown';
|
|
2
|
+
// MarkItDown SVG icon (document-to-markdown conversion)
|
|
3
|
+
export const MarkItDownIcon = `<?xml version="1.0" encoding="utf-8"?>
|
|
4
|
+
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
|
5
|
+
viewBox="0 0 129 129" style="enable-background:new 0 0 129 129;" xml:space="preserve">
|
|
6
|
+
<style type="text/css">
|
|
7
|
+
.st0{fill:#F25022;}
|
|
8
|
+
.st1{fill:#7FBA00;}
|
|
9
|
+
.st2{fill:#00A4EF;}
|
|
10
|
+
.st3{fill:#FFB900;}
|
|
11
|
+
</style>
|
|
12
|
+
<path class="st0" d="M0,0h61.3v61.3H0V0z"/>
|
|
13
|
+
<path class="st1" d="M67.7,0H129v61.3H67.7V0z"/>
|
|
14
|
+
<path class="st2" d="M0,67.7h61.3V129H0V67.7z"/>
|
|
15
|
+
<path class="st3" d="M67.7,67.7H129V129H67.7V67.7z"/>
|
|
16
|
+
</svg>`;
|
package/package.json
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@xpert-ai/plugin-markitdown",
|
|
3
|
+
"version": "0.0.1",
|
|
4
|
+
"author": {
|
|
5
|
+
"name": "XpertAI",
|
|
6
|
+
"url": "https://xpertai.cn"
|
|
7
|
+
},
|
|
8
|
+
"license": "AGPL-3.0",
|
|
9
|
+
"repository": {
|
|
10
|
+
"type": "git",
|
|
11
|
+
"url": "https://github.com/xpert-ai/xpert-plugins.git"
|
|
12
|
+
},
|
|
13
|
+
"bugs": {
|
|
14
|
+
"url": "https://github.com/xpert-ai/xpert-plugins/issues"
|
|
15
|
+
},
|
|
16
|
+
"type": "module",
|
|
17
|
+
"main": "./dist/index.js",
|
|
18
|
+
"module": "./dist/index.js",
|
|
19
|
+
"types": "./dist/index.d.ts",
|
|
20
|
+
"exports": {
|
|
21
|
+
"./package.json": "./package.json",
|
|
22
|
+
".": {
|
|
23
|
+
"@xpert-plugins-starter/source": "./src/index.ts",
|
|
24
|
+
"types": "./dist/index.d.ts",
|
|
25
|
+
"import": "./dist/index.js",
|
|
26
|
+
"default": "./dist/index.js"
|
|
27
|
+
}
|
|
28
|
+
},
|
|
29
|
+
"files": [
|
|
30
|
+
"dist",
|
|
31
|
+
"!**/*.tsbuildinfo"
|
|
32
|
+
],
|
|
33
|
+
"dependencies": {
|
|
34
|
+
"tslib": "^2.3.0"
|
|
35
|
+
},
|
|
36
|
+
"peerDependencies": {
|
|
37
|
+
"@metad/contracts": "^3.8.3",
|
|
38
|
+
"@xpert-ai/plugin-sdk": "^3.8.3"
|
|
39
|
+
},
|
|
40
|
+
"scripts": {}
|
|
41
|
+
}
|