@knpkv/confluence-to-markdown 0.5.0 → 0.7.0

package/CHANGELOG.md

@@ -1,5 +1,66 @@
 # @knpkv/confluence-to-markdown
 
+## 0.7.0
+
+### Minor Changes
+
+- [#81](https://github.com/knpkv/npm/pull/81) [`19c1538`](https://github.com/knpkv/npm/commit/19c153835bc198b9e407a013c16775c3fb7eb357) Thanks [@konopkov](https://github.com/konopkov)! - Ship agent skills alongside each CLI package and add an installer package plus per-CLI `skills install` commands for Codex and Claude.
+
+- [#85](https://github.com/knpkv/npm/pull/85) [`f01d83a`](https://github.com/knpkv/npm/commit/f01d83a091a13808b36aadfd45989240da537e8f) Thanks [@konopkov](https://github.com/konopkov)! - Add `confluence fetch` for printing the latest page markdown without creating a git workspace, support `--url` page parsing for `fetch` and `clone`, and add opt-in `--clean-markdown` output that strips Confluence round-trip metadata comments.
+
+- [#71](https://github.com/knpkv/npm/pull/71) [`e3c3805`](https://github.com/knpkv/npm/commit/e3c3805ee527a6edb69ed91977c95c586b563ff9) Thanks [@konopkov](https://github.com/konopkov)! - Migrate the package workspace to Effect v4 beta.
+
+  This updates runtime and peer dependencies to the Effect v4 beta module layout,
+  adopts Effect platform/runtime services for Node process, HTTP, filesystem, and
+  clock access, and refreshes package export metadata to point published type
+  entries at emitted `dist/*.d.ts` declarations.
+
+  CodeCommit packages now use Effect v4-compatible AWS and cache layers, including
+  typed `distilled-aws` context services, shared cached-comment decoding, and
+  schema-derived config defaults. Jira and Confluence OAuth callback servers bind
+  the expected local callback port range again under the Effect v4 Node HTTP
+  server layer.
+
+  The retired Claude AI packages have been removed from the workspace.
+
+### Patch Changes
+
+- [#72](https://github.com/knpkv/npm/pull/72) [`0e1c5ea`](https://github.com/knpkv/npm/commit/0e1c5eaf6d48c43e6591b6b6260dbfbf6bfb810b) Thanks [@konopkov](https://github.com/konopkov)! - Preserve Confluence-native ADF elements through markdown sync by storing decoded
+  placeholder metadata in per-page `.adf.json` sidecars and hydrating those refs
+  before push.
+
+  The integration test now requires API auth for raw ADF verification, asserts the
+  sidecar file contract, and checks native node and mark metadata across the
+  create/update/reclone cycle.
+
+- [#78](https://github.com/knpkv/npm/pull/78) [`b833841`](https://github.com/knpkv/npm/commit/b8338412b2352188a8505e4ee46ccd3f86a6b58f) Thanks [@konopkov](https://github.com/konopkov)! - Externalize base64-encoded ADF placeholder metadata into `.adf.json` sidecars
+  when pulling Confluence macros and native nodes.
+- Updated dependencies [[`c697d3c`](https://github.com/knpkv/npm/commit/c697d3c4ab779f14f017d3ec8fc8d1bffa1493b5), [`19c1538`](https://github.com/knpkv/npm/commit/19c153835bc198b9e407a013c16775c3fb7eb357), [`e3c3805`](https://github.com/knpkv/npm/commit/e3c3805ee527a6edb69ed91977c95c586b563ff9)]:
+  - @knpkv/agent-skills@0.2.0
+  - @knpkv/confluence-api-client@0.3.0
+
+## 0.6.0
+
+### Minor Changes
+
+- [#67](https://github.com/knpkv/npm/pull/67) [`10a1cc0`](https://github.com/knpkv/npm/commit/10a1cc068e907b4d2fb7e176cc834a0c8cddef3b) Thanks @konopkov! - Switch wire format from Confluence storage format to Atlassian Document
+  Format (ADF). Push (markdown → ADF) is now handled by the official
+  `@atlaskit/editor-markdown-transformer` + `@atlaskit/editor-json-transformer`;
+  pull (ADF → markdown) by an in-package tree walker typed against
+  `@atlaskit/adf-schema`. The bespoke storage-format parsers and serializers
+  (~140 KB) are removed. CLI behavior is unchanged. When `saveSource` is
+  enabled, the `.source` companion file is now `<page>.source.json` instead of
+  `<page>.html`; existing companion `.html` files are harmless and can be
+  deleted.
+
+  Confluence macros now survive a pull → edit → push round-trip: extension
+  placeholders carry the macro's full attrs (parameters, layout, localId) as a
+  base64 blob, and a bodied macro's body is re-attached from the blocks between
+  its `<!-- adf:bodiedExtension … -->` / `<!-- adf:/bodiedExtension -->`
+  markers. Constructs that still degrade on push (panels, task lists, dates,
+  emojis, expand sections, inline cards, media captions) are documented under
+  "Known fidelity limitations" in the README.
+
 ## 0.5.0
 
 ### Minor Changes

package/README.md

@@ -18,6 +18,12 @@ npm install @knpkv/confluence-to-markdown effect
 ```bash
 # Clone pages with full version history
 confluence clone --root-page-id <ID> --base-url <URL>
+confluence clone --url <PAGE_URL>
+
+# Fetch one latest page to stdout without creating a git workspace
+confluence fetch --page-id <ID> --base-url <URL>
+confluence fetch --url <PAGE_URL>
+confluence fetch --url <PAGE_URL> --clean-markdown
 
 # Pull pages from Confluence
 confluence pull
@@ -102,10 +108,48 @@ confluence diff --commit HEAD~1   # compare with commit
 ## How It Works
 
 1. `confluence clone` creates `.confluence/` directory, initializes git, and pulls pages with full version history
-2. `confluence pull` downloads pages and auto-commits changes
-3. `confluence pull --replay-history` replays each Confluence version as a separate git commit with original author/date
-4. Version messages from Confluence are preserved in markdown front-matter
-5. Use standard git commands in `.confluence/` for advanced operations
+2. `confluence fetch` prints the latest markdown for one page without creating `.confluence/` or git commits
+3. `confluence pull` downloads pages and auto-commits changes
+4. `confluence pull --replay-history` replays each Confluence version as a separate git commit with original author/date
+5. Version messages from Confluence are preserved in markdown front-matter
+6. Use standard git commands in `.confluence/` for advanced operations
+
+`clone` and `fetch` accept full page URLs via `--url`, including Confluence paths such as
+`https://yoursite.atlassian.net/wiki/spaces/DEV/pages/123456/Page+Title` and shorthand numeric paths such as
+`https://yoursite.atlassian.com/123456`.
+
+`confluence fetch --clean-markdown` removes Confluence round-trip metadata comments such as `<!-- adf:... -->`
+from the printed output. This is intended for readable exports and is not suitable for editing and pushing back to
+Confluence.
+
+### Conversion pipeline
+
+This package talks to Confluence Cloud in **Atlassian Document Format (ADF)**:
+
+- **Push (markdown → ADF)** delegates to the official `@atlaskit/editor-markdown-transformer` and `@atlaskit/editor-json-transformer`. Atlassian's own libraries author the JSON we send back to Confluence.
+- **Pull (ADF → markdown)** uses an in-package tree walker (`AdfWalker`). The walker covers paragraphs, headings, lists, code blocks, tables, panels, task and decision lists, mentions, emojis, status, dates, expand sections, inline cards, and native Table of Contents macro syntax. Lossy marks (`underline`, `textColor`, `backgroundColor`, `subsup`) fall back to inline HTML; unknown nodes degrade to a placeholder comment plus a logged warning.
+- Both directions validate against the canonical `@atlaskit/adf-schema` JSON Schema, so library bugs and remote drift surface as structured errors instead of silent corruption.
+
+When `saveSource` is enabled, the raw ADF JSON is persisted as `<page>.source.json` (pretty-printed) alongside the markdown.
+
+#### Known fidelity limitations
+
+Nodes that survive a pull → edit → push round-trip structurally intact: status (unless its text contains `<`), mentions (with an accountId), native Table of Contents syntax (`[[toc]]`, `[[toc:min=2,max=4]]`), and macros (`extension` / `bodiedExtension` / `inlineExtension` — the placeholder comment carries the full macro attrs, including `parameters`, as a base64 blob, and a bodied macro's body is re-attached from the blocks between its `<!-- adf:bodiedExtension … -->` / `<!-- adf:/bodiedExtension -->` markers; a bodied macro inside a table cell keeps only the marker, its body is dropped). TOC macros with unrepresentable attrs such as `localId` or `layout` keep using the exact placeholder form instead of readable syntax. Everything below does **not** fully survive:
+
+- **Attachment media.** Pages with images or other attachments use ADF `mediaSingle` / `mediaInline` nodes that reference attachments by `id` + `collection`. Resolving these to download URLs requires a separate Confluence API call per attachment, which this package does not yet make. Such nodes are currently emitted as `<!-- adf:media id=… -->` placeholders plus a logged `MediaWithoutUrl` warning. A media-resolver hook is on the roadmap.
+- **Media captions.** A `mediaSingle` caption is rendered as an italic line under the media on pull, but pushes back as plain italic text — the structured `caption` node is lost.
+- **Link titles.** The official `@atlaskit/editor-markdown-transformer` does not retain link titles when parsing markdown, so `[text](url "title")` round-trips back as `[text](url)`.
+- **Table cell merges.** `colspan`/`rowspan` are flattened — GFM tables have no merged cells.
+
+The pull side logs warnings for unknown node types, lossy marks, macro placeholders, and unresolvable media or inline cards.
+
+#### Migrating from earlier versions
+
+Prior versions of this package wrote a `<page>.html` companion file when `saveSource` was enabled; the current version writes `<page>.source.json` instead. Existing `.html` files are harmless and can be removed manually:
+
+```bash
+find .confluence/docs -name "*.html" -delete
+```
 
 ## Git Implementation
 
@@ -144,7 +188,7 @@ Note: Front-matter (`contentHash`, `version`) is **not** updated at commit time.
 When you push:
 
 1. **Detect changes**: Compares content hash vs stored `contentHash` in front-matter
-2. **Push to Confluence**: Uploads markdown converted to Confluence storage format
+2. **Push to Confluence**: Converts markdown to Atlassian Document Format (ADF) via the official `@atlaskit/editor-markdown-transformer` + `@atlaskit/editor-json-transformer`, validated against `@atlaskit/adf-schema`
 3. **Fetch canonical content**: Downloads what Confluence actually stored (may differ slightly)
 4. **Amend commit**: Updates local file with canonical content so future clones match exactly
 5. **Update tracking branch**: Moves `origin/confluence` to HEAD
@@ -250,15 +294,15 @@ This creates `.confluence/config.json` in your project root:
 
 ### Configuration Options
 
-| Option            | Description                           | Default            |
-| ----------------- | ------------------------------------- | ------------------ |
-| `rootPageId`      | Confluence page ID to sync from       | Required           |
-| `baseUrl`         | Confluence Cloud URL                  | Required           |
-| `spaceKey`        | Optional space key                    | -                  |
-| `docsPath`        | Local path for markdown files         | `.confluence/docs` |
-| `excludePatterns` | Glob patterns to exclude              | `[]`               |
-| `saveSource`      | Save original HTML alongside markdown | `false`            |
-| `trackedPaths`    | Glob patterns for git tracking        | `["**/*.md"]`      |
+| Option            | Description                                                 | Default            |
+| ----------------- | ----------------------------------------------------------- | ------------------ |
+| `rootPageId`      | Confluence page ID to sync from                             | Required           |
+| `baseUrl`         | Confluence Cloud URL                                        | Required           |
+| `spaceKey`        | Optional space key                                          | -                  |
+| `docsPath`        | Local path for markdown files                               | `.confluence/docs` |
+| `excludePatterns` | Glob patterns to exclude                                    | `[]`               |
+| `saveSource`      | Save raw ADF JSON alongside markdown (`<page>.source.json`) | `false`            |
+| `trackedPaths`    | Glob patterns for git tracking                              | `["**/*.md"]`      |
 
 ## Directory Structure
 

package/dist/AdfPlaceholders.d.ts

@@ -0,0 +1,42 @@
+/**
+ * Reverse the AdfWalker's placeholder syntax back into proper ADF nodes
+ * after the @atlaskit markdown transformer has run.
+ *
+ * The walker emits Confluence-only nodes (status, extension, inlineExtension)
+ * as HTML/comment placeholders so they survive a pull. The markdown transformer
+ * has no concept of these nodes, so on push it parses the placeholders as
+ * plain text — Confluence then renders the literal HTML/comment. This module
+ * walks the produced ADF and rewrites those text patterns into the structured
+ * nodes the editor expects.
+ *
+ * Patterns recognized (must match the AdfWalker emission exactly):
+ *  - `<span class="adf-status" data-color="COLOR">TEXT</span>`
+ *  - `<!-- adf:extension key=KEY type=TYPE attrs=BASE64 -->`   (block, when
+ *      the whole paragraph is just this comment; `attrs` is base64 JSON of
+ *      the node's full attrs — parameters, localId, layout — and wins over
+ *      the readable key/type parts; key/type-only is the legacy form)
+ *  - `<!-- adf:paragraph marks=BASE64 --> BODY <!-- adf:/paragraph -->`
+ *      (the body paragraph regains its paragraph-level marks)
+ *  - `<!-- adf:bodiedExtension … --> BODY <!-- adf:/bodiedExtension -->`
+ *      (the sibling blocks between the markers become the extension's body)
+ *  - `<!-- adf:inlineCard attrs=BASE64 -->` (inline)
+ *  - `<!-- adf:inlineExtension key=KEY type=TYPE attrs=BASE64 -->` (inline)
+ *  - `<!-- adf:date node=BASE64 -->` and `<!-- adf:emoji node=BASE64 -->`
+ *      (inline)
+ *  - `<!-- adf:panel type=TYPE attrs=BASE64 --> BODY <!-- adf:/panel -->`
+ *      (the sibling blocks between the markers become the panel's body)
+ *  - `<!-- adf:TYPE node=BASE64 --> BODY <!-- adf:/TYPE -->` for selected
+ *      native block nodes such as task/decision lists, expands, layouts,
+ *      cards, and tables
+ *  - `<u>TEXT</u>`, `<sub>TEXT</sub>`, `<sup>TEXT</sup>`, and exact styled
+ *      spans emitted for Confluence-only inline marks
+ *  - `[@Name](confluence-mention://ACCOUNT_ID)`            (link mark with a
+ *      custom scheme — the only way to round-trip mention accountIds)
+ *  - `[[toc]]`, `[[toc:min=1,max=3]]` (block-level native syntax for the
+ *      Confluence Table of Contents macro)
+ *
+ * @module
+ */
+/** Walk the document tree and rewrite placeholder text into proper ADF nodes. */
+export declare const revertPlaceholders: (doc: unknown) => unknown;
+//# sourceMappingURL=AdfPlaceholders.d.ts.map

package/dist/AdfPlaceholders.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"AdfPlaceholders.d.ts","sourceRoot":"","sources":["../src/AdfPlaceholders.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAsCG;AA2gBH,iFAAiF;AACjF,eAAO,MAAM,kBAAkB,GAAI,KAAK,OAAO,KAAG,OAAoC,CAAA"}