athena-python-pptx 0.5.0__tar.gz → 0.7.1__tar.gz

{athena_python_pptx-0.5.0 → athena_python_pptx-0.7.1}/CHANGELOG.md

@@ -2,6 +2,92 @@
 
 All notable changes to `athena-python-pptx` are documented in this file.
 
+## 0.7.1 — CommandBuffer thread-safety
+
+Fixes a data-loss/duplication race in `CommandBuffer`. `self._commands` was
+shared between the foreground `add`/`flush`/`clear` path and the 2.0s
+auto-flush daemon thread, with only the timer guarded — so a flush that
+snapshotted-then-reset the buffer could drop a command a concurrent `add`
+appended in between (silent slide/shape loss), and a timer flush racing a
+foreground flush could POST the same commands twice (duplication). In
+long-lived Daytona executor processes this silently dropped or duplicated
+slides/shapes with no error. A dedicated `_commands_lock` now guards every
+read/append/reassign of `_commands` (drain atomically under the lock; POST
+outside it). No public API change. (#24278)
+
+## 0.7.0
+
+**python-pptx behavioral-parity sweep — read-path return types, enum
+values, and chart-data builder objects now match python-pptx 1.0.2.**
+
+Surface parity (every public class/method/signature) was already 100%, but a
+deep-dive against python-pptx 1.0.2 found *behavioral* divergences where
+getters returned raw strings/`None` instead of enum members or typed
+defaults, two enums had wrong integer values, and the chart-data builders
+weren't python-pptx-shaped. All fixes are pure-local (no wire/server change);
+`to_series_payload()` output is byte-identical to before.
+
+**Enum value corrections (silent drop-in breakers):**
+
+- **`MSO_THEME_COLOR.TEXT_2` / `BACKGROUND_2` were swapped** — now `15` /
+  `16` to match upstream (the `<a:schemeClr>` read/write maps were updated in
+  lockstep so OOXML output is unchanged). `MSO_THEME_COLOR_INDEX` inherits.
+- **`MSO_TEXT_UNDERLINE_TYPE` had 6 wrong values, 9 missing members, and a
+  bogus `DASHED_LINE`** — rewritten to upstream's exact 19 members/values;
+  added the `MSO_UNDERLINE` alias (`from pptx.enum.text import MSO_UNDERLINE`).
+- **`PP_MEDIA_TYPE`**: `MOVIE` corrected `2`→`3`, added `OTHER = 1`, and the
+  enum is now importable from its canonical `pptx.enum.shapes` location.
+- Added `MSO_SHAPE_TYPE.MIXED = -2` and the `MSO` alias; added the
+  upstream-typo `MSO_PATTERN.ERCENT_40` member.
+
+**Read-path return-type parity (getters now return enums / typed defaults):**
+
+- `FillFormat.type` → `MSO_FILL_TYPE` (was a raw string); `background()` now
+  reports `MSO_FILL_TYPE.BACKGROUND`. `gradient_angle`/`gradient_stops`/
+  `pattern` raise `TypeError` on a wrong-type fill (parity).
+- `LineFormat.dash_style` → `MSO_LINE_DASH_STYLE` (or `None` when unstyled,
+  instead of defaulting to `"solid"`); `LineFormat.width` defaults to
+  `Emu(0)` for an unstyled line (a color-/dash-only edit no longer injects a
+  synthetic 1pt width).
+- `Chart.chart_type` → `XL_CHART_TYPE` member (was a raw server string);
+  `add_chart` now persists `grouping` so stacked/clustered variants read back
+  exactly.
+- `TableCell.vertical_anchor` → `MSO_VERTICAL_ANCHOR`; `TableCell.margin_*`
+  default to the OOXML defaults (`91440`/`45720` EMU) instead of `None`.
+- `Shape.rotation` is normalized to `[0, 360)` and float-coerced.
+- `_Column.width` / `_Row.height` are populated for `add_table(...)` tables
+  (even division, last column/row absorbs the remainder).
+- `RGBColor` is now a real `tuple` subclass (`RGBColor(255,0,0) == (255,0,0)`,
+  `isinstance(c, tuple)`).
+- `ColorFormat.theme_color` returns `MSO_THEME_COLOR.NOT_THEME_COLOR` for an
+  sRGB color (raises `TypeError` when no color is present) instead of `None`.
+- `TextFrame.auto_size` / `vertical_anchor` return their enums; `margin_*`
+  default to the python-pptx `Length` defaults; `Paragraph.line_spacing =
+  None` clears the override without crashing; `Font.language_id` returns
+  `MSO_LANGUAGE_ID.NONE` on an unset run.
+- `SlideLayouts.get_by_name` is exact/case-sensitive (the fuzzy match moved to
+  the Athena-only `by_name`); `Slide.has_notes_slide` flips `True` once
+  `notes_slide` is accessed.
+- `Video.ext` returns the bare extension (`'mp4'`, `'vid'` fallback) and
+  `Video.filename` synthesizes `'movie.{ext}'`.
+
+**`pptx.util` truncation parity:** unit constructors (`Emu`/`Pt`/`Inches`/…)
+truncate toward zero like upstream (`int(Pt(1.9999)) == 25398`) instead of
+rounding; `Length.centipoints` is integer floor-division.
+
+**Chart-data builder objects (`pptx.chart.data`) are now python-pptx-shaped:**
+
+- `CategoryChartData.add_series(name, values)` returns a real
+  `CategorySeriesData` with `.values`, `.add_data_point()`, and `.index`;
+  `XyChartData`/`BubbleChartData.add_series` return `XySeriesData`/
+  `BubbleSeriesData` whose `add_data_point(...)` returns the data point and
+  expose `.x_values` / `.y_values` / `.bubble_sizes`.
+- `CategoryChartData.add_category(label)` and a `Categories`/`Category`
+  sequence with `are_dates` / `are_numeric` / `number_format`; category label
+  types (dates, numbers) are preserved through `cd.categories[i].label`.
+- The three chart-data containers implement the `Sequence` protocol
+  (`for s in chart_data` / `chart_data[0]`).
+
 ## 0.4.2
 
 **Paragraph-level font fidelity — `paragraph.font` now matches stock

{athena_python_pptx-0.5.0 → athena_python_pptx-0.7.1}/CLAUDE.md

@@ -52,8 +52,10 @@ A small number of REST-SDK-specific departures are documented in
   use `slide.notes_slide.notes_text_frame` for notes text.
 - **`XyChartData.add_series(name, values)`** signature drift vs upstream's
   `add_series(name, number_format=None)` — caller passes inline tuples.
-- **`TextFitter.best_fit_font_size()`** returns `max_size` unchanged
-  (auto-fit is server-side); emits `RuntimeWarning`.
+- **`TextFitter.best_fit_font_size()`** is server-measured: with a base
+  URL configured it queries `POST /measure-text-fit` and returns the
+  largest fitting size; only falls back to `max_size` (+ `RuntimeWarning`)
+  when no studio client is reachable.
 - **`_BaseSeriesData.index`** raises `NotImplementedError` (raising
   rather than silently returning 0).
 - **`Package.open()`** raises `UnsupportedFeatureError`.
@@ -69,6 +71,15 @@ local XML / OPC parts:
 
 `Font.language_id` IS now implemented (returns `MSO_LANGUAGE_ID` enum, accepts enum or int LCID) — added in v0.1.68.
 
+### Slide-index conventions
+
+Slide indexing has TWO bases — never conflate them:
+
+- **0-based** — this SDK (python-pptx parity): `prs.slides[0]`, `slide.slide_index`, and every `slide_index=` parameter are 0-based.
+- **1-based** — the Athena citation/anchor system: `SlideAnchor.slideIndex` / `ShapeAnchor.slideIndex` (in `pptx/_references.py`), the citation methods, and anything displayed as "Slide N" are the 1-based display position.
+
+The PPTX Studio server applier converts at the SDK→anchor boundary (`+1`). `slideId` (== the SDK's `slide.slide_id`) is the stable, base-agnostic identity — prefer it when available.
+
 ### If you need a deviation
 
 If there is a genuine technical reason why a deviation from python-pptx is necessary (e.g., the SDK is a REST client and cannot replicate XML-level behavior):

{athena_python_pptx-0.5.0 → athena_python_pptx-0.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: athena-python-pptx
-Version: 0.5.0
+Version: 0.7.1
 Summary: Drop-in replacement for python-pptx that connects to PPTX Studio for real-time collaboration
 Project-URL: Homepage, https://github.com/pptx-studio/python-sdk
 Project-URL: Documentation, https://docs.pptx-studio.com/sdk/python

{athena_python_pptx-0.5.0 → athena_python_pptx-0.7.1}/docs/API_PARITY_EXCEPTIONS.md

@@ -45,48 +45,51 @@ text = slide.notes_slide.notes_text_frame.text
 A future server-side notes-master patch op will plumb the full shape
 tree through. Tracked in `PARITY_QUESTIONS.md`.
 
-### `XyChartData.add_series()` — different signature
+### `XyChartData.add_series()` — both the upstream and inline patterns work (v0.7.0)
 
-Upstream signature: `add_series(name, number_format=None)` — caller adds
-data points via the returned series object.
-
-**Our signature:** `add_series(name, values)` — caller passes the
-(x, y) tuples up front.
-
-**Migration:**
+As of v0.7.0 `add_series` returns a real `XySeriesData` (resp.
+`BubbleSeriesData`), so the **upstream pattern works**:
 
 ```python
-# python-pptx pattern (NOT supported by athena-python-pptx):
+# python-pptx pattern (now supported):
 series = data.add_series("Series 1")
-series.add_data_point(1.0, 2.0)
+series.add_data_point(1.0, 2.0)          # returns the XyDataPoint
 series.add_data_point(3.0, 4.0)
+series.x_values  # [1.0, 3.0]; series.y_values -> [2.0, 4.0]
 
-# athena-python-pptx pattern (use this):
+# athena-python-pptx inline shorthand (still supported):
 data.add_series("Series 1", [(1.0, 2.0), (3.0, 4.0)])
 ```
 
-Same applies to `BubbleChartData.add_series`. Fixing this is a larger
-refactor (the SDK's chart-authoring path expects values inline);
-tracked in `PARITY_QUESTIONS.md`.
+`add_series` additionally accepts an optional `number_format=` for
+signature parity. The same applies to `BubbleChartData.add_series`
+(inline triples are `(x, y, size)`).
 
-### `TextFitter.best_fit_font_size()` — returns `max_size` unchanged
+### `TextFitter.best_fit_font_size()` — server-measured (needs a base URL)
 
-Upstream binary-searches font sizes locally to fit text within bounds.
-Our stub always returns `max_size` and emits a `RuntimeWarning`.
-Auto-fit happens server-side at render time. **Don't rely on this for
-local layout decisions**; use the auto-fit text frame property instead:
+Upstream binary-searches font sizes against locally-installed font
+metrics, which a REST SDK can't read. When a base URL is configured
+(`ATHENA_PPTX_BASE_URL` or an explicit `client`) the classmethod issues
+a `MeasureTextFit` query to the studio's `POST /measure-text-fit`
+endpoint and returns the largest integer size that fits — measured
+against the studio's bundled fonts via the same layouter the renderer
+uses (see the extension entry below). It falls back to `max_size` with a
+`RuntimeWarning` **only** when no studio client is reachable (offline /
+import-time tests). For full-deck layout, prefer letting the text frame
+auto-fit at render time:
 
 ```python
 from pptx.enum.text import MSO_AUTO_SIZE
 text_frame.auto_size = MSO_AUTO_SIZE.TEXT_TO_FIT_SHAPE
 ```
 
-### `_BaseSeriesData.index` — raises `NotImplementedError`
+### `_BaseSeriesData.index` — resolves against the parent chart-data (v0.7.0)
 
-Upstream's `CategorySeriesData.index` returns the series's offset
-within its parent chart-data. Our stub raises rather than silently
-returning `0` (which would corrupt any OOXML `<c:idx>` builder that
-trusts the value). If you need the series index, track it externally.
+As of v0.7.0, a series created via `chart_data.add_series(...)` carries a
+back-reference to its parent, so `series.index` returns the series's
+offset within the chart-data exactly like upstream. A *detached* series
+constructed standalone (no parent) still raises `NotImplementedError`
+rather than lying about the offset.
 
 ### `Package.open(pkg_file)` — raises `UnsupportedFeatureError`
 
@@ -137,6 +140,24 @@ prs.save()              # flush only (Athena-only)
 prs.save("out.pptx")    # flush + export + write to local file (parity)
 ```
 
+### Oversized flushes are split into multiple atomic sub-transactions
+
+The SDK serializes every buffered command into one JSON body and POSTs it
+to `/decks/:id/commands`. pptx-studio's apps/api (Fastify) caps that body
+at 32 MiB, and `AddPicture` / `SubstitutePlaceholder(picture)` carry inline
+base64 image bytes (~4/3 expansion), so a few large images can blow past
+the ceiling. For batches that exceed the limit, `CommandBuffer.flush`
+splits them into multiple atomic sub-transactions (still validating the
+whole batch up front, so an invalid command aborts before anything is
+applied). The all-or-nothing guarantee then holds *per sub-transaction*
+rather than end-to-end — a server-side failure mid-sequence can leave
+earlier chunks committed. The split threshold is tunable via
+`ATHENA_PPTX_MAX_REQUEST_BYTES`; batches within the limit ship as one
+atomic request, unchanged. A single command larger than the budget (one
+very large inline image) cannot be split — it is sent alone and may still
+be rejected with a teaching `IMAGE_TOO_LARGE` error, but it no longer
+takes the rest of the queued edits down with it.
+
 ---
 
 ## Agent-Friendly Additions (not in python-pptx)
@@ -215,34 +236,23 @@ python-pptx's `_Row.height` and `_Column.width` are read/write EMU properties. T
 
 In python-pptx, `Table.rows` returns a `_RowCollection` of `_Row` objects (each with `.height`). This SDK matches that. In a prior version, `Table.rows` returned a flat collection of cell lists.
 
-### `TextFrame.clear()` — also resets bullet on paragraph 0
+### `TextFrame.clear()` — bullet inheritance (now matches upstream)
 
-In upstream python-pptx, `text_frame.clear()` removes every paragraph except the first, and on that first paragraph it removes every child element **except** `<a:pPr>`. Any bullet definition the author wrote into `<a:pPr>` survives the clear, and an inherited bullet from the master / layout's `<a:lstStyle>` is left to be re-resolved by PowerPoint at render time.
+`text_frame.clear()` matches upstream python-pptx: it removes the runs but
+preserves the cleared paragraph's inherited list style. A content/body
+placeholder therefore keeps its master `<p:bodyStyle>` bullet, and a title
+keeps its `<p:titleStyle>` `<a:buNone/>` (no bullet), exactly as PowerPoint
+re-resolves them at render time.
 
-This SDK additionally emits a `SetParagraphStyle(paragraph_index=0, bullet='none')` after the empty `SetText`, explicitly overriding the bullet on the cleared paragraph.
-
-The deviation exists because pptx-studio **pre-bakes** master/layout list-style inheritance into `richContent.paragraphs[i].bullet` at ingest time (and at slide materialization), then preserves that baked field through every subsequent `applySetText` via `preserveParagraphProps`. Without the explicit override, the python-pptx idiom
-
-```python
-tf.clear()
-p = tf.paragraphs[0]
-run = p.add_run()
-run.text = "Title"
-```
-
-renders the title with a stray "•" whenever the placeholder lives on a layout whose master defines a level-1 `<a:buChar/>` (the common case for `<p:bodyStyle>` and any content placeholder). The explicit `bullet='none'` clears the baked inheritance so the literal "no pPr was authored" intent reaches the renderer.
-
-To re-enable a bullet after `clear()`, set it back explicitly:
-
-```python
-tf.clear()
-p = tf.paragraphs[0]
-p.bullet = True             # restores 'disc' default
-run = p.add_run()
-run.text = "Bullet item"
-```
-
-Only `bullet` is overridden — `alignment`, `level`, `line_spacing`, `space_before`, `space_after`, `margin_left`, `indent`, and `bullet_color` are left alone (they continue to follow upstream `<a:pPr>`-preservation semantics, since baked-inheritance surprises are far less common on those fields).
+> **Historical note:** earlier versions emitted a
+> `SetParagraphStyle(paragraph_index=0, bullet='none')` after the empty
+> `SetText` to mask a *stray title bullet*. The real cause was incorrect
+> per-type bullet baking — empty layout/master prompt paragraphs were baked
+> with `bullet:'none'` instead of their master `<p:txStyles>` value, and a
+> blanket override then stripped bullets from **content** placeholders too
+> (where bullets are expected). That baking is now correct per placeholder
+> type (`titleStyle` → no bullet, `bodyStyle` → its `<a:buChar/>`), so the
+> override was removed and `clear()` is faithful to upstream.
 
 ---
 
@@ -328,6 +338,37 @@ img = ws.export_range_as_image_asset("A1:F20", name="Q4 Revenue")
 slide.shapes.add_picture(img.url, Inches(1), Inches(1), width=Inches(6))
 ```
 
+### `Picture.image_url` — owner-scoped media URL for cross-deck image copy
+
+python-pptx upstream exposes `Picture.image` returning an `Image` with
+`.blob` bytes. The REST SDK can't materialize bytes locally, so `.image`
+stays `None` (see the omissions list above). **Additionally**, when a
+picture is loaded from a server snapshot it now carries `image_url`: a
+stable `{base_url}/media?key=<assetId>` URL that streams the picture's
+bytes. It returns `None` for pictures with no backing media asset (an
+SDK-added picture not yet flushed/refreshed, a placeholder with no image,
+or exotic sources like OLE/movie posters that lack a top-level asset id).
+
+The URL is owner/EDIT-scoped (the `/media` route runs the same deck-access
+check as every other deck op), and `add_picture` auto-attaches the
+sandbox bearer for Athena hosts — so it composes directly into the
+canonical "copy every image from deck A into deck B" recipe:
+
+```python
+src = Presentation(asset_id="asset_source")
+dst = Presentation(asset_id="asset_dest")
+for i, slide in enumerate(src.slides):
+    for pic in slide.shapes.get_images():
+        if pic.image_url:
+            dst.slides[i].shapes.add_picture(
+                pic.image_url, pic.left, pic.top, pic.width, pic.height
+            )
+dst.save()
+```
+
+**Portable code that needs to run against stock python-pptx cannot rely
+on `image_url`** — it is an Athena-only accessor.
+
 ### `name=` kwarg on `add_textbox()` / `add_shape()` / `add_picture()`
 
 Optional keyword for setting a stable shape name at creation. python-pptx
@@ -1368,8 +1409,8 @@ A citation record is:
 ```jsonc
 {
   "id": "citation_<uuid12>",
-  "destinationAnchor": { "type": "slide", "slideId": "...", "slideIndex": 0 },
-  // or { "type": "shape", "slideId": "...", "slideIndex": 0, "shapeIds": ["..."] }
+  "destinationAnchor": { "type": "slide", "slideId": "...", "slideIndex": 1 },
+  // or { "type": "shape", "slideId": "...", "slideIndex": 1, "shapeIds": ["..."] }
   "citation_string": "<source Spaces URL>",
   "displayValue": "Q4 Sales",        // optional
   "active": true,
@@ -1399,17 +1440,31 @@ cid = slide.add_citation(
 )
 ```
 
-### `Shape.cite(source, *, anchor=None, display_value=None) -> str`
+### `Shape.add_citation(source, *, anchor=None, display_value=None) -> str`
 
 Shape-scoped citation — emits `AddShapeCitation` carrying this shape's id, so
 the server writes a `shape` destination anchor with `shapeIds`. Returns the
-created citation id.
+created citation id. (Mirrors `Slide.add_citation` / `Cell.add_citation` — the
+citation verb is `add_citation` on every citable object.)
 
 ```python
 shape = slide.shapes.add_textbox(Inches(1), Inches(1), Inches(4), Inches(1))
-cid = shape.cite(AssetReference(id="asset_doc_xyz"))
+cid = shape.add_citation(AssetReference(id="asset_doc_xyz"))
 ```
 
+**Slide-index base shift (citations).** The SDK/command wire `slide_index`
+(`prs.slides[i]`, `slide.slide_index`) is **0-based** — python-pptx array
+parity. But the `destinationAnchor.slideIndex` the server writes into the
+citation record is **1-based** — the "Slide N" display position per the
+athena-references contract. The server applies the `+1` at the SDK→anchor
+boundary, so the stored anchor's `slideIndex` is always one greater than the
+SDK index of the cited slide.
+
+> ⚠️ If the **cited source** is itself a slide/shape (you pass a
+> `SlideAnchor` / `ShapeAnchor` as `anchor=`), that anchor's `slide_index` is
+> the **1-based** display position per the athena-references contract — do
+> **not** pass the SDK's 0-based `slide.slide_index` into it (add 1).
+
 ### `Slide.remove_citation(citation_id, *, soft=True)` / `Presentation.remove_citation(citation_id, *, soft=True)`
 
 Remove a citation from the deck by id. Citations are deck-scoped, so both

{athena_python_pptx-0.5.0 → athena_python_pptx-0.7.1}/pptx/__init__.py

@@ -133,7 +133,7 @@ def flush_all() -> None:
     _active_buffers[:] = alive
 
 
-__version__ = "0.5.0"
+__version__ = "0.7.1"
 
 __all__ = [
     # Main entry point

{athena_python_pptx-0.5.0 → athena_python_pptx-0.7.1}/pptx/batching.py

@@ -8,11 +8,11 @@ context manager interface for batch operations.
 from __future__ import annotations
 
 __athena_extension_module__: bool = True
-__athena_extension_description__: str = (
-    "Command buffer + batching helpers (REST-SDK only)."
-)
+__athena_extension_description__: str = "Command buffer + batching helpers (REST-SDK only)."
 __athena_extension_since__: str = "0.1.0"
 
+import json
+import os
 import threading
 from contextlib import contextmanager
 from threading import local
@@ -20,12 +20,71 @@ from typing import TYPE_CHECKING, Any, Generator, Optional
 
 from . import _ptc
 from .commands import AnyCommand
+from .errors import BatchPartialError
 from .typing import CommandsResponse
 
 if TYPE_CHECKING:
     from .client import Client
 
 
+# The SDK serializes every buffered command into one JSON body and POSTs it to
+# ``/decks/:id/commands``. pptx-studio's apps/api (Fastify) caps that body at
+# 32 MiB, so a single oversized flush 413s and silently loses the entire write
+# (text edits + adds alike) because Fastify rejects the body before the route
+# handler runs. AddPicture / SubstitutePlaceholder(picture) carry inline base64
+# image bytes (~4/3 expansion), so a few large images blow past the ceiling.
+# ``CommandBuffer.flush`` therefore splits the buffer into chunks that stay
+# comfortably under that ceiling; the headroom absorbs the request envelope
+# (client/txn/return), JSON structural characters, and multi-byte expansion.
+_DEFAULT_MAX_REQUEST_BYTES = 24 * 1024 * 1024
+
+
+def _max_request_bytes() -> int:
+    """Per-request command-body budget in bytes.
+
+    ``ATHENA_PPTX_MAX_REQUEST_BYTES`` overrides the default when set to a
+    positive integer (e.g. for a server configured with a different limit).
+    """
+    raw = os.environ.get("ATHENA_PPTX_MAX_REQUEST_BYTES")
+    if raw:
+        try:
+            override = int(raw)
+        except ValueError:
+            override = 0
+        if override > 0:
+            return override
+    return _DEFAULT_MAX_REQUEST_BYTES
+
+
+def _command_size_bytes(command: AnyCommand) -> int:
+    """Serialized JSON byte size of one command's contribution to the body."""
+    return len(json.dumps(command.to_dict()).encode("utf-8"))
+
+
+def _chunk_commands_by_size(commands: list[AnyCommand], max_bytes: int) -> list[list[AnyCommand]]:
+    """Greedily pack ``commands`` into ordered chunks under ``max_bytes``.
+
+    A single command larger than ``max_bytes`` (e.g. a large inline base64
+    image) cannot be split, so it lands alone in its own chunk; that one POST
+    may still be rejected by the server, but only that command is at risk
+    rather than the entire flush.
+    """
+    chunks: list[list[AnyCommand]] = []
+    current: list[AnyCommand] = []
+    current_bytes = 0
+    for command in commands:
+        size = _command_size_bytes(command)
+        if current and current_bytes + size > max_bytes:
+            chunks.append(current)
+            current = []
+            current_bytes = 0
+        current.append(command)
+        current_bytes += size
+    if current:
+        chunks.append(current)
+    return chunks
+
+
 # Only commands whose class name is in this set produce PTC sub-tool-cards.
 # Every Command subclass is a candidate, but each emit is now a synchronous
 # HTTP POST to agora (see ``_ptc._send``), so emitting one card per low-level
@@ -112,20 +171,49 @@ class CommandBuffer:
         self._auto_flush_seconds = auto_flush_seconds
         self._flush_timer: Optional[threading.Timer] = None
         self._timer_lock = threading.Lock()
+        # Guards every read/append/reassignment of ``_commands`` so the
+        # background auto-flush timer thread can't interleave with foreground
+        # add/flush/clear calls and drop, duplicate, or reorder buffered
+        # commands. Held only for the in-memory buffer ops — NEVER across the
+        # network POST in ``flush`` (that would serialize every write behind a
+        # multi-second request and let the timer block the foreground).
+        self._commands_lock = threading.Lock()
         # Exception captured by the background auto-flush timer. Re-raised on
         # the next foreground operation (add/flush/refresh/save) so server-
         # rejected commands aren't silently lost.
         self._deferred_error: Optional[BaseException] = None
+        # True once a shape-creating command has been BUFFERED without a
+        # subsequent refresh. Buffered creates keep their local client id
+        # (the proxy is only reconciled to the server id by a full refresh),
+        # so this flag tells callers like ``Slide.clear_shapes`` whether a
+        # reconciling refresh is actually needed before deleting by id.
+        self._unreconciled_creates = False
 
     @property
     def is_batching(self) -> bool:
         """Return True if currently in batch mode."""
         return self._batch_depth > 0
 
+    @property
+    def has_unreconciled_creates(self) -> bool:
+        """True if a shape-creating command was buffered since the last refresh.
+
+        Buffered creates carry a client id (`shp_<uuid>`) that only reconciles
+        to the server id on a full `refresh()`. Synchronously-flushed creates
+        reconcile from the response immediately and do not set this flag.
+        """
+        return self._unreconciled_creates
+
+    def mark_creates_reconciled(self) -> None:
+        """Clear the unreconciled-creates flag. Called after a full refresh
+        rebuilds every shape proxy from the server snapshot (server ids)."""
+        self._unreconciled_creates = False
+
     @property
     def pending_count(self) -> int:
         """Return the number of pending commands."""
-        return len(self._commands)
+        with self._commands_lock:
+            return len(self._commands)
 
     @property
     def last_response(self) -> Optional[CommandsResponse]:
@@ -166,10 +254,17 @@ class CommandBuffer:
             except Exception:  # noqa: BLE001
                 pass
 
-        self._commands.append(command)
+        with self._commands_lock:
+            self._commands.append(command)
 
         if not self._auto_batch and not self.is_batching:
+            # Synchronous flush — the caller reconciles the proxy's id from the
+            # response, so this create does not leave a stale client id.
             return self.flush()
+        # Buffered create: its client id won't reconcile to the server id until
+        # the next refresh(), so flag it for clear_shapes et al.
+        if getattr(command, "client_id", None) is not None:
+            self._unreconciled_creates = True
         # Schedule auto-flush if enabled and not inside explicit batch()
         if self._auto_flush_seconds and self._auto_flush_seconds > 0 and not self.is_batching:
             self._schedule_auto_flush()
@@ -228,35 +323,99 @@ class CommandBuffer:
         Send all buffered commands to the server.
 
         Returns:
-            Server response, or None if no commands to send
+            Server response from the final POST, or None if no commands to send.
+
+        Large buffers are split into multiple ``/decks/:id/commands`` POSTs so
+        no single request exceeds the server's body-size limit (see
+        :func:`_max_request_bytes`). Each chunk is applied as its own atomic
+        server transaction and chunks are sent in order. No flush requests a
+        snapshot — auto-flushes never read it, so building one is wasted
+        O(whole-deck) work. A buffer that fits within the budget — the common
+        case — is sent as a single atomic POST, unchanged.
+
+        On POST failure (network, 4xx/5xx, auth, or a ``BatchPartialError``
+        for a chunk that committed a prefix), the commands that have not yet
+        been applied are restored to ``self._commands`` and the exception
+        propagates. Earlier chunks already committed server-side, so the
+        restore deliberately excludes them — callers that catch and retry
+        resend only the un-applied tail instead of double-applying.
         """
         self._cancel_timer()
         # Surface a background-flush error before draining the buffer, so
         # the caller sees it instead of a no-op return.
         self._raise_deferred_error()
 
-        if not self._commands:
-            return None
-
-        commands = self._commands
-        self._commands = []
-
+        # Atomically drain the buffer under the lock so a concurrent foreground
+        # ``add`` (or a second flush) can't have its append wiped by the
+        # ``= []`` below, and two flushes can't both POST the same commands.
+        # The POST itself runs OUTSIDE the lock.
+        with self._commands_lock:
+            if not self._commands:
+                return None
+            commands = self._commands
+            self._commands = []
+
+        chunks = _chunk_commands_by_size(commands, _max_request_bytes())
+
+        # A multi-chunk flush cannot be a single atomic server transaction, so
+        # validate the whole batch up front: an invalid command must abort the
+        # flush before any chunk is applied, preserving all-or-nothing
+        # semantics for bad input. Single-chunk flushes skip this —
+        # ``post_commands`` validates that one batch before it reaches the wire.
+        if len(chunks) > 1:
+            try:
+                for command in commands:
+                    command.validate()
+            except Exception:
+                with self._commands_lock:
+                    self._commands = commands + self._commands
+                _ptc_emit_end_batch(commands, is_error=True)
+                raise
+
+        applied = 0
         try:
-            self._last_response = self._client.post_commands(
-                self._deck_id,
-                commands,
-                return_snapshot=True,
-            )
+            for chunk in chunks:
+                self._last_response = self._client.post_commands(
+                    self._deck_id,
+                    chunk,
+                    return_snapshot=False,
+                )
+                applied += len(chunk)
+        except BatchPartialError as exc:
+            # The failing chunk committed its commands before
+            # ``failure_index`` server-side; everything in earlier chunks
+            # already committed too. Restore only the unsent suffix — the
+            # failing command (at ``applied + exc.failure_index``) and the
+            # rest of the buffer — so a retry doesn't re-apply the committed
+            # prefix.
+            committed = applied + exc.failure_index
+            self._restore_unsent(commands[committed:])
+            _ptc_emit_end_batch(commands[:committed], is_error=False)
+            _ptc_emit_end_batch(commands[committed:], is_error=True)
+            raise
         except Exception:
-            _ptc_emit_end_batch(commands, is_error=True)
+            # The current chunk's POST failed atomically (network, 4xx/5xx,
+            # auth), so nothing in it committed; earlier chunks already did.
+            # Restore only the un-applied tail so callers that retry resend
+            # the tail instead of double-applying the committed prefix.
+            self._restore_unsent(commands[applied:])
+            _ptc_emit_end_batch(commands[:applied], is_error=False)
+            _ptc_emit_end_batch(commands[applied:], is_error=True)
             raise
         _ptc_emit_end_batch(commands, is_error=False)
         return self._last_response
 
+    def _restore_unsent(self, commands: list[AnyCommand]) -> None:
+        """Prepend unsent commands back onto the buffer after a failed flush."""
+        if commands:
+            with self._commands_lock:
+                self._commands = commands + self._commands
+
     def clear(self) -> None:
         """Clear all pending commands without sending."""
         self._cancel_timer()
-        self._commands = []
+        with self._commands_lock:
+            self._commands = []
 
     @contextmanager
     def batch(self) -> Generator[CommandBuffer, None, None]: