@letta-ai/letta-code 0.34.16 → 0.34.18

package/skills/maintaining-deferred-memory/SKILL.md

@@ -0,0 +1,84 @@
+---
+name: maintaining-deferred-memory
+description: Maintains deferred memory, which stores accumulated knowledge, analysis, and observations made by the agent over time. Use when saving knowledge (e.g. about a person, project, repository, system, or event). Do NOT use for managing skills, prompts, or for saving information that is already automatically stored in the message history.
+---
+
+Deferred memory contains the agent's knowledge. Use deferred memory for understanding of a subject that matters only when that subject comes up. If the agent needs it on most turns, it is core memory: use the `maintaining-core-memory` skill. If it is a procedure with a trigger, use the `creating-skills` skill.
+
+Deferred memory should NOT be used to store:
+- Information that can be accessed by querying the message history (recall memory). For example, logs of specific interactions can simply be looked up in past messages. Reserve deferred memory to store knowledge that is actually *derived* from experience, or to organize knowledge in a way that optimizes recalling the right information at the right time through progressive disclosure.
+- Justifications or reasons for updating memory. These belong in the git commit history of MemFS.
+
+## Check the limits first
+
+Read `$MEMORY_DIR/.memfs.config.json` if it exists. The memory repository's pre-commit hook enforces it:
+
+- `maxDepth` limits how many directories can sit between the memory root and a file (default 2). `persona.md` is depth 0, `projects/MEMORY.md` is depth 1, and `projects/acme/notes.md` is depth 2. `skills/` is exempt.
+- `maxFileCharacters` caps each memory file, frontmatter included (default 20,000 characters). `fileCharacterLimits` overrides it per glob; the first match wins and `null` removes the cap.
+- `readOnlyFiles` lists globs whose files cannot change. Do not edit them.
+
+When the config file is absent, the defaults apply. When content does not fit, split it into more files or flatten the structure; change the config only when the user asks.
+
+## Layout
+
+```text
+$MEMORY_DIR/
+├── MEMORY.md              # root index (core memory)
+└── projects/
+    ├── MEMORY.md          # index for projects/
+    ├── acme.md
+    └── acme/
+        ├── MEMORY.md      # index for projects/acme/
+        └── incidents.md
+```
+- The top-level `MEMORY.md` is always in context, so it must tell the agent what each subdirectory contains and when to open it. Without that entry the directory is never read.
+- `skills/` is reserved for skills and is never deferred memory.
+- A directory is memory only when it has its own `MEMORY.md`. The agent will not discover a directory without one, and neither will the files in it.
+- Every `MEMORY.md` has no frontmatter.
+- Every other Markdown file starts with frontmatter containing exactly two fields:
+
+  ```markdown
+  ---
+  name: acme
+  description: How the Acme integration works, why it fails, and what to do about it.
+  ---
+  ```
+
+- `description` says what the file is about. The agent sees it when it lists the directory, so make it specific enough to decide whether to open the file.
+- Name files and directories after their subject, not after a date or session. Group related subjects under one directory instead of creating many top-level directories.
+
+## Write the indexes
+
+Discovery runs through the `MEMORY.md` chain, one level at a time:
+
+- The system prompt lists each top-level memory directory only by path. Add a line to root `MEMORY.md` saying when to open the directory, in the words a future request would use. When a deeper file's topic would not be guessable from that line, name the file there too.
+- When the agent opens a directory, it sees that directory's `MEMORY.md`, then each file's `description` and each subdirectory's index summary (the first prose paragraph of its `MEMORY.md`). Start every `MEMORY.md` with that summary paragraph.
+- A directory listing shows at most 10 entries, most recently edited first, so stable files fall out of it. Link every file that matters from its `MEMORY.md` rather than relying on the listing.
+- Use relative Markdown links from the memory root, such as `[Acme](projects/acme.md)`.
+
+After adding or moving a file, trace the path a future request would take from root `MEMORY.md` to the file. If any step depends on knowing the file already exists, fix the index.
+
+## Design for progressive disclosure
+
+Each level should give the agent enough to decide whether to go deeper, and nothing it would have to read past to decide.
+
+- Put frequently needed material shallow and rarely needed detail deeper. Keep related material in one directory so it loads together.
+- A small topic, where most of the content is useful whenever the subject comes up, belongs entirely in its directory's `MEMORY.md`. Splitting it adds reads without saving context.
+- For a large topic, `MEMORY.md` holds only what the agent needs every time it opens the directory, plus a map: what each deeper file or subdirectory covers and when it is worth opening. The map should also tell the agent when it can stop.
+- A `description` states what the file is about, not when to use it; when-to-open belongs in the parent's link. If you can write a "use when the user wants to…" trigger, the content is a procedure and belongs in a skill.
+- Make each statement stand alone. Name the subject instead of writing "it" or "the repo", because the agent may read only part of a file.
+- Aim for fewer than 10 entries per directory so the listing stays complete. At the `maxDepth` limit, use clearly scoped sibling files linked from `MEMORY.md` rather than nesting further.
+
+## Write analysis, not logs
+
+Recall already returns what happened. A deferred memory file is worth keeping only when it holds what recall cannot: the agent's own synthesis of a subject, built across many events and refined as understanding improves.
+
+- Before writing, ask whether searching message history would give a future agent the same thing. If it would, do not write the file.
+- Write the current understanding: how the subject works, the patterns across events, why things failed or succeeded, what to do about it, and the open questions.
+- Revise the file in place when new evidence arrives. Fold the new event into the analysis; never append a dated entry, a quote, or an incident narrative.
+- When the file you are updating is already a log, convert it: write the analysis the entries support and delete the entries. Keeping them under an "incident history" heading leaves the log in place; git and recall already hold it.
+- When a raw event matters, point to it with terms that will find it in recall rather than copying it.
+- Keep one home per subject. Update the existing file instead of adding a near-duplicate, and replace stale conclusions where they live.
+- Leave out content the agent can cheaply rederive from the source of truth: file contents, command lists, and current status. Link to the owning source instead.
+- Never store secrets. Reference them by name, such as `$SECRET_NAME`.
+

package/skills/music-generation/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 ElevenLabs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/skills/music-generation/SKILL.md

@@ -0,0 +1,164 @@
+---
+name: music-generation
+description: Generate music using ElevenLabs Music through Letta. Use when creating instrumental tracks, songs with lyrics, background music, jingles, theme songs, or any AI-generated music composition. Supports prompt-based generation and composition plans for section-by-section control.
+license: MIT
+---
+
+# ElevenLabs Music Generation (via Letta)
+
+Generate music from text prompts: instrumental tracks, songs with lyrics, and fine-grained control via composition plans. Examples use `music_v2_5`, the most advanced ElevenLabs music model and the endpoint's default.
+
+This skill is adapted from ElevenLabs' `music` skill
+([elevenlabs/skills](https://github.com/elevenlabs/skills), MIT; see `LICENSE`).
+Instead of calling ElevenLabs with your own key, you call Letta's hosted endpoint
+`POST /v1/audio/music` (or `POST /v1/audio/music/stream`). It forwards the
+request to ElevenLabs with Letta's key and charges the organization's Letta
+credits, so you do not need an ElevenLabs API key. The request and response are
+the same as ElevenLabs "Compose music": a JSON body with the same field names,
+`output_format` as a query parameter, and the audio file as the response body.
+
+## Quick Start
+
+```bash
+base_url="${LETTA_BASE_URL%/}"
+
+curl -sS -X POST "$base_url/v1/audio/music/stream?output_format=mp3_48000_192" \
+  -H "Authorization: Bearer $LETTA_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"prompt":"A chill lo-fi hip hop beat with jazzy piano chords","music_length_ms":30000,"model_id":"music_v2_5"}' \
+  --output output.mp3 \
+  --write-out '%{http_code} %{content_type}\n'
+```
+
+A `200` with an `audio/...` content type means `output.mp3` is the song. Any
+other status means `output.mp3` holds a JSON error with a `message` instead;
+read it (see "Error Handling") and do not share the file.
+
+Use the runtime-provided `LETTA_BASE_URL` and `LETTA_API_KEY` together. Build
+URLs relative to `${LETTA_BASE_URL%/}` and do not hardcode
+`https://api.letta.com`. If either variable is missing, the user needs to
+authenticate with Letta Cloud; do not ask for an ElevenLabs key.
+
+Generation takes a while (roughly 10-30 seconds per minute of music). Prefer
+`/v1/audio/music/stream`: it sends audio as it is generated, so long songs do
+not sit on an idle connection. `/v1/audio/music` takes the same request and
+returns the same file, but may send nothing until the whole song is done, so
+use it only for short songs. Do not set a short client timeout either way.
+
+Share the saved file the way your current surface supports: attach or upload it
+to the conversation or channel you are replying in. Keep credit amounts out of
+replies unless the user asks about cost.
+
+## Length
+
+- A song is 3 seconds to 10 minutes (600,000 ms).
+- With `prompt`, `music_length_ms` is optional. Leave it out to let the model
+  choose a length that suits the prompt.
+- With `composition_plan`, the length is the sum of the chunk or section
+  `duration_ms` values; do not also send `music_length_ms`.
+- Credits are charged for the length of the audio actually returned, at
+  ElevenLabs' per-minute price. Before generating, the organization needs
+  enough credits for the longest song the request allows: `music_length_ms`,
+  the plan total, or 10 minutes when a prompt sets no length. Setting
+  `music_length_ms` lowers that requirement.
+
+## Models
+
+| Model ID | Best For |
+|----------|----------|
+| `music_v2_5` | **Default.** Most advanced: best audio quality and prompt adherence. Chunk-based composition plans. |
+| `music_v2` | Previous generation. Chunk-based composition plans. |
+| `music_v1` | Oldest. Section-based composition plans. Use only when explicitly requested. |
+
+## Prompting
+
+Describe the music the way a producer would brief a session:
+
+- Genre and mood: "melancholic indie folk", "triumphant orchestral trailer".
+- Instrumentation and vocals: "fingerpicked acoustic guitar, soft female vocals".
+- Tempo and key when they matter: "90 BPM", "D minor".
+- Structure and energy: "starts sparse, builds into a big chorus".
+- Lyrics: put them in the prompt for short songs, or use a composition plan
+  to control which lines land in which section.
+- Set `force_instrumental: true` to guarantee no vocals (prompt mode only).
+
+## Composition Plans
+
+`music_v2_5` composition plans are an ordered list of `chunks`. Each chunk specifies its own
+`text` (section label, lyrics, inline cues), `duration_ms`, `positive_styles`, `negative_styles`,
+and `context_adherence` (`low`, `medium`, or `high`, default `high`). Up to 30 chunks per plan,
+each 3,000–120,000 ms. Each chunk's `text` supports up to 6,132 characters, with up to 30 lines
+of 200 characters each. Send the plan instead of `prompt` and `music_length_ms`:
+
+```json
+{
+  "model_id": "music_v2_5",
+  "composition_plan": {
+    "chunks": [
+      {
+        "text": "[Verse]\nWalking down an empty street",
+        "duration_ms": 15000,
+        "positive_styles": ["pop", "upbeat", "female vocals", "acoustic guitar", "120 BPM", "polished production"],
+        "negative_styles": ["dark", "slow"],
+        "context_adherence": "high"
+      },
+      {
+        "text": "[Chorus]\nThis is my moment",
+        "duration_ms": 15000,
+        "positive_styles": ["powerful vocals", "full band"],
+        "negative_styles": [],
+        "context_adherence": "high"
+      }
+    ]
+  }
+}
+```
+
+- Put broader characteristics (genre, instrumentation, vocal style) in `positive_styles`, not in
+  `text`. The first chunk's styles set the overall tone — include 6–7 styles there.
+- Use later chunks' styles for progression: "bass joins in", "building intensity", "drums softening".
+- Inline cues in curly braces add moments inside a chunk: `{guitar solo}`, `{vinyl crackle}`.
+- Lower `context_adherence` lets a chunk depart from its neighbors, for a bridge or a key change.
+- `seed` works with composition plans (not with `prompt`) to make results more repeatable.
+
+For `music_v1`, plans use `positive_global_styles`, `negative_global_styles`, and `sections`
+(each with `section_name`, `positive_local_styles`, `negative_local_styles`, `duration_ms`, and
+`lines`), and `respect_sections_durations: false` lets the model adjust section lengths.
+
+Inpainting and audio references (`song_id` chunks, `conditioning_ref`, `source_from`) are not
+available through this endpoint and return 400.
+
+## Output Formats
+
+Pass `output_format` as a query parameter, not in the body. The default, `auto`,
+gives `mp3_48000_192` for `music_v2` and later (`mp3_44100_128` for `music_v1`).
+Name the file to match the format.
+
+| Format | Description |
+|--------|-------------|
+| `mp3_48000_192` | MP3 48kHz 192kbps (what `auto` picks for v2 models) |
+| `mp3_48000_320` | MP3 48kHz 320kbps - highest quality MP3 |
+| `mp3_44100_128` | MP3 44.1kHz 128kbps - smaller files |
+| `pcm_48000` | Raw 16-bit PCM 48kHz, no header - for further processing |
+| `opus_48000_128` | Opus 48kHz 128kbps in Ogg - efficient codec, small files |
+
+## Content Restrictions
+
+- Cannot reference specific artists, bands, or copyrighted lyrics. Describe the style instead
+  ("1960s British invasion rock" rather than a band name).
+- A rejected prompt returns 400 with ElevenLabs' `bad_prompt` detail, which includes a
+  `prompt_suggestion` with alternative phrasing. A rejected plan returns `bad_composition_plan`
+  with a `composition_plan_suggestion`. Retry with the suggestion.
+
+## Error Handling
+
+Errors return JSON with a `message`:
+- **400 with an ElevenLabs message**: invalid parameters or restricted content (see "Content Restrictions").
+- **400 "Provide prompt or composition_plan."**: send one of them.
+- **402**: the organization does not have enough Letta credits for the longest song the request
+  allows (`credits_required`). Setting a shorter `music_length_ms` may help; otherwise tell the user
+  and do not retry.
+- **500**: retry once, then report the failure.
+
+If the connection drops partway through `/stream`, the file is truncated; only the audio
+received is charged. Retry the request rather than sharing a partial song.

package/skills/speech-generation/SKILL.md

@@ -14,7 +14,8 @@ Instead of calling ElevenLabs with your own key, you call Letta's hosted endpoin
 `POST /v1/audio/speech`. It forwards the request to ElevenLabs with Letta's key
 and charges the organization's Letta credits, so you do not need an ElevenLabs
 API key. The request body uses the same field names as ElevenLabs "Create
-speech"; the voice ID goes in the body.
+speech" (the voice ID and `output_format` go in the body), and the response is
+the audio file itself, as with ElevenLabs.
 
 ## Quick Start
 
@@ -25,20 +26,9 @@ curl -sS -X POST "$base_url/v1/audio/speech" \
   -H "Authorization: Bearer $LETTA_API_KEY" \
   -H "Content-Type: application/json" \
   -d '{"text":"Hello, welcome to ElevenLabs!","voice_id":"JBFqnCBsd6RMkjVDRZzb","model_id":"eleven_v4"}' \
-  > speech-response.json
-
-python3 - <<'PY'
-import base64, json
-
-with open("speech-response.json") as f:
-    response = json.load(f)
-if "audio_b64" not in response:
-    raise SystemExit(response.get("message", response))
-
-with open("output.mp3", "wb") as f:
-    f.write(base64.b64decode(response["audio_b64"]))
-print("saved output.mp3")
-PY
+  --output output.mp3 \
+  --dump-header speech-headers.txt \
+  --write-out '%{http_code} %{content_type}\n'
 ```
 
 Use the runtime-provided `LETTA_BASE_URL` and `LETTA_API_KEY` together. Build
@@ -46,9 +36,13 @@ URLs relative to `${LETTA_BASE_URL%/}` and do not hardcode
 `https://api.letta.com`. If either variable is missing, the user needs to
 authenticate with Letta Cloud; do not ask for an ElevenLabs key.
 
+A `200` with an `audio/...` content type means `output.mp3` is the speech. Any
+other status means `output.mp3` holds a JSON error with a `message` instead;
+read it (see "Error Handling") and do not share the file.
+
 Share the saved file the way your current surface supports: attach or upload it
-to the conversation or channel you are replying in. Never paste the base64 into
-a reply. Keep credit amounts out of replies unless the user asks about cost.
+to the conversation or channel you are replying in. Keep credit amounts out of
+replies unless the user asks about cost.
 
 ## Models
 
@@ -160,8 +154,9 @@ When generating long audio in multiple requests, the audio can have pops, unnatu
 {"text": "And this continues the story.", "voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_v4", "previous_text": "This is the first part."}
 ```
 
-Each response includes `request_id`. Instead of `previous_text`, you can pass up
-to three earlier `request_id` values in `previous_request_ids` (and later ones in
+Each response has a `request-id` header (saved by `--dump-header` above, e.g.
+`grep -i '^request-id:' speech-headers.txt`). Instead of `previous_text`, you can
+pass up to three earlier request IDs in `previous_request_ids` (and later ones in
 `next_request_ids` when regenerating a middle part). Keep the same voice, model,
 and voice settings for every part, split long text at sentence or paragraph
 boundaries (each request takes up to 10,000 characters), and concatenate the
@@ -169,7 +164,7 @@ files in order, for example with `ffmpeg -f concat`.
 
 ## Output Formats
 
-Pass `output_format` in the body.
+Pass `output_format` in the body, and name the file to match the format.
 
 | Format | Description |
 |--------|-------------|
@@ -187,22 +182,6 @@ Pass `output_format` in the body.
 | `opus_48000_64` | Opus 48kHz 64kbps - efficient codec, small files |
 | `wav_44100` | WAV 44.1kHz - uncompressed with headers |
 
-The Letta endpoint returns the whole file at once (as `audio_b64`); it does not stream.
-
-## Response
-
-```json
-{
-  "model_id": "eleven_v4",
-  "voice_id": "JBFqnCBsd6RMkjVDRZzb",
-  "output_format": "mp3_44100_128",
-  "audio_b64": "<base64>",
-  "request_id": "Frhov0LQNC3M9HcHKY3t",
-  "characters": 46,
-  "billing": { "credits_charged": 4, "transaction_id": "...", "new_credit_balance": "..." }
-}
-```
-
 ## Error Handling
 
 Errors return JSON with a `message`:

package/skills/workflow-authoring/SKILL.md

@@ -80,7 +80,9 @@ Cloud decisions service.
   `phase` (progress group — use this inside concurrent stages), `schema`,
   `json`, `model`, `effort` (`'low'` for mechanical stages, higher for the
   hardest verify/judge stages), `allowedTools`, `systemPrompt` (extra system
-  prompt for this subagent), `timeoutMs` (default 10 minutes), `maxToolCalls`
+  prompt for this subagent), `skills` (skill names whose full content is
+  loaded into the subagent's prompt; workers otherwise have no skills),
+  `timeoutMs` (default 10 minutes), `maxToolCalls`
   (positive safe integer; default 1000 unique tool calls for this subagent),
   `conversationId` (resume a worker — see "Diagnosing a run").
 - `pipeline(items, stage1, stage2, ...)` → run each item through all stages