quiver-cli 0.8.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (200) hide show
  1. package/README.md +127 -45
  2. package/bin/quiver-cli.mjs +3 -1
  3. package/dist/cli.js +666 -436
  4. package/package.json +2 -2
  5. package/template/.agents/AGENTS.md +4 -3
  6. package/template/.agents/config.json +7 -0
  7. package/template/.agents/plugins/opencode/rtk.ts +34 -0
  8. package/template/.agents/skills/agent-browser/SKILL.md +1 -0
  9. package/template/.agents/skills/apps/skybridge/SKILL.md +4 -0
  10. package/template/.agents/skills/design/impeccable/SKILL.md +36 -118
  11. package/template/.agents/skills/design/impeccable/reference/adapt.md +1 -0
  12. package/template/.agents/skills/design/impeccable/reference/adapt.native.md +58 -0
  13. package/template/.agents/skills/design/impeccable/reference/android.md +40 -0
  14. package/template/.agents/skills/design/impeccable/reference/animate.md +73 -188
  15. package/template/.agents/skills/design/impeccable/reference/audit.md +12 -9
  16. package/template/.agents/skills/design/impeccable/reference/audit.native.md +139 -0
  17. package/template/.agents/skills/design/impeccable/reference/bolder.md +19 -101
  18. package/template/.agents/skills/design/impeccable/reference/clarify.md +59 -253
  19. package/template/.agents/skills/design/impeccable/reference/colorize.md +51 -222
  20. package/template/.agents/skills/design/impeccable/reference/craft-floor.md +42 -0
  21. package/template/.agents/skills/design/impeccable/reference/craft.md +3 -121
  22. package/template/.agents/skills/design/impeccable/reference/critique.md +44 -23
  23. package/template/.agents/skills/design/impeccable/reference/degraded/asset-producer.md +90 -0
  24. package/template/.agents/skills/design/impeccable/reference/degraded/documenter.md +24 -0
  25. package/template/.agents/skills/design/impeccable/reference/degraded/finish-reviewer.md +37 -0
  26. package/template/.agents/skills/design/impeccable/reference/degraded/manual-edit-applier.md +92 -0
  27. package/template/.agents/skills/design/impeccable/reference/delight.md +47 -279
  28. package/template/.agents/skills/design/impeccable/reference/distill.md +2 -2
  29. package/template/.agents/skills/design/impeccable/reference/doctor.md +53 -0
  30. package/template/.agents/skills/design/impeccable/reference/document.md +60 -73
  31. package/template/.agents/skills/design/impeccable/reference/harden.md +1 -12
  32. package/template/.agents/skills/design/impeccable/reference/hooks.md +20 -5
  33. package/template/.agents/skills/design/impeccable/reference/init.md +72 -119
  34. package/template/.agents/skills/design/impeccable/reference/ios.md +45 -0
  35. package/template/.agents/skills/design/impeccable/reference/layout.md +54 -131
  36. package/template/.agents/skills/design/impeccable/reference/live-setup.md +102 -0
  37. package/template/.agents/skills/design/impeccable/reference/live.md +116 -511
  38. package/template/.agents/skills/design/impeccable/reference/new-work.md +105 -0
  39. package/template/.agents/skills/design/impeccable/reference/{product.md → operate.md} +6 -5
  40. package/template/.agents/skills/design/impeccable/reference/optimize.md +4 -4
  41. package/template/.agents/skills/design/impeccable/reference/overdrive.md +1 -4
  42. package/template/.agents/skills/design/impeccable/reference/polish.md +68 -212
  43. package/template/.agents/skills/design/impeccable/reference/quieter.md +3 -3
  44. package/template/.agents/skills/design/impeccable/reference/routing.md +18 -0
  45. package/template/.agents/skills/design/impeccable/reference/shape.md +38 -144
  46. package/template/.agents/skills/design/impeccable/reference/typeset.md +51 -250
  47. package/template/.agents/skills/design/impeccable/reference/visualize.md +47 -0
  48. package/template/.agents/skills/design/impeccable/scripts/command-metadata.json +1 -1
  49. package/template/.agents/skills/design/impeccable/scripts/concept-seed.mjs +558 -0
  50. package/template/.agents/skills/design/impeccable/scripts/context-signals.mjs +119 -10
  51. package/template/.agents/skills/design/impeccable/scripts/context.mjs +534 -45
  52. package/template/.agents/skills/design/impeccable/scripts/critique-storage.mjs +18 -47
  53. package/template/.agents/skills/design/impeccable/scripts/detector/browser/injected/index.mjs +96 -10
  54. package/template/.agents/skills/design/impeccable/scripts/detector/cli/main.mjs +174 -26
  55. package/template/.agents/skills/design/impeccable/scripts/detector/design-system.mjs +233 -0
  56. package/template/.agents/skills/design/impeccable/scripts/detector/detect-antipatterns-browser.js +3348 -203
  57. package/template/.agents/skills/design/impeccable/scripts/detector/engines/browser/detect-url.mjs +102 -7
  58. package/template/.agents/skills/design/impeccable/scripts/detector/engines/regex/detect-text.mjs +297 -97
  59. package/template/.agents/skills/design/impeccable/scripts/detector/engines/static-html/css-cascade.mjs +187 -16
  60. package/template/.agents/skills/design/impeccable/scripts/detector/engines/static-html/detect-html.mjs +41 -11
  61. package/template/.agents/skills/design/impeccable/scripts/detector/findings.mjs +7 -1
  62. package/template/.agents/skills/design/impeccable/scripts/detector/node/file-system.mjs +16 -2
  63. package/template/.agents/skills/design/impeccable/scripts/detector/registry/antipatterns.mjs +207 -38
  64. package/template/.agents/skills/design/impeccable/scripts/detector/rules/checks.mjs +3082 -173
  65. package/template/.agents/skills/design/impeccable/scripts/detector/shared/constants.mjs +11 -0
  66. package/template/.agents/skills/design/impeccable/scripts/detector/shared/fonts.mjs +30 -0
  67. package/template/.agents/skills/design/impeccable/scripts/doctor.mjs +336 -0
  68. package/template/.agents/skills/design/impeccable/scripts/embed-prompt.mjs +133 -0
  69. package/template/.agents/skills/design/impeccable/scripts/generate-image.mjs +240 -0
  70. package/template/.agents/skills/design/impeccable/scripts/hook-admin.mjs +98 -18
  71. package/template/.agents/skills/design/impeccable/scripts/hook-before-edit.mjs +46 -6
  72. package/template/.agents/skills/design/impeccable/scripts/hook-lib.mjs +518 -50
  73. package/template/.agents/skills/design/impeccable/scripts/hook.mjs +25 -8
  74. package/template/.agents/skills/design/impeccable/scripts/lib/artifact-schema.mjs +93 -0
  75. package/template/.agents/skills/design/impeccable/scripts/lib/composition-catalog.mjs +200 -0
  76. package/template/.agents/skills/design/impeccable/scripts/lib/concept-catalog.mjs +357 -0
  77. package/template/.agents/skills/design/impeccable/scripts/lib/impeccable-config.mjs +27 -7
  78. package/template/.agents/skills/design/impeccable/scripts/lib/impeccable-paths.mjs +17 -8
  79. package/template/.agents/skills/design/impeccable/scripts/lib/provider.mjs +5 -0
  80. package/template/.agents/skills/design/impeccable/scripts/lib/roll-selection.mjs +362 -0
  81. package/template/.agents/skills/design/impeccable/scripts/lib/staleness-deep.mjs +457 -0
  82. package/template/.agents/skills/design/impeccable/scripts/lib/staleness-notice.mjs +169 -0
  83. package/template/.agents/skills/design/impeccable/scripts/lib/staleness.mjs +457 -0
  84. package/template/.agents/skills/design/impeccable/scripts/lib/surface-briefs.mjs +151 -0
  85. package/template/.agents/skills/design/impeccable/scripts/lib/target-slug.mjs +33 -0
  86. package/template/.agents/skills/design/impeccable/scripts/lib/template-extensions.mjs +146 -0
  87. package/template/.agents/skills/design/impeccable/scripts/live/accept-css.mjs +617 -0
  88. package/template/.agents/skills/design/impeccable/scripts/live/accept-verify.mjs +60 -0
  89. package/template/.agents/skills/design/impeccable/scripts/live/browser-script-parts.mjs +7 -1
  90. package/template/.agents/skills/design/impeccable/scripts/live/completion.mjs +10 -1
  91. package/template/.agents/skills/design/impeccable/scripts/live/event-validation.mjs +67 -5
  92. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/astro.mjs +47 -0
  93. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/detect-utils.mjs +73 -0
  94. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/index.mjs +143 -0
  95. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/journal.mjs +197 -0
  96. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/nextjs.mjs +49 -0
  97. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/nuxt.mjs +161 -0
  98. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/script-src.mjs +17 -0
  99. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/static-html.mjs +26 -0
  100. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/sveltekit.mjs +71 -0
  101. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/tag-strategy.mjs +247 -0
  102. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/tanstack-start.mjs +70 -0
  103. package/template/.agents/skills/design/impeccable/scripts/live/frameworks/vite-generic.mjs +42 -0
  104. package/template/.agents/skills/design/impeccable/scripts/live/generation-preflight.mjs +149 -0
  105. package/template/.agents/skills/design/impeccable/scripts/live/instructions.mjs +142 -0
  106. package/template/.agents/skills/design/impeccable/scripts/live/poll-lanes.mjs +14 -0
  107. package/template/.agents/skills/design/impeccable/scripts/live/roots.mjs +508 -0
  108. package/template/.agents/skills/design/impeccable/scripts/live/session-store.mjs +324 -50
  109. package/template/.agents/skills/design/impeccable/scripts/live/source-lock.mjs +105 -0
  110. package/template/.agents/skills/design/impeccable/scripts/live/source-search.mjs +105 -0
  111. package/template/.agents/skills/design/impeccable/scripts/live/svelte-ast.mjs +961 -0
  112. package/template/.agents/skills/design/impeccable/scripts/live/svelte-component.mjs +588 -72
  113. package/template/.agents/skills/design/impeccable/scripts/live/sveltekit-adapter.mjs +59 -17
  114. package/template/.agents/skills/design/impeccable/scripts/live/tanstack-adapter.mjs +280 -0
  115. package/template/.agents/skills/design/impeccable/scripts/live/vocabulary.mjs +135 -0
  116. package/template/.agents/skills/design/impeccable/scripts/live-accept.mjs +210 -68
  117. package/template/.agents/skills/design/impeccable/scripts/live-browser.js +1562 -223
  118. package/template/.agents/skills/design/impeccable/scripts/live-commit-manual-edits.mjs +3 -0
  119. package/template/.agents/skills/design/impeccable/scripts/live-complete.mjs +33 -1
  120. package/template/.agents/skills/design/impeccable/scripts/live-inject.mjs +175 -255
  121. package/template/.agents/skills/design/impeccable/scripts/live-insert.mjs +26 -6
  122. package/template/.agents/skills/design/impeccable/scripts/live-manual-edit-evidence.mjs +6 -1
  123. package/template/.agents/skills/design/impeccable/scripts/live-poll.mjs +61 -16
  124. package/template/.agents/skills/design/impeccable/scripts/live-resume.mjs +39 -10
  125. package/template/.agents/skills/design/impeccable/scripts/live-server.mjs +573 -47
  126. package/template/.agents/skills/design/impeccable/scripts/live-status.mjs +17 -7
  127. package/template/.agents/skills/design/impeccable/scripts/live-wrap.mjs +124 -91
  128. package/template/.agents/skills/design/impeccable/scripts/live.mjs +88 -26
  129. package/template/.agents/skills/design/impeccable/scripts/palette.mjs +76 -81
  130. package/template/.agents/skills/design/impeccable/scripts/pin.mjs +18 -11
  131. package/template/.agents/skills/design/impeccable/scripts/serve-question.mjs +932 -0
  132. package/template/.agents/skills/design/impeccable/scripts/surface-brief.mjs +74 -0
  133. package/template/.agents/skills/design/shadcn/SKILL.md +46 -11
  134. package/template/.agents/skills/design/shadcn/cli.md +49 -16
  135. package/template/.agents/skills/design/shadcn/customization.md +14 -7
  136. package/template/.agents/skills/design/shadcn/evals/evals.json +30 -0
  137. package/template/.agents/skills/design/shadcn/mcp.md +27 -16
  138. package/template/.agents/skills/design/shadcn/registry.md +277 -0
  139. package/template/.agents/skills/design/shadcn/rules/chat.md +224 -0
  140. package/template/.agents/skills/design/shadcn/rules/composition.md +20 -2
  141. package/template/.agents/skills/design/shadcn/rules/styling.md +23 -0
  142. package/template/.agents/skills/find-skills/SKILL.md +2 -3
  143. package/template/.agents/skills/hono/SKILL.md +579 -0
  144. package/template/.agents/skills/integrations/langfuse/SKILL.md +10 -6
  145. package/template/.agents/skills/integrations/langfuse/references/ci-cd.md +41 -0
  146. package/template/.agents/skills/integrations/langfuse/references/cli.md +8 -0
  147. package/template/.agents/skills/integrations/langfuse/references/error-analysis.md +15 -27
  148. package/template/.agents/skills/integrations/langfuse/references/instrumentation.md +39 -52
  149. package/template/.agents/skills/integrations/langfuse/references/judge-calibration.md +5 -2
  150. package/template/.agents/skills/integrations/langfuse/references/prompt-engineering.md +35 -0
  151. package/template/.agents/skills/integrations/langfuse/references/prompt-migration.md +41 -196
  152. package/template/.agents/skills/integrations/langfuse/references/skill-feedback.md +3 -0
  153. package/template/.agents/skills/integrations/langfuse/references/trace-evaluator-upgrade.md +76 -0
  154. package/template/.agents/skills/integrations/langfuse/references/user-feedback.md +4 -0
  155. package/template/.agents/skills/integrations/langfuse/references/v4-project-migration.md +73 -0
  156. package/template/.agents/skills/supabase/CHANGELOG.md +71 -0
  157. package/template/.agents/skills/supabase/SKILL.md +145 -0
  158. package/template/.agents/skills/supabase/assets/feedback-issue-template.md +17 -0
  159. package/template/.agents/skills/supabase/references/skill-feedback.md +17 -0
  160. package/template/.agents/skills/supabase-postgres-best-practices/CHANGELOG.md +73 -0
  161. package/template/.agents/skills/supabase-postgres-best-practices/SKILL.md +64 -0
  162. package/template/.agents/skills/supabase-postgres-best-practices/references/_contributing.md +170 -0
  163. package/template/.agents/skills/supabase-postgres-best-practices/references/_sections.md +39 -0
  164. package/template/.agents/skills/supabase-postgres-best-practices/references/_template.md +34 -0
  165. package/template/.agents/skills/supabase-postgres-best-practices/references/advanced-full-text-search.md +55 -0
  166. package/template/.agents/skills/supabase-postgres-best-practices/references/advanced-jsonb-indexing.md +49 -0
  167. package/template/.agents/skills/supabase-postgres-best-practices/references/conn-idle-timeout.md +46 -0
  168. package/template/.agents/skills/supabase-postgres-best-practices/references/conn-limits.md +44 -0
  169. package/template/.agents/skills/supabase-postgres-best-practices/references/conn-pooling.md +41 -0
  170. package/template/.agents/skills/supabase-postgres-best-practices/references/conn-prepared-statements.md +46 -0
  171. package/template/.agents/skills/supabase-postgres-best-practices/references/data-batch-inserts.md +54 -0
  172. package/template/.agents/skills/supabase-postgres-best-practices/references/data-n-plus-one.md +53 -0
  173. package/template/.agents/skills/supabase-postgres-best-practices/references/data-pagination.md +50 -0
  174. package/template/.agents/skills/supabase-postgres-best-practices/references/data-upsert.md +50 -0
  175. package/template/.agents/skills/supabase-postgres-best-practices/references/lock-advisory.md +56 -0
  176. package/template/.agents/skills/supabase-postgres-best-practices/references/lock-deadlock-prevention.md +68 -0
  177. package/template/.agents/skills/supabase-postgres-best-practices/references/lock-short-transactions.md +50 -0
  178. package/template/.agents/skills/supabase-postgres-best-practices/references/lock-skip-locked.md +54 -0
  179. package/template/.agents/skills/supabase-postgres-best-practices/references/monitor-explain-analyze.md +45 -0
  180. package/template/.agents/skills/supabase-postgres-best-practices/references/monitor-pg-stat-statements.md +55 -0
  181. package/template/.agents/skills/supabase-postgres-best-practices/references/monitor-vacuum-analyze.md +55 -0
  182. package/template/.agents/skills/supabase-postgres-best-practices/references/query-composite-indexes.md +44 -0
  183. package/template/.agents/skills/supabase-postgres-best-practices/references/query-covering-indexes.md +40 -0
  184. package/template/.agents/skills/supabase-postgres-best-practices/references/query-index-types.md +48 -0
  185. package/template/.agents/skills/supabase-postgres-best-practices/references/query-missing-indexes.md +43 -0
  186. package/template/.agents/skills/supabase-postgres-best-practices/references/query-partial-indexes.md +45 -0
  187. package/template/.agents/skills/supabase-postgres-best-practices/references/schema-constraints.md +80 -0
  188. package/template/.agents/skills/supabase-postgres-best-practices/references/schema-data-types.md +46 -0
  189. package/template/.agents/skills/supabase-postgres-best-practices/references/schema-foreign-key-indexes.md +59 -0
  190. package/template/.agents/skills/supabase-postgres-best-practices/references/schema-lowercase-identifiers.md +55 -0
  191. package/template/.agents/skills/supabase-postgres-best-practices/references/schema-partitioning.md +55 -0
  192. package/template/.agents/skills/supabase-postgres-best-practices/references/schema-primary-keys.md +61 -0
  193. package/template/.agents/skills/supabase-postgres-best-practices/references/security-privileges.md +54 -0
  194. package/template/.agents/skills/supabase-postgres-best-practices/references/security-rls-basics.md +50 -0
  195. package/template/.agents/skills/supabase-postgres-best-practices/references/security-rls-performance.md +63 -0
  196. package/template/.agents/upstreams.json +34 -12
  197. package/template/.agents/skills/design/impeccable/reference/brand.md +0 -108
  198. package/template/.agents/skills/design/impeccable/reference/codex.md +0 -105
  199. package/template/.agents/skills/design/impeccable/reference/interaction-design.md +0 -189
  200. package/template/.agents/skills/integrations/langfuse/references/sdk-upgrade.md +0 -175
@@ -8,6 +8,9 @@ description: Deep-dive error analysis of an LLM pipeline or AI application using
8
8
  errors", "build a failure taxonomy", "what's going wrong with my pipeline", or any
9
9
  request to systematically inspect, annotate, or score Langfuse traces. If the user
10
10
  is trying to understand or improve the quality of an AI system's outputs, use this skill.
11
+ metadata:
12
+ required_access:
13
+ - LANGFUSE_PROJECT_INTERFACE
11
14
  ---
12
15
 
13
16
  # Error Analysis
@@ -24,43 +27,28 @@ Read it in full. It defines the authoritative 5-step process (sample selection
24
27
 
25
28
  **2. Guide the user through this step by step**
26
29
 
27
- You as a coding agent and the user go through this together to perform a full error analysis with their data in langfuse. Do everything you can achieve via CLI (look up traces, create annotation queues, ...) for the user. Provide them with direct links to UI wherever their action is required. Be proactive and narrate what is going on for the user.
30
+ You as a coding agent and the user go through this together to perform a full error analysis with their data in langfuse. Do as much of the work as you can directly for the user (look up traces, create annotation queues, ...). Provide them with direct links to UI wherever their action is required. Be proactive and narrate what is going on for the user.
28
31
 
29
32
  ## Rules CRITICAL
30
- Use Langfuse CLI wherever possible
33
+ Perform interactions with the user's Langfuse instance yourself rather than telling the user to do them — don't say "now do this in Langfuse" when you can do it directly
34
+ But don't barrel through on assumptions: where a step needs the user's judgment or input (e.g. what to fix, how to label, which evaluator), pause and ask before acting
31
35
  Use charts where possible to display data
32
36
 
33
37
  ---
34
38
 
35
39
  ## Langfuse Implementation Notes
36
40
 
37
- The guide describes the process. These notes cover the Langfuse-specific API and CLI mechanics required to execute it.
41
+ The guide describes the process. These notes cover the Langfuse-specific mechanics required to execute it.
38
42
 
39
43
  ### Credentials
40
44
 
41
- ```bash
42
- echo $LANGFUSE_PUBLIC_KEY # pk-lf-...
43
- echo $LANGFUSE_SECRET_KEY # sk-lf-...
44
- echo $LANGFUSE_BASE_URL # https://cloud.langfuse.com (EU), https://us.cloud.langfuse.com (US), https://jp.cloud.langfuse.com (JP) or self-hosted
45
- ```
45
+ Make sure Langfuse credentials are available before starting — a public key (`pk-lf-...`), a secret key (`sk-lf-...`), and the host (e.g. `https://cloud.langfuse.com` for EU, `https://us.cloud.langfuse.com` for US, `https://jp.cloud.langfuse.com` for JP, or a self-hosted URL). If they aren't configured, ask the user to set them — do not ask them to paste secret values into chat.
46
46
 
47
- If not set, check `.env` in the project root: `export $(grep -v '^#' .env | xargs)`. If `LANGFUSE_HOST` is used instead of `LANGFUSE_BASE_URL`, run `export LANGFUSE_BASE_URL="$LANGFUSE_HOST"`.
48
-
49
- ```bash
50
- AUTH=$(echo -n "${LANGFUSE_PUBLIC_KEY}:${LANGFUSE_SECRET_KEY}" | base64)
51
-
52
- # Verify before proceeding
53
- STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
54
- -H "Authorization: Basic $AUTH" \
55
- "${LANGFUSE_BASE_URL}/api/public/projects")
56
- echo "Auth check: $STATUS"
57
- ```
58
-
59
- If status is not `200`, stop and ask the user to check their credentials and host before continuing.
47
+ Verify you can actually reach the user's project before proceeding. If access fails, stop and ask the user to check their credentials and host.
60
48
 
61
49
  ### Annotation target: OBSERVATION versus TRACE
62
50
 
63
- > **CRITICAL:** In OpenTelemetry-instrumented apps, trace-level `input`/`output` can be null — content often lives in a GENERATION observation. Always consider if the right objectType to add is `objectType: OBSERVATION` pointing to the GENERATION observation ID to annotation queues.
51
+ > **CRITICAL:** In OpenTelemetry-instrumented apps, trace-level `input`/`output` can be null — content often lives in a GENERATION observation. In that case, add the GENERATION observation (not the trace) to the annotation queue, so the content being reviewed is actually visible.
64
52
 
65
53
  ### Annotation queues
66
54
 
@@ -86,15 +74,15 @@ When a category warrants a prompt fix, always offer the user two options:
86
74
 
87
75
  ### Setup evaluators
88
76
 
89
- When a category warrants an evaluator setup, propose the type of evaluator and offer to set it up for user via CLI
77
+ When a category warrants an evaluator setup, propose the type of evaluator and offer to set it up for the user
90
78
 
91
79
 
92
80
  ### Common gotchas
93
81
 
94
82
  | Mistake | Fix |
95
83
  |---------|-----|
96
- | `objectType: TRACE` in queue | Use `objectType: OBSERVATION` with GENERATION obs ID |
97
- | Creating score config without checking existing | `GET /api/public/score-configs` first; can't delete |
84
+ | Annotating the trace instead of the observation | Target the GENERATION observation, not the trace, when content lives there |
85
+ | Creating a score config without checking existing ones | Check existing score configs first; they can't be deleted |
98
86
  | Queue created before score configs | Create configs → collect IDs → create queue |
99
- | `--limit` > 100 on traces list | API hard cap; paginate with `--page` |
100
- | No rate limiting on queue item creation | `sleep 0.4` between calls to avoid 429 |
87
+ | Requesting too many traces at once | Page size is capped (max 100); paginate to get more |
88
+ | No rate limiting on bulk queue item creation | Space out requests to avoid hitting rate limits (429) |
@@ -1,6 +1,10 @@
1
1
  ---
2
2
  name: langfuse-observability
3
3
  description: Instrument LLM applications with Langfuse tracing. Use when setting up Langfuse, adding observability to LLM calls, or auditing existing instrumentation.
4
+ metadata:
5
+ required_access:
6
+ - CODEBASE
7
+ - LANGFUSE_PROJECT_SCRIPT
4
8
  ---
5
9
 
6
10
  # Langfuse Observability
@@ -31,7 +35,7 @@ Every trace should have these fundamentals:
31
35
  | Token usage | Are input/output tokens tracked? | Enables automatic cost calculation |
32
36
  | Good trace names | Are names descriptive? (`chat-response`, not `trace-1`) | Makes traces findable and filterable |
33
37
  | Span hierarchy | Are multi-step operations nested properly? | Shows which step is slow or failing |
34
- | Correct observation types | Are generations marked as generations? | Enables model-specific analytics |
38
+ | Correct observation types | Are generations marked as generations, and is each other call given its most specific type (`retriever` for a lookup, `agent` for a subagent, etc.) rather than a generic `tool`/`span`? See the [observation types docs](https://langfuse.com/docs/observability/features/observation-types). | Enables model-specific analytics and drives the Agent Graph |
35
39
  | Sensitive data masked | Is PII/confidential data excluded or masked? | Prevents data leakage |
36
40
  | Trace input/output | Does the trace capture meaningful input/output? Is input explicitly set to show only relevant data (e.g., user message), not all function args? | Makes traces readable in the UI and avoids leaking sensitive args |
37
41
 
@@ -39,72 +43,46 @@ Framework integrations (OpenAI, LangChain, etc.) handle model name, tokens, and
39
43
 
40
44
  Docs: https://langfuse.com/docs/tracing
41
45
 
42
- ### 3. Explore Traces First
46
+ **Beyond the baseline**, add context relevant to the app. Infer from code where possible; only ask when it's not obvious (e.g. how they judge a good vs. bad response, what they'd filter a dashboard by, which user segments they'd compare). These are not baseline — add only what fits.
43
47
 
44
- Once baseline instrumentation is working, encourage the user to explore their traces in the Langfuse UI before adding more context:
48
+ | If code shows... | Add | Why |
49
+ | ---------------------------------------------------- | ------------------- | ------------------------------------------------------------------------------- |
50
+ | Conversation history, chat endpoints, message arrays | `session_id` | Groups conversations — [docs](https://langfuse.com/docs/tracing-features/sessions) |
51
+ | User authentication, `user_id` variables | `user_id` | User filtering and cost attribution — [docs](https://langfuse.com/docs/tracing-features/users) |
52
+ | Multiple distinct endpoints/features | `feature` tag | Per-feature analytics — [docs](https://langfuse.com/docs/tracing-features/tags) |
53
+ | Customer/tenant identifiers | `customer_tier` tag | Cost/quality breakdown by segment — [docs](https://langfuse.com/docs/tracing-features/tags) |
54
+ | Feedback collection, ratings | Feedback score | Quality filtering and trends — [docs](https://langfuse.com/docs/scores/overview) |
45
55
 
46
- "Your traces are now appearing in Langfuse. Take a look at a few of them—see what data is being captured, what's useful, and what's missing. This will help us decide what additional context to add."
56
+ ### 3. Run and Self-Audit the Traces (required)
47
57
 
48
- This helps the user:
58
+ Instrumentation isn't done when the code compiles. This is a loop you own as the agent, it's your responsibility to deliver traces of the highest quality your can produce:
49
59
 
50
- - Understand what they're already getting
51
- - Form opinions about what's missing
52
- - Ask better questions about what they need
60
+ **a.** Execute the instrumented path end-to-end so a trace is actually sent.
53
61
 
54
- ### 4. Discover Additional Context Needs
62
+ **b.** Fetch the trace(s) you just created from Langfuse. Any method works (`langfuse-cli`, REST API, SDK, MCP); the CLI is usually simplest — see [references/cli.md](references/cli.md).
55
63
 
56
- Determine what additional instrumentation would be valuable. **Infer from code when possible, only ask when unclear.**
64
+ **c.** Audit the trace against the best-practices page. **Always fetch it fresh — never audit from memory, this cannot be skipped** (the guidance changes over time):
57
65
 
58
- **Infer from code:**
66
+ https://langfuse.com/docs/observability/best-practices
59
67
 
60
- | If you see in code... | Infer | Suggest |
61
- | ---------------------------------------------------- | ----------------- | ------------------------- |
62
- | Conversation history, chat endpoints, message arrays | Multi-turn app | `session_id` |
63
- | User authentication, `user_id` variables | User-aware app | `user_id` on traces |
64
- | Multiple distinct endpoints/features | Multi-feature app | `feature` tag |
65
- | Customer/tenant identifiers | Multi-tenant app | `customer_id` or tier tag |
66
- | Feedback collection, ratings | Has user feedback | Capture as scores |
68
+ Ask yourself, for each observation: is all data that a user might need in the future, to understand exactly what context the agent had when it made decisions, available in Langfuse?
67
69
 
68
- **Only ask when not obvious from code:**
70
+ **d.** Fix every gap you find, then re-run and re-fetch to confirm. Repeat until the trace clears the guidance. Then report what you audited and changed, and link the final trace.
69
71
 
70
- - "How do you know when a response is good vs bad?" → Determines scoring approach
71
- - "What would you want to filter by in a dashboard?" → Surfaces non-obvious tags
72
- - "Are there different user segments you'd want to compare?" → Customer tiers, plans, etc.
72
+ ### 4. Explore Traces With the User
73
73
 
74
- **Additions and their value:**
74
+ With a clean trace in place, invite the user to explore it in the Langfuse UI:
75
75
 
76
- | Addition | Why | Docs |
77
- | ------------------- | ------------------------------------------- | --------------------------------------------------- |
78
- | `session_id` | Groups conversations together | https://langfuse.com/docs/tracing-features/sessions |
79
- | `user_id` | Enables user filtering and cost attribution | https://langfuse.com/docs/tracing-features/users |
80
- | User feedback score | Enables quality filtering and trends | https://langfuse.com/docs/scores/overview |
81
- | `feature` tag | Per-feature analytics | https://langfuse.com/docs/tracing-features/tags |
82
- | `customer_tier` tag | Cost/quality breakdown by segment | https://langfuse.com/docs/tracing-features/tags |
76
+ "Your traces are now appearing in Langfuse. Take a look at a few of them—see what data is being captured, what's useful, and what's missing."
83
77
 
84
- These are NOT baseline requirements—only add what's relevant based on inference or user input.
78
+ Point them to the relevant views:
85
79
 
86
- ### 5. Guide to UI
80
+ - Traces view: individual requests
81
+ - Sessions view: grouped conversations (if `session_id` added)
82
+ - Dashboard: filtered views using tags
83
+ - Scores: filter by quality metrics
87
84
 
88
- After adding context, point users to relevant UI features:
89
-
90
- - Traces view: See individual requests
91
- - Sessions view: See grouped conversations (if session_id added)
92
- - Dashboard: Build filtered views using tags
93
- - Scores: Filter by quality metrics
94
-
95
- ## Framework Integrations
96
-
97
- Prefer these over manual instrumentation:
98
-
99
- | Framework | Integration | Docs |
100
- | ------------- | ---------------------- | ---------------------------------------------------- |
101
- | OpenAI SDK | Drop-in replacement | https://langfuse.com/docs/integrations/openai |
102
- | LangChain | Callback handler | https://langfuse.com/docs/integrations/langchain |
103
- | LlamaIndex | Callback handler | https://langfuse.com/docs/integrations/llama-index |
104
- | Vercel AI SDK | OpenTelemetry exporter | https://langfuse.com/docs/integrations/vercel-ai-sdk |
105
- | LiteLLM | Callback or proxy | https://langfuse.com/docs/integrations/litellm |
106
-
107
- Full list: https://langfuse.com/docs/integrations
85
+ This helps them understand what they're getting, spot what's missing, and ask better questions about what to add next.
108
86
 
109
87
  ## Always Explain Why
110
88
 
@@ -120,6 +98,15 @@ making it much easier to debug multi-turn interactions.
120
98
  Learn more: https://langfuse.com/docs/tracing-features/sessions"
121
99
  ```
122
100
 
101
+ ## Multi-agent systems (subagent dispatch)
102
+
103
+ When one agent's execution dispatches OTHER agents (coding agents like Claude Code/Codex, research agents, orchestrator/worker architectures), a few extra points on top of the baseline:
104
+
105
+ - **Type a subagent's own execution as `agent`, not `tool`/`span`.** A bare tool/span for a dispatch hides all of the subagent's internal structure; `agent` lets it show up as its own node in the [Agent Graph](https://langfuse.com/docs/observability/features/agent-graphs).
106
+ - **Don't emit duplicate dispatch + execution nodes.** Emitting both a `tool`-typed "dispatch" span and a separate `agent`-typed observation for the same subagent, as siblings, double-represents one event. Emit only the `agent` observation when you have the subagent's actual execution; keep a bare tool span only as a fallback when you have no visibility into what the subagent did.
107
+ - **Nest recursively.** Nest the subagent's `agent` observation under the `agent` or `span` that orchestrates the dispatch, as a sibling of the `generation` that requested it. Within the subagent, put its generations, tool calls, and nested subagent executions under that subagent's `agent` observation; each tool or nested subagent is a sibling of the generation that requested it, not a child of that generation.
108
+ - **Name subagents distinctly.** Frameworks often default every subagent to the same generic role name, making them indistinguishable in the tree and graph (nodes key on name). Derive a specific name from the subagent's actual task/role when the framework doesn't provide one.
109
+
123
110
  ## Common Mistakes
124
111
 
125
112
  | Mistake | Problem | Fix |
@@ -1,7 +1,10 @@
1
1
  ---
2
2
  name: langfuse-judge-calibration
3
- description: Calibrate and validate LLM-as-a-Judge evaluators against dataset ground truth. Runs the judge prompt as a Langfuse dataset experiment, compares judge outputs with dataset item expected outputs, and reports simple accuracy or advanced confusion-matrix metrics. Use this guide whenever a user asks if their LLM judge is actually useful,
4
- aligned with human judgment, or safe to trust for monitoring decisions.
3
+ description: Calibrate and validate LLM-as-a-Judge evaluators against dataset ground truth. Runs the judge prompt as a Langfuse dataset experiment, compares judge outputs with dataset item expected outputs, and reports simple accuracy or advanced confusion-matrix metrics. Use this guide whenever a user asks if their LLM judge is actually useful, aligned with human judgment, or safe to trust for monitoring decisions.
4
+ metadata:
5
+ required_access:
6
+ - CODEBASE
7
+ - LANGFUSE_PROJECT_SCRIPT
5
8
  ---
6
9
 
7
10
  # Judge Calibration (LLM-as-a-Judge)
@@ -0,0 +1,35 @@
1
+ ---
2
+ name: langfuse-prompt-engineering
3
+ description: Write or change prompts in Langfuse or code. Use whenever the user asks to create, edit, rewrite, debug, tune, or otherwise modify a prompt, including a small wording or instruction change. Distinct from prompt-migration and judge-calibration.
4
+ metadata:
5
+ required_access: []
6
+ ---
7
+
8
+ # Prompt Engineering
9
+
10
+ ## Universal principles
11
+
12
+ 1. **Be specific.** State the required output format, constraints, and, when order matters, numbered steps. Write for someone with minimal context.
13
+ 2. **Use labeled sections** for the role, instructions, examples, and context. Use XML tags when they help distinguish rules, data, and variables.
14
+ 3. **Give the model a role.** A focused system-prompt sentence can steer tone and behavior.
15
+ 4. **Explain why important rules exist** so the model can apply them to adjacent cases.
16
+
17
+ ## Adjusting an existing prompt
18
+
19
+ 1. **Pin down the failure first.** Use concrete misbehaving outputs and name the exact failure mode: wrong format, ignored instruction, wrong tone, hallucination, or another observable error. The task and data show what could fail, not what did fail; do not write a fix before observing the failure.
20
+ 2. **Trace the failure to the prompt.** Look for a missing, ambiguous, or conflicting instruction.
21
+ 3. **Fix the failure class, not the example.** Generalize the observed mistake into an error class and write the instruction at that level, in your own words. Cover adjacent cases, but do not add definitions, rules, or examples for unobserved problems or encode the failing example itself. If a generalized instruction does not work, report the failure instead of narrowing the rule to that case.
22
+ 4. **Preserve existing behavior.** Before changing text, identify what behavior it protects. Avoid contradicting other rules or breaking cases that already work.
23
+ 5. **Make the smallest testable edit.** Change one cause per attempt so you can attribute the result. Avoid extra length, emphasis, or all-caps. In Langfuse, save each attempt as a new version or label so you can compare and roll back.
24
+
25
+ ## Model-specific tuning
26
+
27
+ Identify the target model if you can — tuning differs by model, so when you know which model the prompt runs on, follow that model's own prompting guidance.
28
+
29
+ | Technique | Standard (GPT) | Reasoning (Claude latest) |
30
+ |-----------|----------------|---------------------------|
31
+ | Instruction density | High, prescriptive | Goal-level; don't micro-manage steps |
32
+ | Reasoning | Add explicit chain-of-thought | Native thinking; "think thoroughly" + tune `effort` |
33
+ | Emphasis language | Strong directives fine | Dial back `MUST`/`CRITICAL` |
34
+ | Output format | Prefilling/scaffolds | Prefill removed (4.6+); structured outputs + "no preamble" |
35
+ | Roles | `developer` > `user` > `assistant` | `system` role + `user`/`assistant` |
@@ -1,234 +1,79 @@
1
1
  ---
2
2
  name: langfuse-prompt-migration
3
3
  description: Migrate hardcoded prompts to Langfuse for version control and deployment-free iteration. Use when user wants to externalize prompts, move prompts to Langfuse, or set up prompt management.
4
+ metadata:
5
+ required_access:
6
+ - CODEBASE
7
+ - LANGFUSE_PROJECT_SCRIPT
4
8
  ---
5
9
 
6
10
  # Langfuse Prompt Migration
7
11
 
8
- Migrate hardcoded prompts to Langfuse for version control, A/B testing, and deployment-free iteration.
12
+ Migrate hardcoded prompts into Langfuse-managed prompts. The API mechanics (`create_prompt`, `get_prompt`, `.compile()`, linking to traces) are in the docs — fetch them at execution time.
9
13
 
10
14
  ## Prerequisites
11
15
 
12
- Verify credentials are set before starting. Check existence only never print the secret key, since the value would land in the agent's context and transcripts:
16
+ Verify credentials exist check presence only, never print the secret key (its value would land in the agent's context and transcripts):
13
17
 
14
18
  ```bash
15
- [ -n "$LANGFUSE_PUBLIC_KEY" ] && echo "LANGFUSE_PUBLIC_KEY: set" || echo "LANGFUSE_PUBLIC_KEY: missing"
16
- [ -n "$LANGFUSE_SECRET_KEY" ] && echo "LANGFUSE_SECRET_KEY: set" || echo "LANGFUSE_SECRET_KEY: missing"
17
- [ -n "$LANGFUSE_BASE_URL" ] && echo "LANGFUSE_BASE_URL: $LANGFUSE_BASE_URL" || echo "LANGFUSE_BASE_URL: missing"
19
+ [ -n "$LANGFUSE_PUBLIC_KEY" ] && echo "public key: set" || echo "public key: missing"
20
+ [ -n "$LANGFUSE_SECRET_KEY" ] && echo "secret key: set" || echo "secret key: missing"
21
+ [ -n "${LANGFUSE_BASE_URL:-${LANGFUSE_HOST:-}}" ] && echo "base url: set" || echo "base url: missing"
18
22
  ```
19
23
 
20
- If not set, ask the user to configure them in their shell or a `.env` file. Do not ask them to paste keys into chat.
24
+ Use `LANGFUSE_BASE_URL` for current SDKs. If only `LANGFUSE_HOST` is set, export `LANGFUSE_BASE_URL="$LANGFUSE_HOST"`. If a CLI expects `LANGFUSE_HOST` and only `LANGFUSE_BASE_URL` is set, export `LANGFUSE_HOST="$LANGFUSE_BASE_URL"`.
21
25
 
22
- ## Migration Flow
26
+ If credentials or the base URL are missing, ask the user to set them in their shell or a `.env` file. Do not ask them to paste secret keys into chat.
23
27
 
24
- ```
25
- 1. Scan codebase for prompts
26
- 2. Analyze templating compatibility
27
- 3. Propose structure (names, subprompts, variables)
28
- 4. User approves
29
- 5. Create prompts in Langfuse
30
- 6. Refactor code to use get_prompt()
31
- 7. Link prompts to traces (if tracing enabled)
32
- 8. Verify application works
33
- ```
34
-
35
- ## Step 1: Find Prompts and Build an Inventory
36
-
37
- Before writing ANY code, make a complete list of every prompt you found. For each one, note:
38
-
39
- - Name: descriptive, lowercase, hyphenated (e.g. chat-assistant, email-classifier)
40
- - Source file: where the prompt text lives
41
- - Code file to refactor: the Python/JS file that USES the prompt (for asset files like .txt/.yaml/.md, this is the file that reads/loads the asset — NOT the asset file itself)
42
- - Type: chat (used as a message in a chat API) or text (used as a plain string)
43
- - Variables: values interpolated into the prompt, converted to {{var}} syntax:
44
- f-string {var} → {{var}}
45
- .format(var=...) → {{var}}
46
- ${var} → {{var}}
47
- String concatenation + var + → {{var}}
48
- YAML {var} → {{var}}
49
- - Prompt content: the actual text to upload, with variables converted to {{var}} syntax
50
-
51
- Search for these patterns:
28
+ ## 1. Inventory every prompt (before writing any code)
52
29
 
53
- | Framework | Look for |
54
- |-----------|----------|
55
- | OpenAI | `messages=[{"role": "system", "content": "..."}]` |
56
- | Anthropic | `system="..."` |
57
- | LangChain | `ChatPromptTemplate`, `SystemMessage` |
58
- | Vercel AI | `system: "..."`, `prompt: "..."` |
59
- | Raw | Multi-line strings near LLM calls |
30
+ For each prompt, record:
60
31
 
61
- ## Step 2: Check Templating Compatibility
32
+ - **Name**: lowercase, hyphenated (e.g. `chat-assistant`)
33
+ - **Source file**: where the prompt text lives
34
+ - **Code file to refactor**: the file that USES the prompt. For asset files (`.txt`/`.yaml`/`.md`), this is the file that loads the asset, not the asset itself
35
+ - **Type**: `chat` (message array) or `text` (plain string)
36
+ - **Variables**: values interpolated in, converted to `{{var}}`
37
+ - **Content**: the actual text to upload
62
38
 
63
- **CRITICAL:** Langfuse only supports simple `{{variable}}` substitution. No conditionals, loops, or filters.
39
+ Before choosing `text` or `chat`, fetch and follow [Chat vs Text Prompts](https://langfuse.com/docs/prompt-management/data-model#text-vs-chat-prompts). Do not default unrelated prompt flows to one prompt type.
64
40
 
65
- | Template Feature | Langfuse Native | Action |
66
- |------------------|-----------------|--------|
67
- | `{{variable}}` | ✅ | Direct migration |
68
- | `{var}` / `${var}` | ⚠️ | Convert to `{{var}}` |
69
- | `{% if %}` / `{% for %}` | ❌ | Move logic to code |
70
- | `{{ var \| filter }}` | ❌ | Apply filter in code |
41
+ Prompts typically live in OpenAI message arrays, Anthropic system arguments, LangChain prompt templates, Vercel AI system/prompt fields, and raw multi-line strings near LLM calls.
71
42
 
72
- **CRITICAL Variable syntax:** Langfuse uses DOUBLE curly braces for variables: `{{var}}`. When uploading prompt content, you MUST convert every single-brace `{var}` from the original code to double-brace `{{var}}`. Never upload `{var}` — it must be `{{var}}`.
43
+ ## 2. Convert templating and decide structure
73
44
 
74
- ### Decision Tree
45
+ **Variable syntax:** Langfuse substitutes only double-brace `{{var}}`. Convert every single-brace form during upload — `{var}`, `${var}`, f-string `{var}`, `.format(var=...)`, and string concatenation all become `{{var}}`. Uploading `{var}` will silently fail to substitute.
75
46
 
76
- ```
77
- Contains {% if %}, {% for %}, or filters?
78
- ├─ No → Direct migration
79
- └─ Yes → Choose:
80
- ├─ Option A (RECOMMENDED): Move logic to code, pass pre-computed values
81
- └─ Option B: Store raw template, compile client-side with Jinja2
82
- └─ ⚠️ Loses: Playground preview, UI experiments
83
- ```
84
-
85
- ### Simplifying Complex Templates
47
+ **Complex templates:** Langfuse has no conditionals, loops, or filters. If the code uses them (e.g. Jinja `{% if %}`/`{% for %}`), either pre-compute the value in code and pass a plain `{{variable}}` (recommended), or store the raw template and compile client-side — which loses Playground preview and UI experiments. See https://langfuse.com/docs/prompt-management/features/variables and the [external templating FAQ](https://langfuse.com/faq/all/using-external-templating-libraries).
86
48
 
87
- **Conditionals** Pre-compute in code:
88
- ```python
89
- # Instead of {% if user.is_premium %}...{% endif %} in prompt
90
- # Use {{tier_message}} and compute value in code before compile()
91
- ```
92
-
93
- **Loops** → Pre-format in code:
94
- ```python
95
- # Instead of {% for tool in tools %}...{% endfor %} in prompt
96
- # Use {{tools_list}} and format the list in code before compile()
97
- ```
49
+ **What to make a variable vs. keep hardcoded:**
98
50
 
99
- For external templating details, fetch: https://langfuse.com/faq/all/using-external-templating-libraries
100
-
101
- ## Step 3: Propose Structure
102
-
103
- ### Naming Conventions
104
-
105
- | Rule | Example | Bad |
106
- |------|---------|-----|
107
- | Lowercase, hyphenated | `chat-assistant` | `ChatAssistant_v2` |
108
- | Feature-based | `document-summarizer` | `prompt1` |
109
- | Hierarchical for related | `support/triage` | `supportTriage` |
110
- | Prefix subprompts with `_` | `_base-personality` | `shared-personality` |
111
-
112
- ### Identify Subprompts
113
-
114
- Extract when:
115
- - Same text in 2+ prompts
116
- - Represents distinct component (personality, safety rules, format)
117
- - Would need to change together
118
-
119
- ### Variable Extraction
120
-
121
- | Make Variable | Keep Hardcoded |
51
+ | Make variable | Keep hardcoded |
122
52
  |---------------|----------------|
123
53
  | User-specific (`{{user_name}}`) | Output format instructions |
124
54
  | Dynamic content (`{{context}}`) | Safety guardrails |
125
- | Per-request (`{{query}}`) | Persona/personality |
55
+ | Per-request (`{{query}}`) | Persona / personality |
126
56
  | Environment-specific (`{{company_name}}`) | Static examples |
127
57
 
128
- ## Step 4: Present Plan to User
129
-
130
- Format:
131
- ```
132
- Found N prompts across M files:
133
-
134
- src/chat.py:
135
- - System prompt (47 lines) → 'chat-assistant'
136
-
137
- src/support/triage.py:
138
- - Triage prompt (34 lines) → 'support/triage'
139
- ⚠️ Contains {% if %} - will simplify
140
-
141
- Subprompts to extract:
142
- - '_base-personality' - used by: chat-assistant, support/triage
143
-
144
- Variables to add:
145
- - {{user_name}} - hardcoded in 2 prompts
146
-
147
- Proceed?
148
- ```
149
-
150
- ## Step 5: Create Prompts in Langfuse
151
-
152
- Use `langfuse.create_prompt()` with:
153
- - `name`: Your chosen name
154
- - `prompt`: Template text (or message array for chat type)
155
- - `type`: `"text"` or `"chat"`
156
- - `labels`: `["production"]` (they're already live)
157
- - `config`: Optional model settings
158
-
159
- **Labeling strategy:**
160
- - `production` → All migrated prompts
161
- - `staging` → Add later for testing
162
- - `latest` → Auto-applied by Langfuse
163
-
164
- For full API: fetch https://langfuse.com/docs/prompts/get-started
165
-
166
- ## Step 6: Refactor Code
167
-
168
- Replace hardcoded prompts with:
169
-
170
- ```python
171
- prompt = langfuse.get_prompt("name", label="production")
172
- messages = prompt.compile(var1=value1, var2=value2)
173
- ```
174
-
175
- **Key points:**
176
- - Always use `label="production"` (not `latest`) for stability
177
- - Call `.compile()` to substitute variables
178
- - For chat prompts, result is message array ready for API
179
-
180
- For SDK examples (Python/JS/TS): fetch https://langfuse.com/docs/prompts/get-started
181
-
182
- ## Step 7: Link Prompts to Traces
183
-
184
- If codebase uses Langfuse tracing, link prompts so you can see which version produced each response.
185
-
186
- ### Detect Existing Tracing
187
-
188
- Look for:
189
- - `@observe()` decorators
190
- - `langfuse.trace()` calls
191
- - `from langfuse.openai import openai` (instrumented client)
192
-
193
- ### Link Methods
194
-
195
- | Setup | How to Link |
196
- |-------|-------------|
197
- | `@observe()` decorator | `langfuse_context.update_current_observation(prompt=prompt)` |
198
- | Manual tracing | `trace.generation(prompt=prompt, ...)` |
199
- | OpenAI integration | `openai.chat.completions.create(..., langfuse_prompt=prompt)` |
200
-
201
- ### Verify in UI
202
-
203
- 1. Go to **Traces** → select a trace
204
- 2. Click on **Generation**
205
- 3. Check **Prompt** field shows name and version
58
+ **Naming:** lowercase-hyphenated, feature-based (`document-summarizer`), hierarchical for related prompts (`support/triage`), prefix subprompts with `_` (`_base-personality`).
206
59
 
207
- For tracing details: fetch https://langfuse.com/docs/prompts/get-started#link-with-langfuse-tracing
60
+ Before extracting subprompts, fetch and follow [Prompt Composability](https://langfuse.com/docs/prompt-management/features/composability). Use composition for the reuse and shared-maintenance cases described there, not merely to decompose one coherent prompt flow.
208
61
 
209
- ## Step 8: Verify Migration
62
+ ## 3. Present the plan, then create and refactor
210
63
 
211
- ### Checklist
64
+ When the user explicitly requests a migration and credentials work, treat that request as authorization to create prompts and refactor the call sites. Present the inventory and plan as a progress update, then continue.
212
65
 
213
- - [ ] All prompts created with `production` label
214
- - [ ] Code fetches with `label="production"`
215
- - [ ] Variables compile without errors
216
- - [ ] Subprompts resolve correctly
217
- - [ ] Application behavior unchanged
218
- - [ ] Generations show linked prompt in UI (if tracing)
66
+ Ask only when a materially different design choice would change behavior, required credentials are unavailable, or destructive cleanup needs approval. Do not stop merely to confirm names, prompt types, or optional tracing.
219
67
 
220
- ### Common Issues
68
+ Then:
221
69
 
222
- | Issue | Solution |
223
- |-------|----------|
224
- | `PromptNotFoundError` | Check name spelling |
225
- | Variables not replaced | Use `{{var}}` not `{var}`, call `.compile()` |
226
- | Subprompt not resolved | Must exist with same label |
227
- | Old prompt cached | Restart app |
70
+ - Create the prompts (label migrated prompts `production` — they're already live) and refactor call sites to fetch each prompt from Langfuse and compile its variables in. The SDK calls differ across Python and JS/TS — fetch the current docs: https://langfuse.com/docs/prompt-management/get-started
71
+ - Fetch by the `production` label
72
+ - If the codebase already has Langfuse tracing (decorators, an instrumented client, or manual spans), link prompts so you can see which version produced each response. See https://langfuse.com/docs/prompt-management/features/link-to-traces
228
73
 
229
- ## Out of Scope
74
+ ## 4. Verify
230
75
 
231
- - Prompt engineering (writing better prompts)
232
- - Evaluation setup
233
- - A/B testing workflow
234
- - Non-LLM string templates
76
+ - All prompts created with the `production` label; code fetches with `label="production"`
77
+ - Variables and subprompts compile without errors
78
+ - Application behavior is unchanged
79
+ - Generations show the linked prompt in the UI (if tracing enabled)
@@ -1,6 +1,9 @@
1
1
  ---
2
2
  name: langfuse-skill-feedback
3
3
  description: Submit feedback about the Langfuse skill to its maintainers via GitHub Discussions. Use when the user indicates the skill gave incorrect guidance, is missing information, or could be improved.
4
+ metadata:
5
+ required_access:
6
+ - GITHUB
4
7
  ---
5
8
 
6
9
  # Skill Feedback