langfuse-agent-cli 0.3.0__tar.gz → 0.4.0__tar.gz

{langfuse_agent_cli-0.3.0 → langfuse_agent_cli-0.4.0}/PKG-INFO

@@ -1,10 +1,10 @@
 Metadata-Version: 2.4
 Name: langfuse-agent-cli
-Version: 0.3.0
+Version: 0.4.0
 Summary: Agent-oriented CLI and skill for Langfuse traces, prompts, datasets and scores.
 Author-email: Tarkeshwar Narayan <122815908+tns15june@users.noreply.github.com>
 License-Expression: MIT
-Project-URL: Homepage, https://github.com/tns15june/langfuse-agent-cli
+Project-URL: Homepage, https://tns15june.github.io/langfuse-agent-cli/
 Project-URL: Repository, https://github.com/tns15june/langfuse-agent-cli
 Project-URL: Issues, https://github.com/tns15june/langfuse-agent-cli/issues
 Keywords: langfuse,llm,observability,agent,cli,skills
@@ -28,9 +28,13 @@ Dynamic: license-file
 
 # langfuse-agent-cli (`lf`)
 
-Command-line access to [Langfuse](https://langfuse.com) traces, prompts, datasets and scores, shaped
-for AI coding agents: JSON by default (`--compact` trims it), failing traces with their error messages
-in one command, a trace as a tree, and an agent skill that teaches the commands.
+Ask your coding agent about your LLM app in plain English, such as "Why did my agent fail in the last
+hour?" or "What did we spend on each model this week?", and it answers from your
+[Langfuse](https://langfuse.com) data with `lf`. You get failing traces with their error messages, a
+trace as a tree, conversations, cost, latency, prompts, datasets and scores. `lf` is a command-line tool
+plus an agent skill that teaches your agent the commands.
+
+![lf trace get --timeline --io marks the failing tool call with its error message and its input](https://raw.githubusercontent.com/tns15june/langfuse-agent-cli/main/docs/lf-timeline.png)
 
 > **Status: prototype.** Langfuse ships an official CLI,
 > [`@langfuse/cli`](https://github.com/langfuse/langfuse-cli), which covers the whole API, and official
@@ -48,47 +52,77 @@ in one command, a trace as a tree, and an agent skill that teaches the commands.
 
 ## Install
 
+### Option 1: one line (macOS and Linux)
+
 ```bash
-pipx install langfuse-agent-cli   # or: uv tool install langfuse-agent-cli
-lf setup
+curl -fsSL https://tns15june.github.io/langfuse-agent-cli/install.sh | sh
 ```
 
-`lf setup` takes three steps and is safe to run again:
+It installs `lf` with uv or pipx (and installs uv first when you have neither), then runs `lf setup`.
 
-1. **Credentials.** It keeps working credentials that it finds. When no profile is saved, or the
-   saved profile's keys are rejected, it asks for the host and the keys (the secret is not echoed)
-   and saves them. It never replaces saved keys when the server is merely unreachable, and keys from
-   `LANGFUSE_*` variables, `--env-file` or key flags, which take precedence over profiles, are left for
-   you to fix.
-2. **Agent skill.** It installs the `langfuse-agent-cli` skill with the
-   [skills CLI](https://github.com/vercel-labs/skills), which asks which coding agents to install it
-   for when it finds more than one. The skill comes from the release tag that matches your `lf`
-   version.
-3. **Check.** It runs `lf doctor`.
+### Option 2: with uv or pipx
+
+```bash
+uv tool install langfuse-agent-cli    # or: pipx install langfuse-agent-cli
+lf setup
+```
+
+### Option 3: let your coding agent install it
 
-To have a coding agent install it, give the agent this prompt:
+Give your agent this prompt:
 
 ```text
-Install langfuse-agent-cli by following https://raw.githubusercontent.com/tns15june/langfuse-agent-cli/main/INSTALL.md
+Install langfuse-agent-cli using https://tns15june.github.io/langfuse-agent-cli/AGENTS.md
 ```
 
-The agent installs the CLI and runs `lf setup --yes`, which never prompts and installs the skill for
-that agent. If no working credentials are configured, the agent asks you to run `lf setup` yourself,
-which asks for the host and the keys.
+The agent installs `lf` and runs `lf setup --yes`, which never prompts. If no credentials are
+configured, it asks you to run `lf setup` in your own terminal, which asks for the host and the keys.
 
-To install the latest code from GitHub instead of PyPI:
-`pipx install git+https://github.com/tns15june/langfuse-agent-cli`.
+### What `lf setup` does
+
+1. **Credentials.** It keeps working credentials that it finds. When no profile is saved, or the saved
+   profile's keys are rejected, it asks for the host and the keys (the secret is not echoed) and saves
+   them. It never replaces saved keys when the server is merely unreachable, and keys from `LANGFUSE_*`
+   variables, `--env-file` or key flags, which take precedence over profiles, are left for you to fix.
+2. **Agent skill.** It installs the `langfuse-agent-cli` skill with the
+   [skills CLI](https://github.com/vercel-labs/skills), which asks which coding agents to install it for
+   when it finds more than one. The skill comes from the release tag that matches your `lf` version.
+3. **Check.** It runs `lf doctor` and suggests questions to try.
 
-### Upgrade and uninstall
+It is safe to run again: steps that are already done are skipped. For a second Langfuse instance, run
+`lf setup --profile-name staging --host https://langfuse.example.com`; the new profile becomes the
+current one, and `lf auth switch default` switches back.
+
+### Verify
 
 ```bash
-pipx upgrade langfuse-agent-cli && lf setup   # lf setup also installs the matching skill
-npx skills remove --global langfuse-agent-cli
-pipx uninstall langfuse-agent-cli
+lf doctor
 ```
 
-With uv, use `uv tool upgrade langfuse-agent-cli` and `uv tool uninstall langfuse-agent-cli`. Saved
-profiles stay in `~/.config/langfuse/profiles.json` until you delete that file.
+It shows where `lf` is installed and whether a newer version exists, checks the host, the keys and the
+API, and lists where the skill is installed.
+
+## Try it
+
+Start a new session of your coding agent and ask, for example:
+
+- "Why did my LLM app fail in the last hour?"
+- "Which tool fails most often this week, and since when?"
+- "What did we spend on each model this week?"
+- "Which step of my agent is the slowest?"
+- "Walk me through the conversation in session sess-42."
+- "Which version of the welcome-bot prompt is live, and how does it read for Alice?"
+
+More questions, with the commands your agent runs for them:
+[Agent examples](https://github.com/tns15june/langfuse-agent-cli/blob/main/skills/langfuse-agent-cli/references/EXAMPLES.md).
+
+## Upgrade and uninstall
+
+```bash
+lf upgrade              # upgrades lf with uv or pipx, then installs the matching skill
+lf uninstall --dry-run  # lists the skill, the saved profiles and lf itself
+lf uninstall            # removes them after you confirm
+```
 
 ## Credentials
 
@@ -115,8 +149,8 @@ Profiles live in `~/.config/langfuse/profiles.json`, readable only by you (mode
 
 ## Commands
 
-Data commands print JSON by default and short lines with `-o text`; `doctor`, `setup`, `auth` and
-`help-map` print text. Three options keep JSON small:
+Data commands print JSON by default and short lines with `-o text`; `doctor`, `setup`, `upgrade`,
+`uninstall`, `auth` and `help-map` print text. Three options keep JSON small:
 
 - `--compact` drops null and empty fields.
 - `--agent-fields name,level` keeps only the listed fields of each row.
@@ -139,6 +173,11 @@ lf trace session <session-id> --compact
 # Recent traces, filtered
 lf trace list --user-id user-42 --from-ts 2026-10-24T00:00:00Z --limit 20
 
+# Cost, tokens, latency and error counts, aggregated (the last 24 hours unless --from-ts is given)
+lf metrics --metric totalCost,totalTokens --by providedModelName --from-ts 2026-10-20T00:00:00Z
+lf metrics --metric latency:p95 --by name --order-by p95_latency
+lf metrics --where level=ERROR --by name --granularity hour -o text
+
 # Prompts
 lf prompt list -o text
 lf prompt get support/greeter
@@ -172,6 +211,18 @@ Exit codes: `0` success, `1` API or configuration error, `2` invalid command. Th
 on stderr, except that `doctor`, `setup --yes` and `auth status` print their whole report, failures
 included, on stdout. `setup` also exits `1` when the skill was not installed.
 
+## Troubleshooting
+
+| What you see | What to do |
+| --- | --- |
+| `command not found: lf` | `lf` is in `~/.local/bin`. Run `uv tool update-shell` or `pipx ensurepath`, then open a new terminal. |
+| `HTTP 401: Invalid credentials` | The keys do not belong to that host. Run `lf setup`; `lf doctor` shows where the keys came from. |
+| "is a login page or proxy answering instead of Langfuse?" | The host is a company login page or proxy. Use the address of the Langfuse API. |
+| "This server has no /api/public/v2/observations" | The server predates Langfuse v4. The trace commands need an upgraded server. |
+| "npx not found" or "Skill not installed" | Install Node.js 22.20 or later, then run `lf setup`. |
+| Your saved profile is ignored | `LANGFUSE_PUBLIC_KEY` or `LANGFUSE_SECRET_KEY` is set; environment variables take precedence over profiles. |
+| "The installed skill is v0.3.0" | Run `lf setup` to install the skill that matches your `lf` version. |
+
 ## API endpoints used
 
 | Command | Endpoint |
@@ -180,10 +231,11 @@ included, on stdout. `setup` also exits `1` when the skill was not installed.
 | `trace list --status error` | `GET /api/public/v2/observations?level=ERROR`, grouped by trace |
 | `trace get` | `GET /api/public/v2/observations?traceId=…`, all pages |
 | `trace session` | `GET /api/public/v2/observations?sessionId=…&isRootObservation=true` |
+| `metrics` | `GET /api/public/v2/metrics` |
 | `prompt list`, `prompt get`, `prompt push` | `/api/public/v2/prompts` |
 | `dataset list`, `dataset get` | `/api/public/v2/datasets`, `/api/public/dataset-items` |
 | `score create` | `POST /api/public/scores` |
-| `doctor` | `/api/public/health`, `/api/public/projects`, `/api/public/v2/observations?limit=1` |
+| `doctor` | `/api/public/health`, `/api/public/projects`, `/api/public/v2/observations?limit=1`, and PyPI for the version check (`LF_NO_UPDATE_CHECK=1` turns it off) |
 | `auth login`, `auth status` | `/api/public/projects` |
 | `setup` | `/api/public/projects`, then the `doctor` requests |
 
@@ -206,8 +258,8 @@ python scripts/seed_langfuse.py   # synthetic prompts, a dataset and traces: use
 LF_LIVE=1 pytest tests/test_live.py -v
 ```
 
-CI runs the unit tests on Python 3.10 to 3.13 and the live tests against the latest Langfuse release,
-on every push and weekly.
+CI runs the unit tests on Python 3.10 to 3.13, shellcheck on the installer, and the live tests against
+the latest Langfuse release, on every push and weekly.
 
 ### Releasing
 
@@ -216,17 +268,21 @@ lists `publish.yml` and the GitHub environment `pypi` as its trusted publisher.
 `langfuse_cli/__init__.py`, commit, then push that commit and its tag together:
 
 ```bash
-git tag v0.3.0 && git push --atomic origin main v0.3.0
+git tag v0.4.0 && git push --atomic origin main v0.4.0
 ```
 
 The workflow checks that the tag matches the version, runs the tests, builds and checks the
 distributions, installs the wheel, and uploads the wheel and the source archive. `main` should never
 carry a version without its tag, because `lf setup` and `lf doctor` point at the skill under that tag.
 
+## Feedback
+
+Questions, bugs and ideas: [GitHub issues](https://github.com/tns15june/langfuse-agent-cli/issues).
+
 ## Prior art
 
-The command map (`lf help-map`), the `--output-summary` option, `lf setup` and the agent install prompt
-follow Atlassian's
+The command map (`lf help-map`), the `--output-summary` option, `lf setup`, the one-line installer and
+the agent install guide follow Atlassian's
 [Teamwork Graph CLI](https://developer.atlassian.com/platform/teamwork-graph/twg-cli/getting-started/installation/),
 whose `--select` option does what `lf`'s `--agent-fields` does.
 

{langfuse_agent_cli-0.3.0 → langfuse_agent_cli-0.4.0}/README.md

@@ -1,8 +1,12 @@
 # langfuse-agent-cli (`lf`)
 
-Command-line access to [Langfuse](https://langfuse.com) traces, prompts, datasets and scores, shaped
-for AI coding agents: JSON by default (`--compact` trims it), failing traces with their error messages
-in one command, a trace as a tree, and an agent skill that teaches the commands.
+Ask your coding agent about your LLM app in plain English, such as "Why did my agent fail in the last
+hour?" or "What did we spend on each model this week?", and it answers from your
+[Langfuse](https://langfuse.com) data with `lf`. You get failing traces with their error messages, a
+trace as a tree, conversations, cost, latency, prompts, datasets and scores. `lf` is a command-line tool
+plus an agent skill that teaches your agent the commands.
+
+![lf trace get --timeline --io marks the failing tool call with its error message and its input](https://raw.githubusercontent.com/tns15june/langfuse-agent-cli/main/docs/lf-timeline.png)
 
 > **Status: prototype.** Langfuse ships an official CLI,
 > [`@langfuse/cli`](https://github.com/langfuse/langfuse-cli), which covers the whole API, and official
@@ -20,47 +24,77 @@ in one command, a trace as a tree, and an agent skill that teaches the commands.
 
 ## Install
 
+### Option 1: one line (macOS and Linux)
+
 ```bash
-pipx install langfuse-agent-cli   # or: uv tool install langfuse-agent-cli
-lf setup
+curl -fsSL https://tns15june.github.io/langfuse-agent-cli/install.sh | sh
 ```
 
-`lf setup` takes three steps and is safe to run again:
+It installs `lf` with uv or pipx (and installs uv first when you have neither), then runs `lf setup`.
 
-1. **Credentials.** It keeps working credentials that it finds. When no profile is saved, or the
-   saved profile's keys are rejected, it asks for the host and the keys (the secret is not echoed)
-   and saves them. It never replaces saved keys when the server is merely unreachable, and keys from
-   `LANGFUSE_*` variables, `--env-file` or key flags, which take precedence over profiles, are left for
-   you to fix.
-2. **Agent skill.** It installs the `langfuse-agent-cli` skill with the
-   [skills CLI](https://github.com/vercel-labs/skills), which asks which coding agents to install it
-   for when it finds more than one. The skill comes from the release tag that matches your `lf`
-   version.
-3. **Check.** It runs `lf doctor`.
+### Option 2: with uv or pipx
+
+```bash
+uv tool install langfuse-agent-cli    # or: pipx install langfuse-agent-cli
+lf setup
+```
+
+### Option 3: let your coding agent install it
 
-To have a coding agent install it, give the agent this prompt:
+Give your agent this prompt:
 
 ```text
-Install langfuse-agent-cli by following https://raw.githubusercontent.com/tns15june/langfuse-agent-cli/main/INSTALL.md
+Install langfuse-agent-cli using https://tns15june.github.io/langfuse-agent-cli/AGENTS.md
 ```
 
-The agent installs the CLI and runs `lf setup --yes`, which never prompts and installs the skill for
-that agent. If no working credentials are configured, the agent asks you to run `lf setup` yourself,
-which asks for the host and the keys.
+The agent installs `lf` and runs `lf setup --yes`, which never prompts. If no credentials are
+configured, it asks you to run `lf setup` in your own terminal, which asks for the host and the keys.
 
-To install the latest code from GitHub instead of PyPI:
-`pipx install git+https://github.com/tns15june/langfuse-agent-cli`.
+### What `lf setup` does
+
+1. **Credentials.** It keeps working credentials that it finds. When no profile is saved, or the saved
+   profile's keys are rejected, it asks for the host and the keys (the secret is not echoed) and saves
+   them. It never replaces saved keys when the server is merely unreachable, and keys from `LANGFUSE_*`
+   variables, `--env-file` or key flags, which take precedence over profiles, are left for you to fix.
+2. **Agent skill.** It installs the `langfuse-agent-cli` skill with the
+   [skills CLI](https://github.com/vercel-labs/skills), which asks which coding agents to install it for
+   when it finds more than one. The skill comes from the release tag that matches your `lf` version.
+3. **Check.** It runs `lf doctor` and suggests questions to try.
 
-### Upgrade and uninstall
+It is safe to run again: steps that are already done are skipped. For a second Langfuse instance, run
+`lf setup --profile-name staging --host https://langfuse.example.com`; the new profile becomes the
+current one, and `lf auth switch default` switches back.
+
+### Verify
 
 ```bash
-pipx upgrade langfuse-agent-cli && lf setup   # lf setup also installs the matching skill
-npx skills remove --global langfuse-agent-cli
-pipx uninstall langfuse-agent-cli
+lf doctor
 ```
 
-With uv, use `uv tool upgrade langfuse-agent-cli` and `uv tool uninstall langfuse-agent-cli`. Saved
-profiles stay in `~/.config/langfuse/profiles.json` until you delete that file.
+It shows where `lf` is installed and whether a newer version exists, checks the host, the keys and the
+API, and lists where the skill is installed.
+
+## Try it
+
+Start a new session of your coding agent and ask, for example:
+
+- "Why did my LLM app fail in the last hour?"
+- "Which tool fails most often this week, and since when?"
+- "What did we spend on each model this week?"
+- "Which step of my agent is the slowest?"
+- "Walk me through the conversation in session sess-42."
+- "Which version of the welcome-bot prompt is live, and how does it read for Alice?"
+
+More questions, with the commands your agent runs for them:
+[Agent examples](https://github.com/tns15june/langfuse-agent-cli/blob/main/skills/langfuse-agent-cli/references/EXAMPLES.md).
+
+## Upgrade and uninstall
+
+```bash
+lf upgrade              # upgrades lf with uv or pipx, then installs the matching skill
+lf uninstall --dry-run  # lists the skill, the saved profiles and lf itself
+lf uninstall            # removes them after you confirm
+```
 
 ## Credentials
 
@@ -87,8 +121,8 @@ Profiles live in `~/.config/langfuse/profiles.json`, readable only by you (mode
 
 ## Commands
 
-Data commands print JSON by default and short lines with `-o text`; `doctor`, `setup`, `auth` and
-`help-map` print text. Three options keep JSON small:
+Data commands print JSON by default and short lines with `-o text`; `doctor`, `setup`, `upgrade`,
+`uninstall`, `auth` and `help-map` print text. Three options keep JSON small:
 
 - `--compact` drops null and empty fields.
 - `--agent-fields name,level` keeps only the listed fields of each row.
@@ -111,6 +145,11 @@ lf trace session <session-id> --compact
 # Recent traces, filtered
 lf trace list --user-id user-42 --from-ts 2026-10-24T00:00:00Z --limit 20
 
+# Cost, tokens, latency and error counts, aggregated (the last 24 hours unless --from-ts is given)
+lf metrics --metric totalCost,totalTokens --by providedModelName --from-ts 2026-10-20T00:00:00Z
+lf metrics --metric latency:p95 --by name --order-by p95_latency
+lf metrics --where level=ERROR --by name --granularity hour -o text
+
 # Prompts
 lf prompt list -o text
 lf prompt get support/greeter
@@ -144,6 +183,18 @@ Exit codes: `0` success, `1` API or configuration error, `2` invalid command. Th
 on stderr, except that `doctor`, `setup --yes` and `auth status` print their whole report, failures
 included, on stdout. `setup` also exits `1` when the skill was not installed.
 
+## Troubleshooting
+
+| What you see | What to do |
+| --- | --- |
+| `command not found: lf` | `lf` is in `~/.local/bin`. Run `uv tool update-shell` or `pipx ensurepath`, then open a new terminal. |
+| `HTTP 401: Invalid credentials` | The keys do not belong to that host. Run `lf setup`; `lf doctor` shows where the keys came from. |
+| "is a login page or proxy answering instead of Langfuse?" | The host is a company login page or proxy. Use the address of the Langfuse API. |
+| "This server has no /api/public/v2/observations" | The server predates Langfuse v4. The trace commands need an upgraded server. |
+| "npx not found" or "Skill not installed" | Install Node.js 22.20 or later, then run `lf setup`. |
+| Your saved profile is ignored | `LANGFUSE_PUBLIC_KEY` or `LANGFUSE_SECRET_KEY` is set; environment variables take precedence over profiles. |
+| "The installed skill is v0.3.0" | Run `lf setup` to install the skill that matches your `lf` version. |
+
 ## API endpoints used
 
 | Command | Endpoint |
@@ -152,10 +203,11 @@ included, on stdout. `setup` also exits `1` when the skill was not installed.
 | `trace list --status error` | `GET /api/public/v2/observations?level=ERROR`, grouped by trace |
 | `trace get` | `GET /api/public/v2/observations?traceId=…`, all pages |
 | `trace session` | `GET /api/public/v2/observations?sessionId=…&isRootObservation=true` |
+| `metrics` | `GET /api/public/v2/metrics` |
 | `prompt list`, `prompt get`, `prompt push` | `/api/public/v2/prompts` |
 | `dataset list`, `dataset get` | `/api/public/v2/datasets`, `/api/public/dataset-items` |
 | `score create` | `POST /api/public/scores` |
-| `doctor` | `/api/public/health`, `/api/public/projects`, `/api/public/v2/observations?limit=1` |
+| `doctor` | `/api/public/health`, `/api/public/projects`, `/api/public/v2/observations?limit=1`, and PyPI for the version check (`LF_NO_UPDATE_CHECK=1` turns it off) |
 | `auth login`, `auth status` | `/api/public/projects` |
 | `setup` | `/api/public/projects`, then the `doctor` requests |
 
@@ -178,8 +230,8 @@ python scripts/seed_langfuse.py   # synthetic prompts, a dataset and traces: use
 LF_LIVE=1 pytest tests/test_live.py -v
 ```
 
-CI runs the unit tests on Python 3.10 to 3.13 and the live tests against the latest Langfuse release,
-on every push and weekly.
+CI runs the unit tests on Python 3.10 to 3.13, shellcheck on the installer, and the live tests against
+the latest Langfuse release, on every push and weekly.
 
 ### Releasing
 
@@ -188,17 +240,21 @@ lists `publish.yml` and the GitHub environment `pypi` as its trusted publisher.
 `langfuse_cli/__init__.py`, commit, then push that commit and its tag together:
 
 ```bash
-git tag v0.3.0 && git push --atomic origin main v0.3.0
+git tag v0.4.0 && git push --atomic origin main v0.4.0
 ```
 
 The workflow checks that the tag matches the version, runs the tests, builds and checks the
 distributions, installs the wheel, and uploads the wheel and the source archive. `main` should never
 carry a version without its tag, because `lf setup` and `lf doctor` point at the skill under that tag.
 
+## Feedback
+
+Questions, bugs and ideas: [GitHub issues](https://github.com/tns15june/langfuse-agent-cli/issues).
+
 ## Prior art
 
-The command map (`lf help-map`), the `--output-summary` option, `lf setup` and the agent install prompt
-follow Atlassian's
+The command map (`lf help-map`), the `--output-summary` option, `lf setup`, the one-line installer and
+the agent install guide follow Atlassian's
 [Teamwork Graph CLI](https://developer.atlassian.com/platform/teamwork-graph/twg-cli/getting-started/installation/),
 whose `--select` option does what `lf`'s `--agent-fields` does.