margin-meter 0.6.2__tar.gz → 0.6.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {margin_meter-0.6.2 → margin_meter-0.6.3}/PKG-INFO +45 -1
- {margin_meter-0.6.2 → margin_meter-0.6.3}/README.md +44 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/__init__.py +1 -1
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/client.py +19 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter.egg-info/PKG-INFO +45 -1
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/__main__.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/batching.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/braintrust.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/deepeval.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/doctor.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/evals.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/instrument.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/langfuse.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/langsmith.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/model_override.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/promptfoo.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/providers.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/pytest_plugin.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/ragas.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/substrate.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter/values.py +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter.egg-info/SOURCES.txt +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter.egg-info/dependency_links.txt +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter.egg-info/entry_points.txt +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/margin_meter.egg-info/top_level.txt +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/pyproject.toml +0 -0
- {margin_meter-0.6.2 → margin_meter-0.6.3}/setup.cfg +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: margin-meter
|
|
3
|
-
Version: 0.6.
|
|
3
|
+
Version: 0.6.3
|
|
4
4
|
Summary: Tiny stdlib-only client that auto-instruments OpenAI/Anthropic/Gemini and emits LLM call + outcome economics to a Margin ingest API.
|
|
5
5
|
Author: Margin
|
|
6
6
|
License: MIT
|
|
@@ -452,6 +452,50 @@ current_task.set(ticket.topic) # "billing" | "password-reset" | …
|
|
|
452
452
|
Leave it off and the row's `task_key` is simply unset (`unknown`) — never a
|
|
453
453
|
fabricated bucket.
|
|
454
454
|
|
|
455
|
+
## Say where each call RAN (`environment`)
|
|
456
|
+
|
|
457
|
+
`environment` splits the work you sell from the work you did to decide what to
|
|
458
|
+
sell. A spike, an eval pass and your production traffic are three different kinds
|
|
459
|
+
of spend, and only one of them belongs in the number you report upward.
|
|
460
|
+
|
|
461
|
+
```python
|
|
462
|
+
meter.record_call(
|
|
463
|
+
workflow_id="support-triage",
|
|
464
|
+
provider="openai",
|
|
465
|
+
model="gpt-4o-mini",
|
|
466
|
+
input_tokens=812,
|
|
467
|
+
output_tokens=96,
|
|
468
|
+
environment="prod", # "prod" | "dev" | "eval"
|
|
469
|
+
)
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
Setting it once per process is usually easier. A deployment already knows whether
|
|
473
|
+
it is production; the call site does not:
|
|
474
|
+
|
|
475
|
+
```bash
|
|
476
|
+
export MARGIN_ENVIRONMENT=prod
|
|
477
|
+
```
|
|
478
|
+
|
|
479
|
+
An explicit argument wins over the variable, so one process can still label a
|
|
480
|
+
single call differently.
|
|
481
|
+
|
|
482
|
+
| value | send it for |
|
|
483
|
+
| --- | --- |
|
|
484
|
+
| `prod` | traffic your customers cause |
|
|
485
|
+
| `dev` | a spike, a local run, a branch deploy |
|
|
486
|
+
| `eval` | a graded pass, in CI or from an eval harness |
|
|
487
|
+
|
|
488
|
+
Why it earns a field. `cost_per_outcome` is total call spend divided by passed
|
|
489
|
+
outcomes. With `environment` unset, your R&D and your operations land in that one
|
|
490
|
+
number, so "your AI workforce costs $X per outcome" describes neither. Margin
|
|
491
|
+
already keeps its own face-off spend out of your number; this is how you get the
|
|
492
|
+
same split on your side.
|
|
493
|
+
|
|
494
|
+
Two things it will not do. It does not default: leave it off and the row is
|
|
495
|
+
`unknown`, which is honest and costs you nothing, where guessing `prod` would book
|
|
496
|
+
your spike as production. And it is never priced and never rolled into cost. It is
|
|
497
|
+
a label, like `team`.
|
|
498
|
+
|
|
455
499
|
## API
|
|
456
500
|
|
|
457
501
|
| Method | Emits to | Notes |
|
|
@@ -440,6 +440,50 @@ current_task.set(ticket.topic) # "billing" | "password-reset" | …
|
|
|
440
440
|
Leave it off and the row's `task_key` is simply unset (`unknown`) — never a
|
|
441
441
|
fabricated bucket.
|
|
442
442
|
|
|
443
|
+
## Say where each call RAN (`environment`)
|
|
444
|
+
|
|
445
|
+
`environment` splits the work you sell from the work you did to decide what to
|
|
446
|
+
sell. A spike, an eval pass and your production traffic are three different kinds
|
|
447
|
+
of spend, and only one of them belongs in the number you report upward.
|
|
448
|
+
|
|
449
|
+
```python
|
|
450
|
+
meter.record_call(
|
|
451
|
+
workflow_id="support-triage",
|
|
452
|
+
provider="openai",
|
|
453
|
+
model="gpt-4o-mini",
|
|
454
|
+
input_tokens=812,
|
|
455
|
+
output_tokens=96,
|
|
456
|
+
environment="prod", # "prod" | "dev" | "eval"
|
|
457
|
+
)
|
|
458
|
+
```
|
|
459
|
+
|
|
460
|
+
Setting it once per process is usually easier. A deployment already knows whether
|
|
461
|
+
it is production; the call site does not:
|
|
462
|
+
|
|
463
|
+
```bash
|
|
464
|
+
export MARGIN_ENVIRONMENT=prod
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
An explicit argument wins over the variable, so one process can still label a
|
|
468
|
+
single call differently.
|
|
469
|
+
|
|
470
|
+
| value | send it for |
|
|
471
|
+
| --- | --- |
|
|
472
|
+
| `prod` | traffic your customers cause |
|
|
473
|
+
| `dev` | a spike, a local run, a branch deploy |
|
|
474
|
+
| `eval` | a graded pass, in CI or from an eval harness |
|
|
475
|
+
|
|
476
|
+
Why it earns a field. `cost_per_outcome` is total call spend divided by passed
|
|
477
|
+
outcomes. With `environment` unset, your R&D and your operations land in that one
|
|
478
|
+
number, so "your AI workforce costs $X per outcome" describes neither. Margin
|
|
479
|
+
already keeps its own face-off spend out of your number; this is how you get the
|
|
480
|
+
same split on your side.
|
|
481
|
+
|
|
482
|
+
Two things it will not do. It does not default: leave it off and the row is
|
|
483
|
+
`unknown`, which is honest and costs you nothing, where guessing `prod` would book
|
|
484
|
+
your spike as production. And it is never priced and never rolled into cost. It is
|
|
485
|
+
a label, like `team`.
|
|
486
|
+
|
|
443
487
|
## API
|
|
444
488
|
|
|
445
489
|
| Method | Emits to | Notes |
|
|
@@ -106,4 +106,4 @@ __all__ = [
|
|
|
106
106
|
# closing the invoice-reconcile coverage gap is "ship the fix, wait for the seed's
|
|
107
107
|
# dependency refresh, re-run the reconcile", and a runtime that misreports which
|
|
108
108
|
# SDK metered a call makes that unanswerable.
|
|
109
|
-
__version__ = "0.6.
|
|
109
|
+
__version__ = "0.6.3"
|
|
@@ -1719,6 +1719,7 @@ class MarginMeter:
|
|
|
1719
1719
|
is_simulated: bool = False,
|
|
1720
1720
|
event_id: str | None = None,
|
|
1721
1721
|
span_id: str | None = None,
|
|
1722
|
+
environment: str | None = None,
|
|
1722
1723
|
value_usd: float | None = None,
|
|
1723
1724
|
) -> IngestResult:
|
|
1724
1725
|
"""Emit one outcome (a unit of productivity) to ``POST /api/ingest/outcomes``.
|
|
@@ -1748,11 +1749,25 @@ class MarginMeter:
|
|
|
1748
1749
|
value (revenue booked, a ticket's handling cost avoided); omit it and the
|
|
1749
1750
|
outcome is measured as cost-per-outcome, the honest default. It is optional
|
|
1750
1751
|
by construction — absent is the normal case, never defaulted to 0.
|
|
1752
|
+
|
|
1753
|
+
``environment`` is WHERE this outcome was produced — the DENOMINATOR twin of
|
|
1754
|
+
:meth:`record_call`'s (MAR-1562): ``prod`` for the work the customer sells,
|
|
1755
|
+
``dev``/``eval`` for the work they did to decide what to sell. Same resolution
|
|
1756
|
+
rules as the call path — explicit wins, then ``MARGIN_ENVIRONMENT``, then
|
|
1757
|
+
absent. ⛔ Omit it and the server stores NULL (UNKNOWN); it is never assumed to
|
|
1758
|
+
be ``prod``.
|
|
1751
1759
|
"""
|
|
1752
1760
|
if span_id is None:
|
|
1753
1761
|
_span = _SPAN.get()
|
|
1754
1762
|
if _span is not None:
|
|
1755
1763
|
span_id = _span.span_id
|
|
1764
|
+
# Same env-var fallback and same server-width clamp as `record_call`, for the same
|
|
1765
|
+
# reason: `ingest._require_str` RAISES (422) on an over-long value, so an outcome
|
|
1766
|
+
# label longer than the column would drop the metered row rather than mislabel it —
|
|
1767
|
+
# and a dropped outcome is a silently smaller denominator, the flattering direction.
|
|
1768
|
+
environment = environment or (os.environ.get("MARGIN_ENVIRONMENT") or "").strip() or None
|
|
1769
|
+
if environment:
|
|
1770
|
+
environment = environment[:32]
|
|
1756
1771
|
payload: dict[str, Any] = {
|
|
1757
1772
|
"workflow_id": workflow_id,
|
|
1758
1773
|
"passed": passed,
|
|
@@ -1781,6 +1796,10 @@ class MarginMeter:
|
|
|
1781
1796
|
# send. An older server that predates the column ignores an unknown key.
|
|
1782
1797
|
if value_usd is not None:
|
|
1783
1798
|
payload["value_usd"] = value_usd
|
|
1799
|
+
# Rides only when one was declared (explicitly or via the process env). An
|
|
1800
|
+
# unlabelled outcome sends nothing and the server stores NULL, never `prod`.
|
|
1801
|
+
if environment is not None:
|
|
1802
|
+
payload["environment"] = environment
|
|
1784
1803
|
return self._emit(OUTCOMES_PATH, payload)
|
|
1785
1804
|
|
|
1786
1805
|
def measure(
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: margin-meter
|
|
3
|
-
Version: 0.6.
|
|
3
|
+
Version: 0.6.3
|
|
4
4
|
Summary: Tiny stdlib-only client that auto-instruments OpenAI/Anthropic/Gemini and emits LLM call + outcome economics to a Margin ingest API.
|
|
5
5
|
Author: Margin
|
|
6
6
|
License: MIT
|
|
@@ -452,6 +452,50 @@ current_task.set(ticket.topic) # "billing" | "password-reset" | …
|
|
|
452
452
|
Leave it off and the row's `task_key` is simply unset (`unknown`) — never a
|
|
453
453
|
fabricated bucket.
|
|
454
454
|
|
|
455
|
+
## Say where each call RAN (`environment`)
|
|
456
|
+
|
|
457
|
+
`environment` splits the work you sell from the work you did to decide what to
|
|
458
|
+
sell. A spike, an eval pass and your production traffic are three different kinds
|
|
459
|
+
of spend, and only one of them belongs in the number you report upward.
|
|
460
|
+
|
|
461
|
+
```python
|
|
462
|
+
meter.record_call(
|
|
463
|
+
workflow_id="support-triage",
|
|
464
|
+
provider="openai",
|
|
465
|
+
model="gpt-4o-mini",
|
|
466
|
+
input_tokens=812,
|
|
467
|
+
output_tokens=96,
|
|
468
|
+
environment="prod", # "prod" | "dev" | "eval"
|
|
469
|
+
)
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
Setting it once per process is usually easier. A deployment already knows whether
|
|
473
|
+
it is production; the call site does not:
|
|
474
|
+
|
|
475
|
+
```bash
|
|
476
|
+
export MARGIN_ENVIRONMENT=prod
|
|
477
|
+
```
|
|
478
|
+
|
|
479
|
+
An explicit argument wins over the variable, so one process can still label a
|
|
480
|
+
single call differently.
|
|
481
|
+
|
|
482
|
+
| value | send it for |
|
|
483
|
+
| --- | --- |
|
|
484
|
+
| `prod` | traffic your customers cause |
|
|
485
|
+
| `dev` | a spike, a local run, a branch deploy |
|
|
486
|
+
| `eval` | a graded pass, in CI or from an eval harness |
|
|
487
|
+
|
|
488
|
+
Why it earns a field. `cost_per_outcome` is total call spend divided by passed
|
|
489
|
+
outcomes. With `environment` unset, your R&D and your operations land in that one
|
|
490
|
+
number, so "your AI workforce costs $X per outcome" describes neither. Margin
|
|
491
|
+
already keeps its own face-off spend out of your number; this is how you get the
|
|
492
|
+
same split on your side.
|
|
493
|
+
|
|
494
|
+
Two things it will not do. It does not default: leave it off and the row is
|
|
495
|
+
`unknown`, which is honest and costs you nothing, where guessing `prod` would book
|
|
496
|
+
your spike as production. And it is never priced and never rolled into cost. It is
|
|
497
|
+
a label, like `team`.
|
|
498
|
+
|
|
455
499
|
## API
|
|
456
500
|
|
|
457
501
|
| Method | Emits to | Notes |
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|