sooth 0.2.2__tar.gz → 0.2.3__tar.gz

{sooth-0.2.2 → sooth-0.2.3}/.github/workflows/ci.yml

@@ -19,3 +19,4 @@ jobs:
       - run: pip install -e ".[dev]"
       - run: ruff check src tests
       - run: pytest
+      - run: bash tests/test_action.sh

{sooth-0.2.2 → sooth-0.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: sooth
-Version: 0.2.2
+Version: 0.2.3
 Summary: Verify AI-generated text against source material. Claim-by-claim trust reports powered by Jev (TypeSafe System One).
 Project-URL: Homepage, https://github.com/naufalhilmiaji/sooth
 Project-URL: Documentation, https://github.com/naufalhilmiaji/sooth/tree/main/docs
@@ -48,10 +48,18 @@ Real output — seven claims in an AI-written news summary, three numbers quietl
 
 | # | Claim | Verdict | P | Why (P distribution) | Evidence |
 |---|-------|---------|---|----------------------|----------|
-| 3 | BNBR baru menuntaskan rights issue bernilai besar di harga Rp 33. | ❌ FAIL | ████████ 1.00 | contradicts 1.00 / not_found 0.00 / supports 0.00 · details 0.01 · missing #s: 33 | `examples/news-1.md:25` "Saham ini juga baru menyelesaikan rights issue dalam juml…" |
-| 5 | BUMI hanya perlu turun sekitar 10% untuk menyentuh level Rp 50. | ❌ FAIL | ████████ 1.00 | contradicts 1.00 / not_found 0.00 / supports 0.00 · details 0.01 · missing #s: 10 | `examples/news-1.md:23` "Adapun PT Bumi Resources Tbk (BUMI) di sekitar Rp192 haru…" |
-| 6 | BEI juga menetapkan batas atas harga saham Rp 5.000 per saham. | ⚠️ REVIEW | ████████ 1.00 | contradicts 0.00 / not_found 1.00 / supports 0.00 · details 0.02 · missing #s: 5.000 |  |
-| 7 | Para investor ritel sangat senang dengan aturan baru ini. | ➖ UNCHECKABLE | ██░░░░░░ 0.24 | P(checkable)=0.24 |  |
+| 1 | Aturan baru BEI menurunkan batas bawah harga saham menjadi Rp 1 per saham mulai 28 September 2026. | ✅ PASS | ████████ 1.00 | supports 1.00 / not_found 0.00 / contradicts 0.00 · details 0.98 | `examples/news-1.md:3` "Saham-saham Grup Bakrie kembali menjadi sorotan menjelang…" |
+| 2 | Menurut analis Bumiputera Sekuritas, aturan ini berlaku untuk seluruh emiten di BEI. | ✅ PASS | ████████ 1.00 | supports 1.00 / not_found 0.00 / contradicts 0.00 · details 0.82 | `examples/news-1.md:7` "Ia menyebut aturan ini berlaku untuk seluruh saham di BEI…" |
+| 3 | BNBR baru menuntaskan rights issue bernilai besar di harga Rp 33. | ❌ FAIL | ████████ 1.00 | supports 0.00 / not_found 0.00 / contradicts 1.00 · details 0.01 · missing #s: 33 | `examples/news-1.md:25` "Saham ini juga baru menyelesaikan rights issue dalam juml…" |
+| 4 | Saham VIVA disuspensi di harga Rp 50. | ❌ FAIL | ████████ 1.00 | supports 0.00 / not_found 0.00 / contradicts 1.00 · details 0.02 |  |
+| 5 | BUMI hanya perlu turun sekitar 10% untuk menyentuh level Rp 50. | ❌ FAIL | ████████ 1.00 | supports 0.00 / not_found 0.00 / contradicts 1.00 · details 0.01 · missing #s: 10 | `examples/news-1.md:23` "Adapun PT Bumi Resources Tbk (BUMI) di sekitar Rp192 haru…" |
+| 6 | BEI juga menetapkan batas atas harga saham Rp 5.000 per saham di pasar reguler. | ⚠️ REVIEW | ████████ 1.00 | supports 0.00 / not_found 1.00 / contradicts 0.00 · details 0.02 · missing #s: 5.000 |  |
+| 7 | Para investor ritel sangat senang dengan aturan baru ini. | ➖ UNCHECKABLE | ██░░░░░░ 0.25 | P(checkable)=0.25 |  |
+
+## Needs review
+
+- line 13: BEI juga menetapkan batas atas harga saham Rp 5.000 per saham di pasar reguler.
+- line 15: Para investor ritel sangat senang dengan aturan baru ini.
 ```
 
 Every verdict carries its probability distribution and the source span it was judged against. When Sooth is unsure, it says `REVIEW` instead of guessing.
@@ -85,15 +93,22 @@ sooth --source policy.md --text draft.md --format plain
 #   --log FILE       append full judgment trace (one JSONL line per run)
 ```
 
-Wire it into CI as a quality gate for generated content:
+Use it in CI as a quality gate for generated content:
 
 ```yaml
 - name: Verify AI output
-  run: sooth --source docs/policy.md --text generated-reply.md
+  uses: naufalhilmiaji/sooth@v0.2.3
+  with:
+    source: docs/policy.md
+    text: generated-reply.md
   env:
     TYPESAFE_API_KEY: ${{ secrets.TYPESAFE_API_KEY }}
 ```
 
+`source` takes newline-separated paths when you have several documents. The step fails the build on any `FAIL` — and on any `REVIEW`, so add `continue-on-error: true` if you only want to block on hard contradictions. The full report is appended to the job summary.
+
+Prefer plain shell? `sooth --source policy.md --text draft.md` gives the same exit codes.
+
 ## How it works
 
 1. Draft is split into claims (one sentence each).
@@ -149,9 +164,9 @@ Docs: [PRD](https://github.com/naufalhilmiaji/sooth/blob/main/docs/PRD.md) · [D
 
 ## Roadmap
 
+- v0.2.3 shipped: GitHub Action — CI quality gate in one `uses:` line
 - v0.2.2 shipped: `sooth demo` (offline, no API key)
 - v0.2.1 shipped: source-span evidence, published on PyPI
-- GitHub Action
 - later: hosted web app — paste UI, history, team review queues
 
 ## License

{sooth-0.2.2 → sooth-0.2.3}/README.md

@@ -25,10 +25,18 @@ Real output — seven claims in an AI-written news summary, three numbers quietl
 
 | # | Claim | Verdict | P | Why (P distribution) | Evidence |
 |---|-------|---------|---|----------------------|----------|
-| 3 | BNBR baru menuntaskan rights issue bernilai besar di harga Rp 33. | ❌ FAIL | ████████ 1.00 | contradicts 1.00 / not_found 0.00 / supports 0.00 · details 0.01 · missing #s: 33 | `examples/news-1.md:25` "Saham ini juga baru menyelesaikan rights issue dalam juml…" |
-| 5 | BUMI hanya perlu turun sekitar 10% untuk menyentuh level Rp 50. | ❌ FAIL | ████████ 1.00 | contradicts 1.00 / not_found 0.00 / supports 0.00 · details 0.01 · missing #s: 10 | `examples/news-1.md:23` "Adapun PT Bumi Resources Tbk (BUMI) di sekitar Rp192 haru…" |
-| 6 | BEI juga menetapkan batas atas harga saham Rp 5.000 per saham. | ⚠️ REVIEW | ████████ 1.00 | contradicts 0.00 / not_found 1.00 / supports 0.00 · details 0.02 · missing #s: 5.000 |  |
-| 7 | Para investor ritel sangat senang dengan aturan baru ini. | ➖ UNCHECKABLE | ██░░░░░░ 0.24 | P(checkable)=0.24 |  |
+| 1 | Aturan baru BEI menurunkan batas bawah harga saham menjadi Rp 1 per saham mulai 28 September 2026. | ✅ PASS | ████████ 1.00 | supports 1.00 / not_found 0.00 / contradicts 0.00 · details 0.98 | `examples/news-1.md:3` "Saham-saham Grup Bakrie kembali menjadi sorotan menjelang…" |
+| 2 | Menurut analis Bumiputera Sekuritas, aturan ini berlaku untuk seluruh emiten di BEI. | ✅ PASS | ████████ 1.00 | supports 1.00 / not_found 0.00 / contradicts 0.00 · details 0.82 | `examples/news-1.md:7` "Ia menyebut aturan ini berlaku untuk seluruh saham di BEI…" |
+| 3 | BNBR baru menuntaskan rights issue bernilai besar di harga Rp 33. | ❌ FAIL | ████████ 1.00 | supports 0.00 / not_found 0.00 / contradicts 1.00 · details 0.01 · missing #s: 33 | `examples/news-1.md:25` "Saham ini juga baru menyelesaikan rights issue dalam juml…" |
+| 4 | Saham VIVA disuspensi di harga Rp 50. | ❌ FAIL | ████████ 1.00 | supports 0.00 / not_found 0.00 / contradicts 1.00 · details 0.02 |  |
+| 5 | BUMI hanya perlu turun sekitar 10% untuk menyentuh level Rp 50. | ❌ FAIL | ████████ 1.00 | supports 0.00 / not_found 0.00 / contradicts 1.00 · details 0.01 · missing #s: 10 | `examples/news-1.md:23` "Adapun PT Bumi Resources Tbk (BUMI) di sekitar Rp192 haru…" |
+| 6 | BEI juga menetapkan batas atas harga saham Rp 5.000 per saham di pasar reguler. | ⚠️ REVIEW | ████████ 1.00 | supports 0.00 / not_found 1.00 / contradicts 0.00 · details 0.02 · missing #s: 5.000 |  |
+| 7 | Para investor ritel sangat senang dengan aturan baru ini. | ➖ UNCHECKABLE | ██░░░░░░ 0.25 | P(checkable)=0.25 |  |
+
+## Needs review
+
+- line 13: BEI juga menetapkan batas atas harga saham Rp 5.000 per saham di pasar reguler.
+- line 15: Para investor ritel sangat senang dengan aturan baru ini.
 ```
 
 Every verdict carries its probability distribution and the source span it was judged against. When Sooth is unsure, it says `REVIEW` instead of guessing.
@@ -62,15 +70,22 @@ sooth --source policy.md --text draft.md --format plain
 #   --log FILE       append full judgment trace (one JSONL line per run)
 ```
 
-Wire it into CI as a quality gate for generated content:
+Use it in CI as a quality gate for generated content:
 
 ```yaml
 - name: Verify AI output
-  run: sooth --source docs/policy.md --text generated-reply.md
+  uses: naufalhilmiaji/sooth@v0.2.3
+  with:
+    source: docs/policy.md
+    text: generated-reply.md
   env:
     TYPESAFE_API_KEY: ${{ secrets.TYPESAFE_API_KEY }}
 ```
 
+`source` takes newline-separated paths when you have several documents. The step fails the build on any `FAIL` — and on any `REVIEW`, so add `continue-on-error: true` if you only want to block on hard contradictions. The full report is appended to the job summary.
+
+Prefer plain shell? `sooth --source policy.md --text draft.md` gives the same exit codes.
+
 ## How it works
 
 1. Draft is split into claims (one sentence each).
@@ -126,9 +141,9 @@ Docs: [PRD](https://github.com/naufalhilmiaji/sooth/blob/main/docs/PRD.md) · [D
 
 ## Roadmap
 
+- v0.2.3 shipped: GitHub Action — CI quality gate in one `uses:` line
 - v0.2.2 shipped: `sooth demo` (offline, no API key)
 - v0.2.1 shipped: source-span evidence, published on PyPI
-- GitHub Action
 - later: hosted web app — paste UI, history, team review queues
 
 ## License

sooth-0.2.3/action.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+# Wire SOOTH_* inputs into the sooth CLI, surface the report, propagate the exit code.
+# Inputs arrive via env (never interpolated into this script) so untrusted input
+# cannot be executed as shell.
+set -uo pipefail
+
+args=(--format md)
+while IFS= read -r path; do
+  [ -z "$path" ] && continue
+  args+=(--source "$path")
+done <<< "$SOOTH_SOURCE"
+
+args+=(--text "$SOOTH_TEXT")
+if [ -n "${SOOTH_CONFIDENCE:-}" ]; then
+  args+=(--confidence "$SOOTH_CONFIDENCE")
+fi
+
+report="${RUNNER_TEMP:-/tmp}/sooth-report.md"
+sooth "${args[@]}" -o "$report"
+status=$?
+
+if [ -f "$report" ]; then
+  cat "$report"
+  if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
+    cat "$report" >> "$GITHUB_STEP_SUMMARY"
+  fi
+fi
+
+exit "$status"

sooth-0.2.3/action.yml

@@ -0,0 +1,35 @@
+name: "Sooth"
+description: "Claim-by-claim verification of AI-generated text against source material. Fails CI on hallucinations."
+author: "Naufal Hilmiaji"
+branding:
+  icon: "check-circle"
+  color: "green"
+
+inputs:
+  source:
+    description: "Ground-truth source file(s). Repeat by separating paths with newlines."
+    required: true
+  text:
+    description: "AI-generated draft to check."
+    required: true
+  confidence:
+    description: "REVIEW below this confidence."
+    required: false
+    default: "0.7"
+
+runs:
+  using: "composite"
+  steps:
+    - uses: actions/setup-python@v5
+      with:
+        python-version: "3.12"
+    - name: Install sooth
+      shell: bash
+      run: python -m pip install --quiet sooth
+    - name: Verify draft
+      shell: bash
+      env:
+        SOOTH_SOURCE: ${{ inputs.source }}
+        SOOTH_TEXT: ${{ inputs.text }}
+        SOOTH_CONFIDENCE: ${{ inputs.confidence }}
+      run: bash "$GITHUB_ACTION_PATH/action.sh"

{sooth-0.2.2 → sooth-0.2.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sooth"
-version = "0.2.2"
+version = "0.2.3"
 description = "Verify AI-generated text against source material. Claim-by-claim trust reports powered by Jev (TypeSafe System One)."
 readme = "README.md"
 license = { text = "MIT" }

sooth-0.2.3/tests/test_action.sh

@@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+# Arg-wiring checks for action.sh. No network — a fake `sooth` records its argv.
+# Run: bash tests/test_action.sh
+set -uo pipefail
+
+here="$(cd "$(dirname "$0")" && pwd)"
+tmp="$(mktemp -d)"
+trap 'rm -rf "$tmp"' EXIT
+
+write_fake() {
+  cat > "$tmp/sooth" <<'EOF'
+#!/usr/bin/env bash
+printf '%s\n' "$@" > "$ARGV_FILE"
+prev=""
+for a in "$@"; do
+  if [ "$prev" = "-o" ]; then echo "# report" > "$a"; fi
+  prev="$a"
+done
+exit "${FAKE_STATUS:-0}"
+EOF
+  chmod +x "$tmp/sooth"
+}
+
+# argv as one line, plus the same with the dynamic "-o <report>" tail dropped
+argv_full() { tr '\n' ' ' < "$ARGV_FILE"; }
+args_only() { argv_full | sed 's/ -o [^ ]*//'; }
+
+export PATH="$tmp:$PATH"
+export ARGV_FILE="$tmp/argv"
+export RUNNER_TEMP="$tmp"
+export SOOTH_TEXT="draft.md"
+export SOOTH_CONFIDENCE="0.75"
+unset GITHUB_STEP_SUMMARY 2>/dev/null || true
+
+fail() { echo "FAIL: $*" >&2; exit 1; }
+
+write_fake
+
+# single source
+export SOOTH_SOURCE="policy.md"
+bash "$here/../action.sh" >/dev/null || fail "single source: exit $?"
+got="$(args_only)"
+want="--format md --source policy.md --text draft.md --confidence 0.75 "
+[ "$got" = "$want" ] || fail "single source argv: got [$got] want [$want]"
+
+# multiple sources, newline-separated, blanks ignored
+export SOOTH_SOURCE="a.md
+b.md
+
+c.md"
+bash "$here/../action.sh" >/dev/null || fail "multi source: exit $?"
+got="$(args_only)"
+want="--format md --source a.md --source b.md --source c.md --text draft.md --confidence 0.75 "
+[ "$got" = "$want" ] || fail "multi source argv: got [$got] want [$want]"
+
+# confidence is optional
+unset SOOTH_CONFIDENCE
+export SOOTH_SOURCE="policy.md"
+bash "$here/../action.sh" >/dev/null || fail "no confidence: exit $?"
+got="$(args_only)"
+want="--format md --source policy.md --text draft.md "
+[ "$got" = "$want" ] || fail "no confidence argv: got [$got] want [$want]"
+
+# a FAIL from sooth must fail the step (CI gate), and the report still prints
+export SOOTH_CONFIDENCE="0.7"
+export SOOTH_SOURCE="policy.md"
+export FAKE_STATUS=1
+out="$(bash "$here/../action.sh")"
+status=$?
+[ "$status" -eq 1 ] || fail "exit code not propagated: got $status"
+[ -n "$out" ] || fail "report not printed on failure"
+
+# hostile input must arrive as a literal argument and never execute
+export FAKE_STATUS=0
+export SOOTH_TEXT='draft.md; echo PWNED'
+out="$(bash "$here/../action.sh")" || fail "injection: exit $?"
+case "$out" in
+  *PWNED*) fail "input was executed as shell" ;;
+esac
+full="$(argv_full)"
+case "$full" in
+  *"--text draft.md; echo PWNED "*) : ;;
+  *) fail "injection argv: got [$full]" ;;
+esac
+
+echo "ok — action.sh arg wiring (5 checks)"