threadsense 0.1.0a0__tar.gz

threadsense-0.1.0a0/.gitignore

@@ -0,0 +1,220 @@
+# macOS
+.DS_Store
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[codz]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py.cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# UV
+#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#uv.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+#poetry.toml
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#   pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
+#   https://pdm-project.org/en/latest/usage/project/#working-with-version-control
+#pdm.lock
+#pdm.toml
+.pdm-python
+.pdm-build/
+
+# pixi
+#   Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
+#pixi.lock
+#   Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
+#   in the .venv directory. It is recommended not to include this directory in version control.
+.pixi
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.envrc
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+# Abstra
+# Abstra is an AI-powered process automation framework.
+# Ignore directories containing user credentials, local state, and settings.
+# Learn more at https://abstra.io/docs
+.abstra/
+
+# Visual Studio Code
+#  Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore 
+#  that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
+#  and can be added to the global gitignore or merged into this file. However, if you prefer, 
+#  you could uncomment the following to ignore the entire vscode folder
+# .vscode/
+
+# Ruff stuff:
+.ruff_cache/
+
+# PyPI configuration file
+.pypirc
+
+# Cursor
+#  Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
+#  exclude from AI features like autocomplete and code analysis. Recommended for sensitive data
+#  refer to https://docs.cursor.com/context/ignore-files
+.cursorignore
+.cursorindexingignore
+
+# Marimo
+marimo/_static/
+marimo/_lsp/
+__marimo__/
+
+# Agentic Systems
+.agent
+.claude/
+.gemini
+.opencode/
+
+# Developer workspace
+.docs/
+.threadsense/

threadsense-0.1.0a0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tom Mathews
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

threadsense-0.1.0a0/PKG-INFO

@@ -0,0 +1,258 @@
+Metadata-Version: 2.4
+Name: threadsense
+Version: 0.1.0a0
+Summary: Evidence-first discussion intelligence tooling.
+License: MIT License
+        
+        Copyright (c) 2026 Tom Mathews
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Requires-Python: >=3.11
+Requires-Dist: rich>=14.3.3
+Description-Content-Type: text/markdown
+
+# ThreadSense
+
+ThreadSense is an evidence-first discussion intelligence pipeline for turning long community threads into structured, inspectable product intelligence.
+
+The current implementation supports:
+
+- Reddit thread ingestion through the public JSON API
+- canonical normalization and persisted artifacts
+- deterministic analysis with evidence-linked findings
+- optional local-model summary generation through an OpenAI-compatible runtime
+- JSON and Markdown report generation
+- reproducible batch runs
+- a local HTTP API surface
+
+ThreadSense is local-first. It is optimized for operator workflows, replayability, and evidence traceability rather than cloud-first convenience.
+
+## Happy Path
+
+Run the full single-thread workflow with one command:
+
+```bash
+uv run threadsense run reddit <reddit-url> \
+  [--with-summary] \
+  [--summary-required] \
+  [--format markdown|json]
+```
+
+Example:
+
+```bash
+uv run threadsense run reddit \
+  "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/" \
+  --format markdown \
+  --with-summary \
+  --summary-required
+```
+
+## What It Does
+
+ThreadSense runs a staged pipeline:
+
+1. fetch a Reddit thread
+2. persist the raw artifact
+3. normalize it into a canonical thread model
+4. run deterministic analysis
+5. optionally run local inference for summary synthesis
+6. persist a JSON or Markdown report
+
+Artifacts remain separate at each stage so the pipeline is inspectable and rerunnable.
+
+## Architecture
+
+Core modules:
+
+- [src/threadsense/connectors/reddit.py](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/connectors/reddit.py): Reddit ingestion, retries, response validation, `morechildren` expansion
+- [src/threadsense/models/canonical.py](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/models/canonical.py): canonical thread schema
+- [src/threadsense/pipeline/normalize.py](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/pipeline/normalize.py): Reddit-to-canonical normalization
+- [src/threadsense/pipeline/analyze.py](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/pipeline/analyze.py): deterministic analysis baseline
+- [src/threadsense/inference](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/inference): local inference contracts, prompts, runtime adapter, and routing
+- [src/threadsense/reporting](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/reporting): report assembly, rendering, and quality checks
+- [src/threadsense/workflows.py](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/workflows.py): shared fetch, normalize, analyze, infer, and report execution
+- [src/threadsense/batching.py](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/batching.py): batch manifests and bounded parallel runs
+- [src/threadsense/api_server.py](/Users/druk/WorkSpace/AetherForge/ThreadSense/src/threadsense/api_server.py): local HTTP surface
+
+Supporting docs:
+
+- [docs/overview.md](/Users/druk/WorkSpace/AetherForge/ThreadSense/docs/overview.md)
+- [docs/system-design.md](/Users/druk/WorkSpace/AetherForge/ThreadSense/docs/system-design.md)
+- [docs/local-runtime-contract.md](/Users/druk/WorkSpace/AetherForge/ThreadSense/docs/local-runtime-contract.md)
+- [docs/batch-api-runtime.md](/Users/druk/WorkSpace/AetherForge/ThreadSense/docs/batch-api-runtime.md)
+- [.docs/2026-04-04-system-enhancement-analysis.md](/Users/druk/WorkSpace/AetherForge/ThreadSense/.docs/2026-04-04-system-enhancement-analysis.md)
+
+## Requirements
+
+- Python `>=3.11`
+- `uv`
+- optional local OpenAI-compatible runtime at `http://127.0.0.1:8080/v1/chat/completions`
+
+## Installation
+
+```bash
+uv sync
+```
+
+Run the CLI through `uv`:
+
+```bash
+uv run threadsense --help
+```
+
+## Quickstart
+
+### 1. Validate local setup
+
+```bash
+uv run threadsense preflight
+```
+
+Skip runtime probing if you only want config validation:
+
+```bash
+uv run threadsense preflight --skip-runtime
+```
+
+### 2. Run the full workflow
+
+```bash
+uv run threadsense run reddit \
+  "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/" \
+  --format markdown
+```
+
+The command prints one JSON payload containing the stage outputs and final artifact paths.
+
+Use `--with-summary --summary-required` if you want the final report to require a live local-model summary.
+
+## Usage Reference
+
+For the detailed command-by-command reference, including:
+
+- `preflight`
+- `fetch`
+- `normalize`
+- `analyze`
+- `infer`
+- `report`
+- `inspect`
+- `batch run`
+- `serve`
+- `run reddit`
+
+see [docs/usage.md](/Users/druk/WorkSpace/AetherForge/ThreadSense/docs/usage.md).
+
+## Configuration
+
+Default config file:
+
+- [threadsense.toml](/Users/druk/WorkSpace/AetherForge/ThreadSense/threadsense.toml)
+
+Important sections:
+
+- `[runtime]`: local inference backend URL, path, model, timeout, repair retries
+- `[reddit]`: user agent, timeout, retries, backoff, request delay, listing limit
+- `[storage]`: raw, normalized, analysis, report, and batch directories
+- `[batch]`: worker count, max jobs, fail-fast behavior
+- `[api]`: host, port, request size limit
+- `[limits]`: runtime concurrency guard
+
+Common environment overrides:
+
+- `THREADSENSE_RUNTIME_ENABLED`
+- `THREADSENSE_RUNTIME_BASE_URL`
+- `THREADSENSE_RUNTIME_CHAT_PATH`
+- `THREADSENSE_RUNTIME_MODEL`
+- `THREADSENSE_REDDIT_TIMEOUT`
+- `THREADSENSE_REDDIT_REQUEST_DELAY`
+- `THREADSENSE_STORAGE_ROOT`
+- `THREADSENSE_BATCH_MAX_WORKERS`
+- `THREADSENSE_BATCH_MAX_JOBS`
+- `THREADSENSE_API_PORT`
+- `THREADSENSE_RUNTIME_CONCURRENCY`
+
+## Artifact Layout
+
+Default storage root:
+
+- `.threadsense`
+
+Default structure:
+
+- `.threadsense/raw/reddit/<thread-id>.json`
+- `.threadsense/normalized/reddit/<thread-id>.json`
+- `.threadsense/analysis/reddit/<thread-id>.json`
+- `.threadsense/reports/reddit/<thread-id>.json`
+- `.threadsense/reports/reddit/<thread-id>.md`
+- `.threadsense/batches/<run-name>.json`
+
+## Local Runtime Contract
+
+ThreadSense expects a local OpenAI-compatible chat completion endpoint.
+
+Default target:
+
+- base URL: `http://127.0.0.1:8080`
+- chat path: `/v1/chat/completions`
+- resolved endpoint: `http://127.0.0.1:8080/v1/chat/completions`
+
+The local runtime contract is documented in [docs/local-runtime-contract.md](/Users/druk/WorkSpace/AetherForge/ThreadSense/docs/local-runtime-contract.md).
+
+## Validation
+
+Run the full local validation stack:
+
+```bash
+uv run ruff check
+uv run ruff format --check .
+uv run mypy --strict src tests
+uv run pytest
+./scripts/validate.sh
+```
+
+## Tests
+
+The repo includes:
+
+- unit tests for config, connectors, normalization, analysis, inference, reporting, observability, batching, and API validation
+- integration tests for fetch, normalization, inference, reporting, batch runs, and API workflows
+- fixture-backed Reddit payloads and replay manifests under [tests/fixtures](/Users/druk/WorkSpace/AetherForge/ThreadSense/tests/fixtures)
+
+## Current Limits
+
+The current system is intentionally constrained:
+
+- only Reddit is implemented as a source
+- analysis is still a deterministic heuristic baseline
+- the main product unit is still a single thread or local batch run
+- the API is local-first and not hardened for untrusted callers
+- report presentation is Markdown and JSON, not yet an interactive HTML review surface
+
+## Roadmap Direction
+
+The completed phased plan established the system backbone. The next high-value areas are:
+
+- claim-level evidence attribution
+- evaluation and replay benchmarking
+- corpus-level analysis across threads
+- better decision-oriented report presentation
+- stronger onboarding and developer ergonomics