threadsense 0.2.0__tar.gz → 0.2.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (150) hide show
  1. threadsense-0.2.1/PKG-INFO +223 -0
  2. threadsense-0.2.1/README.md +190 -0
  3. {threadsense-0.2.0 → threadsense-0.2.1}/pyproject.toml +1 -1
  4. threadsense-0.2.0/PKG-INFO +0 -249
  5. threadsense-0.2.0/README.md +0 -216
  6. {threadsense-0.2.0 → threadsense-0.2.1}/.approval_tests_temp/.gitignore +0 -0
  7. {threadsense-0.2.0 → threadsense-0.2.1}/.approval_tests_temp/approve_all.py +0 -0
  8. {threadsense-0.2.0 → threadsense-0.2.1}/.approval_tests_temp/remove_abandoned_files.py +0 -0
  9. {threadsense-0.2.0 → threadsense-0.2.1}/.gitignore +0 -0
  10. {threadsense-0.2.0 → threadsense-0.2.1}/LICENSE +0 -0
  11. {threadsense-0.2.0 → threadsense-0.2.1}/docs/artifacts.md +0 -0
  12. {threadsense-0.2.0 → threadsense-0.2.1}/docs/batch-api-runtime.md +0 -0
  13. {threadsense-0.2.0 → threadsense-0.2.1}/docs/local-runtime-contract.md +0 -0
  14. {threadsense-0.2.0 → threadsense-0.2.1}/docs/output-modes.md +0 -0
  15. {threadsense-0.2.0 → threadsense-0.2.1}/docs/overview.md +0 -0
  16. {threadsense-0.2.0 → threadsense-0.2.1}/docs/pitch.md +0 -0
  17. {threadsense-0.2.0 → threadsense-0.2.1}/docs/research-reddit.md +0 -0
  18. {threadsense-0.2.0 → threadsense-0.2.1}/docs/system-design.md +0 -0
  19. {threadsense-0.2.0 → threadsense-0.2.1}/docs/usage.md +0 -0
  20. {threadsense-0.2.0 → threadsense-0.2.1}/scripts/validate.sh +0 -0
  21. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/__init__.py +0 -0
  22. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/__main__.py +0 -0
  23. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/action_signals.py +0 -0
  24. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/api_server.py +0 -0
  25. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/batching.py +0 -0
  26. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/cli.py +0 -0
  27. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/cli_display.py +0 -0
  28. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/config.py +0 -0
  29. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/connectors/__init__.py +0 -0
  30. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/connectors/cache.py +0 -0
  31. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/connectors/github_discussions.py +0 -0
  32. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/connectors/hackernews.py +0 -0
  33. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/connectors/reddit.py +0 -0
  34. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/connectors/registry.py +0 -0
  35. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/contracts/__init__.py +0 -0
  36. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/contracts/analysis_contract.py +0 -0
  37. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/contracts/defaults.py +0 -0
  38. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/__init__.py +0 -0
  39. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/definitions/developer_tools.toml +0 -0
  40. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/definitions/financial_markets.toml +0 -0
  41. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/definitions/gaming.toml +0 -0
  42. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/definitions/hiring_careers.toml +0 -0
  43. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/definitions/product_feedback.toml +0 -0
  44. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/definitions/research_academic.toml +0 -0
  45. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/loader.py +0 -0
  46. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/domains/vocabulary.py +0 -0
  47. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/errors.py +0 -0
  48. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/evaluation/__init__.py +0 -0
  49. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/evaluation/compare.py +0 -0
  50. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/evaluation/golden.py +0 -0
  51. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/evaluation/inference_quality.py +0 -0
  52. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/evaluation/metrics.py +0 -0
  53. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/inference/__init__.py +0 -0
  54. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/inference/contracts.py +0 -0
  55. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/inference/local_runtime.py +0 -0
  56. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/inference/prompts.py +0 -0
  57. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/inference/router.py +0 -0
  58. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/logging_config.py +0 -0
  59. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/models/__init__.py +0 -0
  60. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/models/analysis.py +0 -0
  61. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/models/canonical.py +0 -0
  62. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/models/corpus.py +0 -0
  63. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/models/report.py +0 -0
  64. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/models/results.py +0 -0
  65. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/observability.py +0 -0
  66. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/__init__.py +0 -0
  67. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/alignment.py +0 -0
  68. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/analyze.py +0 -0
  69. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/corpus.py +0 -0
  70. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/corpus_index.py +0 -0
  71. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/domain_detect.py +0 -0
  72. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/normalize.py +0 -0
  73. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/replay.py +0 -0
  74. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/storage.py +0 -0
  75. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/strategies/__init__.py +0 -0
  76. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/strategies/keyword_heuristic.py +0 -0
  77. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/tree.py +0 -0
  78. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/pipeline/versioning.py +0 -0
  79. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/preflight.py +0 -0
  80. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/py.typed +0 -0
  81. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/reporting/__init__.py +0 -0
  82. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/reporting/build.py +0 -0
  83. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/reporting/corpus_render.py +0 -0
  84. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/reporting/quality.py +0 -0
  85. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/reporting/render.py +0 -0
  86. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/schema_utils.py +0 -0
  87. {threadsense-0.2.0 → threadsense-0.2.1}/src/threadsense/workflows.py +0 -0
  88. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/README.md +0 -0
  89. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/analysis/canonical_feedback_thread.json +0 -0
  90. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/batch/reddit_manifest.json +0 -0
  91. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/hackernews/raw/story_thread.json +0 -0
  92. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/inference/analysis_summary_invalid_response.json +0 -0
  93. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/inference/analysis_summary_response.json +0 -0
  94. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/inference/chat_completion_request.json +0 -0
  95. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/inference/chat_completion_response.json +0 -0
  96. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/normalized/.gitkeep +0 -0
  97. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/reddit/raw/.gitkeep +0 -0
  98. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/reddit/raw/deleted_thread.json +0 -0
  99. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/reddit/raw/large_thread.json +0 -0
  100. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/reddit/raw/morechildren_response.json +0 -0
  101. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/reddit/raw/normal_thread.json +0 -0
  102. {threadsense-0.2.0 → threadsense-0.2.1}/tests/fixtures/reddit/raw/removed_thread.json +0 -0
  103. {threadsense-0.2.0 → threadsense-0.2.1}/tests/golden/README.md +0 -0
  104. {threadsense-0.2.0 → threadsense-0.2.1}/tests/golden/developer_tools/reddit_feedback_thread.json +0 -0
  105. {threadsense-0.2.0 → threadsense-0.2.1}/tests/golden/manifest.json +0 -0
  106. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_batch_api.py +0 -0
  107. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_bootstrap.py +0 -0
  108. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_canonical_persistence.py +0 -0
  109. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_full_pipeline.py +0 -0
  110. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_local_inference.py +0 -0
  111. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_preflight.py +0 -0
  112. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_reddit_fetch.py +0 -0
  113. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_reddit_research.py +0 -0
  114. {threadsense-0.2.0 → threadsense-0.2.1}/tests/integration/test_reporting.py +0 -0
  115. {threadsense-0.2.0 → threadsense-0.2.1}/tests/property/__init__.py +0 -0
  116. {threadsense-0.2.0 → threadsense-0.2.1}/tests/property/test_schema_properties.py +0 -0
  117. {threadsense-0.2.0 → threadsense-0.2.1}/tests/regression/expected_outputs/reddit_feedback_thread.json +0 -0
  118. {threadsense-0.2.0 → threadsense-0.2.1}/tests/regression/test_analysis_replay.py +0 -0
  119. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_alignment.py +0 -0
  120. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_analysis_pipeline.py +0 -0
  121. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_api_server.py +0 -0
  122. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_artifact_versioning.py +0 -0
  123. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_batching.py +0 -0
  124. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_canonical_models.py +0 -0
  125. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_cli.py +0 -0
  126. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_cli_display.py +0 -0
  127. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_config.py +0 -0
  128. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_corpus_analysis.py +0 -0
  129. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_corpus_index.py +0 -0
  130. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_corpus_model.py +0 -0
  131. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_domain_detect.py +0 -0
  132. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_errors.py +0 -0
  133. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_fetch_cache.py +0 -0
  134. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_github_discussions_connector.py +0 -0
  135. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_golden_evaluation.py +0 -0
  136. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_hackernews_connector.py +0 -0
  137. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_inference_quality.py +0 -0
  138. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_inference_router.py +0 -0
  139. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_local_runtime.py +0 -0
  140. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_minhash.py +0 -0
  141. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_normalize_pipeline.py +0 -0
  142. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_observability.py +0 -0
  143. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_reddit_connector.py +0 -0
  144. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_replay.py +0 -0
  145. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_report_rendering.py +0 -0
  146. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_source_registry.py +0 -0
  147. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_strategy_comparison.py +0 -0
  148. {threadsense-0.2.0 → threadsense-0.2.1}/tests/unit/test_tree.py +0 -0
  149. {threadsense-0.2.0 → threadsense-0.2.1}/threadsense.toml +0 -0
  150. {threadsense-0.2.0 → threadsense-0.2.1}/uv.lock +0 -0
@@ -0,0 +1,223 @@
1
+ Metadata-Version: 2.4
2
+ Name: threadsense
3
+ Version: 0.2.1
4
+ Summary: Evidence-first discussion intelligence tooling.
5
+ License: MIT License
6
+
7
+ Copyright (c) 2026 Tom Mathews
8
+
9
+ Permission is hereby granted, free of charge, to any person obtaining a copy
10
+ of this software and associated documentation files (the "Software"), to deal
11
+ in the Software without restriction, including without limitation the rights
12
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
13
+ copies of the Software, and to permit persons to whom the Software is
14
+ furnished to do so, subject to the following conditions:
15
+
16
+ The above copyright notice and this permission notice shall be included in all
17
+ copies or substantial portions of the Software.
18
+
19
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
20
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
21
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
22
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
23
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
24
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
25
+ SOFTWARE.
26
+ License-File: LICENSE
27
+ Requires-Python: >=3.11
28
+ Requires-Dist: httpx>=0.28.1
29
+ Requires-Dist: pydantic-settings>=2.13.1
30
+ Requires-Dist: pydantic>=2.12.5
31
+ Requires-Dist: rich>=14.3.3
32
+ Description-Content-Type: text/markdown
33
+
34
+ # ThreadSense
35
+
36
+ **Discussion intelligence, not scraping.** ThreadSense is a reproducible pipeline that turns public discussion threads into structured, evidence-backed product and research intelligence.
37
+
38
+ ## Why ThreadSense
39
+
40
+ Scrapers give you raw payloads. AI summarizers give you prose. Neither gives you a defensible basis for decisions.
41
+
42
+ ThreadSense keeps the evidence chain intact at every stage:
43
+
44
+ 1. **Acquire** — source connectors fetch threads from Reddit, Hacker News, and GitHub Discussions
45
+ 2. **Normalize** — source-specific payloads are mapped into a canonical thread model with provenance metadata
46
+ 3. **Analyze** — deterministic extraction identifies issues, feature requests, themes, and sentiment — each linked to the comment that produced it
47
+ 4. **Synthesize** — optional local-model inference adds summaries on top of the deterministic evidence layer
48
+ 5. **Report** — structured outputs in Markdown, HTML, or JSON, with full traceability from finding back to source comment
49
+
50
+ Every stage produces a persisted, inspectable artifact. Rerun any stage independently. Diff results across runs. Audit exactly where a finding came from.
51
+
52
+ ## What This Enables
53
+
54
+ ### Single-thread analysis
55
+
56
+ Feed a discussion URL. Get structured findings — issues, requests, themes, severity — with every claim linked to the comment that produced it.
57
+
58
+ ```bash
59
+ uv run threadsense run reddit \
60
+ "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/" \
61
+ --format markdown \
62
+ --with-summary \
63
+ --summary-required
64
+ ```
65
+
66
+ ### Cross-thread research
67
+
68
+ Search a topic across multiple subreddits. ThreadSense deterministically selects, ranks, and analyzes matching threads, then synthesizes a corpus-level report.
69
+
70
+ ```bash
71
+ uv run threadsense --output-format human research reddit \
72
+ --query "second brain OR agentic PKM" \
73
+ --subreddit ClaudeCode \
74
+ --subreddit LocalLLaMA \
75
+ --subreddit AI_Agents \
76
+ --limit 5 \
77
+ --per-subreddit-limit 3 \
78
+ --with-summary
79
+ ```
80
+
81
+ ### Domain-aware analysis
82
+
83
+ The analysis layer uses a contract system with domain-specific vocabularies (developer tools, product feedback, hiring, research, financial markets, gaming). Each domain defines its own theme keywords, issue markers, and severity calibration — so analysis adapts to context rather than applying one-size-fits-all heuristics.
84
+
85
+ ## Architecture
86
+
87
+ ```text
88
+ fetch → normalize → analyze → [optional inference] → report
89
+ ↓ ↓ ↓ ↓
90
+ raw artifact canonical analysis report artifact
91
+ artifact artifact
92
+ ```
93
+
94
+ - **Deterministic core** — parsing, normalization, scoring, and selection are reproducible across runs
95
+ - **Inference on top** — LLM synthesis is optional and layered over deterministic evidence, never a substitute for it
96
+ - **Stable artifacts** — each stage persists a separate JSON artifact with `schema_version` and SHA256 provenance
97
+ - **Fail fast** — invalid URLs, malformed payloads, and schema inconsistencies surface immediately
98
+
99
+ ## Sources and Discovery
100
+
101
+ | Capability | Reddit | Hacker News | GitHub Discussions |
102
+ | --------------- | :----: | :---------: | :----------------: |
103
+ | Thread analysis | yes | yes | yes |
104
+ | Topic research | yes | — | — |
105
+
106
+ ## Output Modes
107
+
108
+ | Mode | Purpose |
109
+ | ------- | ------------------------------------------------ |
110
+ | `json` | Machine-readable payloads for downstream tooling |
111
+ | `human` | Rich terminal panels and summaries for operators |
112
+ | `quiet` | Status-only output for scripts and CI |
113
+
114
+ ```bash
115
+ uv run threadsense --output-format human research reddit ...
116
+ ```
117
+
118
+ See [docs/output-modes.md](docs/output-modes.md) for details.
119
+
120
+ ## Who This Is For
121
+
122
+ - **Product teams** validating pain points and feature demand from community discussions
123
+ - **Founders** doing market and competitor research across technical communities
124
+ - **DevRel teams** tracking developer workflow friction and tooling sentiment
125
+ - **Researchers** studying technical communities with reproducible methodology
126
+
127
+ ## Quickstart
128
+
129
+ ```bash
130
+ # 1. Install
131
+ uv sync
132
+
133
+ # 2. Validate local setup
134
+ uv run threadsense preflight
135
+
136
+ # 3. Analyze a single thread
137
+ uv run threadsense run reddit \
138
+ "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/"
139
+
140
+ # 4. Research a topic across subreddits
141
+ uv run threadsense research reddit \
142
+ --query "second brain OR agentic PKM" \
143
+ --subreddit ClaudeCode \
144
+ --subreddit LocalLLaMA \
145
+ --subreddit AI_Agents
146
+ ```
147
+
148
+ ## CLI Commands
149
+
150
+ | Command | Purpose |
151
+ | ----------------- | --------------------------------------------------- |
152
+ | `run` | End-to-end single-thread pipeline |
153
+ | `research reddit` | Cross-subreddit topic research and corpus synthesis |
154
+ | `fetch` | Acquire raw thread data |
155
+ | `normalize` | Map raw data to canonical model |
156
+ | `analyze` | Deterministic evidence extraction |
157
+ | `infer` | LLM-assisted synthesis |
158
+ | `report` | Generate output reports |
159
+ | `corpus` | Build and analyze cross-thread corpora |
160
+ | `inspect` | Examine persisted artifacts |
161
+ | `batch run` | Process multiple threads |
162
+ | `preflight` | Validate local environment |
163
+ | `serve` | Local API server |
164
+
165
+ Full command reference: [docs/usage.md](docs/usage.md)
166
+
167
+ ## Artifact Storage
168
+
169
+ Every pipeline run produces inspectable artifacts under `.threadsense/`:
170
+
171
+ ```text
172
+ .threadsense/
173
+ ├── raw/<source>/ # Source payloads as fetched
174
+ ├── normalized/<source>/ # Canonical thread model
175
+ ├── analysis/<source>/ # Evidence-linked findings
176
+ ├── reports/<source>/ # Rendered reports
177
+ ├── corpora/<corpus-id>/ # Manifest, analysis, and report
178
+ └── batches/ # Batch run metadata
179
+ ```
180
+
181
+ Details: [docs/artifacts.md](docs/artifacts.md)
182
+
183
+ ## Local Runtime
184
+
185
+ ThreadSense runs without a local model for deterministic analysis. Summaries and synthesis require a local OpenAI-compatible endpoint (default: `http://127.0.0.1:8080/v1/chat/completions`).
186
+
187
+ Details: [docs/local-runtime-contract.md](docs/local-runtime-contract.md)
188
+
189
+ ## Documentation
190
+
191
+ | Document | Content |
192
+ | ----------------------------------------------------------- | ----------------------------------- |
193
+ | [usage.md](docs/usage.md) | Command reference |
194
+ | [research-reddit.md](docs/research-reddit.md) | Reddit topic research workflow |
195
+ | [output-modes.md](docs/output-modes.md) | JSON, human, and quiet output modes |
196
+ | [artifacts.md](docs/artifacts.md) | Artifact types and storage layout |
197
+ | [overview.md](docs/overview.md) | Product and workflow overview |
198
+ | [system-design.md](docs/system-design.md) | Architecture and system boundaries |
199
+ | [local-runtime-contract.md](docs/local-runtime-contract.md) | Local inference contract |
200
+ | [pitch.md](docs/pitch.md) | Product positioning |
201
+
202
+ ## Validation
203
+
204
+ ```bash
205
+ uv run ruff check
206
+ uv run ruff format --check .
207
+ uv run mypy --strict src tests
208
+ uv run pytest
209
+ ```
210
+
211
+ ## Current Limits
212
+
213
+ - Topic research is implemented for Reddit; other source discovery workflows are planned
214
+ - Reddit research queries support `OR`/`|` clause unions only (intentionally narrow for deterministic alignment)
215
+ - Corpus reports are Markdown only
216
+ - The local API is a trusted local surface, not a hardened public service
217
+
218
+ ## Direction
219
+
220
+ - Richer corpus presentation and operator workflows
221
+ - Discovery workflows beyond Reddit
222
+ - Evaluation and replay benchmarking
223
+ - Source-distribution and research-quality reporting
@@ -0,0 +1,190 @@
1
+ # ThreadSense
2
+
3
+ **Discussion intelligence, not scraping.** ThreadSense is a reproducible pipeline that turns public discussion threads into structured, evidence-backed product and research intelligence.
4
+
5
+ ## Why ThreadSense
6
+
7
+ Scrapers give you raw payloads. AI summarizers give you prose. Neither gives you a defensible basis for decisions.
8
+
9
+ ThreadSense keeps the evidence chain intact at every stage:
10
+
11
+ 1. **Acquire** — source connectors fetch threads from Reddit, Hacker News, and GitHub Discussions
12
+ 2. **Normalize** — source-specific payloads are mapped into a canonical thread model with provenance metadata
13
+ 3. **Analyze** — deterministic extraction identifies issues, feature requests, themes, and sentiment — each linked to the comment that produced it
14
+ 4. **Synthesize** — optional local-model inference adds summaries on top of the deterministic evidence layer
15
+ 5. **Report** — structured outputs in Markdown, HTML, or JSON, with full traceability from finding back to source comment
16
+
17
+ Every stage produces a persisted, inspectable artifact. Rerun any stage independently. Diff results across runs. Audit exactly where a finding came from.
18
+
19
+ ## What This Enables
20
+
21
+ ### Single-thread analysis
22
+
23
+ Feed a discussion URL. Get structured findings — issues, requests, themes, severity — with every claim linked to the comment that produced it.
24
+
25
+ ```bash
26
+ uv run threadsense run reddit \
27
+ "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/" \
28
+ --format markdown \
29
+ --with-summary \
30
+ --summary-required
31
+ ```
32
+
33
+ ### Cross-thread research
34
+
35
+ Search a topic across multiple subreddits. ThreadSense deterministically selects, ranks, and analyzes matching threads, then synthesizes a corpus-level report.
36
+
37
+ ```bash
38
+ uv run threadsense --output-format human research reddit \
39
+ --query "second brain OR agentic PKM" \
40
+ --subreddit ClaudeCode \
41
+ --subreddit LocalLLaMA \
42
+ --subreddit AI_Agents \
43
+ --limit 5 \
44
+ --per-subreddit-limit 3 \
45
+ --with-summary
46
+ ```
47
+
48
+ ### Domain-aware analysis
49
+
50
+ The analysis layer uses a contract system with domain-specific vocabularies (developer tools, product feedback, hiring, research, financial markets, gaming). Each domain defines its own theme keywords, issue markers, and severity calibration — so analysis adapts to context rather than applying one-size-fits-all heuristics.
51
+
52
+ ## Architecture
53
+
54
+ ```text
55
+ fetch → normalize → analyze → [optional inference] → report
56
+ ↓ ↓ ↓ ↓
57
+ raw artifact canonical analysis report artifact
58
+ artifact artifact
59
+ ```
60
+
61
+ - **Deterministic core** — parsing, normalization, scoring, and selection are reproducible across runs
62
+ - **Inference on top** — LLM synthesis is optional and layered over deterministic evidence, never a substitute for it
63
+ - **Stable artifacts** — each stage persists a separate JSON artifact with `schema_version` and SHA256 provenance
64
+ - **Fail fast** — invalid URLs, malformed payloads, and schema inconsistencies surface immediately
65
+
66
+ ## Sources and Discovery
67
+
68
+ | Capability | Reddit | Hacker News | GitHub Discussions |
69
+ | --------------- | :----: | :---------: | :----------------: |
70
+ | Thread analysis | yes | yes | yes |
71
+ | Topic research | yes | — | — |
72
+
73
+ ## Output Modes
74
+
75
+ | Mode | Purpose |
76
+ | ------- | ------------------------------------------------ |
77
+ | `json` | Machine-readable payloads for downstream tooling |
78
+ | `human` | Rich terminal panels and summaries for operators |
79
+ | `quiet` | Status-only output for scripts and CI |
80
+
81
+ ```bash
82
+ uv run threadsense --output-format human research reddit ...
83
+ ```
84
+
85
+ See [docs/output-modes.md](docs/output-modes.md) for details.
86
+
87
+ ## Who This Is For
88
+
89
+ - **Product teams** validating pain points and feature demand from community discussions
90
+ - **Founders** doing market and competitor research across technical communities
91
+ - **DevRel teams** tracking developer workflow friction and tooling sentiment
92
+ - **Researchers** studying technical communities with reproducible methodology
93
+
94
+ ## Quickstart
95
+
96
+ ```bash
97
+ # 1. Install
98
+ uv sync
99
+
100
+ # 2. Validate local setup
101
+ uv run threadsense preflight
102
+
103
+ # 3. Analyze a single thread
104
+ uv run threadsense run reddit \
105
+ "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/"
106
+
107
+ # 4. Research a topic across subreddits
108
+ uv run threadsense research reddit \
109
+ --query "second brain OR agentic PKM" \
110
+ --subreddit ClaudeCode \
111
+ --subreddit LocalLLaMA \
112
+ --subreddit AI_Agents
113
+ ```
114
+
115
+ ## CLI Commands
116
+
117
+ | Command | Purpose |
118
+ | ----------------- | --------------------------------------------------- |
119
+ | `run` | End-to-end single-thread pipeline |
120
+ | `research reddit` | Cross-subreddit topic research and corpus synthesis |
121
+ | `fetch` | Acquire raw thread data |
122
+ | `normalize` | Map raw data to canonical model |
123
+ | `analyze` | Deterministic evidence extraction |
124
+ | `infer` | LLM-assisted synthesis |
125
+ | `report` | Generate output reports |
126
+ | `corpus` | Build and analyze cross-thread corpora |
127
+ | `inspect` | Examine persisted artifacts |
128
+ | `batch run` | Process multiple threads |
129
+ | `preflight` | Validate local environment |
130
+ | `serve` | Local API server |
131
+
132
+ Full command reference: [docs/usage.md](docs/usage.md)
133
+
134
+ ## Artifact Storage
135
+
136
+ Every pipeline run produces inspectable artifacts under `.threadsense/`:
137
+
138
+ ```text
139
+ .threadsense/
140
+ ├── raw/<source>/ # Source payloads as fetched
141
+ ├── normalized/<source>/ # Canonical thread model
142
+ ├── analysis/<source>/ # Evidence-linked findings
143
+ ├── reports/<source>/ # Rendered reports
144
+ ├── corpora/<corpus-id>/ # Manifest, analysis, and report
145
+ └── batches/ # Batch run metadata
146
+ ```
147
+
148
+ Details: [docs/artifacts.md](docs/artifacts.md)
149
+
150
+ ## Local Runtime
151
+
152
+ ThreadSense runs without a local model for deterministic analysis. Summaries and synthesis require a local OpenAI-compatible endpoint (default: `http://127.0.0.1:8080/v1/chat/completions`).
153
+
154
+ Details: [docs/local-runtime-contract.md](docs/local-runtime-contract.md)
155
+
156
+ ## Documentation
157
+
158
+ | Document | Content |
159
+ | ----------------------------------------------------------- | ----------------------------------- |
160
+ | [usage.md](docs/usage.md) | Command reference |
161
+ | [research-reddit.md](docs/research-reddit.md) | Reddit topic research workflow |
162
+ | [output-modes.md](docs/output-modes.md) | JSON, human, and quiet output modes |
163
+ | [artifacts.md](docs/artifacts.md) | Artifact types and storage layout |
164
+ | [overview.md](docs/overview.md) | Product and workflow overview |
165
+ | [system-design.md](docs/system-design.md) | Architecture and system boundaries |
166
+ | [local-runtime-contract.md](docs/local-runtime-contract.md) | Local inference contract |
167
+ | [pitch.md](docs/pitch.md) | Product positioning |
168
+
169
+ ## Validation
170
+
171
+ ```bash
172
+ uv run ruff check
173
+ uv run ruff format --check .
174
+ uv run mypy --strict src tests
175
+ uv run pytest
176
+ ```
177
+
178
+ ## Current Limits
179
+
180
+ - Topic research is implemented for Reddit; other source discovery workflows are planned
181
+ - Reddit research queries support `OR`/`|` clause unions only (intentionally narrow for deterministic alignment)
182
+ - Corpus reports are Markdown only
183
+ - The local API is a trusted local surface, not a hardened public service
184
+
185
+ ## Direction
186
+
187
+ - Richer corpus presentation and operator workflows
188
+ - Discovery workflows beyond Reddit
189
+ - Evaluation and replay benchmarking
190
+ - Source-distribution and research-quality reporting
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
4
4
 
5
5
  [project]
6
6
  name = "threadsense"
7
- version = "0.2.0"
7
+ version = "0.2.1"
8
8
  description = "Evidence-first discussion intelligence tooling."
9
9
  readme = "README.md"
10
10
  license = { file = "LICENSE" }
@@ -1,249 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: threadsense
3
- Version: 0.2.0
4
- Summary: Evidence-first discussion intelligence tooling.
5
- License: MIT License
6
-
7
- Copyright (c) 2026 Tom Mathews
8
-
9
- Permission is hereby granted, free of charge, to any person obtaining a copy
10
- of this software and associated documentation files (the "Software"), to deal
11
- in the Software without restriction, including without limitation the rights
12
- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
13
- copies of the Software, and to permit persons to whom the Software is
14
- furnished to do so, subject to the following conditions:
15
-
16
- The above copyright notice and this permission notice shall be included in all
17
- copies or substantial portions of the Software.
18
-
19
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
20
- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
21
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
22
- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
23
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
24
- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
25
- SOFTWARE.
26
- License-File: LICENSE
27
- Requires-Python: >=3.11
28
- Requires-Dist: httpx>=0.28.1
29
- Requires-Dist: pydantic-settings>=2.13.1
30
- Requires-Dist: pydantic>=2.12.5
31
- Requires-Dist: rich>=14.3.3
32
- Description-Content-Type: text/markdown
33
-
34
- # ThreadSense
35
-
36
- ThreadSense is an evidence-first discussion intelligence system for turning public discussion threads into structured, inspectable product and research intelligence.
37
-
38
- It combines:
39
-
40
- - source connectors for Reddit, Hacker News, and GitHub Discussions
41
- - canonical normalization into a stable thread model
42
- - deterministic analysis with evidence-linked findings
43
- - optional local-model synthesis through an OpenAI-compatible runtime
44
- - report generation for single threads and cross-thread corpora
45
- - operator-facing CLI output in JSON, human, and quiet modes
46
-
47
- ## What You Can Do
48
-
49
- ### Analyze one thread end to end
50
-
51
- ```bash
52
- uv run threadsense run reddit \
53
- "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/" \
54
- --format markdown \
55
- --with-summary \
56
- --summary-required
57
- ```
58
-
59
- ### Research a topic across selected subreddits
60
-
61
- ```bash
62
- uv run threadsense --output-format human research reddit \
63
- --query "second brain OR agentic PKM" \
64
- --subreddit ClaudeCode \
65
- --subreddit LocalLLaMA \
66
- --subreddit AI_Agents \
67
- --limit 5 \
68
- --per-subreddit-limit 3 \
69
- --with-summary
70
- ```
71
-
72
- This workflow:
73
-
74
- 1. searches the selected subreddits
75
- 2. ranks and selects matching threads deterministically
76
- 3. fetches and analyzes each selected thread
77
- 4. builds a corpus manifest and corpus analysis
78
- 5. writes a corpus markdown report
79
- 6. prints a compact terminal summary in human mode
80
-
81
- ## Core Features
82
-
83
- - Reddit thread ingestion through the public JSON API
84
- - Reddit topic research across selected subreddits
85
- - canonical thread artifacts with top-level thread body support
86
- - deterministic issue/request/theme extraction
87
- - optional local-runtime summaries for thread reports and corpus reports
88
- - Markdown, HTML, and JSON report outputs for single-thread runs
89
- - Markdown corpus reports for topic research and corpus reporting
90
- - terminal summaries for single-thread `run` and `research reddit` in human mode
91
-
92
- ## Supported Sources
93
-
94
- Implemented connector support:
95
-
96
- - Reddit
97
- - Hacker News
98
- - GitHub Discussions
99
-
100
- Current research discovery support:
101
-
102
- - Reddit only
103
-
104
- ## Output Modes
105
-
106
- ThreadSense supports three output modes:
107
-
108
- - `json`: machine-readable payloads
109
- - `human`: Rich terminal panels and summaries
110
- - `quiet`: status-only output
111
-
112
- Force human mode explicitly when you want the terminal summary panels:
113
-
114
- ```bash
115
- uv run threadsense --output-format human research reddit ...
116
- ```
117
-
118
- See [docs/output-modes.md](docs/output-modes.md) for details.
119
-
120
- ## Quickstart
121
-
122
- ### 1. Install dependencies
123
-
124
- ```bash
125
- uv sync
126
- ```
127
-
128
- ### 2. Validate your local setup
129
-
130
- ```bash
131
- uv run threadsense preflight
132
- ```
133
-
134
- ### 3. Run a single-thread workflow
135
-
136
- ```bash
137
- uv run threadsense run reddit \
138
- "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/"
139
- ```
140
-
141
- ### 4. Run topic research
142
-
143
- ```bash
144
- uv run threadsense research reddit \
145
- --query "second brain OR agentic PKM" \
146
- --subreddit ClaudeCode \
147
- --subreddit LocalLLaMA \
148
- --subreddit AI_Agents
149
- ```
150
-
151
- ## Research Query Notes
152
-
153
- `research reddit` supports a deliberately narrow query style so local deterministic matching stays aligned with retrieval:
154
-
155
- - supported: `OR`, `|`
156
- - unsupported: quotes, parentheses, `title:`, `selftext:`, negation-style advanced Reddit syntax
157
-
158
- Example supported query:
159
-
160
- ```text
161
- second brain OR agentic PKM
162
- ```
163
-
164
- This is executed as a union of clause searches, then deterministically filtered and ranked locally.
165
-
166
- See [docs/research-reddit.md](docs/research-reddit.md) for the full workflow.
167
-
168
- ## Main CLI Workflows
169
-
170
- - `preflight`
171
- - `fetch`
172
- - `normalize`
173
- - `analyze`
174
- - `infer`
175
- - `report`
176
- - `inspect`
177
- - `corpus`
178
- - `batch run`
179
- - `research reddit`
180
- - `serve`
181
- - `run`
182
-
183
- The detailed command reference lives in [docs/usage.md](docs/usage.md).
184
-
185
- ## Artifacts
186
-
187
- ThreadSense persists separate artifacts for each stage so runs are inspectable and rerunnable.
188
-
189
- Typical layout under `.threadsense`:
190
-
191
- - `raw/<source>/...`
192
- - `normalized/<source>/...`
193
- - `analysis/<source>/...`
194
- - `reports/<source>/...`
195
- - `corpora/<corpus-id>/manifest.json`
196
- - `corpora/<corpus-id>/analysis.json`
197
- - `corpora/<corpus-id>/report.md`
198
- - `batches/<run-name>.json`
199
-
200
- See [docs/artifacts.md](docs/artifacts.md) for the full layout and artifact responsibilities.
201
-
202
- ## Runtime
203
-
204
- ThreadSense can run without a local model, but summaries require a local OpenAI-compatible runtime.
205
-
206
- Default endpoint:
207
-
208
- - base URL: `http://127.0.0.1:8080`
209
- - chat path: `/v1/chat/completions`
210
-
211
- See [docs/local-runtime-contract.md](docs/local-runtime-contract.md).
212
-
213
- ## Documentation Map
214
-
215
- - [docs/usage.md](docs/usage.md): command reference
216
- - [docs/research-reddit.md](docs/research-reddit.md): subreddit topic research workflow
217
- - [docs/output-modes.md](docs/output-modes.md): JSON, human, and quiet output modes
218
- - [docs/artifacts.md](docs/artifacts.md): persisted artifact types and storage layout
219
- - [docs/overview.md](docs/overview.md): product and workflow overview
220
- - [docs/system-design.md](docs/system-design.md): implemented architecture and boundaries
221
- - [docs/local-runtime-contract.md](docs/local-runtime-contract.md): local inference contract
222
- - [docs/batch-api-runtime.md](docs/batch-api-runtime.md): batch, API, and runtime notes
223
- - [docs/pitch.md](docs/pitch.md): product positioning
224
-
225
- ## Validation
226
-
227
- ```bash
228
- uv run ruff check
229
- uv run ruff format --check .
230
- uv run mypy --strict src tests
231
- uv run pytest
232
- ./scripts/validate.sh
233
- ```
234
-
235
- ## Current Limits
236
-
237
- - Reddit topic research is implemented; broader cross-source research discovery is not
238
- - Reddit research query grammar is intentionally limited
239
- - corpus reports are currently Markdown only
240
- - the local API remains a trusted local surface, not a hardened public service
241
-
242
- ## Direction
243
-
244
- The current system is beyond the original single-thread MVP. The highest-value next areas are:
245
-
246
- - richer corpus presentation and operator workflows
247
- - more discovery workflows beyond Reddit subreddit search
248
- - stronger evaluation and replay benchmarking
249
- - better source-distribution and research-quality reporting