coolhand 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/AGENTS.harness.md +139 -0
- data/CHANGELOG.md +64 -0
- data/README.md +46 -105
- data/docs/configuration.md +40 -0
- data/docs/openai.md +91 -0
- data/docs/template-search.md +218 -0
- data/docs/vertex.md +54 -0
- data/lib/coolhand/api_service.rb +56 -16
- data/lib/coolhand/base_interceptor.rb +58 -38
- data/lib/coolhand/configuration.rb +39 -6
- data/lib/coolhand/default_exclude_api_patterns.yml +3 -2
- data/lib/coolhand/default_intercept_addresses.yml +10 -5
- data/lib/coolhand/default_intercept_path_patterns.yml +12 -0
- data/lib/coolhand/errors.rb +19 -0
- data/lib/coolhand/logger_service.rb +1 -1
- data/lib/coolhand/net_http_interceptor.rb +154 -24
- data/lib/coolhand/open_ai/batch_result_processor.rb +3 -1
- data/lib/coolhand/open_ai/webhook_id_store.rb +45 -0
- data/lib/coolhand/open_ai/webhook_validator.rb +50 -12
- data/lib/coolhand/pagination.rb +82 -0
- data/lib/coolhand/read_requests.rb +77 -0
- data/lib/coolhand/template_service.rb +76 -0
- data/lib/coolhand/version.rb +1 -1
- data/lib/coolhand/vertex/batch_result_processor.rb +97 -40
- data/lib/coolhand/webhook_interceptor.rb +11 -0
- data/lib/coolhand.rb +10 -5
- metadata +16 -9
- data/.claude/skills/loop-review/SKILL.md +0 -112
- data/.claude/skills/prep-release/SKILL.md +0 -160
- data/.idea/coolhand-ruby.iml +0 -6
- data/CLAUDE.md +0 -47
|
@@ -1,160 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: prep-release
|
|
3
|
-
description: |
|
|
4
|
-
Prepares this gem for a release: runs the full test suite, updates and
|
|
5
|
-
cleans up the docs (README, CHANGELOG, docs/*.md) to reflect every change
|
|
6
|
-
since the last tag, bumps the version if that hasn't already been done,
|
|
7
|
-
and red-teams the whole package for security - not just this release's
|
|
8
|
-
diff. Never tags or pushes. Use when the user types /prep-release, asks
|
|
9
|
-
to "prep a release", "get ready to cut a release", "release checklist",
|
|
10
|
-
or wants a pre-release audit before tagging/publishing a new version.
|
|
11
|
-
user_invocable: true
|
|
12
|
-
version: 0.2.0
|
|
13
|
-
---
|
|
14
|
-
|
|
15
|
-
# Prep Release
|
|
16
|
-
|
|
17
|
-
Three phases, run in order. This is a whole-package audit, not a diff
|
|
18
|
-
review — do not scope any phase to just what changed since the last
|
|
19
|
-
commit. For an iterative diff-scoped review during normal development, use
|
|
20
|
-
`/loop-review` instead; this skill is for the release boundary.
|
|
21
|
-
|
|
22
|
-
## Phase 1: Run all tests
|
|
23
|
-
|
|
24
|
-
Run `bundle exec rake` (`rspec` then `rubocop`, per the `Rakefile`). If a
|
|
25
|
-
fix round changed something outside this repo's usual toolchain, discover
|
|
26
|
-
the right command instead of assuming. All specs must pass and RuboCop
|
|
27
|
-
must report zero offenses before continuing — a release doesn't ship on a
|
|
28
|
-
red build. If either fails, stop here and report the failures; fixing
|
|
29
|
-
genuine bugs takes priority over Phase 2/3 work.
|
|
30
|
-
|
|
31
|
-
Then judge coverage on quality, not just the SimpleCov percentage the rake
|
|
32
|
-
run reports (written to `coverage/`):
|
|
33
|
-
|
|
34
|
-
1. **Find the gaps.** List files/lines SimpleCov marks uncovered. Weight
|
|
35
|
-
by risk: an uncovered error-handling branch or security check
|
|
36
|
-
(signature/header validation) matters more than an uncovered
|
|
37
|
-
`attr_reader`.
|
|
38
|
-
2. **Audit existing tests for meaningfulness, not just count.** Flag tests
|
|
39
|
-
that only assert a stub returns what it was stubbed to return without
|
|
40
|
-
exercising real conditional logic in the subject under test; missing
|
|
41
|
-
negative/error-path cases (invalid input, malformed provider responses,
|
|
42
|
-
network failure); missing domain edge cases (empty batch results,
|
|
43
|
-
duplicate-request prevention, concurrent access, streaming vs
|
|
44
|
-
non-streaming shapes).
|
|
45
|
-
3. **Recommend, don't pad.** Propose specific specs for the highest-risk
|
|
46
|
-
gaps, named by `file:describe/context`. Don't add tests purely to move
|
|
47
|
-
the percentage — a test with no failure mode it would catch adds
|
|
48
|
-
maintenance cost without adding signal.
|
|
49
|
-
|
|
50
|
-
## Phase 2: Update and clean the docs
|
|
51
|
-
|
|
52
|
-
1. Find the last release tag: `git describe --tags --abbrev=0`.
|
|
53
|
-
2. Diff everything since that tag: `git log <last-tag>..HEAD --oneline` and
|
|
54
|
-
`git diff <last-tag>..HEAD -- lib/` to see every behavioral change, not
|
|
55
|
-
just the most recent commit.
|
|
56
|
-
3. For each change, check it's reflected in:
|
|
57
|
-
- `CHANGELOG.md` — every notable change since the last tag needs an
|
|
58
|
-
entry under `[Unreleased]` (or a new version heading), in Keep a
|
|
59
|
-
Changelog format matching this repo's existing entries (see past
|
|
60
|
-
entries for style — plain-English migration notes for anything
|
|
61
|
-
behavior-affecting).
|
|
62
|
-
- `README.md` / `docs/*.md` — any new config option, public method, or
|
|
63
|
-
behavior change needs the relevant section updated. Follow this
|
|
64
|
-
repo's docs philosophy from `CLAUDE.md`: the README stays a scannable
|
|
65
|
-
landing page (basic config/feedback snippets only); anything needing
|
|
66
|
-
more than one code block belongs in `docs/`.
|
|
67
|
-
4. **Clean, don't just append.** Look for docs that are now stale,
|
|
68
|
-
contradictory, or redundant given the accumulated changes since the
|
|
69
|
-
last tag — consolidate/rewrite rather than layering a new paragraph on
|
|
70
|
-
top of an outdated one. Remove docs for anything removed from the gem.
|
|
71
|
-
5. **Bump the version if it hasn't already been done.** Check whether
|
|
72
|
-
`lib/coolhand/version.rb` was already bumped for the changes
|
|
73
|
-
accumulated since the last tag (e.g. by an earlier commit on this
|
|
74
|
-
branch) — if so, leave it. If not, determine the SemVer bump this
|
|
75
|
-
repo's convention implies (patch = fix, minor = backward-compatible
|
|
76
|
-
addition or breaking change while pre-1.0), write it to
|
|
77
|
-
`lib/coolhand/version.rb`, turn the `[Unreleased]` CHANGELOG heading
|
|
78
|
-
into `## [X.Y.Z] - <today's date>`, and run `bundle install` so
|
|
79
|
-
`Gemfile.lock`'s `coolhand (X.Y.Z)` line matches. State the version and
|
|
80
|
-
bump rationale in the wrap-up summary so the user can override it if
|
|
81
|
-
they'd have picked differently — don't ask before writing it, since
|
|
82
|
-
this is a mechanical, reversible edit gated by Phase 1's green build.
|
|
83
|
-
|
|
84
|
-
## Phase 3: Red-team the whole package
|
|
85
|
-
|
|
86
|
-
Adversarially review the entire `lib/` tree (not just this release's
|
|
87
|
-
diff) for security issues. This gem intercepts outgoing LLM API traffic
|
|
88
|
-
and logs it to Coolhand, so hunt specifically for:
|
|
89
|
-
|
|
90
|
-
- **Credential/secret leakage**: does any interceptor, logger, or error
|
|
91
|
-
handler write an API key, bearer token, or provider auth header value
|
|
92
|
-
into a log line, exception message, or the payload sent to Coolhand?
|
|
93
|
-
Check every header-sanitization path actually strips what it claims to
|
|
94
|
-
(e.g. `WebhookValidator`, provider header redaction) rather than
|
|
95
|
-
sanitizing a differently-cased or differently-named header.
|
|
96
|
-
- **Webhook/signature validation**: can `WebhookValidator#valid?` (or
|
|
97
|
-
equivalent) be bypassed — timing-unsafe comparison instead of a
|
|
98
|
-
constant-time compare, an environment where an empty/missing signature
|
|
99
|
-
is treated as valid, or a fallback path meant for development that's
|
|
100
|
-
reachable in production.
|
|
101
|
-
- **SSRF / address matching**: the default and configurable intercept
|
|
102
|
-
address lists — can a crafted URL (redirect, unicode homograph,
|
|
103
|
-
userinfo trick, subdomain confusion) match or evade the intended
|
|
104
|
-
host-matching logic in a way that intercepts (or fails to intercept)
|
|
105
|
-
the wrong destination?
|
|
106
|
-
- **ReDoS**: any regex built from configurable or user-influenced input
|
|
107
|
-
(intercept patterns, header names) — check for catastrophic backtracking
|
|
108
|
-
shapes (nested quantifiers, overlapping alternation).
|
|
109
|
-
- **Thread safety**: this gem documents thread-safe operation and
|
|
110
|
-
duplicate-request prevention — look for unsynchronized shared mutable
|
|
111
|
-
state (class-level `@@` vars, memoized `@client` on a shared instance)
|
|
112
|
-
that a concurrent request could race on.
|
|
113
|
-
- **Unsafe deserialization**: any `JSON.parse` without checking for
|
|
114
|
-
`Marshal.load`/`YAML.load` (unsafe) usage, and any parsing of
|
|
115
|
-
webhook/batch-result payloads that trusts attacker-controlled shape
|
|
116
|
-
without validation.
|
|
117
|
-
- **Fail-open vs fail-closed**: when Coolhand's API is unreachable, rate
|
|
118
|
-
limited, or returns malformed data, does the gem fail open in a way that
|
|
119
|
-
silently drops security-relevant logging, or fail in a way that breaks
|
|
120
|
-
the host application's actual LLM call (the interceptor must never break
|
|
121
|
-
the underlying request)?
|
|
122
|
-
|
|
123
|
-
For each finding, report file, line, a concrete failure scenario, and
|
|
124
|
-
severity. Apply safe, mechanical, low-risk fixes directly (e.g. a missing
|
|
125
|
-
header-redaction pattern, a missing timeout). Flag but do not silently
|
|
126
|
-
apply anything that's a behavior/architecture decision (e.g. changing a
|
|
127
|
-
fail-open security default, adding replay protection, moving synchronous
|
|
128
|
-
work to a background thread) — surface these to the user for a decision,
|
|
129
|
-
the same "hand it to a human" rule `/loop-review` uses for stuck findings.
|
|
130
|
-
|
|
131
|
-
## Wrap-up
|
|
132
|
-
|
|
133
|
-
Report one consolidated summary: test/lint result, coverage-quality gaps
|
|
134
|
-
plus recommended specs, docs updated, the version (bumped or already
|
|
135
|
-
current, and why), and security findings split into fixed vs.
|
|
136
|
-
flagged-for-decision.
|
|
137
|
-
|
|
138
|
-
## Safety
|
|
139
|
-
|
|
140
|
-
- Bumping `lib/coolhand/version.rb`, finalizing the CHANGELOG heading, and
|
|
141
|
-
running `bundle install` for the lockfile are all in scope and don't need
|
|
142
|
-
a stop-and-ask — they're mechanical, reversible, and gated on Phase 1
|
|
143
|
-
already being green.
|
|
144
|
-
- Never create or push a git tag, never push commits, and never run
|
|
145
|
-
`rake release`, `gem push`, or anything else that publishes the gem or
|
|
146
|
-
touches the remote. Tagging and publishing are the user's action once
|
|
147
|
-
they've reviewed this skill's report, not something this skill does.
|
|
148
|
-
|
|
149
|
-
## Rationalizations to resist
|
|
150
|
-
|
|
151
|
-
- *"The diff since the last tag is small, I'll skip the red-team."* Small
|
|
152
|
-
diffs can still sit on top of latent issues in code nobody's touched
|
|
153
|
-
recently — that's exactly what "whole package, not just the diff" means.
|
|
154
|
-
- *"Tests pass, so coverage is fine."* Passing tests and meaningful
|
|
155
|
-
coverage are different questions. A red build blocks release; a green
|
|
156
|
-
build with hollow tests doesn't guarantee anything.
|
|
157
|
-
- *"Docs are close enough, I'll skip the cleanup pass."* Accumulated
|
|
158
|
-
changes since the last tag are exactly when docs drift from behavior —
|
|
159
|
-
this phase exists because per-PR doc updates miss the cross-cutting
|
|
160
|
-
view.
|
data/.idea/coolhand-ruby.iml
DELETED
data/CLAUDE.md
DELETED
|
@@ -1,47 +0,0 @@
|
|
|
1
|
-
# Development Guidelines
|
|
2
|
-
|
|
3
|
-
## Optional Provider Dependencies
|
|
4
|
-
|
|
5
|
-
Coolhand supports multiple LLM providers (OpenAI, Anthropic, Google Gemini, etc.). These provider gems should **never** be required at gem load time, as clients may not use all providers and shouldn't be forced to install unnecessary dependencies.
|
|
6
|
-
|
|
7
|
-
**Rule**: Any require for provider SDKs (openai, anthropic, google-generativeai, etc.) must be:
|
|
8
|
-
1. Placed in the file where it's actually used (not in the main coolhand.rb)
|
|
9
|
-
2. Only executed when that provider's functionality is accessed
|
|
10
|
-
3. Not declared as a hard dependency in coolhand-ruby.gemspec
|
|
11
|
-
|
|
12
|
-
Example pattern:
|
|
13
|
-
```ruby
|
|
14
|
-
# ❌ DON'T: In lib/coolhand.rb (loads unconditionally)
|
|
15
|
-
require "openai"
|
|
16
|
-
|
|
17
|
-
# ✅ DO: In lib/coolhand/open_ai/batch_result_processor.rb (only when needed)
|
|
18
|
-
require "openai"
|
|
19
|
-
|
|
20
|
-
module Coolhand
|
|
21
|
-
module OpenAi
|
|
22
|
-
class BatchResultProcessor
|
|
23
|
-
def client
|
|
24
|
-
@client ||= OpenAI::Client.new
|
|
25
|
-
end
|
|
26
|
-
end
|
|
27
|
-
end
|
|
28
|
-
end
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
This ensures:
|
|
32
|
-
- Gem loads cleanly regardless of what providers are installed
|
|
33
|
-
- Apps using path gems (local development) don't break from missing optional dependencies
|
|
34
|
-
- Users only need gems for providers they actually use
|
|
35
|
-
|
|
36
|
-
## README and docs philosophy
|
|
37
|
-
|
|
38
|
-
The README is a landing page — install, quick start, what it supports, where to go next. Keep it scannable. When in doubt, link rather than expand.
|
|
39
|
-
|
|
40
|
-
**Three rules:**
|
|
41
|
-
- **Config**: the basic `Coolhand.configure` snippet belongs in the README. Anything requiring more than one code block (self-hosted `base_url`, custom intercept addresses) goes in `docs/configuration.md`.
|
|
42
|
-
- **Feedback**: the basic `create_feedback` snippet belongs in the README. The full field table, matching strategies, and sentiment conversion details go in `docs/feedback.md`.
|
|
43
|
-
- **Integrations**: each integration gets its own `docs/<name>.md` file. The README links to them from the Documentation section.
|
|
44
|
-
|
|
45
|
-
**Align with coolhand-node.** When adding a section that exists in the Node README, match its structure and tone. The two READMEs should feel like siblings.
|
|
46
|
-
|
|
47
|
-
**Discoverability (SEO / AEO).** Write headings, the package description, and the supported-libraries list with search engines and AI agents in mind: use full provider/framework names (e.g. "OpenAI", "Anthropic", "Google Gemini", "Cohere") rather than abbreviations. The goal is that searches for "Ruby LLM monitoring", "Anthropic Ruby logging", or "OpenAI Ruby observability" surface this gem.
|