opensearch-sugar 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.agents/skills/diataxis/SKILL.md +142 -0
- data/.agents/skills/diataxis/references/examples.md +420 -0
- data/.agents/skills/diataxis/references/explanation-template.md +96 -0
- data/.agents/skills/diataxis/references/framework.md +400 -0
- data/.agents/skills/diataxis/references/how-to-guide-template.md +105 -0
- data/.agents/skills/diataxis/references/reference-template.md +110 -0
- data/.agents/skills/diataxis/references/tutorial-template.md +101 -0
- data/.agents/skills/diataxis/scripts/generate_index.py +139 -0
- data/.rspec +3 -0
- data/.standard.yml +3 -0
- data/AGENTS.md +120 -0
- data/CHANGELOG.md +5 -0
- data/Dockerfile.opensearch +4 -0
- data/Increase_Coverage.md +311 -0
- data/README.md +143 -0
- data/Rakefile +27 -0
- data/Steepfile +23 -0
- data/adrs/ADR-000-template.md +87 -0
- data/adrs/ADR-001-simpledelegator-for-client.md +138 -0
- data/adrs/ADR-002-facade-pattern-for-index.md +126 -0
- data/adrs/ADR-003-repository-pattern-for-models.md +148 -0
- data/adrs/ADR-004-integration-tests-no-mocking.md +91 -0
- data/adrs/ADR-005-exceptions-over-result-objects.md +107 -0
- data/adrs/ADR-006-ssl-on-by-default.md +95 -0
- data/adrs/ADR-007-selective-sugar-surface.md +118 -0
- data/adrs/ADR-008-integration-test-design.md +178 -0
- data/compose.yml +2 -0
- data/compose_opensearch.yml +31 -0
- data/docs/HOWTO.md +844 -0
- data/docs/REFERENCE.md +725 -0
- data/docs/TUTORIAL.md +327 -0
- data/docs/alias-api-design-notes.md +119 -0
- data/lib/opensearch/sugar/client.rb +300 -0
- data/lib/opensearch/sugar/index/include/utilities.rb +6 -0
- data/lib/opensearch/sugar/index.rb +339 -0
- data/lib/opensearch/sugar/models.rb +209 -0
- data/lib/opensearch/sugar/version.rb +8 -0
- data/lib/opensearch/sugar.rb +61 -0
- data/old_docs/DELEGATED_METHODS_ANALYSIS.md +361 -0
- data/old_docs/EXPLANATION.md +685 -0
- data/old_docs/README.md +155 -0
- data/old_docs/docs/CLI-PROPOSAL.md +257 -0
- data/old_docs/docs/HOWTO.md +798 -0
- data/old_docs/docs/REFERENCE.md +901 -0
- data/old_docs/docs/TUTORIAL.md +493 -0
- data/sig/opensearch/sugar.rbs +162 -0
- metadata +240 -0
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# ADR-004: Integration Tests Against a Real OpenSearch Node; No HTTP Mocking
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted
|
|
6
|
+
|
|
7
|
+
## Date
|
|
8
|
+
|
|
9
|
+
2026-04-28
|
|
10
|
+
|
|
11
|
+
## Context
|
|
12
|
+
|
|
13
|
+
`opensearch-sugar` wraps the OpenSearch HTTP API. Its correctness depends on real
|
|
14
|
+
request/response semantics: field ordering in bodies, HTTP status codes, error shapes,
|
|
15
|
+
and side effects (index creation, document persistence, model deployment state). The question
|
|
16
|
+
is whether tests should run against a live cluster or use some form of HTTP-level mocking.
|
|
17
|
+
|
|
18
|
+
The main options evaluated were:
|
|
19
|
+
|
|
20
|
+
- **VCR (cassette recording)**: record real HTTP interactions once, replay from cassettes in CI
|
|
21
|
+
- **WebMock / stub_request**: hand-write HTTP stubs for each tested scenario
|
|
22
|
+
- **Live integration tests with a Dockerized OpenSearch node**: run all tests against a real
|
|
23
|
+
cluster spun up via `docker compose`
|
|
24
|
+
|
|
25
|
+
## Decision
|
|
26
|
+
|
|
27
|
+
All tests are integration tests that run against a live OpenSearch node. There is no mocking
|
|
28
|
+
layer. A `docker compose up -d` brings up the cluster before running `bundle exec rspec`.
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
# Start the cluster
|
|
32
|
+
docker compose up -d
|
|
33
|
+
|
|
34
|
+
# Run the full suite
|
|
35
|
+
bundle exec rspec
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Tests are responsible for cleaning up their own fixtures (indexes, models, pipelines) and must
|
|
39
|
+
not leave traces in the cluster after they complete.
|
|
40
|
+
|
|
41
|
+
VCR was explicitly evaluated and rejected.
|
|
42
|
+
|
|
43
|
+
## Consequences
|
|
44
|
+
|
|
45
|
+
### Positive
|
|
46
|
+
|
|
47
|
+
- **High fidelity**: tests exercise the actual HTTP client, auth, TLS, serialization, and
|
|
48
|
+
OpenSearch response parsing — the exact code paths that matter in production.
|
|
49
|
+
- **No cassette maintenance**: recorded cassettes drift as the OpenSearch API evolves; with live
|
|
50
|
+
tests there is nothing to regenerate or keep in sync.
|
|
51
|
+
- **Catches real regressions**: version bumps of `opensearch-ruby` or OpenSearch itself are
|
|
52
|
+
immediately visible in the test results.
|
|
53
|
+
- **Simple setup**: `docker compose up -d && bundle exec rspec` is the entire test workflow;
|
|
54
|
+
no cassette directory, no VCR configuration, no per-test recording modes.
|
|
55
|
+
|
|
56
|
+
### Negative
|
|
57
|
+
|
|
58
|
+
- **Requires Docker**: contributors without Docker cannot run the suite locally without
|
|
59
|
+
additional setup.
|
|
60
|
+
- **Slower than mocked tests**: the suite waits for network I/O and OpenSearch processing;
|
|
61
|
+
ML model deployment tests are particularly slow due to polling.
|
|
62
|
+
- **Flakiness risk**: tests are sensitive to cluster state. Poorly isolated tests can interfere
|
|
63
|
+
with each other if cleanup is missed.
|
|
64
|
+
- **No offline CI without a real cluster**: CI pipelines must be able to start a Docker service
|
|
65
|
+
container; environments that cannot do so cannot run the suite.
|
|
66
|
+
|
|
67
|
+
### Neutral
|
|
68
|
+
|
|
69
|
+
- Unit tests for pure Ruby logic (no I/O) are welcome alongside integration tests. If logic
|
|
70
|
+
that can be tested in isolation is added to `lib/`, unit specs should be added as well.
|
|
71
|
+
- The `spec/env.testing` file (loaded via `dotenv`) supplies cluster credentials; this file
|
|
72
|
+
must not be committed with production credentials.
|
|
73
|
+
|
|
74
|
+
## Alternatives Considered
|
|
75
|
+
|
|
76
|
+
**VCR (cassette recording)**
|
|
77
|
+
Evaluated and rejected. VCR adds complexity (cassette storage, recording modes, sensitive-data
|
|
78
|
+
filtering) without sufficient benefit for this gem. The core value of these tests is verifying
|
|
79
|
+
real interactions with OpenSearch; replaying stale cassettes undermines that value and creates
|
|
80
|
+
a maintenance burden.
|
|
81
|
+
|
|
82
|
+
**WebMock / hand-written stubs**
|
|
83
|
+
Rejected for the same reasons: stubs are approximations of the real API and would require
|
|
84
|
+
manual updates whenever the response shape changes. For a thin wrapper gem, testing against
|
|
85
|
+
the real thing is more reliable than maintaining an approximate fake.
|
|
86
|
+
|
|
87
|
+
## Documentation Requirements
|
|
88
|
+
|
|
89
|
+
- The README and CONTRIBUTING guide must document the Docker prerequisite and the
|
|
90
|
+
`docker compose up -d` + `bundle exec rspec` workflow.
|
|
91
|
+
- The `spec/env.testing` file must document all required environment variables.
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
# ADR-005: Exceptions for Error Handling; No Result/Either Objects
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted
|
|
6
|
+
|
|
7
|
+
## Date
|
|
8
|
+
|
|
9
|
+
2026-04-28
|
|
10
|
+
|
|
11
|
+
## Context
|
|
12
|
+
|
|
13
|
+
When a Sugar method fails — invalid arguments, OpenSearch returns an error, a multi-step
|
|
14
|
+
sequence partially succeeds — the gem must communicate that failure to the caller. Two broad
|
|
15
|
+
approaches were considered:
|
|
16
|
+
|
|
17
|
+
- **Exceptions**: raise a Ruby exception; callers use `begin/rescue` to handle errors
|
|
18
|
+
- **Result/Either objects**: return a value that is either a success wrapper or a failure
|
|
19
|
+
wrapper, forcing callers to inspect the return value (e.g., `result.success?`,
|
|
20
|
+
`result.value`, `result.error`)
|
|
21
|
+
|
|
22
|
+
The Result pattern (sometimes called Dry::Monads `Result`, `Either`, or a custom `Ok`/`Err`
|
|
23
|
+
type) is popular in functional-leaning Ruby code. Before settling on exceptions, a full
|
|
24
|
+
comparison of what the codebase would look like under each approach was produced and reviewed.
|
|
25
|
+
|
|
26
|
+
## Decision
|
|
27
|
+
|
|
28
|
+
We use Ruby exceptions for all error handling. Sugar raises `OpenSearch::Sugar::Error` (or a
|
|
29
|
+
more specific subclass) for errors that originate within the gem; errors from the underlying
|
|
30
|
+
`opensearch-ruby` transport layer are allowed to propagate as-is (i.e., as
|
|
31
|
+
`OpenSearch::Transport::Transport::Error` subclasses).
|
|
32
|
+
|
|
33
|
+
```ruby
|
|
34
|
+
# Sugar raises on failure
|
|
35
|
+
begin
|
|
36
|
+
index = client.open_or_create_index("my_index")
|
|
37
|
+
index.update_settings(settings)
|
|
38
|
+
rescue OpenSearch::Sugar::Error => e
|
|
39
|
+
# Sugar-level error (e.g., invalid arguments, sequence failure)
|
|
40
|
+
logger.error("Sugar error: #{e.message}")
|
|
41
|
+
rescue OpenSearch::Transport::Transport::Error => e
|
|
42
|
+
# Raw transport error (e.g., connection refused, 4xx/5xx from OpenSearch)
|
|
43
|
+
logger.error("OpenSearch error: #{e.message}")
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# Methods that succeed simply return the result; no wrapper needed
|
|
47
|
+
count = index.count # Integer
|
|
48
|
+
settings = index.settings # Hash
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
The Result pattern was considered and summarized in full detail, but not adopted.
|
|
52
|
+
|
|
53
|
+
## Consequences
|
|
54
|
+
|
|
55
|
+
### Positive
|
|
56
|
+
|
|
57
|
+
- **Idiomatic Ruby**: exceptions are the standard Ruby error-handling mechanism; contributors
|
|
58
|
+
and users familiar with Ruby (or any Ruby gem) will immediately understand the contract.
|
|
59
|
+
- **Clean happy path**: methods return their values directly with no wrapping. Callers that
|
|
60
|
+
don't care about errors don't need to unwrap anything.
|
|
61
|
+
- **Chainable interfaces**: because successful methods return their natural value (an `Index`,
|
|
62
|
+
an `Integer`, a `Hash`) rather than a `Result` wrapper, return values can be used directly
|
|
63
|
+
in chains and compositions without a prior unwrapping step. For example,
|
|
64
|
+
`client.open_or_create_index("my_index").update_settings(settings)` works naturally. With
|
|
65
|
+
`Result` objects, each step would require unwrapping before the next call.
|
|
66
|
+
- **No new dependencies**: Result types would require either `dry-monads` or a custom
|
|
67
|
+
implementation. Exceptions are built into the language.
|
|
68
|
+
- **Interoperability**: code that mixes Sugar calls with other Ruby libraries doesn't need
|
|
69
|
+
an adapter layer to convert `Result` values to exceptions or vice versa.
|
|
70
|
+
|
|
71
|
+
### Negative
|
|
72
|
+
|
|
73
|
+
- **Errors are invisible in signatures**: a method's return type doesn't communicate what it
|
|
74
|
+
can raise. Callers must read documentation or source to know which exceptions to handle.
|
|
75
|
+
- **Easy to ignore**: unlike a `Result` type, exceptions can be silently swallowed by an
|
|
76
|
+
overly broad `rescue Exception` or `rescue StandardError` in the caller's code.
|
|
77
|
+
- **No enforced handling**: a compiler/type checker cannot require the caller to handle the
|
|
78
|
+
error case (unlike `Result` in Rust or Haskell). Ruby's type system provides no enforcement.
|
|
79
|
+
|
|
80
|
+
### Neutral
|
|
81
|
+
|
|
82
|
+
- `OpenSearch::Sugar::Error < StandardError` is the base error class for all Sugar-originated
|
|
83
|
+
exceptions. More specific subclasses should be introduced as distinct error categories emerge.
|
|
84
|
+
- Errors from the raw `opensearch-ruby` client propagate unchanged. Sugar does not wrap or
|
|
85
|
+
re-raise transport exceptions except in cases where it needs to add context.
|
|
86
|
+
- The Result pattern summary produced during evaluation is preserved in `vibe/` as a reference
|
|
87
|
+
for contributors who want to understand the tradeoff in detail.
|
|
88
|
+
|
|
89
|
+
## Alternatives Considered
|
|
90
|
+
|
|
91
|
+
**Result/Either objects (e.g., `dry-monads`)**
|
|
92
|
+
Evaluated in full. Rejected because:
|
|
93
|
+
- It adds a `dry-monads` dependency (or requires a custom implementation) to a gem that
|
|
94
|
+
otherwise has minimal dependencies.
|
|
95
|
+
- It forces callers to adopt a monadic style that is not idiomatic in most Ruby codebases,
|
|
96
|
+
especially non-Rails ones.
|
|
97
|
+
- The happy-path ergonomics worsen: every return value must be unwrapped before use.
|
|
98
|
+
- `opensearch-ruby` itself raises exceptions, so callers would still need `rescue` for
|
|
99
|
+
transport errors; mixing `Result` and exceptions produces an inconsistent interface.
|
|
100
|
+
|
|
101
|
+
## Documentation Requirements
|
|
102
|
+
|
|
103
|
+
- REFERENCE must document `OpenSearch::Sugar::Error` and any subclasses with their meanings.
|
|
104
|
+
- HOWTO must include an error-handling section showing the recommended `rescue` pattern for
|
|
105
|
+
both Sugar errors and raw transport errors.
|
|
106
|
+
- EXPLANATION should note that Result objects were considered and explain the rationale for
|
|
107
|
+
choosing exceptions.
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# ADR-006: SSL Enabled by Default; Explicit Opt-Out for Development
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted
|
|
6
|
+
|
|
7
|
+
## Date
|
|
8
|
+
|
|
9
|
+
2026-04-28
|
|
10
|
+
|
|
11
|
+
## Context
|
|
12
|
+
|
|
13
|
+
`OpenSearch::Sugar::Client` wraps `OpenSearch::Client` and must decide what default transport
|
|
14
|
+
security settings to apply when the caller does not specify them. OpenSearch clusters in
|
|
15
|
+
production are almost universally deployed with TLS. The risk of accidentally connecting
|
|
16
|
+
without TLS — exposing credentials or data in transit — is meaningful and asymmetric: the
|
|
17
|
+
cost of a misconfigured production cluster far outweighs the minor inconvenience of explicitly
|
|
18
|
+
disabling SSL for local development.
|
|
19
|
+
|
|
20
|
+
The two realistic positions are:
|
|
21
|
+
|
|
22
|
+
1. **SSL off by default, opt-in for production** — easy local setup, but a forgotten
|
|
23
|
+
production configuration is a silent security failure.
|
|
24
|
+
2. **SSL on by default, explicit opt-out for development** — requires one extra line in
|
|
25
|
+
development, but the default is always secure.
|
|
26
|
+
|
|
27
|
+
## Decision
|
|
28
|
+
|
|
29
|
+
SSL is enabled by default. Callers who need to connect to a development cluster without
|
|
30
|
+
certificate verification must explicitly pass `ssl: { verify: false }` in transport options.
|
|
31
|
+
|
|
32
|
+
```ruby
|
|
33
|
+
# Production — SSL on, certificate verified (default; no extra config needed)
|
|
34
|
+
client = OpenSearch::Sugar::Client.new(
|
|
35
|
+
host: "https://search.production.example.com:9200",
|
|
36
|
+
user: ENV["OPENSEARCH_USER"],
|
|
37
|
+
password: ENV["OPENSEARCH_PASSWORD"]
|
|
38
|
+
)
|
|
39
|
+
|
|
40
|
+
# Development — SSL on but verification disabled
|
|
41
|
+
client = OpenSearch::Sugar::Client.new(
|
|
42
|
+
host: "https://localhost:9200",
|
|
43
|
+
user: "admin",
|
|
44
|
+
password: ENV["OPENSEARCH_PASSWORD"],
|
|
45
|
+
transport_options: {
|
|
46
|
+
ssl: { verify: false }
|
|
47
|
+
}
|
|
48
|
+
)
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Documentation leads with the production-safe form and explicitly explains how to disable
|
|
52
|
+
verification for development. The Docker Compose test environment uses `ssl: { verify: false }`
|
|
53
|
+
in `spec/env.testing`.
|
|
54
|
+
|
|
55
|
+
## Consequences
|
|
56
|
+
|
|
57
|
+
### Positive
|
|
58
|
+
|
|
59
|
+
- **Secure by default**: a caller who copies the simplest example from the README will get
|
|
60
|
+
a TLS-secured connection. There is no "forgot to enable SSL in prod" failure mode.
|
|
61
|
+
- **Explicit intent in dev code**: `ssl: { verify: false }` in development configuration
|
|
62
|
+
is a visible, searchable signal that certificate verification is intentionally disabled.
|
|
63
|
+
Code review can flag it if it appears in production configuration.
|
|
64
|
+
|
|
65
|
+
### Negative
|
|
66
|
+
|
|
67
|
+
- **Friction for local development**: first-time setup requires understanding why the default
|
|
68
|
+
connection attempt to a local Docker cluster fails and how to disable verification. Without
|
|
69
|
+
clear documentation this is confusing.
|
|
70
|
+
- **No plaintext (`http://`) shortcut**: callers who want plain HTTP (no TLS at all) must
|
|
71
|
+
also configure transport options explicitly. The documentation must cover this case.
|
|
72
|
+
|
|
73
|
+
### Neutral
|
|
74
|
+
|
|
75
|
+
- The underlying `OpenSearch::Client` and the `Faraday` HTTP adapter handle the actual TLS
|
|
76
|
+
negotiation; Sugar does not implement TLS logic itself.
|
|
77
|
+
- Credentials must always be provided via environment variables (e.g., loaded from
|
|
78
|
+
`spec/env.testing` with `dotenv`); hardcoding passwords in source is never acceptable
|
|
79
|
+
regardless of environment.
|
|
80
|
+
|
|
81
|
+
## Alternatives Considered
|
|
82
|
+
|
|
83
|
+
**SSL off by default**
|
|
84
|
+
Rejected. The security asymmetry is clear: forgetting to enable SSL in production is a serious
|
|
85
|
+
vulnerability with no automatic safeguard. Forgetting to disable SSL verification in development
|
|
86
|
+
is a minor inconvenience that produces a loud, obvious error.
|
|
87
|
+
|
|
88
|
+
## Documentation Requirements
|
|
89
|
+
|
|
90
|
+
- README must include both the production (SSL on) and development (verify: false) connection
|
|
91
|
+
examples, with a prominent note that `verify: false` is for development only.
|
|
92
|
+
- HOWTO "Connection and Configuration" section must appear at the top of the guide and cover
|
|
93
|
+
both cases.
|
|
94
|
+
- EXPLANATION must explain why SSL is on by default and what `verify: false` does (and does
|
|
95
|
+
not) protect against.
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# ADR-007: Selective Sugar Surface — Intentionally Incomplete API Coverage
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted
|
|
6
|
+
|
|
7
|
+
## Date
|
|
8
|
+
|
|
9
|
+
2026-04-28
|
|
10
|
+
|
|
11
|
+
## Context
|
|
12
|
+
|
|
13
|
+
`opensearch-sugar` is a convenience wrapper, not a complete replacement for `opensearch-ruby`.
|
|
14
|
+
The OpenSearch HTTP API is large and continues to grow: document CRUD, search, aggregations,
|
|
15
|
+
bulk operations, cluster management, snapshots, ingest pipelines, ML Commons, and more.
|
|
16
|
+
|
|
17
|
+
Two approaches to API coverage were considered:
|
|
18
|
+
|
|
19
|
+
1. **Comprehensive wrapping**: provide Sugar methods for every (or nearly every) OpenSearch
|
|
20
|
+
operation, so callers never need to call the raw client.
|
|
21
|
+
2. **Selective wrapping**: provide Sugar methods only for operations that genuinely benefit
|
|
22
|
+
from a higher-level interface (multi-step sequences, complex defaults, common boilerplate).
|
|
23
|
+
For everything else, rely on `SimpleDelegator` to pass calls through to the raw
|
|
24
|
+
`OpenSearch::Client` transparently.
|
|
25
|
+
|
|
26
|
+
## Decision
|
|
27
|
+
|
|
28
|
+
Sugar wraps only operations where a higher-level interface provides meaningful value:
|
|
29
|
+
|
|
30
|
+
- **Multi-step sequences** that would otherwise be error-prone (e.g., close/update/reopen
|
|
31
|
+
for settings changes — see ADR-002)
|
|
32
|
+
- **Lifecycle management** that requires orchestration (e.g., model registration and
|
|
33
|
+
deployment — see ADR-003)
|
|
34
|
+
- **Common index management boilerplate** (`open_or_create_index`, `has_index?`, `index_names`)
|
|
35
|
+
- **Text analysis helpers** (`test_analyzer_by_name`, `test_analyzer_by_fieldname`, `test_analyzer_by_definition`)
|
|
36
|
+
|
|
37
|
+
Document CRUD, search, bulk operations, aggregations, and cluster management are intentionally
|
|
38
|
+
left as raw-client operations:
|
|
39
|
+
|
|
40
|
+
```ruby
|
|
41
|
+
# Sugar — meaningful abstraction
|
|
42
|
+
index = client.open_or_create_index("products")
|
|
43
|
+
index.update_settings(settings) # hides close/reopen
|
|
44
|
+
index.count # hides query body construction
|
|
45
|
+
model = client.models.register(...) # hides polling loop
|
|
46
|
+
|
|
47
|
+
# Delegation — use the raw client directly
|
|
48
|
+
client.index(index: "products", body: doc) # no benefit to wrapping
|
|
49
|
+
client.search(index: "products", body: query) # DSL is already expressive
|
|
50
|
+
client.bulk(body: operations) # no benefit to wrapping
|
|
51
|
+
client.cluster.health # no benefit to wrapping
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
This is an explicit design choice, not an omission. The guiding principle:
|
|
55
|
+
> **"Use sugar where you want it, raw client where you need it."**
|
|
56
|
+
|
|
57
|
+
## Consequences
|
|
58
|
+
|
|
59
|
+
### Positive
|
|
60
|
+
|
|
61
|
+
- **Maintenance stays manageable**: every Sugar method is a liability that must be kept in
|
|
62
|
+
sync with the OpenSearch API. Wrapping only what adds value keeps the surface small.
|
|
63
|
+
- **Full API always accessible**: because `Sugar::Client` uses `SimpleDelegator` (ADR-001),
|
|
64
|
+
callers are never blocked by a missing Sugar method; the full raw API is always one call
|
|
65
|
+
away.
|
|
66
|
+
- **Avoids redundant abstraction**: wrapping `client.search` into a Sugar method would not
|
|
67
|
+
simplify anything — the OpenSearch query DSL is already a well-documented, expressive
|
|
68
|
+
interface.
|
|
69
|
+
- **Clearer mental model**: callers learn which operations benefit from Sugar and which don't,
|
|
70
|
+
rather than hunting for Sugar wrappers that don't exist or produce no benefit.
|
|
71
|
+
|
|
72
|
+
### Negative
|
|
73
|
+
|
|
74
|
+
- **Callers must know the raw client API**: for operations without Sugar equivalents, callers
|
|
75
|
+
need to read `opensearch-ruby` documentation. Sugar does not hide this complexity.
|
|
76
|
+
- **Inconsistent abstraction level**: some operations are "high-level Sugar" and others are
|
|
77
|
+
"raw client" in the same code. This boundary can feel arbitrary to new contributors.
|
|
78
|
+
- **No guided discovery for raw operations**: there is no machine-readable declaration of
|
|
79
|
+
which operations are delegated vs. implemented; callers discover this through docs or
|
|
80
|
+
source inspection.
|
|
81
|
+
|
|
82
|
+
### Neutral
|
|
83
|
+
|
|
84
|
+
- The `old_docs/DELEGATED_METHODS_ANALYSIS.md` file documents 19 specific examples of raw
|
|
85
|
+
client calls used in the HOWTO guide and categorizes which have Sugar alternatives.
|
|
86
|
+
- New Sugar methods should be added only when they hide genuine complexity. A "thin wrapper
|
|
87
|
+
that just passes arguments through" should not be added.
|
|
88
|
+
- The boundary will shift over time as new use cases emerge. Revisit this ADR before adding
|
|
89
|
+
Sugar methods for document CRUD or basic search.
|
|
90
|
+
|
|
91
|
+
## Alternatives Considered
|
|
92
|
+
|
|
93
|
+
**Comprehensive wrapping**
|
|
94
|
+
Rejected. A comprehensive wrapper would need to track every OpenSearch API change, producing
|
|
95
|
+
a large maintenance surface and introducing lag between upstream API additions and Sugar
|
|
96
|
+
availability. It would also need to either replicate the entire query DSL (impractical) or
|
|
97
|
+
provide a lossy subset of it (worse than the raw API).
|
|
98
|
+
|
|
99
|
+
## Diagram
|
|
100
|
+
|
|
101
|
+
```mermaid
|
|
102
|
+
flowchart LR
|
|
103
|
+
Caller --> SugarClient["Sugar::Client"]
|
|
104
|
+
|
|
105
|
+
SugarClient -->|"Sugar method\n(adds value)"| SugarImpl["Sugar implementation\n(open_or_create, update_settings,\nmodels.register, ...)"]
|
|
106
|
+
SugarClient -->|"Delegated\n(no wrapping needed)"| RawClient["OpenSearch::Client\n(search, index, bulk,\ncluster.health, ...)"]
|
|
107
|
+
|
|
108
|
+
SugarImpl --> RawClient
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## Documentation Requirements
|
|
112
|
+
|
|
113
|
+
- HOWTO must demonstrate both Sugar methods and delegated raw-client calls side by side,
|
|
114
|
+
making clear which is which.
|
|
115
|
+
- EXPLANATION must articulate the "use sugar where you want it, raw client where you need it"
|
|
116
|
+
philosophy and explain why document CRUD and search are intentionally unwrapped.
|
|
117
|
+
- The `DELEGATED_METHODS_ANALYSIS.md` analysis in `old_docs/` should be kept or migrated to
|
|
118
|
+
an appropriate location as a reference for which operations are and are not wrapped.
|
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
# ADR-008: Integration Test Design
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted
|
|
6
|
+
|
|
7
|
+
## Date
|
|
8
|
+
|
|
9
|
+
2026-04-28
|
|
10
|
+
|
|
11
|
+
## Context
|
|
12
|
+
|
|
13
|
+
Having established in ADR-004 that all tests run against a live OpenSearch node with no HTTP
|
|
14
|
+
mocking, this ADR records the specific structural and scoping decisions made when designing
|
|
15
|
+
the integration test suite for `opensearch-sugar`.
|
|
16
|
+
|
|
17
|
+
The key questions were:
|
|
18
|
+
- How to organize spec files (per class vs. per feature)
|
|
19
|
+
- How to isolate tests from each other
|
|
20
|
+
- How to handle slow ML model tests
|
|
21
|
+
- How to share the client setup across specs
|
|
22
|
+
- What Sugar-owned behavior is in scope vs. what is delegated and therefore out of scope
|
|
23
|
+
- What to do about unimplemented stub methods (`index_document`, `index_jsonl_file`)
|
|
24
|
+
|
|
25
|
+
## Decision
|
|
26
|
+
|
|
27
|
+
### File organization: per feature, not per class
|
|
28
|
+
|
|
29
|
+
Spec files are organized by feature cluster rather than by class. The `Client` and `Index`
|
|
30
|
+
classes are large enough that a single file per class would become unwieldy. Feature-based
|
|
31
|
+
files are independently runnable via `--pattern` and keep each file focused on one concern
|
|
32
|
+
with its own setup.
|
|
33
|
+
|
|
34
|
+
```
|
|
35
|
+
spec/
|
|
36
|
+
support/
|
|
37
|
+
opensearch_client.rb # shared client setup helper
|
|
38
|
+
opensearch/sugar/
|
|
39
|
+
client/
|
|
40
|
+
connection_spec.rb # ping, raw_client type
|
|
41
|
+
index_management_spec.rb # has_index?, index_names, open_or_create, []
|
|
42
|
+
settings_spec.rb # update_settings (cluster-level settings)
|
|
43
|
+
index/
|
|
44
|
+
lifecycle_spec.rb # Index.open, Index.create, delete!
|
|
45
|
+
document_spec.rb # count, clear!, delete_by_id, index_document, index_jsonl_file
|
|
46
|
+
settings_spec.rb # settings, update_settings (index-level)
|
|
47
|
+
mappings_spec.rb # mappings, update_mappings
|
|
48
|
+
aliases_spec.rb # aliases, create_alias
|
|
49
|
+
analyzer_spec.rb # all_available_analyzers, test_analyzer_by_name, test_analyzer_by_fieldname, test_analyzer_by_definition
|
|
50
|
+
models/ # tagged :slow, :models — excluded from default run
|
|
51
|
+
registration_spec.rb
|
|
52
|
+
lookup_spec.rb
|
|
53
|
+
lifecycle_spec.rb
|
|
54
|
+
pipeline_spec.rb
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
### Test isolation
|
|
58
|
+
|
|
59
|
+
Each spec that creates an index uses a uniquely named index per example:
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
let(:index_name) { "sugar_test_#{SecureRandom.hex(6)}" }
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Cleanup is registered with an `after` hook:
|
|
66
|
+
|
|
67
|
+
```ruby
|
|
68
|
+
after { client.indices.delete(index: index_name) rescue nil }
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
The `rescue nil` is intentional: if a test fails before the index is created, cleanup must
|
|
72
|
+
not itself raise. No global sweep is used.
|
|
73
|
+
|
|
74
|
+
### No raw OpenSearch::Client in specs
|
|
75
|
+
|
|
76
|
+
Specs interact exclusively through `OpenSearch::Sugar::Client`. No test may call
|
|
77
|
+
`OpenSearch::Client` directly. This keeps specs honest about what Sugar provides vs. what
|
|
78
|
+
would require bypassing the abstraction.
|
|
79
|
+
|
|
80
|
+
### Shared client setup
|
|
81
|
+
|
|
82
|
+
A `spec/support/opensearch_client.rb` helper loads `spec/env.testing` via `dotenv` and
|
|
83
|
+
defines an RSpec shared context that exposes `let(:client)`. Every spec file includes this
|
|
84
|
+
shared context. The client is configured for the local Docker cluster with SSL verification
|
|
85
|
+
disabled.
|
|
86
|
+
|
|
87
|
+
### ML model tests: tagged and excluded by default
|
|
88
|
+
|
|
89
|
+
All specs in `spec/opensearch/sugar/models/` are tagged `:slow` and `:models`. They are
|
|
90
|
+
excluded from the default `bundle exec rspec` run via RSpec filter configuration. To run
|
|
91
|
+
them:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
bundle exec rspec --tag models
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
### Scope: Sugar-owned behavior only
|
|
98
|
+
|
|
99
|
+
Tests cover only behavior implemented by Sugar. Delegated methods (e.g., `client.search`,
|
|
100
|
+
`client.index`, `client.bulk`) are not tested — `opensearch-ruby` is responsible for those.
|
|
101
|
+
|
|
102
|
+
The guiding question for inclusion: *"Does Sugar add logic here, or is it a straight
|
|
103
|
+
pass-through to the underlying client?"*
|
|
104
|
+
|
|
105
|
+
### `update_settings` and `update_mappings` raise on failure
|
|
106
|
+
|
|
107
|
+
The existing implementation returns an error Hash on failure, which is inconsistent with
|
|
108
|
+
ADR-005. As part of this work, both methods are changed to raise `OpenSearch::Sugar::Error`
|
|
109
|
+
on failure (after attempting to reopen the index). This is a breaking change from the
|
|
110
|
+
previous behavior.
|
|
111
|
+
|
|
112
|
+
### `index_document` and `index_jsonl_file`
|
|
113
|
+
|
|
114
|
+
Both stub methods are implemented as simple, intentionally inefficient implementations
|
|
115
|
+
suitable for small-scale and testing use only. A TODO is left in the code for a future
|
|
116
|
+
bulk-API implementation.
|
|
117
|
+
|
|
118
|
+
```ruby
|
|
119
|
+
# index_document: requires both arguments
|
|
120
|
+
def index_document(doc, id)
|
|
121
|
+
# TODO: inefficient; replace with bulk API for large-scale use
|
|
122
|
+
client.index(index: name, id: id, body: doc)
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
# index_jsonl_file: accepts a String path or any IO-like object (StringIO, File)
|
|
126
|
+
def index_jsonl_file(source, id_field:)
|
|
127
|
+
# TODO: inefficient; replace with bulk API for large-scale use
|
|
128
|
+
io = source.is_a?(String) ? File.open(source) : source
|
|
129
|
+
io.each_line do |line|
|
|
130
|
+
doc = JSON.parse(line, symbolize_names: true)
|
|
131
|
+
id = doc.fetch(id_field.to_sym) {
|
|
132
|
+
raise ArgumentError, "id_field :#{id_field} not found in document"
|
|
133
|
+
}
|
|
134
|
+
index_document(doc, id.to_s)
|
|
135
|
+
end
|
|
136
|
+
end
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Specs for these methods use `StringIO` to avoid filesystem dependencies.
|
|
140
|
+
|
|
141
|
+
## Consequences
|
|
142
|
+
|
|
143
|
+
### Positive
|
|
144
|
+
|
|
145
|
+
- Feature-based files are independently runnable and focused
|
|
146
|
+
- Unique index names eliminate inter-test interference without a global sweep
|
|
147
|
+
- ML tests are opt-in; default suite runs fast
|
|
148
|
+
- Restricting specs to Sugar's API surface keeps the test suite honest
|
|
149
|
+
- Implementing the stubs closes a gap in the public API and gives tests something real to run against
|
|
150
|
+
|
|
151
|
+
### Negative
|
|
152
|
+
|
|
153
|
+
- `update_settings`/`update_mappings` behavior change (raise vs. return Hash) is breaking for
|
|
154
|
+
any existing callers relying on checking `result[:status]`
|
|
155
|
+
- `index_document` and `index_jsonl_file` are explicitly not production-grade; callers doing
|
|
156
|
+
bulk loads must use the raw `client.bulk` API until a proper bulk implementation is added
|
|
157
|
+
|
|
158
|
+
### Neutral
|
|
159
|
+
|
|
160
|
+
- `spec/env.testing` must not be committed with real credentials; `.gitignore` covers it
|
|
161
|
+
- The `spec/opensearch/sugar_spec.rb` skeleton (with wrong constant casing and a hardcoded
|
|
162
|
+
failure) is deleted and replaced by the new structure
|
|
163
|
+
|
|
164
|
+
## Alternatives Considered
|
|
165
|
+
|
|
166
|
+
**Per-class spec files**
|
|
167
|
+
Considered and rejected; the `Client` and `Index` classes are large enough that single-file
|
|
168
|
+
specs would become hard to navigate and run selectively.
|
|
169
|
+
|
|
170
|
+
**Global `after(:suite)` sweep to delete all `sugar_test_*` indexes**
|
|
171
|
+
Considered as a safety net. Rejected as primary strategy in favor of per-example cleanup,
|
|
172
|
+
which makes cleanup intent explicit. A sweep could be added as a belt-and-suspenders measure
|
|
173
|
+
in the future.
|
|
174
|
+
|
|
175
|
+
## Documentation Requirements
|
|
176
|
+
|
|
177
|
+
- `CONTRIBUTING` guide must document the `:models` tag and how to run ML tests.
|
|
178
|
+
- `spec/env.testing` must document all required environment variables inline.
|
data/compose.yml
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
services:
|
|
2
|
+
dor-opensearch:
|
|
3
|
+
image: opensearchproject/opensearch:latest
|
|
4
|
+
container_name: opensearch-opensearch-sugar
|
|
5
|
+
environment:
|
|
6
|
+
- cluster.name=opensearch
|
|
7
|
+
- node.name=opensearch-node
|
|
8
|
+
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
|
|
9
|
+
- discovery.type=single-node
|
|
10
|
+
env_file:
|
|
11
|
+
- spec/docker/env.development
|
|
12
|
+
ulimits:
|
|
13
|
+
memlock:
|
|
14
|
+
soft: -1
|
|
15
|
+
hard: -1
|
|
16
|
+
nofile:
|
|
17
|
+
soft: 65536
|
|
18
|
+
hard: 65536
|
|
19
|
+
volumes:
|
|
20
|
+
- opensearch-data1:/usr/share/opensearch/data
|
|
21
|
+
ports:
|
|
22
|
+
- 29200:9200 # REST API
|
|
23
|
+
- 29600:9600 # Performance Analyzer
|
|
24
|
+
networks:
|
|
25
|
+
- opensearch-net
|
|
26
|
+
|
|
27
|
+
volumes:
|
|
28
|
+
opensearch-data1:
|
|
29
|
+
|
|
30
|
+
networks:
|
|
31
|
+
opensearch-net:
|