sec_api 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.devcontainer/Dockerfile +54 -0
- data/.devcontainer/README.md +178 -0
- data/.devcontainer/devcontainer.json +46 -0
- data/.devcontainer/docker-compose.yml +28 -0
- data/.devcontainer/post-create.sh +51 -0
- data/.devcontainer/post-start.sh +44 -0
- data/.rspec +3 -0
- data/.standard.yml +3 -0
- data/CHANGELOG.md +5 -0
- data/CLAUDE.md +0 -0
- data/LICENSE.txt +21 -0
- data/MIGRATION.md +274 -0
- data/README.md +370 -0
- data/Rakefile +10 -0
- data/config/secapi.yml.example +57 -0
- data/docs/development-guide.md +291 -0
- data/docs/enumerator_pattern_design.md +483 -0
- data/docs/examples/README.md +58 -0
- data/docs/examples/backfill_filings.rb +419 -0
- data/docs/examples/instrumentation.rb +583 -0
- data/docs/examples/query_builder.rb +308 -0
- data/docs/examples/streaming_notifications.rb +491 -0
- data/docs/index.md +244 -0
- data/docs/migration-guide-v1.md +1091 -0
- data/docs/pre-review-checklist.md +145 -0
- data/docs/project-overview.md +90 -0
- data/docs/project-scan-report.json +60 -0
- data/docs/source-tree-analysis.md +190 -0
- data/lib/sec_api/callback_helper.rb +49 -0
- data/lib/sec_api/client.rb +606 -0
- data/lib/sec_api/collections/filings.rb +267 -0
- data/lib/sec_api/collections/fulltext_results.rb +86 -0
- data/lib/sec_api/config.rb +590 -0
- data/lib/sec_api/deep_freezable.rb +42 -0
- data/lib/sec_api/errors/authentication_error.rb +24 -0
- data/lib/sec_api/errors/configuration_error.rb +5 -0
- data/lib/sec_api/errors/error.rb +75 -0
- data/lib/sec_api/errors/network_error.rb +26 -0
- data/lib/sec_api/errors/not_found_error.rb +23 -0
- data/lib/sec_api/errors/pagination_error.rb +28 -0
- data/lib/sec_api/errors/permanent_error.rb +29 -0
- data/lib/sec_api/errors/rate_limit_error.rb +57 -0
- data/lib/sec_api/errors/reconnection_error.rb +34 -0
- data/lib/sec_api/errors/server_error.rb +25 -0
- data/lib/sec_api/errors/transient_error.rb +28 -0
- data/lib/sec_api/errors/validation_error.rb +23 -0
- data/lib/sec_api/extractor.rb +122 -0
- data/lib/sec_api/filing_journey.rb +477 -0
- data/lib/sec_api/mapping.rb +125 -0
- data/lib/sec_api/metrics_collector.rb +411 -0
- data/lib/sec_api/middleware/error_handler.rb +250 -0
- data/lib/sec_api/middleware/instrumentation.rb +186 -0
- data/lib/sec_api/middleware/rate_limiter.rb +541 -0
- data/lib/sec_api/objects/data_file.rb +34 -0
- data/lib/sec_api/objects/document_format_file.rb +45 -0
- data/lib/sec_api/objects/entity.rb +92 -0
- data/lib/sec_api/objects/extracted_data.rb +118 -0
- data/lib/sec_api/objects/fact.rb +147 -0
- data/lib/sec_api/objects/filing.rb +197 -0
- data/lib/sec_api/objects/fulltext_result.rb +66 -0
- data/lib/sec_api/objects/period.rb +96 -0
- data/lib/sec_api/objects/stream_filing.rb +194 -0
- data/lib/sec_api/objects/xbrl_data.rb +356 -0
- data/lib/sec_api/query.rb +423 -0
- data/lib/sec_api/rate_limit_state.rb +130 -0
- data/lib/sec_api/rate_limit_tracker.rb +154 -0
- data/lib/sec_api/stream.rb +841 -0
- data/lib/sec_api/structured_logger.rb +199 -0
- data/lib/sec_api/types.rb +32 -0
- data/lib/sec_api/version.rb +42 -0
- data/lib/sec_api/xbrl.rb +220 -0
- data/lib/sec_api.rb +137 -0
- data/sig/sec_api.rbs +4 -0
- metadata +217 -0
|
@@ -0,0 +1,291 @@
|
|
|
1
|
+
# sec_api - Development Guide
|
|
2
|
+
|
|
3
|
+
**Generated:** 2026-01-05
|
|
4
|
+
|
|
5
|
+
## Prerequisites
|
|
6
|
+
|
|
7
|
+
### Required
|
|
8
|
+
- **Ruby:** 3.1.0 or higher (project uses 3.2.3)
|
|
9
|
+
- **Bundler:** Latest version
|
|
10
|
+
- **Git:** For version control
|
|
11
|
+
|
|
12
|
+
### Optional
|
|
13
|
+
- **sec-api.io API Key:** Required for live API testing (get from https://sec-api.io)
|
|
14
|
+
|
|
15
|
+
## Initial Setup
|
|
16
|
+
|
|
17
|
+
### 1. Clone and Install Dependencies
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
# Clone the repository
|
|
21
|
+
git clone https://github.com/ljuti/sec_api.git
|
|
22
|
+
cd sec_api
|
|
23
|
+
|
|
24
|
+
# Run setup script
|
|
25
|
+
bin/setup
|
|
26
|
+
|
|
27
|
+
# Or manual setup:
|
|
28
|
+
bundle install
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### 2. Configuration
|
|
32
|
+
|
|
33
|
+
Create a local configuration file:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
# Copy example configuration
|
|
37
|
+
cp config/secapi.yml.example config/secapi.local.yml
|
|
38
|
+
|
|
39
|
+
# Edit with your API key
|
|
40
|
+
# config/secapi.local.yml
|
|
41
|
+
api_key: YOUR_API_KEY_HERE
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
**Environment Variable Alternative:**
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
export SECAPI_API_KEY=your_api_key_here
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
**Note:** `config/secapi.local.yml` is gitignored and won't be committed.
|
|
51
|
+
|
|
52
|
+
## Development Workflow
|
|
53
|
+
|
|
54
|
+
### Interactive Console
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
bin/console
|
|
58
|
+
|
|
59
|
+
# Inside console:
|
|
60
|
+
client = SecApi::Client.new
|
|
61
|
+
client.query.ticker("AAPL").search
|
|
62
|
+
# => SecApi::Filings collection
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
### Running Tests
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
# Run all tests
|
|
69
|
+
bundle exec rspec
|
|
70
|
+
|
|
71
|
+
# Run specific test file
|
|
72
|
+
bundle exec rspec spec/sec_api/client_spec.rb
|
|
73
|
+
|
|
74
|
+
# Run with coverage report
|
|
75
|
+
bundle exec rspec --format documentation
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
### Linting
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
# Run Standard Ruby linter
|
|
82
|
+
bundle exec standardrb
|
|
83
|
+
|
|
84
|
+
# Auto-fix issues
|
|
85
|
+
bundle exec standardrb --fix
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
### Building the Gem
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
# Build gem file
|
|
92
|
+
gem build sec_api.gemspec
|
|
93
|
+
|
|
94
|
+
# Install locally for testing
|
|
95
|
+
gem install sec_api-0.1.0.gem
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
## Project Structure
|
|
99
|
+
|
|
100
|
+
See [Source Tree Analysis](./source-tree-analysis.md) for complete directory structure.
|
|
101
|
+
|
|
102
|
+
**Key Directories:**
|
|
103
|
+
- `lib/sec_api/` - Main library code
|
|
104
|
+
- `spec/` - RSpec tests
|
|
105
|
+
- `config/` - Configuration files
|
|
106
|
+
- `_bmad-output/` - Planning artifacts (PRD, Architecture, Epics)
|
|
107
|
+
|
|
108
|
+
## Testing Strategy
|
|
109
|
+
|
|
110
|
+
### VCR Cassettes
|
|
111
|
+
|
|
112
|
+
The project uses VCR to record HTTP interactions for deterministic, offline-runnable tests.
|
|
113
|
+
|
|
114
|
+
**Recording New Cassettes:**
|
|
115
|
+
|
|
116
|
+
```ruby
|
|
117
|
+
# spec/sec_api/query_spec.rb
|
|
118
|
+
VCR.use_cassette("query_proxy/ticker_search") do
|
|
119
|
+
client.query.ticker("AAPL").search
|
|
120
|
+
end
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
**Cassette Organization:**
|
|
124
|
+
- `spec/fixtures/vcr_cassettes/query_proxy/` - Query API cassettes
|
|
125
|
+
- `spec/fixtures/vcr_cassettes/mapping_proxy/` - Mapping API cassettes
|
|
126
|
+
|
|
127
|
+
### Running Tests Without API Key
|
|
128
|
+
|
|
129
|
+
VCR cassettes allow tests to run without a live API connection:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
# Tests use recorded cassettes
|
|
133
|
+
bundle exec rspec
|
|
134
|
+
|
|
135
|
+
# No API key needed if cassettes exist
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
### Re-recording Cassettes
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
# Delete old cassettes
|
|
142
|
+
rm -rf spec/fixtures/vcr_cassettes/
|
|
143
|
+
|
|
144
|
+
# Run tests with live API (requires API key)
|
|
145
|
+
SECAPI_API_KEY=your_key bundle exec rspec
|
|
146
|
+
|
|
147
|
+
# New cassettes will be recorded
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
## Code Style
|
|
151
|
+
|
|
152
|
+
### Standard Ruby
|
|
153
|
+
|
|
154
|
+
The project follows [Standard Ruby](https://github.com/testdouble/standard) style guide.
|
|
155
|
+
|
|
156
|
+
**Enforced by:** `bundle exec standardrb`
|
|
157
|
+
|
|
158
|
+
**Key Rules:**
|
|
159
|
+
- 2-space indentation
|
|
160
|
+
- No trailing whitespace
|
|
161
|
+
- snake_case for methods and variables
|
|
162
|
+
- PascalCase for classes and modules
|
|
163
|
+
- SCREAMING_SNAKE_CASE for constants
|
|
164
|
+
|
|
165
|
+
### Naming Conventions
|
|
166
|
+
|
|
167
|
+
```ruby
|
|
168
|
+
# Classes and Modules
|
|
169
|
+
class SecApi::Client
|
|
170
|
+
module SecApi::Errors
|
|
171
|
+
|
|
172
|
+
# Methods and Variables
|
|
173
|
+
def query_filings
|
|
174
|
+
api_key = config.api_key
|
|
175
|
+
|
|
176
|
+
# Constants
|
|
177
|
+
DEFAULT_MAX_RETRIES = 5
|
|
178
|
+
RETRYABLE_STATUS_CODES = [429, 500, 502, 503, 504]
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
## Common Development Tasks
|
|
182
|
+
|
|
183
|
+
### Adding a New API Endpoint
|
|
184
|
+
|
|
185
|
+
1. **Create proxy class:** `lib/sec_api/my_endpoint.rb`
|
|
186
|
+
2. **Wire to Client:** Add delegator in `lib/sec_api/client.rb`
|
|
187
|
+
3. **Add response object:** `lib/sec_api/objects/my_response.rb` (Dry::Struct)
|
|
188
|
+
4. **Write tests:** `spec/sec_api/my_endpoint_spec.rb`
|
|
189
|
+
5. **Record VCR cassette:** Run tests with live API
|
|
190
|
+
|
|
191
|
+
### Adding New Error Type
|
|
192
|
+
|
|
193
|
+
1. **Create error class:** `lib/sec_api/errors/my_error.rb`
|
|
194
|
+
2. **Inherit from TransientError or PermanentError:**
|
|
195
|
+
|
|
196
|
+
```ruby
|
|
197
|
+
module SecApi
|
|
198
|
+
class MyError < TransientError # or PermanentError
|
|
199
|
+
end
|
|
200
|
+
end
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
3. **Add to error handler middleware:** `lib/sec_api/middleware/error_handler.rb`
|
|
204
|
+
4. **Write tests:** `spec/sec_api/errors/my_error_spec.rb`
|
|
205
|
+
|
|
206
|
+
### Adding Configuration Option
|
|
207
|
+
|
|
208
|
+
1. **Update config class:** `lib/sec_api/config.rb`
|
|
209
|
+
|
|
210
|
+
```ruby
|
|
211
|
+
class Config < Anyway::Config
|
|
212
|
+
attr_config :my_new_option,
|
|
213
|
+
:existing_option
|
|
214
|
+
end
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
2. **Add to YAML:** `config/secapi.yml`
|
|
218
|
+
3. **Document in README**
|
|
219
|
+
|
|
220
|
+
## CI/CD (Planned for v1.0.0)
|
|
221
|
+
|
|
222
|
+
**GitHub Actions workflows:**
|
|
223
|
+
- `.github/workflows/ci.yml` - Run tests, linting, coverage check
|
|
224
|
+
- `.github/workflows/release.yml` - Publish gem to RubyGems
|
|
225
|
+
|
|
226
|
+
## Release Process (Planned for v1.0.0)
|
|
227
|
+
|
|
228
|
+
1. **Update version:** `lib/sec_api/version.rb`
|
|
229
|
+
2. **Update CHANGELOG:** Add release notes
|
|
230
|
+
3. **Commit changes:** `git commit -m "Release v1.0.0"`
|
|
231
|
+
4. **Tag release:** `git tag v1.0.0`
|
|
232
|
+
5. **Push to GitHub:** `git push && git push --tags`
|
|
233
|
+
6. **GitHub Actions:** Automatically builds and publishes gem
|
|
234
|
+
|
|
235
|
+
## Documentation
|
|
236
|
+
|
|
237
|
+
### YARD Documentation
|
|
238
|
+
|
|
239
|
+
Generate YARD docs:
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
bundle exec yard doc
|
|
243
|
+
|
|
244
|
+
# View locally
|
|
245
|
+
open doc/index.html
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
**YARD Coverage Goal:** 100% for public APIs (v1.0.0 requirement)
|
|
249
|
+
|
|
250
|
+
### Updating Planning Documents
|
|
251
|
+
|
|
252
|
+
Planning artifacts are in `_bmad-output/planning-artifacts/`:
|
|
253
|
+
- **PRD:** Product requirements
|
|
254
|
+
- **Architecture:** Architectural decisions
|
|
255
|
+
- **Epics:** Story breakdown
|
|
256
|
+
|
|
257
|
+
**These are living documents** - update as the project evolves.
|
|
258
|
+
|
|
259
|
+
## Troubleshooting
|
|
260
|
+
|
|
261
|
+
### Bundle Install Fails
|
|
262
|
+
|
|
263
|
+
```bash
|
|
264
|
+
# Try updating bundler
|
|
265
|
+
gem install bundler
|
|
266
|
+
bundle update
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
### Tests Fail with VCR Errors
|
|
270
|
+
|
|
271
|
+
```bash
|
|
272
|
+
# Re-record cassettes with live API
|
|
273
|
+
rm -rf spec/fixtures/vcr_cassettes/
|
|
274
|
+
SECAPI_API_KEY=your_key bundle exec rspec
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
### Configuration Not Loading
|
|
278
|
+
|
|
279
|
+
Check configuration priority:
|
|
280
|
+
1. Environment variables (`SECAPI_*`)
|
|
281
|
+
2. Local YAML (`config/secapi.local.yml`)
|
|
282
|
+
3. Default YAML (`config/secapi.yml`)
|
|
283
|
+
|
|
284
|
+
## Additional Resources
|
|
285
|
+
|
|
286
|
+
- **[Project Overview](./project-overview.md)** - High-level project context
|
|
287
|
+
- **[Source Tree Analysis](./source-tree-analysis.md)** - Code organization
|
|
288
|
+
- **[README](../README.md)** - Quick start and usage
|
|
289
|
+
- **[PRD](../_bmad-output/planning-artifacts/prd.md)** - Complete requirements
|
|
290
|
+
- **[Architecture](../_bmad-output/planning-artifacts/architecture.md)** - Technical decisions
|
|
291
|
+
- **[Epics](../_bmad-output/planning-artifacts/epics.md)** - Implementation stories
|