async-http-capture 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/context/falcon-integration.md +115 -0
- data/context/getting-started.md +40 -53
- data/context/index.yaml +4 -0
- data/design.md +16 -16
- data/lib/async/http/capture/cassette.rb +33 -6
- data/lib/async/http/capture/cassette_store.rb +7 -7
- data/lib/async/http/capture/environment.rb +104 -0
- data/lib/async/http/capture/interaction.rb +25 -16
- data/lib/async/http/capture/interaction_tracker.rb +15 -0
- data/lib/async/http/capture/middleware.rb +11 -5
- data/lib/async/http/capture/version.rb +1 -1
- data/lib/async/http/capture.rb +1 -0
- data/readme.md +14 -13
- metadata +17 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 53f67ea6516d2dabe2edca496f827aefa4324bec0388be36e4771d6d5564aea3
|
4
|
+
data.tar.gz: dd344bbf8bca118a37e214fee92dc0d743c264ff011aed8517a9bf4db2a2b098
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 02234f7f20f20a03768322275c9dc76d1501b8ea220185279b5f0a1d9a0d911646ee2777b64f26b0ac0a36e09613f457d6393a5775340915cd9d9fa0d1ad869d
|
7
|
+
data.tar.gz: a319dd4cc411dca4e4fcd1d7f347a5437f825263d6a5685d2529f48d2d296993c01ee764b7e8a45fd5b33d4bdfd47c42b7996a437190cad20b8332ff28b081be
|
@@ -0,0 +1,115 @@
|
|
1
|
+
# Falcon Integration
|
2
|
+
|
3
|
+
This guide explains how to integrate `async-http-capture` with Falcon web server for recording and replaying HTTP interactions.
|
4
|
+
|
5
|
+
## Core Concepts
|
6
|
+
|
7
|
+
Falcon integration with `async-http-capture` uses the {ruby Async::HTTP::Capture::Environment} to provide clean, declarative configuration for both recording and replay functionality.
|
8
|
+
|
9
|
+
## Usage
|
10
|
+
|
11
|
+
Create a `falcon.rb` configuration file:
|
12
|
+
|
13
|
+
~~~ ruby
|
14
|
+
#!/usr/bin/env falcon-host
|
15
|
+
# frozen_string_literal: true
|
16
|
+
|
17
|
+
require "falcon/environment/rack"
|
18
|
+
require "async/http/capture"
|
19
|
+
|
20
|
+
service "my-app" do
|
21
|
+
include Falcon::Environment::Rack
|
22
|
+
include Async::HTTP::Capture::Environment
|
23
|
+
end
|
24
|
+
~~~
|
25
|
+
|
26
|
+
This minimal configuration automatically:
|
27
|
+
|
28
|
+
- **Records interactions** to `cassette/recordings/` directory
|
29
|
+
- **Replays existing recordings** from `cassette/warmup/` for application warmup
|
30
|
+
- **Logs interactions** to console for visibility
|
31
|
+
|
32
|
+
### Custom Configuration
|
33
|
+
|
34
|
+
Override the environment methods to customize behavior:
|
35
|
+
|
36
|
+
~~~ ruby
|
37
|
+
service "my-app" do
|
38
|
+
include Falcon::Environment::Rack
|
39
|
+
include Async::HTTP::Capture::Environment
|
40
|
+
|
41
|
+
def capture_cassette_directory
|
42
|
+
# Custom warmup directory:
|
43
|
+
"recordings/warmup"
|
44
|
+
end
|
45
|
+
|
46
|
+
def capture_recordings_directory
|
47
|
+
# Custom recording directory:
|
48
|
+
"recordings/production"
|
49
|
+
end
|
50
|
+
|
51
|
+
def capture_console_logging
|
52
|
+
# Disable console output for production:
|
53
|
+
false
|
54
|
+
end
|
55
|
+
end
|
56
|
+
~~~
|
57
|
+
|
58
|
+
## Running the Server
|
59
|
+
|
60
|
+
Start your Falcon server with recording enabled:
|
61
|
+
|
62
|
+
~~~ bash
|
63
|
+
$ bundle exec ./falcon.rb
|
64
|
+
~~~
|
65
|
+
|
66
|
+
The server will:
|
67
|
+
|
68
|
+
1. **Start up**: Load and replay any existing recordings for warmup
|
69
|
+
2. **Accept requests**: Process incoming HTTP requests normally
|
70
|
+
3. **Record interactions**: Save all new interactions to timestamped files
|
71
|
+
4. **Log activity**: Show recording activity in console (if enabled)
|
72
|
+
|
73
|
+
## Application Warmup
|
74
|
+
|
75
|
+
One of the most powerful features is automatic application warmup using recorded interactions:
|
76
|
+
|
77
|
+
### Recording Phase
|
78
|
+
|
79
|
+
During development or testing, make requests to capture typical usage patterns:
|
80
|
+
|
81
|
+
~~~ bash
|
82
|
+
# Make various requests to capture typical traffic patterns
|
83
|
+
curl http://localhost:9292/health
|
84
|
+
curl http://localhost:9292/api/users
|
85
|
+
curl -X POST http://localhost:9292/api/orders \
|
86
|
+
-H "content-type: application/json" \
|
87
|
+
-d '{"product_id": 123, "quantity": 2}'
|
88
|
+
~~~
|
89
|
+
|
90
|
+
These requests are recorded to `cassette/recordings/` as:
|
91
|
+
|
92
|
+
~~~
|
93
|
+
cassette/recordings/
|
94
|
+
├── 20250821-105406-271633-12345-67890.json # GET /health
|
95
|
+
├── 20250821-105407-123456-12345-67890.json # GET /api/users
|
96
|
+
└── 20250821-105408-654321-12345-67890.json # POST /api/orders
|
97
|
+
~~~
|
98
|
+
|
99
|
+
Files are named: `YYYYMMDD-HHMMSS-MICROSECONDS-PID-OBJECT_ID.json`
|
100
|
+
|
101
|
+
Where:
|
102
|
+
- `YYYYMMDD-HHMMSS-MICROSECONDS` = timestamp with microsecond precision.
|
103
|
+
- `PID` = process ID.
|
104
|
+
- `OBJECT_ID` = store instance object ID.
|
105
|
+
|
106
|
+
### Warmup Phase
|
107
|
+
|
108
|
+
On subsequent server starts, the Environment automatically replays recorded interactions to warm up:
|
109
|
+
|
110
|
+
- **Database connections**
|
111
|
+
- **Application caches**
|
112
|
+
- **JIT compilation**
|
113
|
+
- **Service dependencies**
|
114
|
+
|
115
|
+
This dramatically improves first-request latency and provides predictable application startup behavior in production.
|
data/context/getting-started.md
CHANGED
@@ -17,7 +17,7 @@ $ bundle add async-http-capture
|
|
17
17
|
- A {ruby Async::HTTP::Capture::Middleware} which captures HTTP requests and responses as they pass through your application.
|
18
18
|
- An {ruby Async::HTTP::Capture::Interaction} which represents a single HTTP request/response pair with lazy Protocol::HTTP object construction.
|
19
19
|
- A {ruby Async::HTTP::Capture::Cassette} which is a collection of interactions that can be loaded from and saved to JSON files.
|
20
|
-
- A {ruby Async::HTTP::Capture::CassetteStore} which provides
|
20
|
+
- A {ruby Async::HTTP::Capture::CassetteStore} which provides timestamped storage, saving each interaction to a separate file named by timestamp.
|
21
21
|
- A {ruby Async::HTTP::Capture::ConsoleStore} which logs interactions to the console for debugging purposes.
|
22
22
|
|
23
23
|
## Usage
|
@@ -35,11 +35,11 @@ Here's how to record HTTP interactions to files:
|
|
35
35
|
~~~ ruby
|
36
36
|
require "async/http/capture"
|
37
37
|
|
38
|
-
# Create a store that saves to
|
38
|
+
# Create a store that saves to timestamped files:
|
39
39
|
store = Async::HTTP::Capture::CassetteStore.new("interactions")
|
40
40
|
|
41
41
|
# Create your application
|
42
|
-
app = ->(request) {
|
42
|
+
app = ->(request) {Protocol::HTTP::Response[200, {}, ["OK"]]}
|
43
43
|
|
44
44
|
# Wrap it with recording middleware:
|
45
45
|
middleware = Async::HTTP::Capture::Middleware.new(app, store: store)
|
@@ -47,7 +47,7 @@ middleware = Async::HTTP::Capture::Middleware.new(app, store: store)
|
|
47
47
|
# Make requests - they will be automatically recorded:
|
48
48
|
request = Protocol::HTTP::Request["GET", "/users"]
|
49
49
|
response = middleware.call(request)
|
50
|
-
# This creates a file like
|
50
|
+
# This creates a file like recordings/20250821-105406-271633-12345-67890.json
|
51
51
|
~~~
|
52
52
|
|
53
53
|
### Recording with Console Output
|
@@ -70,45 +70,46 @@ middleware.call(request)
|
|
70
70
|
# Load recorded interactions:
|
71
71
|
cassette = Async::HTTP::Capture::Cassette.load("interactions")
|
72
72
|
|
73
|
-
#
|
73
|
+
# Option 1: Use the built-in replay method for application warmup
|
74
|
+
cassette.replay(app)
|
75
|
+
|
76
|
+
# Option 2: Manual iteration for custom processing
|
74
77
|
cassette.each do |interaction|
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
+
request = interaction.request # Lazy Protocol::HTTP::Request construction
|
79
|
+
response = app.call(request) # Send to your app
|
80
|
+
puts "#{request.method} #{request.path} -> #{response.status}"
|
78
81
|
end
|
79
82
|
~~~
|
80
83
|
|
81
84
|
## Recording HTTP Requests and Responses
|
82
85
|
|
83
|
-
|
86
|
+
The middleware automatically records both requests and responses:
|
84
87
|
|
85
88
|
~~~ ruby
|
86
89
|
middleware = Async::HTTP::Capture::Middleware.new(
|
87
|
-
|
88
|
-
|
89
|
-
record_response: true
|
90
|
+
app,
|
91
|
+
store: store
|
90
92
|
)
|
91
93
|
|
92
94
|
response = middleware.call(request)
|
93
|
-
# Both request and response are
|
95
|
+
# Both request and response are recorded.
|
94
96
|
~~~
|
95
97
|
|
96
|
-
##
|
98
|
+
## Timestamped Storage
|
97
99
|
|
98
|
-
Each interaction is saved to a file named
|
100
|
+
Each interaction is saved to a file named with timestamp, process ID, and object ID, providing several benefits:
|
99
101
|
|
100
102
|
~~~
|
101
|
-
|
102
|
-
├──
|
103
|
-
├──
|
104
|
-
└──
|
103
|
+
recordings/
|
104
|
+
├── 20250821-105406-271633-12345-67890.json # GET /users
|
105
|
+
├── 20250821-105006-257022-12346-67891.json # POST /orders
|
106
|
+
└── 20250820-101234-567890-12347-67892.json # GET /health
|
105
107
|
~~~
|
106
108
|
|
107
109
|
Benefits:
|
108
|
-
- **
|
110
|
+
- **Chronological ordering**: Files sorted by timestamp
|
109
111
|
- **Parallel-safe**: Multiple processes can write without conflicts
|
110
|
-
- **
|
111
|
-
- **Git-friendly**: Stable filenames for version control
|
112
|
+
- **Human-readable**: Timestamps are easy to understand
|
112
113
|
|
113
114
|
## Application Warmup
|
114
115
|
|
@@ -122,17 +123,17 @@ endpoint = Async::HTTP::Endpoint.parse("https://api.example.com")
|
|
122
123
|
store = Async::HTTP::Capture::CassetteStore.new("warmup_interactions")
|
123
124
|
|
124
125
|
recording_middleware = Async::HTTP::Capture::Middleware.new(
|
125
|
-
|
126
|
-
|
126
|
+
nil,
|
127
|
+
store: store
|
127
128
|
)
|
128
129
|
|
129
130
|
client = Async::HTTP::Client.new(endpoint, middleware: [recording_middleware])
|
130
131
|
|
131
132
|
# Make the requests you want to record
|
132
133
|
Async do
|
133
|
-
|
134
|
-
|
135
|
-
|
134
|
+
client.get("/health")
|
135
|
+
client.get("/api/popular-items")
|
136
|
+
client.post("/api/user-sessions", {user_id: 123})
|
136
137
|
end
|
137
138
|
|
138
139
|
# Step 2: Use recorded interactions to warm up your application
|
@@ -141,13 +142,13 @@ app = MyApplication.new
|
|
141
142
|
|
142
143
|
puts "Warming up with #{cassette.interactions.size} recorded interactions..."
|
143
144
|
cassette.each do |interaction|
|
144
|
-
|
145
|
-
|
146
|
-
|
147
|
-
|
148
|
-
|
149
|
-
|
150
|
-
|
145
|
+
request = interaction.request
|
146
|
+
begin
|
147
|
+
app_response = app.call(request)
|
148
|
+
puts "Warmed up #{request.method} #{request.path} -> #{app_response.status}"
|
149
|
+
rescue => error
|
150
|
+
puts "Warning: #{request.method} #{request.path} -> #{error.message}"
|
151
|
+
end
|
151
152
|
end
|
152
153
|
|
153
154
|
puts "Warmup complete!"
|
@@ -159,28 +160,14 @@ You can create custom storage backends by implementing the {ruby Async::HTTP::Ca
|
|
159
160
|
|
160
161
|
~~~ ruby
|
161
162
|
class MyCustomStore
|
162
|
-
|
163
|
-
|
164
|
-
|
165
|
-
|
166
|
-
|
167
|
-
puts "Custom handling: #{interaction.request.method} #{interaction.request.path}"
|
168
|
-
end
|
163
|
+
def call(interaction)
|
164
|
+
# Handle the interaction as needed
|
165
|
+
# e.g., send to a database, external service, etc.
|
166
|
+
puts "Custom handling: #{interaction.request.method} #{interaction.request.path}"
|
167
|
+
end
|
169
168
|
end
|
170
169
|
|
171
170
|
# Use your custom store
|
172
171
|
custom_store = MyCustomStore.new
|
173
172
|
middleware = Async::HTTP::Capture::Middleware.new(app, store: custom_store)
|
174
173
|
~~~
|
175
|
-
|
176
|
-
## Key Features
|
177
|
-
|
178
|
-
- **Pure Protocol::HTTP**: Works directly with Protocol::HTTP objects, no lossy conversions
|
179
|
-
- **Content-Addressed Storage**: Each interaction saved as separate JSON file with content hash
|
180
|
-
- **Parallel-Safe**: Multiple processes can record simultaneously without conflicts
|
181
|
-
- **Flexible Stores**: Pluggable storage backends (files, console logging, etc.)
|
182
|
-
- **Complete Headers**: Full round-trip serialization including `fields` and `tail`
|
183
|
-
- **Error Handling**: Captures network errors and connection issues
|
184
|
-
- **Lazy Construction**: Protocol::HTTP objects are constructed on-demand for memory efficiency
|
185
|
-
|
186
|
-
This makes `async-http-capture` ideal for testing, debugging, application warmup, and HTTP traffic analysis scenarios.
|
data/context/index.yaml
CHANGED
@@ -10,3 +10,7 @@ files:
|
|
10
10
|
title: Getting Started
|
11
11
|
description: This guide explains how to get started with `async-http-capture`, a
|
12
12
|
Ruby gem for recording and replaying HTTP requests using Protocol::HTTP.
|
13
|
+
- path: falcon-integration.md
|
14
|
+
title: Falcon Integration
|
15
|
+
description: This guide explains how to integrate `async-http-capture` with Falcon
|
16
|
+
web server for recording and replaying HTTP interactions.
|
data/design.md
CHANGED
@@ -115,7 +115,7 @@ end
|
|
115
115
|
|
116
116
|
```ruby
|
117
117
|
# Load recorded interactions and replay them (no middleware needed)
|
118
|
-
cassette = Async::HTTP::
|
118
|
+
cassette = Async::HTTP::Capture::Cassette.load("recordings")
|
119
119
|
|
120
120
|
# Simple replay: interactions construct Protocol::HTTP objects lazily
|
121
121
|
cassette.each do |interaction|
|
@@ -124,7 +124,7 @@ cassette.each do |interaction|
|
|
124
124
|
# Your app handles the request normally (warming up caches, etc.)
|
125
125
|
end
|
126
126
|
|
127
|
-
# Manual cassette creation using data hashes
|
127
|
+
# Manual cassette creation using data hashes:
|
128
128
|
interactions = [
|
129
129
|
Async::HTTP::Record::Interaction.new({
|
130
130
|
request: {
|
@@ -144,7 +144,7 @@ interactions = [
|
|
144
144
|
]
|
145
145
|
|
146
146
|
cassette = Async::HTTP::Record::Cassette.new(interactions)
|
147
|
-
cassette.save("
|
147
|
+
cassette.save("recordings")
|
148
148
|
```
|
149
149
|
|
150
150
|
### Recording Middleware
|
@@ -170,7 +170,7 @@ class Async::HTTP::Record::Middleware < Protocol::HTTP::Middleware
|
|
170
170
|
# Capture request body if present
|
171
171
|
captured_request = capture_request_body(request)
|
172
172
|
|
173
|
-
# Get response from downstream middleware/app
|
173
|
+
# Get response from downstream middleware/app
|
174
174
|
response = super(captured_request)
|
175
175
|
|
176
176
|
if @record_response
|
@@ -480,11 +480,11 @@ For warmup scenarios where you only need requests:
|
|
480
480
|
|
481
481
|
## Implementation Strategy
|
482
482
|
|
483
|
-
### Phase 1: Core Components
|
483
|
+
### Phase 1: Core Components
|
484
484
|
- [ ] `Interaction` class as immutable data holder using Protocol::HTTP objects with bodies
|
485
485
|
- [ ] `Cassette` class with JSON serialization and loading
|
486
486
|
|
487
|
-
### Phase 2: Replay Integration
|
487
|
+
### Phase 2: Replay Integration
|
488
488
|
- [ ] Simple replay by iterating through interactions and calling `app.call(request)`
|
489
489
|
- [ ] No middleware needed - direct request sending to your application
|
490
490
|
- [ ] Proper error handling during replay for robust warmup scenarios
|
@@ -539,7 +539,7 @@ cassette.save("warmup.json")
|
|
539
539
|
require "async/http/record"
|
540
540
|
|
541
541
|
# Load recorded interactions
|
542
|
-
cassette = Async::HTTP::
|
542
|
+
cassette = Async::HTTP::Capture::Cassette.load("recordings")
|
543
543
|
|
544
544
|
# Your application
|
545
545
|
app = MyApplication.new
|
@@ -569,7 +569,7 @@ require "async/http/record"
|
|
569
569
|
endpoint = Async::HTTP::Endpoint.parse("https://api.example.com")
|
570
570
|
recording_middleware = Async::HTTP::Record::Middleware.new(
|
571
571
|
nil,
|
572
|
-
cassette_path: "
|
572
|
+
cassette_path: "recordings"
|
573
573
|
)
|
574
574
|
|
575
575
|
client = Async::HTTP::Client.new(endpoint, middleware: [recording_middleware])
|
@@ -585,8 +585,8 @@ end
|
|
585
585
|
# Step 2: Use recorded interactions to warm up your application
|
586
586
|
require "async/http/record"
|
587
587
|
|
588
|
-
# Load recorded interactions
|
589
|
-
cassette = Async::HTTP::
|
588
|
+
# Load recorded interactions
|
589
|
+
cassette = Async::HTTP::Capture::Cassette.load("recordings")
|
590
590
|
|
591
591
|
# Your application
|
592
592
|
app = MyApplication.new
|
@@ -610,11 +610,11 @@ puts "Warmup complete! Starting server..."
|
|
610
610
|
|
611
611
|
### Simple Error Strategy
|
612
612
|
- Missing cassette file: raise clear error with path
|
613
|
-
- Invalid JSON: raise parsing error with line number
|
613
|
+
- Invalid JSON: raise parsing error with line number
|
614
614
|
- Invalid request data: raise validation error
|
615
615
|
- Keep error messages focused and actionable
|
616
616
|
|
617
|
-
## Testing Strategy
|
617
|
+
## Testing Strategy
|
618
618
|
|
619
619
|
### Unit Tests
|
620
620
|
```ruby
|
@@ -724,13 +724,13 @@ Following async-http-cache's proven approach:
|
|
724
724
|
- Use `Protocol::HTTP::Body::Completable` for completion callbacks
|
725
725
|
- Use `Protocol::HTTP::Body::Buffered` for final storage as arrays of strings
|
726
726
|
|
727
|
-
### 5. Simple Replay Pattern
|
727
|
+
### 5. Simple Replay Pattern
|
728
728
|
No middleware needed for replay - just iterate through recorded interactions and call `app.call(request)` to warm up your application directly.
|
729
729
|
|
730
|
-
### 6. Optional Response Recording
|
730
|
+
### 6. Optional Response Recording
|
731
731
|
Responses are not recorded by default (`record_response: false`) since many use cases only need request recording for testing or mocking.
|
732
732
|
|
733
|
-
### 7. JSON-Only Storage
|
733
|
+
### 7. JSON-Only Storage
|
734
734
|
Simple, human-readable format that's easy to inspect and version control.
|
735
735
|
|
736
736
|
### 8. No Global State
|
@@ -760,7 +760,7 @@ All components are explicit about their dependencies and configuration.
|
|
760
760
|
|
761
761
|
This design enables a clean workflow:
|
762
762
|
|
763
|
-
1. **Recording**: Use `Middleware` with default `record_response: false` to capture requests during development/testing
|
763
|
+
1. **Recording**: Use `Middleware` with default `record_response: false` to capture requests during development/testing
|
764
764
|
2. **Replay**: Simple iteration - `cassette.each { |interaction| app.call(interaction.request) }` with lazy Protocol::HTTP object construction
|
765
765
|
3. **Lazy Construction**: `Interaction` stores data and builds Protocol::HTTP objects on first access via `request`/`response` methods
|
766
766
|
4. **Leverages Existing APIs**: Uses `Protocol::HTTP::Body::Buffered.wrap()` and standard Protocol::HTTP constructors
|
@@ -6,6 +6,7 @@
|
|
6
6
|
require "json"
|
7
7
|
require "time"
|
8
8
|
require "fileutils"
|
9
|
+
require "console"
|
9
10
|
|
10
11
|
module Async
|
11
12
|
module HTTP
|
@@ -47,22 +48,48 @@ module Async
|
|
47
48
|
data = JSON.parse(File.read(file_path), symbolize_names: true)
|
48
49
|
Interaction.new(data)
|
49
50
|
end
|
50
|
-
|
51
|
+
|
52
|
+
return self.new(interactions)
|
51
53
|
end
|
52
54
|
|
53
|
-
# Save the cassette to a directory using
|
54
|
-
# Each interaction is saved as a separate JSON file
|
55
|
-
# This approach provides
|
55
|
+
# Save the cassette to a directory using timestamped files.
|
56
|
+
# Each interaction is saved as a separate JSON file with a timestamp-based name.
|
57
|
+
# This approach provides parallel-safe recording.
|
56
58
|
# @parameter directory_path [String] The path to the directory where interactions should be saved.
|
57
59
|
def save(directory_path)
|
58
60
|
FileUtils.mkdir_p(directory_path)
|
59
61
|
|
60
|
-
@interactions.
|
61
|
-
|
62
|
+
@interactions.each_with_index do |interaction, index|
|
63
|
+
timestamp = Time.now.strftime("%Y%m%d-%H%M%S-%6N")
|
64
|
+
filename = "#{timestamp}-#{index}.json"
|
62
65
|
file_path = File.join(directory_path, filename)
|
63
66
|
File.write(file_path, JSON.pretty_generate(interaction.to_h))
|
64
67
|
end
|
65
68
|
end
|
69
|
+
|
70
|
+
# Replay all interactions against the provided application.
|
71
|
+
# This is useful for warming up applications by replaying recorded traffic.
|
72
|
+
# @parameter app [#call] The application to replay interactions against.
|
73
|
+
def replay(app)
|
74
|
+
count = @interactions.length
|
75
|
+
Console.info(self) {"Replaying #{count} interactions for warmup..."}
|
76
|
+
|
77
|
+
@interactions.each do |interaction|
|
78
|
+
Console.debug(self, "Replaying interaction:", interaction)
|
79
|
+
|
80
|
+
# Replay the interaction against the app:
|
81
|
+
if request = interaction.request
|
82
|
+
begin
|
83
|
+
response = app.call(request)
|
84
|
+
response.finish
|
85
|
+
rescue => error
|
86
|
+
Console.warn(self, "Failed to replay interaction:", error)
|
87
|
+
end
|
88
|
+
end
|
89
|
+
end
|
90
|
+
|
91
|
+
Console.info(self) {"Warmup complete."}
|
92
|
+
end
|
66
93
|
end
|
67
94
|
end
|
68
95
|
end
|
@@ -12,15 +12,16 @@ require_relative "cassette"
|
|
12
12
|
module Async
|
13
13
|
module HTTP
|
14
14
|
module Capture
|
15
|
-
# Store implementation that saves interactions to
|
15
|
+
# Store implementation that saves interactions to timestamped files in a directory.
|
16
16
|
#
|
17
|
-
# Each interaction is saved as a separate JSON file named by
|
18
|
-
#
|
17
|
+
# Each interaction is saved as a separate JSON file named by timestamp,
|
18
|
+
# using atomic write + move operations for parallel-safe recording.
|
19
19
|
class CassetteStore
|
20
20
|
# Initialize the cassette store.
|
21
21
|
# @parameter directory_path [String] The directory path where interactions should be saved.
|
22
22
|
def initialize(directory_path)
|
23
23
|
@directory_path = directory_path
|
24
|
+
@suffix = "#{Process.pid}-#{self.object_id}"
|
24
25
|
end
|
25
26
|
|
26
27
|
# @returns [Cassette] A cassette object representing the recorded interactions.
|
@@ -28,15 +29,14 @@ module Async
|
|
28
29
|
Cassette.load(@directory_path)
|
29
30
|
end
|
30
31
|
|
31
|
-
# Save an interaction to a
|
32
|
+
# Save an interaction to a timestamped file using PID and object ID for uniqueness.
|
32
33
|
# @parameter interaction [Interaction] The interaction to save.
|
33
34
|
def call(interaction)
|
34
35
|
FileUtils.mkdir_p(@directory_path)
|
35
36
|
|
36
|
-
# Create filename with timestamp
|
37
|
+
# Create filename with timestamp, PID, and object ID for complete uniqueness:
|
37
38
|
timestamp = Time.now.strftime("%Y%m%d-%H%M%S-%6N") # Include microseconds
|
38
|
-
|
39
|
-
filename = "#{timestamp}-#{content_hash}.json"
|
39
|
+
filename = "#{timestamp}-#{@suffix}.json"
|
40
40
|
file_path = File.join(@directory_path, filename)
|
41
41
|
|
42
42
|
File.write(file_path, JSON.pretty_generate(interaction.serialize))
|
@@ -0,0 +1,104 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Released under the MIT License.
|
4
|
+
# Copyright, 2025, by Samuel Williams.
|
5
|
+
|
6
|
+
require_relative "cassette"
|
7
|
+
require_relative "cassette_store"
|
8
|
+
require_relative "console_store"
|
9
|
+
require_relative "middleware"
|
10
|
+
require "console"
|
11
|
+
|
12
|
+
module Async
|
13
|
+
module HTTP
|
14
|
+
module Capture
|
15
|
+
# A flat environment module for HTTP capture services.
|
16
|
+
#
|
17
|
+
# Provides simple, declarative configuration for recording and replaying HTTP interactions.
|
18
|
+
# Override these methods in your service to customize behavior.
|
19
|
+
module Environment
|
20
|
+
# Directory path to load cassettes from for replay warmup.
|
21
|
+
# Override this method to specify a different directory.
|
22
|
+
def capture_cassette_directory
|
23
|
+
"cassette/warmup"
|
24
|
+
end
|
25
|
+
|
26
|
+
# Directory path to save recordings to.
|
27
|
+
# Override this method to specify where recordings should be saved.
|
28
|
+
def capture_recordings_directory
|
29
|
+
"cassette/recordings"
|
30
|
+
end
|
31
|
+
|
32
|
+
# Whether to enable console logging of interactions (default: false).
|
33
|
+
# Override this method to enable console output.
|
34
|
+
def capture_console_logging
|
35
|
+
true
|
36
|
+
end
|
37
|
+
|
38
|
+
# Load the cassette for replay if configured
|
39
|
+
def capture_cassette
|
40
|
+
if capture_cassette_directory && File.directory?(capture_cassette_directory)
|
41
|
+
Cassette.load(capture_cassette_directory)
|
42
|
+
end
|
43
|
+
end
|
44
|
+
|
45
|
+
# Get the recording store if configured
|
46
|
+
def capture_recording_store
|
47
|
+
stores = []
|
48
|
+
|
49
|
+
# Add file storage if directory configured
|
50
|
+
if capture_recordings_directory
|
51
|
+
stores << CassetteStore.new(capture_recordings_directory)
|
52
|
+
end
|
53
|
+
|
54
|
+
# Add console logging if enabled
|
55
|
+
if capture_console_logging
|
56
|
+
stores << ConsoleStore.new
|
57
|
+
end
|
58
|
+
|
59
|
+
# Return combined store or nil
|
60
|
+
case stores.length
|
61
|
+
when 0
|
62
|
+
nil
|
63
|
+
when 1
|
64
|
+
stores.first
|
65
|
+
else
|
66
|
+
# Multiple stores - combine them
|
67
|
+
proc do |interaction|
|
68
|
+
stores.each {|store| store.call(interaction)}
|
69
|
+
end
|
70
|
+
end
|
71
|
+
end
|
72
|
+
|
73
|
+
# The middleware class to use for recording.
|
74
|
+
def capture_middleware_class
|
75
|
+
Middleware
|
76
|
+
end
|
77
|
+
|
78
|
+
# Wrap the middleware with the recording middleware.
|
79
|
+
# @parameter middleware [Middleware] The middleware to wrap.
|
80
|
+
# @parameter store [CassetteStore] The store to use for recording.
|
81
|
+
# @returns [Middleware] The wrapped middleware.
|
82
|
+
def capture_middleware(middleware)
|
83
|
+
if store = capture_recording_store
|
84
|
+
middleware = capture_middleware_class.new(middleware, store: store)
|
85
|
+
end
|
86
|
+
|
87
|
+
return middleware
|
88
|
+
end
|
89
|
+
|
90
|
+
# Set up middleware chain with recording support.
|
91
|
+
def middleware
|
92
|
+
# Get the underlying application by calling super:
|
93
|
+
middleware = super
|
94
|
+
|
95
|
+
# Warm up the application with recorded interactions if available:
|
96
|
+
capture_cassette&.replay(middleware)
|
97
|
+
|
98
|
+
# Wrap with recording middleware if store is configured
|
99
|
+
return capture_middleware(middleware)
|
100
|
+
end
|
101
|
+
end
|
102
|
+
end
|
103
|
+
end
|
104
|
+
end
|
@@ -3,8 +3,8 @@
|
|
3
3
|
# Released under the MIT License.
|
4
4
|
# Copyright, 2025, by Samuel Williams.
|
5
5
|
|
6
|
-
require "digest/sha2"
|
7
6
|
require "json"
|
7
|
+
|
8
8
|
require "protocol/http/request"
|
9
9
|
require "protocol/http/response"
|
10
10
|
require "protocol/http/headers"
|
@@ -61,16 +61,7 @@ module Async
|
|
61
61
|
new(hash)
|
62
62
|
end
|
63
63
|
|
64
|
-
#
|
65
|
-
# This hash can be used as a unique filename for content-addressed storage.
|
66
|
-
# @returns [String] A 16-character hexadecimal hash of the interaction content.
|
67
|
-
def content_hash
|
68
|
-
# Create a consistent JSON representation for hashing:
|
69
|
-
json_string = JSON.generate(serialize, sort_keys: true)
|
70
|
-
Digest::SHA256.hexdigest(json_string)[0, 16]
|
71
|
-
end
|
72
|
-
|
73
|
-
# Serialize the interaction to a data format suitable for storage or hashing.
|
64
|
+
# Serialize the interaction to a data format suitable for storage.
|
74
65
|
# Converts Protocol::HTTP objects to plain data structures.
|
75
66
|
# @returns [Hash] The serialized interaction data.
|
76
67
|
def serialize
|
@@ -80,7 +71,7 @@ module Async
|
|
80
71
|
data[:request] = serialize_request(request_obj)
|
81
72
|
end
|
82
73
|
|
83
|
-
if response_obj = self.response
|
74
|
+
if response_obj = self.response
|
84
75
|
data[:response] = serialize_response(response_obj)
|
85
76
|
end
|
86
77
|
|
@@ -147,7 +138,7 @@ module Async
|
|
147
138
|
|
148
139
|
# Add body chunks if present:
|
149
140
|
if request.body && request.body.is_a?(::Protocol::HTTP::Body::Buffered)
|
150
|
-
data[:body] = request.body.chunks
|
141
|
+
data[:body] = serialize_body_chunks(request.body.chunks)
|
151
142
|
end
|
152
143
|
|
153
144
|
data
|
@@ -173,12 +164,30 @@ module Async
|
|
173
164
|
|
174
165
|
# Add body chunks if present:
|
175
166
|
if response.body && response.body.is_a?(::Protocol::HTTP::Body::Buffered)
|
176
|
-
data[:body] = response.body.chunks
|
167
|
+
data[:body] = serialize_body_chunks(response.body.chunks)
|
177
168
|
end
|
178
169
|
|
179
170
|
data
|
180
171
|
end
|
181
172
|
|
173
|
+
private
|
174
|
+
|
175
|
+
# Serialize body chunks using built-in pack for base64 encoding.
|
176
|
+
# @parameter chunks [Array(String)] The chunks to serialize.
|
177
|
+
# @returns [Array(String)] Base64 encoded chunks.
|
178
|
+
def serialize_body_chunks(chunks)
|
179
|
+
chunks.map {|chunk| [chunk].pack("m0")}
|
180
|
+
end
|
181
|
+
|
182
|
+
# Deserialize body chunks from base64 using built-in unpack1.
|
183
|
+
# @parameter chunks [Array(String)] Base64 encoded chunks.
|
184
|
+
# @returns [Protocol::HTTP::Body::Buffered] Reconstructed buffered body.
|
185
|
+
def deserialize_body_chunks(chunks)
|
186
|
+
::Protocol::HTTP::Body::Buffered.wrap(
|
187
|
+
chunks.map {|encoded_chunk| encoded_chunk.unpack1("m0")}
|
188
|
+
)
|
189
|
+
end
|
190
|
+
|
182
191
|
# Create a Protocol::HTTP::Request from the stored request data.
|
183
192
|
# @returns [Protocol::HTTP::Request] The constructed request object.
|
184
193
|
def make_request
|
@@ -206,7 +215,7 @@ module Async
|
|
206
215
|
# @parameter protocol [String | Array | Nil] The protocol information.
|
207
216
|
# @returns [Protocol::HTTP::Request] The constructed request object.
|
208
217
|
def build_request(scheme: nil, authority: nil, method:, path:, version: nil, headers: nil, body: nil, protocol: nil)
|
209
|
-
body =
|
218
|
+
body = deserialize_body_chunks(body) if body
|
210
219
|
headers = build_headers(headers) if headers
|
211
220
|
|
212
221
|
::Protocol::HTTP::Request.new(
|
@@ -229,7 +238,7 @@ module Async
|
|
229
238
|
# @parameter protocol [String | Array | Nil] The protocol information.
|
230
239
|
# @returns [Protocol::HTTP::Response] The constructed response object.
|
231
240
|
def build_response(version: nil, status:, headers: nil, body: nil, protocol: nil)
|
232
|
-
body =
|
241
|
+
body = deserialize_body_chunks(body) if body
|
233
242
|
headers = build_headers(headers) if headers
|
234
243
|
|
235
244
|
::Protocol::HTTP::Response.new(
|
@@ -27,6 +27,10 @@ module Async
|
|
27
27
|
@response_body = nil
|
28
28
|
@error = nil
|
29
29
|
@clock = Async::Clock.start
|
30
|
+
|
31
|
+
# Will capture body as_json data at completion time for accurate state:
|
32
|
+
@debug_request_body = nil
|
33
|
+
@debug_response_body = nil
|
30
34
|
end
|
31
35
|
|
32
36
|
# Mark the request as ready (no body to process).
|
@@ -55,6 +59,9 @@ module Async
|
|
55
59
|
@request_complete = true
|
56
60
|
@request_body = body
|
57
61
|
|
62
|
+
# Capture as_json at completion time for accurate stateful information:
|
63
|
+
@debug_request_body = @original_request.body&.as_json
|
64
|
+
|
58
65
|
if error
|
59
66
|
@error = capture_error_context(error, :request_body)
|
60
67
|
end
|
@@ -69,6 +76,9 @@ module Async
|
|
69
76
|
@response_complete = true
|
70
77
|
@response_body = body
|
71
78
|
|
79
|
+
# Capture as_json at completion time for accurate stateful information:
|
80
|
+
@debug_response_body = @original_response&.body&.as_json
|
81
|
+
|
72
82
|
if error
|
73
83
|
@error = capture_error_context(error, :response_body)
|
74
84
|
end
|
@@ -130,6 +140,11 @@ module Async
|
|
130
140
|
interaction_data = {}
|
131
141
|
interaction_data[:error] = @error if @error
|
132
142
|
|
143
|
+
# Add as_json data for debugging (captured at completion time):
|
144
|
+
interaction_data[:debug] = {}
|
145
|
+
interaction_data[:debug][:request_body] = @debug_request_body if @debug_request_body
|
146
|
+
interaction_data[:debug][:response_body] = @debug_response_body if @debug_response_body
|
147
|
+
|
133
148
|
interaction = Interaction.new(
|
134
149
|
interaction_data,
|
135
150
|
request: final_request,
|
@@ -16,7 +16,7 @@ module Async
|
|
16
16
|
#
|
17
17
|
# This middleware captures both HTTP requests and responses, waiting for both
|
18
18
|
# to be fully processed before recording the complete interaction.
|
19
|
-
class Middleware < Protocol::HTTP::Middleware
|
19
|
+
class Middleware < ::Protocol::HTTP::Middleware
|
20
20
|
# Initialize the recording middleware.
|
21
21
|
# @parameter app [Protocol::HTTP::Middleware] The next middleware in the chain.
|
22
22
|
# @parameter store [Object] An object that responds to #call(interaction) to handle recorded interactions.
|
@@ -76,10 +76,13 @@ module Async
|
|
76
76
|
|
77
77
|
# Wrap with completion callback:
|
78
78
|
::Protocol::HTTP::Body::Completable.wrap(request) do |error|
|
79
|
+
# Always capture whatever body data we have, even if there was an error
|
80
|
+
captured_body = rewindable_body.buffered rescue nil
|
81
|
+
|
79
82
|
if error
|
80
|
-
tracker.request_completed(error: error)
|
83
|
+
tracker.request_completed(body: captured_body, error: error)
|
81
84
|
else
|
82
|
-
tracker.request_completed(body:
|
85
|
+
tracker.request_completed(body: captured_body)
|
83
86
|
end
|
84
87
|
end
|
85
88
|
|
@@ -102,10 +105,13 @@ module Async
|
|
102
105
|
|
103
106
|
# Wrap with completion callback:
|
104
107
|
::Protocol::HTTP::Body::Completable.wrap(response) do |error|
|
108
|
+
# Always capture whatever body data we have, even if there was an error
|
109
|
+
captured_body = rewindable_body.buffered rescue nil
|
110
|
+
|
105
111
|
if error
|
106
|
-
tracker.response_completed(error: error)
|
112
|
+
tracker.response_completed(body: captured_body, error: error)
|
107
113
|
else
|
108
|
-
tracker.response_completed(body:
|
114
|
+
tracker.response_completed(body: captured_body)
|
109
115
|
end
|
110
116
|
end
|
111
117
|
|
data/lib/async/http/capture.rb
CHANGED
data/readme.md
CHANGED
@@ -1,13 +1,13 @@
|
|
1
1
|
# Async::HTTP::Capture
|
2
2
|
|
3
|
-
A Ruby gem for recording and replaying HTTP requests using `Protocol::HTTP`. Features
|
3
|
+
A Ruby gem for recording and replaying HTTP requests using `Protocol::HTTP`. Features timestamped storage, parallel-safe recording, and flexible store backends.
|
4
4
|
|
5
5
|
[](https://github.com/socketry/async-http-capture/actions?workflow=Test)
|
6
6
|
|
7
7
|
## Features
|
8
8
|
|
9
9
|
- **Pure Protocol::HTTP**: Works directly with Protocol::HTTP objects, no lossy conversions
|
10
|
-
- **
|
10
|
+
- **Timestamped Storage**: Each interaction saved as separate JSON file with timestamp
|
11
11
|
- **Parallel-Safe**: Multiple processes can record simultaneously without conflicts
|
12
12
|
- **Flexible Stores**: Pluggable storage backends (files, console logging, etc.)
|
13
13
|
- **Complete Headers**: Full round-trip serialization including `fields` and `tail`
|
@@ -19,12 +19,14 @@ Please see the [project documentation](https://socketry.github.io/async-http-cap
|
|
19
19
|
|
20
20
|
- [Getting Started](https://socketry.github.io/async-http-capture/guides/getting-started/index) - This guide explains how to get started with `async-http-capture`, a Ruby gem for recording and replaying HTTP requests using Protocol::HTTP.
|
21
21
|
|
22
|
+
- [Falcon Integration](https://socketry.github.io/async-http-capture/guides/falcon-integration/index) - This guide explains how to integrate `async-http-capture` with Falcon web server for recording and replaying HTTP interactions.
|
23
|
+
|
22
24
|
### Basic Recording to Files
|
23
25
|
|
24
26
|
``` ruby
|
25
27
|
require "async/http/capture"
|
26
28
|
|
27
|
-
# Create a store that saves to
|
29
|
+
# Create a store that saves to timestamped files:
|
28
30
|
store = Async::HTTP::Capture::CassetteStore.new("interactions")
|
29
31
|
|
30
32
|
# Create middleware:
|
@@ -84,27 +86,26 @@ response = middleware.call(request)
|
|
84
86
|
- **Stores**: Handle serialization, filtering, persistence, or logging
|
85
87
|
- **Interaction**: Simple data container with lazy Protocol::HTTP object construction
|
86
88
|
|
87
|
-
##
|
89
|
+
## Timestamped Storage
|
88
90
|
|
89
|
-
Each interaction is saved to a file named
|
91
|
+
Each interaction is saved to a file named with timestamp, process ID, and object ID:
|
90
92
|
|
91
|
-
|
92
|
-
├──
|
93
|
-
├──
|
94
|
-
└──
|
93
|
+
recordings/
|
94
|
+
├── 20250821-105406-271633-12345-67890.json # GET /users
|
95
|
+
├── 20250821-105006-257022-12346-67891.json # POST /orders
|
96
|
+
└── 20250820-101234-567890-12347-67892.json # GET /health
|
95
97
|
|
96
98
|
Benefits:
|
97
99
|
|
98
|
-
- **
|
100
|
+
- **Chronological ordering**: Files sorted by timestamp
|
99
101
|
- **Parallel-safe**: Multiple processes can write without conflicts
|
100
|
-
- **
|
101
|
-
- **Git-friendly**: Stable filenames for version control
|
102
|
+
- **Human-readable**: Timestamps are easy to understand
|
102
103
|
|
103
104
|
## Store Implementations
|
104
105
|
|
105
106
|
### CassetteStore
|
106
107
|
|
107
|
-
Saves interactions to
|
108
|
+
Saves interactions to timestamped JSON files in a directory.
|
108
109
|
|
109
110
|
### ConsoleStore
|
110
111
|
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: async-http-capture
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Samuel Williams
|
@@ -23,10 +23,25 @@ dependencies:
|
|
23
23
|
- - "~>"
|
24
24
|
- !ruby/object:Gem::Version
|
25
25
|
version: '0.90'
|
26
|
+
- !ruby/object:Gem::Dependency
|
27
|
+
name: protocol-http
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
29
|
+
requirements:
|
30
|
+
- - ">="
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: 0.53.0
|
33
|
+
type: :runtime
|
34
|
+
prerelease: false
|
35
|
+
version_requirements: !ruby/object:Gem::Requirement
|
36
|
+
requirements:
|
37
|
+
- - ">="
|
38
|
+
- !ruby/object:Gem::Version
|
39
|
+
version: 0.53.0
|
26
40
|
executables: []
|
27
41
|
extensions: []
|
28
42
|
extra_rdoc_files: []
|
29
43
|
files:
|
44
|
+
- context/falcon-integration.md
|
30
45
|
- context/getting-started.md
|
31
46
|
- context/index.yaml
|
32
47
|
- design.md
|
@@ -34,6 +49,7 @@ files:
|
|
34
49
|
- lib/async/http/capture/cassette.rb
|
35
50
|
- lib/async/http/capture/cassette_store.rb
|
36
51
|
- lib/async/http/capture/console_store.rb
|
52
|
+
- lib/async/http/capture/environment.rb
|
37
53
|
- lib/async/http/capture/interaction.rb
|
38
54
|
- lib/async/http/capture/interaction_tracker.rb
|
39
55
|
- lib/async/http/capture/middleware.rb
|