brine-dsl 0.7.0 → 0.8.0

Sign up to get free protection for your applications and to get access to all the features.
data/docs/specs.html CHANGED
@@ -440,6 +440,14 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b
440
440
  <li><a href="#__span_class_feature_name_resource_cleanup_span"><span class="feature name">Resource Cleanup</span></a></li>
441
441
  </ul>
442
442
  </li>
443
+ <li><a href="#_assignment">Assignment</a>
444
+ <ul class="sectlevel2">
445
+ <li><a href="#__span_class_feature_name_a_parameter_span"><span class="feature name">A Parameter</span></a></li>
446
+ <li><a href="#__span_class_feature_name_a_random_string_span"><span class="feature name">A Random String</span></a></li>
447
+ <li><a href="#__span_class_feature_name_a_timestamp_span"><span class="feature name">A Timestamp</span></a></li>
448
+ <li><a href="#__span_class_feature_name_a_value_from_a_response_attribute_span"><span class="feature name">A Value From A Response Attribute.</span></a></li>
449
+ </ul>
450
+ </li>
443
451
  <li><a href="#_selection">Selection</a>
444
452
  <ul class="sectlevel2">
445
453
  <li><a href="#__span_class_feature_name_any_element_span"><span class="feature name">Any Element</span></a></li>
@@ -1065,6 +1073,224 @@ When a resource is created at `/some/path`</code></pre>
1065
1073
  </div>
1066
1074
  </div>
1067
1075
  <div class="sect1">
1076
+ <h2 id="_assignment">Assignment</h2>
1077
+ <div class="sectionbody">
1078
+ <div class="sect2">
1079
+ <h3 id="__span_class_feature_name_a_parameter_span"><span class="feature name">A Parameter</span></h3>
1080
+ <div class="paragraph">
1081
+ <p>An identifier can be assigned the value of the provided parameter.</p>
1082
+ </div>
1083
+ <div class="sect3">
1084
+ <h4 id="__span_class_scenario_name_parameter_assignment_span"><span class="scenario name">Parameter assignment.</span></h4>
1085
+ <div class="ulist step-list">
1086
+ <ul>
1087
+ <li>
1088
+ <p><strong>Given</strong> a file named "features/parameter.feature" with:</p>
1089
+ </li>
1090
+ </ul>
1091
+ </div>
1092
+ <div class="listingblock">
1093
+ <div class="content">
1094
+ <pre class="highlightjs highlight"><code class="language-gherkin" data-lang="gherkin">Feature: Parameter Assignment.
1095
+ Scenario: Simple assignment.
1096
+ Given `foo` is assigned `bar`
1097
+ When the response body is assigned `{{ foo }}`
1098
+ Then the value of the response body is equal to `bar`</code></pre>
1099
+ </div>
1100
+ </div>
1101
+ <div class="ulist">
1102
+ <ul>
1103
+ <li>
1104
+ <p><strong>When</strong> I run <code>cucumber --strict features/parameter.feature</code></p>
1105
+ </li>
1106
+ <li>
1107
+ <p><strong>Then</strong> the output should contain:</p>
1108
+ </li>
1109
+ </ul>
1110
+ </div>
1111
+ <div class="listingblock">
1112
+ <div class="content">
1113
+ <pre class="highlightjs highlight"><code class="language-gherkin" data-lang="gherkin">1 passed</code></pre>
1114
+ </div>
1115
+ </div>
1116
+ <div class="ulist">
1117
+ <ul>
1118
+ <li>
1119
+ <p><strong>And</strong> it should pass</p>
1120
+ </li>
1121
+ </ul>
1122
+ </div>
1123
+ </div>
1124
+ </div>
1125
+ <div class="sect2">
1126
+ <h3 id="__span_class_feature_name_a_random_string_span"><span class="feature name">A Random String</span></h3>
1127
+ <div class="paragraph">
1128
+ <p>An identifier can be assigned a random string with decent entropy.</p>
1129
+ </div>
1130
+ <div class="sect3">
1131
+ <h4 id="__span_class_scenario_name_random_string_assignment_span"><span class="scenario name">Random string assignment.</span></h4>
1132
+ <div class="ulist step-list">
1133
+ <ul>
1134
+ <li>
1135
+ <p><strong>Given</strong> a file named "features/random.feature" with:</p>
1136
+ </li>
1137
+ </ul>
1138
+ </div>
1139
+ <div class="listingblock">
1140
+ <div class="content">
1141
+ <pre class="highlightjs highlight"><code class="language-gherkin" data-lang="gherkin">Feature: Random Assignment.
1142
+ Scenario: Several unique variables.
1143
+ Given `v1` is assigned a random string
1144
+ And `v2` is assigned a random string
1145
+ And `v3` is assigned a random string
1146
+ When the response body is assigned `[ "{{ v1 }}","{{ v2 }}","{{ v3 }}" ]`
1147
+ Then the value of the response body does not have any element that is empty
1148
+ And the value of the response body child `.[0]` is equal to `{{ v1 }}`
1149
+ And the value of the response body children `.[1:2]` does not have any element that is equal to `{{ v1 }}`
1150
+ And the value of the response body child `.[2]` is not equal to `{{ v2 }}`</code></pre>
1151
+ </div>
1152
+ </div>
1153
+ <div class="ulist">
1154
+ <ul>
1155
+ <li>
1156
+ <p><strong>When</strong> I run <code>cucumber --strict features/random.feature</code></p>
1157
+ </li>
1158
+ <li>
1159
+ <p><strong>Then</strong> the output should contain:</p>
1160
+ </li>
1161
+ </ul>
1162
+ </div>
1163
+ <div class="listingblock">
1164
+ <div class="content">
1165
+ <pre class="highlightjs highlight"><code class="language-gherkin" data-lang="gherkin">8 passed</code></pre>
1166
+ </div>
1167
+ </div>
1168
+ <div class="ulist">
1169
+ <ul>
1170
+ <li>
1171
+ <p><strong>And</strong> it should pass</p>
1172
+ </li>
1173
+ </ul>
1174
+ </div>
1175
+ </div>
1176
+ </div>
1177
+ <div class="sect2">
1178
+ <h3 id="__span_class_feature_name_a_timestamp_span"><span class="feature name">A Timestamp</span></h3>
1179
+ <div class="paragraph">
1180
+ <p>An identifier can be assigned a current timestamp.</p>
1181
+ </div>
1182
+ <div class="sect3">
1183
+ <h4 id="__span_class_scenario_name_timestamp_assignment_span"><span class="scenario name">Timestamp assignment.</span></h4>
1184
+ <div class="ulist step-list">
1185
+ <ul>
1186
+ <li>
1187
+ <p><strong>Given</strong> a file named "features/timestamp.feature" with:</p>
1188
+ </li>
1189
+ </ul>
1190
+ </div>
1191
+ <div class="listingblock">
1192
+ <div class="content">
1193
+ <pre class="highlightjs highlight"><code class="language-gherkin" data-lang="gherkin">Feature: Timestamp Assignment.
1194
+ Scenario: Newer than some old date.
1195
+ Given `v1` is assigned a timestamp
1196
+ When the response body is assigned `{{ v1 }}`
1197
+ Then value of the response body is greater than `2018-06-17T12:00:00Z`
1198
+
1199
+ Scenario: Values increase.
1200
+ Given `v1` is assigned a timestamp
1201
+ When the response body is assigned `{{ v1 }}`
1202
+ Then the value of the response body is not empty
1203
+
1204
+ When `v2` is assigned a timestamp
1205
+ And the response body is assigned `{{ v2 }}`
1206
+ Then the value of the response body is greater than or equal to `{{ v1 }}`
1207
+
1208
+ When `v3` is assigned a timestamp
1209
+ And the response body is assigned `{{ v3 }}`
1210
+ Then the value of the response body is greater than or equal to `{{ v1 }}`
1211
+ And the value of the response body is greater than or equal to `{{ v2 }}`</code></pre>
1212
+ </div>
1213
+ </div>
1214
+ <div class="ulist">
1215
+ <ul>
1216
+ <li>
1217
+ <p><strong>When</strong> I run <code>cucumber --strict features/timestamp.feature</code></p>
1218
+ </li>
1219
+ <li>
1220
+ <p><strong>Then</strong> the output should contain:</p>
1221
+ </li>
1222
+ </ul>
1223
+ </div>
1224
+ <div class="listingblock">
1225
+ <div class="content">
1226
+ <pre class="highlightjs highlight"><code class="language-gherkin" data-lang="gherkin">2 passed</code></pre>
1227
+ </div>
1228
+ </div>
1229
+ </div>
1230
+ </div>
1231
+ <div class="sect2">
1232
+ <h3 id="__span_class_feature_name_a_value_from_a_response_attribute_span"><span class="feature name">A Value From A Response Attribute.</span></h3>
1233
+ <div class="paragraph">
1234
+ <p>An identifier can be assigned a value extracted from a response attribute.</p>
1235
+ </div>
1236
+ <div class="sect3">
1237
+ <h4 id="__span_class_scenario_name_assorted_attribute_extractions_span"><span class="scenario name">Assorted attribute extractions.</span></h4>
1238
+ <div class="ulist step-list">
1239
+ <ul>
1240
+ <li>
1241
+ <p><strong>Given</strong> a file named "features/response_attribute.feature" with:</p>
1242
+ </li>
1243
+ </ul>
1244
+ </div>
1245
+ <div class="listingblock">
1246
+ <div class="content">
1247
+ <pre class="highlightjs highlight"><code class="language-gherkin" data-lang="gherkin">Feature: Reponse Attribute Path Assignment.
1248
+ Scenario: Response body.
1249
+ Given the response body is assigned `foo`
1250
+ When `myVar` is assigned the response body
1251
+ And the response body is assigned `{{ myVar }}`
1252
+ Then the value of the response body is equal to `foo`
1253
+
1254
+ Scenario: Response body child.
1255
+ Given the response body is assigned `{"response": "foo"}`
1256
+ When `myVar` is assigned the response body child `.response`
1257
+ And the response body is assigned `{{ myVar }}`
1258
+ Then the value of the response body is equal to `foo`
1259
+
1260
+ Scenario: Response body children.
1261
+ Given the response body is assigned `{"response": "foo"}`
1262
+ When `myVar` is assigned the response body children `.response`
1263
+ And the response body is assigned `{{{ myVar }}}`
1264
+ Then the value of the response body is equal to `["foo"]`</code></pre>
1265
+ </div>
1266
+ </div>
1267
+ <div class="ulist">
1268
+ <ul>
1269
+ <li>
1270
+ <p><strong>When</strong> I run <code>cucumber --strict features/response_attribute.feature</code></p>
1271
+ </li>
1272
+ <li>
1273
+ <p><strong>Then</strong> the output should contain:</p>
1274
+ </li>
1275
+ </ul>
1276
+ </div>
1277
+ <div class="listingblock">
1278
+ <div class="content">
1279
+ <pre class="highlightjs highlight"><code class="language-gherkin" data-lang="gherkin">3 passed</code></pre>
1280
+ </div>
1281
+ </div>
1282
+ <div class="ulist">
1283
+ <ul>
1284
+ <li>
1285
+ <p><strong>And</strong> it should pass</p>
1286
+ </li>
1287
+ </ul>
1288
+ </div>
1289
+ </div>
1290
+ </div>
1291
+ </div>
1292
+ </div>
1293
+ <div class="sect1">
1068
1294
  <h2 id="_selection">Selection</h2>
1069
1295
  <div class="sectionbody">
1070
1296
  <div class="sect2">
@@ -1833,7 +2059,7 @@ Then the value of the response body children `.val` has elements which are all a
1833
2059
  </div>
1834
2060
  <div id="footer">
1835
2061
  <div id="footer-text">
1836
- Last updated 2018-05-30 13:07:46 EDT
2062
+ Last updated 2018-06-17 22:22:57 EDT
1837
2063
  </div>
1838
2064
  </div>
1839
2065
  </body>
@@ -2,16 +2,17 @@
2
2
  Matt Whipple <http://github.com/mwhipple[@mwhipple]>
3
3
  :description: Cookbook for the Brine REST Testing DSL
4
4
  :keywords: Brine, Cucumber, REST, DSL
5
+ :gh_repo: https://github.com/brightcove/brine
5
6
 
6
7
  == Overview
7
8
 
8
9
  The following are some recipes to address issues which may arise while using
9
- Brine but are not considered part of Brine's responsibility (at least presently).
10
- Many of these are focused on simplicity and ease rather than robustness or elegance,
11
- and modification to address specific cases should be expected.
10
+ Brine but are not considered part of Brine's (current) responsibility.
11
+ Many of these are focused on simplicity and ease rather than robustness or
12
+ elegance, and modification to address specific cases should be expected.
12
13
 
13
- Many of these will also be platform specific and be subject to further organization
14
- once platforms beyond ruby are supported.
14
+ Many of these will also be platform specific and be subject to further
15
+ organization once platforms beyond ruby are supported.
15
16
 
16
17
  == Intercepting of HTTP Calls
17
18
 
@@ -27,11 +28,13 @@ of anything that belongs in the specification.
27
28
  This can be done through the registration of custom Faraday middleware on
28
29
  the Brine client. All of the registered middleware attaches
29
30
  to the generated connection through an array of functions in
30
- https://github.com/brightcove/brine/blob/master/lib/brine/requester.rb[requester.rb].
31
- Each function accepts the connection object as its single parameter. The array of functions
32
- is exposed as `connection_handlers` and can be modified through getting that property
33
- from either the default `World` (for the default client) or from an appropriate `ClientBuilder`
34
- if creating custom clients and mutating it accordingly (setting the reference is not suported).
31
+ {gh_repo}/blob/master/lib/brine/requester.rb[requester.rb].
32
+ Each function accepts the connection object as its single parameter.
33
+ The array of functions is exposed as `connection_handlers` and can be modified
34
+ through getting that property from either the default `World`
35
+ (for the default client) or from an appropriate `ClientBuilder`
36
+ if creating custom clients and mutating it accordingly (setting the reference
37
+ is not suported).
35
38
  You could just build your own client without the facilities provided by Brine.
36
39
 
37
40
  === Recipe
@@ -78,66 +81,80 @@ additional overhead to provision a new account. When testing a secured
78
81
  system valid accounts are needed for representative testing, but provisioning
79
82
  a new account may be difficult or outside the scope of the system that is being
80
83
  actively tested. If tested functionality involves enacting account-wide changes
81
- and the number of accounts is limited, then that is likely to unfortunately prevent
82
- complete test isolation.
84
+ and the number of accounts is limited, then that is likely to unfortunately
85
+ prevent complete test isolation.
83
86
 
84
87
  === Solution
85
88
 
86
- In such cases a standard solution is to designate certain resources to be reused for
87
- certain tests. These are analogous to the concept of "fixtures" in some test suites
88
- though there may be slight differences in implementation and reliance on them.
89
- Here the term "assurances" is used..primarily because it starts with `a` which lends
90
- itself to relevant files being listed towards the beginning alphabetically in a given directory.
89
+ In such cases a standard solution is to designate certain resources to be
90
+ reused for certain tests. These are analogous to the concept of "fixtures" in
91
+ some test suites though there may be slight differences in implementation and
92
+ reliance on them. Here the term "assurances" is used...primarily because it
93
+ starts with `a` which lends itself to relevant files being listed towards the
94
+ beginning alphabetically in a given directory.
91
95
 
92
96
  The goal of assurances is to specify conditions which are expected before other
93
- tests are to be run. Preferably the dependent tests should also explicitly declare the
94
- dependency but a significant solution for that is not established. Assurances therefore
95
- validate that preconditions are met; ideally if such preconditions can be established
96
- idempotently then the assurances can do so before the validation.
97
+ tests are to be run. Preferably the dependent tests should also explicitly
98
+ declare the dependency but a significant solution for that is not established.
99
+ Assurances therefore validate that preconditions are met; ideally if such
100
+ preconditions can be established idempotently then the assurances can do so
101
+ before the validation.
97
102
 
98
103
  ==== Assurances are NOT Tests
99
104
 
100
- **Assurances validate a state which is desired to be consistently retained within the
101
- system rather than being changed**. This means that they should _not_ be used for tests
102
- as that would require state changes, nor should they clean up after themselves (as that
103
- would also imply a state change). If assurances are configured for a system which should
104
- also be tested, then appropriate tests should exist (including those that may validate any
105
- behavior relied upon by the assurance).
105
+ **Assurances validate a state which is desired to be consistently retained
106
+ within the system rather than being changed**. This means that they should _not_
107
+ be used for tests as that would require state changes, nor should they clean up
108
+ after themselves (as that would also imply a state change). If assurances are
109
+ configured for a system which should also be tested, then appropriate tests
110
+ should exist (including those that may validate any behavior relied upon by
111
+ the assurance).
106
112
 
107
113
  ==== Consequences
108
- As mentioned previously assertions help in cases where tests cannot be fully isolated,
109
- and therefore some known state must be established and reused across tests (and such state
110
- should *not* change). A practical reason for this is to allow for overlapping test executions.
111
- If tests are not fully isolated, state is being changed, and test runs overlap, then tests
112
- may fail non-deterministically due to one test run pulling the state out from another. This
113
- in the simplest form can be a nuisance but it also effectively precludes the ability to speed
114
- up test runs through the use of parallelism/asynchronicity.
114
+
115
+ As mentioned previously assertions help in cases where tests cannot be fully
116
+ isolated, and therefore some known state must be established and reused across
117
+ tests (and such state should *not* change). A practical reason for this is to
118
+ allow for overlapping test executions.
119
+ If tests are not fully isolated, state is being changed, and test runs overlap,
120
+ then tests may fail non-deterministically due to one test run pulling the state
121
+ out from another. This in the simplest form can be a nuisance but it also
122
+ effectively precludes the ability to speed up test runs through the use of
123
+ parallelism/asynchronicity.
115
124
 
116
125
  _TODO: Enumerate drawbacks_
117
126
 
118
127
  === Recipe
119
- This can be done using standard cucumber tags. Assurances can be defined in designated
120
- `assure_*.feature` files where each Feature is appropriately tagged:
128
+
129
+ This can be done using standard cucumber tags. Assurances can be defined in
130
+ designated `assure_*.feature` files where each Feature is appropriately tagged:
131
+
121
132
  [source,gherkin]
122
133
  ----
123
134
  @assure
124
135
  Feature: Some preconditions are verified...
125
136
  ----
137
+
126
138
  And then a Rake task is added to run those tagged features:
139
+
127
140
  [source,ruby]
128
141
  ----
129
142
  Cucumber::Rake::Task.new(:assure) do |t|
130
143
  t.cucumber_opts = "#{ENV['CUCUMBER_OPTS']} --tags @assure"
131
144
  end
132
145
  ----
146
+
133
147
  The Rake task that runs the other tests then depends on that task:
148
+
134
149
  [source,ruby]
135
150
  ----
136
151
  task :invoke_cuke => [:assure] do
137
152
  #Run cucumber, potentially in parallel and likely with --tags `@assure`
138
153
  end
139
154
  ----
140
- This approach tests preconditions and will avoid running the rest of the tests if they
141
- are not (relying on standard Rake behavior). The assurances can also be run with different
142
- Cucumber behavior so that the full test suite can be more stochastic
143
- (randomized/non-serialized) while the assurances can be more controlled.
155
+
156
+ This approach tests preconditions and will avoid running the rest of the tests
157
+ if they are not (relying on standard Rake behavior). The assurances can also be
158
+ run with different Cucumber behavior so that the full test suite can be more
159
+ stochastic (randomized/non-serialized) while the assurances can be more
160
+ controlled.