robotframework-openapitools 0.1.2__py3-none-any.whl → 0.2.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,1512 +1,1570 @@
1
- """
2
- # OpenApiLibCore for Robot Framework
3
-
4
- The OpenApiLibCore library is a utility library that is meant to simplify creation
5
- of other Robot Framework libraries for API testing based on the information in
6
- an OpenAPI document (also known as Swagger document).
7
- This document explains how to use the OpenApiLibCore library.
8
-
9
- My RoboCon 2022 talk about OpenApiDriver and OpenApiLibCore can be found
10
- [here](https://www.youtube.com/watch?v=7YWZEHxk9Ps)
11
-
12
- For more information about Robot Framework, see http://robotframework.org.
13
-
14
- ---
15
-
16
- > Note: OpenApiLibCore is still being developed so there are currently
17
- restrictions / limitations that you may encounter when using this library to run
18
- tests against an API. See [Limitations](#limitations) for details.
19
-
20
- ---
21
-
22
- ## Installation
23
-
24
- If you already have Python >= 3.8 with pip installed, you can simply run:
25
-
26
- `pip install --upgrade robotframework-openapi-libcore`
27
-
28
- ---
29
-
30
- ## OpenAPI (aka Swagger)
31
-
32
- The OpenAPI Specification (OAS) defines a standard, language-agnostic interface
33
- to RESTful APIs, see https://swagger.io/specification/
34
-
35
- The OpenApiLibCore implements a number of Robot Framework keywords that make it
36
- easy to interact with an OpenAPI implementation by using the information in the
37
- openapi document (Swagger file), for examply by automatic generation of valid values
38
- for requests based on the schema information in the document.
39
-
40
- > Note: OpenApiLibCore is designed for APIs based on the OAS v3
41
- The library has not been tested for APIs based on the OAS v2.
42
-
43
- ---
44
-
45
- ## Getting started
46
-
47
- Before trying to use the keywords exposed by OpenApiLibCore on the target API
48
- it's recommended to first ensure that the openapi document for the API is valid
49
- under the OpenAPI Specification.
50
-
51
- This can be done using the command line interface of a package that is installed as
52
- a prerequisite for OpenApiLibCore.
53
- Both a local openapi.json or openapi.yaml file or one hosted by the API server
54
- can be checked using the `prance validate <reference_to_file>` shell command:
55
-
56
- ```shell
57
- prance validate --backend=openapi-spec-validator http://localhost:8000/openapi.json
58
- Processing "http://localhost:8000/openapi.json"...
59
- -> Resolving external references.
60
- Validates OK as OpenAPI 3.0.2!
61
-
62
- prance validate --backend=openapi-spec-validator /tests/files/petstore_openapi.yaml
63
- Processing "/tests/files/petstore_openapi.yaml"...
64
- -> Resolving external references.
65
- Validates OK as OpenAPI 3.0.2!
66
- ```
67
-
68
- You'll have to change the url or file reference to the location of the openapi
69
- document for your API.
70
-
71
- > Note: Although recursion is technically allowed under the OAS, tool support is limited
72
- and changing the OAS to not use recursion is recommended.
73
- OpenApiLibCore has limited support for parsing OpenAPI documents with
74
- recursion in them. See the `recursion_limit` and `recursion_default` parameters.
75
-
76
- If the openapi document passes this validation, the next step is trying to do a test
77
- run with a minimal test suite.
78
- The example below can be used, with `source`, `origin` and 'endpoint' altered to
79
- fit your situation.
80
-
81
- ``` robotframework
82
- *** Settings ***
83
- Library OpenApiLibCore
84
- ... source=http://localhost:8000/openapi.json
85
- ... origin=http://localhost:8000
86
-
87
- *** Test Cases ***
88
- Getting Started
89
- ${url}= Get Valid Url endpoint=/employees/{employee_id} method=get
90
-
91
- ```
92
-
93
- Running the above suite for the first time may result in an error / failed test.
94
- You should look at the Robot Framework `log.html` to determine the reasons
95
- for the failing tests.
96
- Depending on the reasons for the failures, different solutions are possible.
97
-
98
- Details about the OpenApiLibCore library parameters and keywords that you may need can be found
99
- [here](https://marketsquare.github.io/robotframework-openapi-libcore/openapi_libcore.html).
100
-
101
- The OpenApiLibCore also support handling of relations between resources within the scope
102
- of the API being validated as well as handling dependencies on resources outside the
103
- scope of the API. In addition there is support for handling restrictions on the values
104
- of parameters and properties.
105
-
106
- Details about the `mappings_path` variable usage can be found
107
- [here](https://marketsquare.github.io/robotframework-openapi-libcore/advanced_use.html).
108
-
109
- ---
110
-
111
- ## Limitations
112
-
113
- There are currently a number of limitations to supported API structures, supported
114
- data types and properties. The following list details the most important ones:
115
- - Only JSON request and response bodies are supported.
116
- - No support for per-endpoint authorization levels.
117
- - Parsing of OAS 3.1 documents is supported by the parsing tools, but runtime behavior is untested.
118
-
119
- """
120
-
121
- import json as _json
122
- import re
123
- import sys
124
- from copy import deepcopy
125
- from dataclasses import Field, dataclass, field, make_dataclass
126
- from functools import cached_property
127
- from itertools import zip_longest
128
- from logging import getLogger
129
- from pathlib import Path
130
- from random import choice
131
- from typing import Any, Dict, List, Optional, Set, Tuple, Type, Union
132
- from uuid import uuid4
133
-
134
- from openapi_core import Spec, validate_response
135
- from openapi_core.contrib.requests import (
136
- RequestsOpenAPIRequest,
137
- RequestsOpenAPIResponse,
138
- )
139
- from prance import ResolvingParser, ValidationError
140
- from prance.util.url import ResolutionError
141
- from requests import Response, Session
142
- from requests.auth import AuthBase, HTTPBasicAuth
143
- from requests.cookies import RequestsCookieJar as CookieJar
144
- from robot.api.deco import keyword, library
145
- from robot.libraries.BuiltIn import BuiltIn
146
-
147
- from OpenApiLibCore import value_utils
148
- from OpenApiLibCore.dto_base import (
149
- NOT_SET,
150
- Dto,
151
- IdDependency,
152
- IdReference,
153
- PathPropertiesConstraint,
154
- PropertyValueConstraint,
155
- Relation,
156
- UniquePropertyValueConstraint,
157
- resolve_schema,
158
- )
159
- from OpenApiLibCore.dto_utils import (
160
- DEFAULT_ID_PROPERTY_NAME,
161
- DefaultDto,
162
- get_dto_class,
163
- get_id_property_name,
164
- )
165
- from OpenApiLibCore.oas_cache import PARSER_CACHE
166
- from OpenApiLibCore.value_utils import FAKE, IGNORE, JSON
167
-
168
- run_keyword = BuiltIn().run_keyword
169
-
170
- logger = getLogger(__name__)
171
-
172
-
173
- def get_safe_key(key: str) -> str:
174
- """
175
- Helper function to convert a valid JSON property name to a string that can be used
176
- as a Python variable or function / method name.
177
- """
178
- key = key.replace("-", "_")
179
- key = key.replace("@", "_")
180
- if key[0].isdigit():
181
- key = f"_{key}"
182
- return key
183
-
184
-
185
- @dataclass
186
- class RequestValues:
187
- """Helper class to hold parameter values needed to make a request."""
188
-
189
- url: str
190
- method: str
191
- params: Optional[Dict[str, Any]]
192
- headers: Optional[Dict[str, str]]
193
- json_data: Optional[Dict[str, Any]]
194
-
195
-
196
- @dataclass
197
- class RequestData:
198
- """Helper class to manage parameters used when making requests."""
199
-
200
- dto: Union[Dto, DefaultDto] = field(default_factory=DefaultDto)
201
- dto_schema: Dict[str, Any] = field(default_factory=dict)
202
- parameters: List[Dict[str, Any]] = field(default_factory=list)
203
- params: Dict[str, Any] = field(default_factory=dict)
204
- headers: Dict[str, Any] = field(default_factory=dict)
205
- has_body: bool = True
206
-
207
- def __post_init__(self) -> None:
208
- # prevent modification by reference
209
- self.dto_schema = deepcopy(self.dto_schema)
210
- self.parameters = deepcopy(self.parameters)
211
- self.params = deepcopy(self.params)
212
- self.headers = deepcopy(self.headers)
213
-
214
- @property
215
- def has_optional_properties(self) -> bool:
216
- """Whether or not the dto data (json data) contains optional properties."""
217
-
218
- def is_required_property(property_name: str) -> bool:
219
- return property_name in self.dto_schema.get("required", [])
220
-
221
- properties = (self.dto.as_dict()).keys()
222
- return not all(map(is_required_property, properties))
223
-
224
- @property
225
- def has_optional_params(self) -> bool:
226
- """Whether or not any of the query parameters are optional."""
227
-
228
- def is_optional_param(query_param: str) -> bool:
229
- optional_params = [
230
- p.get("name")
231
- for p in self.parameters
232
- if p.get("in") == "query" and not p.get("required")
233
- ]
234
- return query_param in optional_params
235
-
236
- return any(map(is_optional_param, self.params))
237
-
238
- @cached_property
239
- def params_that_can_be_invalidated(self) -> Set[str]:
240
- """
241
- The query parameters that can be invalidated by violating data
242
- restrictions, data type or by not providing them in a request.
243
- """
244
- result = set()
245
- params = [h for h in self.parameters if h.get("in") == "query"]
246
- for param in params:
247
- # required params can be omitted to invalidate a request
248
- if param["required"]:
249
- result.add(param["name"])
250
- continue
251
-
252
- schema = resolve_schema(param["schema"])
253
- if schema.get("type", None):
254
- param_types = [schema]
255
- else:
256
- param_types = schema["types"]
257
- for param_type in param_types:
258
- # any basic non-string type except "null" can be invalidated by
259
- # replacing it with a string
260
- if param_type["type"] not in ["string", "array", "object", "null"]:
261
- result.add(param["name"])
262
- continue
263
- # enums, strings and arrays with boundaries can be invalidated
264
- if set(param_type.keys()).intersection(
265
- {
266
- "enum",
267
- "minLength",
268
- "maxLength",
269
- "minItems",
270
- "maxItems",
271
- }
272
- ):
273
- result.add(param["name"])
274
- continue
275
- # an array of basic non-string type can be invalidated by replacing the
276
- # items in the array with strings
277
- if param_type["type"] == "array" and param_type["items"][
278
- "type"
279
- ] not in [
280
- "string",
281
- "array",
282
- "object",
283
- "null",
284
- ]:
285
- result.add(param["name"])
286
- return result
287
-
288
- @property
289
- def has_optional_headers(self) -> bool:
290
- """Whether or not any of the headers are optional."""
291
-
292
- def is_optional_header(header: str) -> bool:
293
- optional_headers = [
294
- p.get("name")
295
- for p in self.parameters
296
- if p.get("in") == "header" and not p.get("required")
297
- ]
298
- return header in optional_headers
299
-
300
- return any(map(is_optional_header, self.headers))
301
-
302
- @cached_property
303
- def headers_that_can_be_invalidated(self) -> Set[str]:
304
- """
305
- The header parameters that can be invalidated by violating data
306
- restrictions or by not providing them in a request.
307
- """
308
- result = set()
309
- headers = [h for h in self.parameters if h.get("in") == "header"]
310
- for header in headers:
311
- # required headers can be omitted to invalidate a request
312
- if header["required"]:
313
- result.add(header["name"])
314
- continue
315
-
316
- schema = resolve_schema(header["schema"])
317
- if schema.get("type", None):
318
- header_types = [schema]
319
- else:
320
- header_types = schema["types"]
321
- for header_type in header_types:
322
- # any basic non-string type except "null" can be invalidated by
323
- # replacing it with a string
324
- if header_type["type"] not in ["string", "array", "object", "null"]:
325
- result.add(header["name"])
326
- continue
327
- # enums, strings and arrays with boundaries can be invalidated
328
- if set(header_type.keys()).intersection(
329
- {
330
- "enum",
331
- "minLength",
332
- "maxLength",
333
- "minItems",
334
- "maxItems",
335
- }
336
- ):
337
- result.add(header["name"])
338
- continue
339
- # an array of basic non-string type can be invalidated by replacing the
340
- # items in the array with strings
341
- if header_type["type"] == "array" and header_type["items"][
342
- "type"
343
- ] not in [
344
- "string",
345
- "array",
346
- "object",
347
- "null",
348
- ]:
349
- result.add(header["name"])
350
- return result
351
-
352
- def get_required_properties_dict(self) -> Dict[str, Any]:
353
- """Get the json-compatible dto data containing only the required properties."""
354
- required_properties = self.dto_schema.get("required", [])
355
- required_properties_dict: Dict[str, Any] = {}
356
- for key, value in (self.dto.as_dict()).items():
357
- if key in required_properties:
358
- required_properties_dict[key] = value
359
- return required_properties_dict
360
-
361
- def get_required_params(self) -> Dict[str, str]:
362
- """Get the params dict containing only the required query parameters."""
363
- required_parameters = [
364
- p.get("name") for p in self.parameters if p.get("required")
365
- ]
366
- return {k: v for k, v in self.params.items() if k in required_parameters}
367
-
368
- def get_required_headers(self) -> Dict[str, str]:
369
- """Get the headers dict containing only the required headers."""
370
- required_parameters = [
371
- p.get("name") for p in self.parameters if p.get("required")
372
- ]
373
- return {k: v for k, v in self.headers.items() if k in required_parameters}
374
-
375
-
376
- @library(scope="TEST SUITE", doc_format="ROBOT")
377
- class OpenApiLibCore: # pylint: disable=too-many-instance-attributes
378
- """
379
- Main class providing the keywords and core logic to interact with an OpenAPI server.
380
-
381
- Visit the [https://github.com/MarketSquare/robotframework-openapi-libcore | library page]
382
- for an introduction.
383
- """
384
-
385
- def __init__( # pylint: disable=too-many-arguments, too-many-locals, dangerous-default-value
386
- self,
387
- source: str,
388
- origin: str = "",
389
- base_path: str = "",
390
- mappings_path: Union[str, Path] = "",
391
- invalid_property_default_response: int = 422,
392
- default_id_property_name: str = "id",
393
- faker_locale: Optional[Union[str, List[str]]] = None,
394
- recursion_limit: int = 1,
395
- recursion_default: Any = {},
396
- username: str = "",
397
- password: str = "",
398
- security_token: str = "",
399
- auth: Optional[AuthBase] = None,
400
- cert: Optional[Union[str, Tuple[str, str]]] = None,
401
- verify_tls: Optional[Union[bool, str]] = True,
402
- extra_headers: Optional[Dict[str, str]] = None,
403
- cookies: Optional[Union[Dict[str, str], CookieJar]] = None,
404
- proxies: Optional[Dict[str, str]] = None,
405
- ) -> None:
406
- """
407
- == Base parameters ==
408
-
409
- === source ===
410
- An absolute path to an openapi.json or openapi.yaml file or an url to such a file.
411
-
412
- === origin ===
413
- The server (and port) of the target server. E.g. ``https://localhost:8000``
414
-
415
- === base_path ===
416
- The routing between ``origin`` and the endpoints as found in the ``paths``
417
- section in the openapi document.
418
- E.g. ``/petshop/v2``.
419
-
420
- == API-specific configurations ==
421
-
422
- === mappings_path ===
423
- See [https://marketsquare.github.io/robotframework-openapi-libcore/advanced_use.html | this page]
424
- for an in-depth explanation.
425
-
426
- === invalid_property_default_response ===
427
- The default response code for requests with a JSON body that does not comply
428
- with the schema.
429
- Example: a value outside the specified range or a string value
430
- for a property defined as integer in the schema.
431
-
432
- === default_id_property_name ===
433
- The default name for the property that identifies a resource (i.e. a unique
434
- entity) within the API.
435
- The default value for this property name is ``id``.
436
- If the target API uses a different name for all the resources within the API,
437
- you can configure it globally using this property.
438
-
439
- If different property names are used for the unique identifier for different
440
- types of resources, an ``ID_MAPPING`` can be implemented using the ``mappings_path``.
441
-
442
- === faker_locale ===
443
- A locale string or list of locale strings to pass to the Faker library to be
444
- used in generation of string data for supported format types.
445
-
446
- == Parsing parameters ==
447
-
448
- === recursion_limit ===
449
- The recursion depth to which to fully parse recursive references before the
450
- `recursion_default` is used to end the recursion.
451
-
452
- === recursion_default ===
453
- The value that is used instead of the referenced schema when the
454
- `recursion_limit` has been reached.
455
- The default `{}` represents an empty object in JSON.
456
- Depending on schema definitions, this may cause schema validation errors.
457
- If this is the case, 'None' (``${NONE}`` in Robot Framework) or an empty list
458
- can be tried as an alternative.
459
-
460
- == Security-related parameters ==
461
- _Note: these parameters are equivalent to those in the ``requests`` library._
462
-
463
- === username ===
464
- The username to be used for Basic Authentication.
465
-
466
- === password ===
467
- The password to be used for Basic Authentication.
468
-
469
- === security_token ===
470
- The token to be used for token based security using the ``Authorization`` header.
471
-
472
- === auth ===
473
- A [https://requests.readthedocs.io/en/latest/api/#authentication | requests ``AuthBase`` instance]
474
- to be used for authentication instead of the ``username`` and ``password``.
475
-
476
- === cert ===
477
- The SSL certificate to use with all requests.
478
- If string: the path to ssl client cert file (.pem).
479
- If tuple: the ('cert', 'key') pair.
480
-
481
- === verify_tls ===
482
- Whether or not to verify the TLS / SSL certificate of the server.
483
- If boolean: whether or not to verify the server TLS certificate.
484
- If string: path to a CA bundle to use for verification.
485
-
486
- === extra_headers ===
487
- A dictionary with extra / custom headers that will be send with every request.
488
- This parameter can be used to send headers that are not documented in the
489
- openapi document or to provide an API-key.
490
-
491
- === cookies ===
492
- A dictionary or
493
- [https://docs.python.org/3/library/http.cookiejar.html#http.cookiejar.CookieJar | CookieJar object]
494
- to send with all requests.
495
-
496
- === proxies ===
497
- A dictionary of 'protocol': 'proxy url' to use for all requests.
498
- """
499
- self._source = source
500
- self._origin = origin
501
- self._base_path = base_path
502
- self._recursion_limit = recursion_limit
503
- self._recursion_default = recursion_default
504
- self.session = Session()
505
- # only username and password, security_token or auth object should be provided
506
- # if multiple are provided, username and password take precedence
507
- self.security_token = security_token
508
- self.auth = auth
509
- if username and password:
510
- self.auth = HTTPBasicAuth(username, password)
511
- # Robot Framework does not allow users to create tuples and requests
512
- # does not accept lists, so perform the conversion here
513
- if isinstance(cert, list):
514
- cert = tuple(cert)
515
- self.cert = cert
516
- self.verify = verify_tls
517
- self.extra_headers = extra_headers
518
- self.cookies = cookies
519
- self.proxies = proxies
520
- self.invalid_property_default_response = invalid_property_default_response
521
- if mappings_path and str(mappings_path) != ".":
522
- mappings_path = Path(mappings_path)
523
- if not mappings_path.is_file():
524
- logger.warning(
525
- f"mappings_path '{mappings_path}' is not a Python module."
526
- )
527
- # intermediate variable to ensure path.append is possible so we'll never
528
- # path.pop a location that we didn't append
529
- mappings_folder = str(mappings_path.parent)
530
- sys.path.append(mappings_folder)
531
- mappings_module_name = mappings_path.stem
532
- self.get_dto_class = get_dto_class(
533
- mappings_module_name=mappings_module_name
534
- )
535
- self.get_id_property_name = get_id_property_name(
536
- mappings_module_name=mappings_module_name
537
- )
538
- sys.path.pop()
539
- else:
540
- self.get_dto_class = get_dto_class(mappings_module_name="no mapping")
541
- self.get_id_property_name = get_id_property_name(
542
- mappings_module_name="no mapping"
543
- )
544
- if faker_locale:
545
- FAKE.set_locale(locale=faker_locale)
546
- # update the globally available DEFAULT_ID_PROPERTY_NAME to the provided value
547
- DEFAULT_ID_PROPERTY_NAME.id_property_name = default_id_property_name
548
-
549
- @property
550
- def origin(self) -> str:
551
- return self._origin
552
-
553
- @keyword
554
- def set_origin(self, origin: str) -> None:
555
- """
556
- Update the `origin` after the library is imported.
557
-
558
- This can be done during the `Suite setup` when using DataDriver in situations
559
- where the OpenAPI document is available on disk but the target host address is
560
- not known before the test starts.
561
-
562
- In combination with OpenApiLibCore, the `origin` can be used at any point to
563
- target another server that hosts an API that complies to the same OAS.
564
- """
565
- self._origin = origin
566
-
567
- @property
568
- def base_url(self) -> str:
569
- return f"{self.origin}{self._base_path}"
570
-
571
- @cached_property
572
- def validation_spec(self) -> Spec:
573
- return Spec.from_dict(self.openapi_spec)
574
-
575
- @property
576
- def openapi_spec(self) -> Dict[str, Any]:
577
- """Return a deepcopy of the parsed openapi document."""
578
- # protect the parsed openapi spec from being mutated by reference
579
- return deepcopy(self._openapi_spec)
580
-
581
- @cached_property
582
- def _openapi_spec(self) -> Dict[str, Any]:
583
- parser = self._load_parser()
584
- return parser.specification
585
-
586
- def read_paths(self) -> Dict[str, Any]:
587
- return self.openapi_spec["paths"]
588
-
589
- def _load_parser(self) -> ResolvingParser:
590
- try:
591
-
592
- def recursion_limit_handler(
593
- limit: int, refstring: str, recursions: Any
594
- ) -> Any:
595
- return self._recursion_default
596
-
597
- # Since parsing of the OAS and creating the Spec can take a long time,
598
- # they are cached. This is done by storing them in an imported module that
599
- # will have a global scope due to how the Python import system works. This
600
- # ensures that in a Suite of Suites where multiple Suites use the same
601
- # `source`, that OAS is only parsed / loaded once.
602
- parser = PARSER_CACHE.get(self._source, None)
603
- if parser is None:
604
- parser = ResolvingParser(
605
- self._source,
606
- backend="openapi-spec-validator",
607
- recursion_limit=self._recursion_limit,
608
- recursion_limit_handler=recursion_limit_handler,
609
- )
610
-
611
- if parser.specification is None: # pragma: no cover
612
- BuiltIn().fatal_error(
613
- "Source was loaded, but no specification was present after parsing."
614
- )
615
-
616
- PARSER_CACHE[self._source] = parser
617
-
618
- return parser
619
-
620
- except ResolutionError as exception:
621
- BuiltIn().fatal_error(
622
- f"ResolutionError while trying to load openapi spec: {exception}"
623
- )
624
- except ValidationError as exception:
625
- BuiltIn().fatal_error(
626
- f"ValidationError while trying to load openapi spec: {exception}"
627
- )
628
-
629
- def validate_response_vs_spec(
630
- self, request: RequestsOpenAPIRequest, response: RequestsOpenAPIResponse
631
- ) -> None:
632
- """
633
- Validate the reponse for a given request against the OpenAPI Spec that is
634
- loaded during library initialization.
635
- """
636
- _ = validate_response(
637
- spec=self.validation_spec,
638
- request=request,
639
- response=response,
640
- )
641
-
642
- @keyword
643
- def get_valid_url(self, endpoint: str, method: str) -> str:
644
- """
645
- This keyword returns a valid url for the given `endpoint` and `method`.
646
-
647
- If the `endpoint` contains path parameters the Get Valid Id For Endpoint
648
- keyword will be executed to retrieve valid ids for the path parameters.
649
-
650
- > Note: if valid ids cannot be retrieved within the scope of the API, the
651
- `PathPropertiesConstraint` Relation can be used. More information can be found
652
- [https://marketsquare.github.io/robotframework-openapi-libcore/advanced_use.html | here].
653
- """
654
- method = method.lower()
655
- try:
656
- # endpoint can be partially resolved or provided by a PathPropertiesConstraint
657
- parametrized_endpoint = self.get_parametrized_endpoint(endpoint=endpoint)
658
- _ = self.openapi_spec["paths"][parametrized_endpoint]
659
- except KeyError:
660
- raise ValueError(
661
- f"{endpoint} not found in paths section of the OpenAPI document."
662
- ) from None
663
- dto_class = self.get_dto_class(endpoint=endpoint, method=method)
664
- relations = dto_class.get_relations()
665
- paths = [p.path for p in relations if isinstance(p, PathPropertiesConstraint)]
666
- if paths:
667
- url = f"{self.base_url}{choice(paths)}"
668
- return url
669
- endpoint_parts = list(endpoint.split("/"))
670
- for index, part in enumerate(endpoint_parts):
671
- if part.startswith("{") and part.endswith("}"):
672
- type_endpoint_parts = endpoint_parts[slice(index)]
673
- type_endpoint = "/".join(type_endpoint_parts)
674
- existing_id: Union[str, int, float] = run_keyword(
675
- "get_valid_id_for_endpoint", type_endpoint, method
676
- )
677
- endpoint_parts[index] = str(existing_id)
678
- resolved_endpoint = "/".join(endpoint_parts)
679
- url = f"{self.base_url}{resolved_endpoint}"
680
- return url
681
-
682
- @keyword
683
- def get_valid_id_for_endpoint(
684
- self, endpoint: str, method: str
685
- ) -> Union[str, int, float]:
686
- """
687
- Support keyword that returns the `id` for an existing resource at `endpoint`.
688
-
689
- To prevent resource conflicts with other test cases, a new resource is created
690
- (POST) if possible.
691
- """
692
-
693
- def dummy_transformer(
694
- valid_id: Union[str, int, float]
695
- ) -> Union[str, int, float]:
696
- return valid_id
697
-
698
- method = method.lower()
699
- url: str = run_keyword("get_valid_url", endpoint, method)
700
- # Try to create a new resource to prevent conflicts caused by
701
- # operations performed on the same resource by other test cases
702
- request_data = self.get_request_data(endpoint=endpoint, method="post")
703
-
704
- response: Response = run_keyword(
705
- "authorized_request",
706
- url,
707
- "post",
708
- request_data.get_required_params(),
709
- request_data.get_required_headers(),
710
- request_data.get_required_properties_dict(),
711
- )
712
-
713
- # determine the id property name for this path and whether or not a transformer is used
714
- mapping = self.get_id_property_name(endpoint=endpoint)
715
- if isinstance(mapping, str):
716
- id_property = mapping
717
- # set the transformer to a dummy callable that returns the original value so
718
- # the transformer can be applied on any returned id
719
- id_transformer = dummy_transformer
720
- else:
721
- id_property, id_transformer = mapping
722
-
723
- if not response.ok:
724
- # If a new resource cannot be created using POST, try to retrieve a
725
- # valid id using a GET request.
726
- try:
727
- valid_id = choice(run_keyword("get_ids_from_url", url))
728
- return id_transformer(valid_id)
729
- except Exception as exception:
730
- raise AssertionError(
731
- f"Failed to get a valid id using GET on {url}"
732
- ) from exception
733
-
734
- response_data = response.json()
735
- if prepared_body := response.request.body:
736
- if isinstance(prepared_body, bytes):
737
- send_json = _json.loads(prepared_body.decode("UTF-8"))
738
- else:
739
- send_json = _json.loads(prepared_body)
740
- else:
741
- send_json = None
742
-
743
- # no support for retrieving an id from an array returned on a POST request
744
- if isinstance(response_data, list):
745
- raise NotImplementedError(
746
- f"Unexpected response body for POST request: expected an object but "
747
- f"received an array ({response_data})"
748
- )
749
-
750
- # POST on /resource_type/{id}/array_item/ will return the updated {id} resource
751
- # instead of a newly created resource. In this case, the send_json must be
752
- # in the array of the 'array_item' property on {id}
753
- send_path: str = response.request.path_url
754
- response_href: Optional[str] = response_data.get("href", None)
755
- if response_href and (send_path not in response_href) and send_json:
756
- try:
757
- property_to_check = send_path.replace(response_href, "")[1:]
758
- item_list: List[Dict[str, Any]] = response_data[property_to_check]
759
- # Use the (mandatory) id to get the POSTed resource from the list
760
- [valid_id] = [
761
- item[id_property]
762
- for item in item_list
763
- if item[id_property] == send_json[id_property]
764
- ]
765
- except Exception as exception:
766
- raise AssertionError(
767
- f"Failed to get a valid id from {response_href}"
768
- ) from exception
769
- else:
770
- try:
771
- valid_id = response_data[id_property]
772
- except KeyError:
773
- raise AssertionError(
774
- f"Failed to get a valid id from {response_data}"
775
- ) from None
776
- return id_transformer(valid_id)
777
-
778
- @keyword
779
- def get_ids_from_url(self, url: str) -> List[str]:
780
- """
781
- Perform a GET request on the `url` and return the list of resource
782
- `ids` from the response.
783
- """
784
- endpoint = self.get_parameterized_endpoint_from_url(url)
785
- request_data = self.get_request_data(endpoint=endpoint, method="get")
786
- response = run_keyword(
787
- "authorized_request",
788
- url,
789
- "get",
790
- request_data.get_required_params(),
791
- request_data.get_required_headers(),
792
- )
793
- response.raise_for_status()
794
- response_data: Union[Dict[str, Any], List[Dict[str, Any]]] = response.json()
795
-
796
- # determine the property name to use
797
- mapping = self.get_id_property_name(endpoint=endpoint)
798
- if isinstance(mapping, str):
799
- id_property = mapping
800
- else:
801
- id_property, _ = mapping
802
-
803
- if isinstance(response_data, list):
804
- valid_ids: List[str] = [item[id_property] for item in response_data]
805
- return valid_ids
806
- # if the response is an object (dict), check if it's hal+json
807
- if embedded := response_data.get("_embedded"):
808
- # there should be 1 item in the dict that has a value that's a list
809
- for value in embedded.values():
810
- if isinstance(value, list):
811
- valid_ids = [item[id_property] for item in value]
812
- return valid_ids
813
- if (valid_id := response_data.get(id_property)) is not None:
814
- return [valid_id]
815
- valid_ids = [item[id_property] for item in response_data["items"]]
816
- return valid_ids
817
-
818
- @keyword
819
- def get_request_data(self, endpoint: str, method: str) -> RequestData:
820
- """Return an object with valid request data for body, headers and query params."""
821
- method = method.lower()
822
- dto_cls_name = self._get_dto_cls_name(endpoint=endpoint, method=method)
823
- # The endpoint can contain already resolved Ids that have to be matched
824
- # against the parametrized endpoints in the paths section.
825
- spec_endpoint = self.get_parametrized_endpoint(endpoint)
826
- dto_class = self.get_dto_class(endpoint=spec_endpoint, method=method)
827
- try:
828
- method_spec = self.openapi_spec["paths"][spec_endpoint][method]
829
- except KeyError:
830
- logger.info(
831
- f"method '{method}' not supported on '{spec_endpoint}, using empty spec."
832
- )
833
- method_spec = {}
834
-
835
- parameters, params, headers = self.get_request_parameters(
836
- dto_class=dto_class, method_spec=method_spec
837
- )
838
- if (body_spec := method_spec.get("requestBody", None)) is None:
839
- if dto_class == DefaultDto:
840
- dto_instance: Dto = DefaultDto()
841
- else:
842
- dto_class = make_dataclass(
843
- cls_name=method_spec.get("operationId", dto_cls_name),
844
- fields=[],
845
- bases=(dto_class,),
846
- )
847
- dto_instance = dto_class()
848
- return RequestData(
849
- dto=dto_instance,
850
- parameters=parameters,
851
- params=params,
852
- headers=headers,
853
- has_body=False,
854
- )
855
- content_schema = resolve_schema(self.get_content_schema(body_spec))
856
- headers.update({"content-type": self.get_content_type(body_spec)})
857
- dto_data = self.get_json_data_for_dto_class(
858
- schema=content_schema,
859
- dto_class=dto_class,
860
- operation_id=method_spec.get("operationId", ""),
861
- )
862
- if dto_data is None:
863
- dto_instance = DefaultDto()
864
- else:
865
- fields = self.get_fields_from_dto_data(content_schema, dto_data)
866
- dto_class = make_dataclass(
867
- cls_name=method_spec.get("operationId", dto_cls_name),
868
- fields=fields,
869
- bases=(dto_class,),
870
- )
871
- dto_data = {get_safe_key(key): value for key, value in dto_data.items()}
872
- dto_instance = dto_class(**dto_data)
873
- return RequestData(
874
- dto=dto_instance,
875
- dto_schema=content_schema,
876
- parameters=parameters,
877
- params=params,
878
- headers=headers,
879
- )
880
-
881
- @staticmethod
882
- def _get_dto_cls_name(endpoint: str, method: str) -> str:
883
- method = method.capitalize()
884
- path = endpoint.translate({ord(i): None for i in "{}"})
885
- path_parts = path.split("/")
886
- path_parts = [p.capitalize() for p in path_parts]
887
- result = "".join([method, *path_parts])
888
- return result
889
-
890
- @staticmethod
891
- def get_fields_from_dto_data(
892
- content_schema: Dict[str, Any], dto_data: Dict[str, Any]
893
- ):
894
- # FIXME: annotation is not Pyhon 3.8-compatible
895
- # ) -> List[Union[str, Tuple[str, Type[Any]], Tuple[str, Type[Any], Field[Any]]]]:
896
- """Get a dataclasses fields list based on the content_schema and dto_data."""
897
- fields: List[
898
- Union[str, Tuple[str, Type[Any]], Tuple[str, Type[Any], Field[Any]]]
899
- ] = []
900
- for key, value in dto_data.items():
901
- required_properties = content_schema.get("required", [])
902
- safe_key = get_safe_key(key)
903
- metadata = {"original_property_name": key}
904
- if key in required_properties:
905
- # The fields list is used to create a dataclass, so non-default fields
906
- # must go before fields with a default
907
- fields.insert(0, (safe_key, type(value), field(metadata=metadata)))
908
- else:
909
- fields.append((safe_key, type(value), field(default=None, metadata=metadata))) # type: ignore[arg-type]
910
- return fields
911
-
912
- def get_request_parameters(
913
- self, dto_class: Union[Dto, Type[Dto]], method_spec: Dict[str, Any]
914
- ) -> Tuple[List[Dict[str, Any]], Dict[str, Any], Dict[str, str]]:
915
- """Get the methods parameter spec and params and headers with valid data."""
916
- parameters = method_spec.get("parameters", [])
917
- parameter_relations = dto_class.get_parameter_relations()
918
- query_params = [p for p in parameters if p.get("in") == "query"]
919
- header_params = [p for p in parameters if p.get("in") == "header"]
920
- params = self.get_parameter_data(query_params, parameter_relations)
921
- headers = self.get_parameter_data(header_params, parameter_relations)
922
- return parameters, params, headers
923
-
924
- @classmethod
925
- def get_content_schema(cls, body_spec: Dict[str, Any]) -> Dict[str, Any]:
926
- """Get the content schema from the requestBody spec."""
927
- content_type = cls.get_content_type(body_spec)
928
- content_schema = body_spec["content"][content_type]["schema"]
929
- return resolve_schema(content_schema)
930
-
931
- @staticmethod
932
- def get_content_type(body_spec: Dict[str, Any]) -> str:
933
- """Get and validate the first supported content type from the requested body spec
934
-
935
- Should be application/json like content type,
936
- e.g "application/json;charset=utf-8" or "application/merge-patch+json"
937
- """
938
- content_types: List[str] = body_spec["content"].keys()
939
- json_regex = r"application/([a-z\-]+\+)?json(;\s?charset=(.+))?"
940
- for content_type in content_types:
941
- if re.search(json_regex, content_type):
942
- return content_type
943
-
944
- # At present no supported for other types.
945
- raise NotImplementedError(
946
- f"Only content types like 'application/json' are supported. "
947
- f"Content types definded in the spec are '{content_types}'."
948
- )
949
-
950
- def get_parametrized_endpoint(self, endpoint: str) -> str:
951
- """
952
- Get the parametrized endpoint as found in the `paths` section of the openapi
953
- document from a (partially) resolved endpoint.
954
- """
955
-
956
- def match_parts(parts: List[str], spec_parts: List[str]) -> bool:
957
- for part, spec_part in zip_longest(parts, spec_parts, fillvalue="Filler"):
958
- if part == "Filler" or spec_part == "Filler":
959
- return False
960
- if part != spec_part and not spec_part.startswith("{"):
961
- return False
962
- return True
963
-
964
- endpoint_parts = endpoint.split("/")
965
- # if the last part is empty, the path has a trailing `/` that
966
- # should be ignored during matching
967
- if endpoint_parts[-1] == "":
968
- _ = endpoint_parts.pop(-1)
969
-
970
- spec_endpoints: List[str] = {**self.openapi_spec}["paths"].keys()
971
-
972
- candidates: List[str] = []
973
-
974
- for spec_endpoint in spec_endpoints:
975
- spec_endpoint_parts = spec_endpoint.split("/")
976
- # ignore trailing `/` the same way as for endpoint_parts
977
- if spec_endpoint_parts[-1] == "":
978
- _ = spec_endpoint_parts.pop(-1)
979
- if match_parts(endpoint_parts, spec_endpoint_parts):
980
- candidates.append(spec_endpoint)
981
-
982
- if not candidates:
983
- raise ValueError(
984
- f"{endpoint} not found in paths section of the OpenAPI document."
985
- )
986
-
987
- if len(candidates) == 1:
988
- return candidates[0]
989
- # Multiple matches can happen in APIs with overloaded endpoints, e.g.
990
- # /users/me
991
- # /users/${user_id}
992
- # In this case, find the closest (or exact) match
993
- exact_match = [c for c in candidates if c == endpoint]
994
- if exact_match:
995
- return exact_match[0]
996
- # TODO: Implement a decision mechanism when real-world examples become available
997
- # In the face of ambiguity, refuse the temptation to guess.
998
- raise ValueError(f"{endpoint} matched to multiple paths: {candidates}")
999
-
1000
- @staticmethod
1001
- def get_parameter_data(
1002
- parameters: List[Dict[str, Any]],
1003
- parameter_relations: List[Relation],
1004
- ) -> Dict[str, str]:
1005
- """Generate a valid list of key-value pairs for all parameters."""
1006
- result: Dict[str, str] = {}
1007
- value: Any = None
1008
- for parameter in parameters:
1009
- parameter_name = parameter["name"]
1010
- parameter_schema = resolve_schema(parameter["schema"])
1011
- relations = [
1012
- r for r in parameter_relations if r.property_name == parameter_name
1013
- ]
1014
- if constrained_values := [
1015
- r.values for r in relations if isinstance(r, PropertyValueConstraint)
1016
- ]:
1017
- value = choice(*constrained_values)
1018
- if value is IGNORE:
1019
- continue
1020
- result[parameter_name] = value
1021
- continue
1022
- value = value_utils.get_valid_value(parameter_schema)
1023
- result[parameter_name] = value
1024
- return result
1025
-
1026
- @keyword
1027
- def get_json_data_for_dto_class(
1028
- self,
1029
- schema: Dict[str, Any],
1030
- dto_class: Union[Dto, Type[Dto]],
1031
- operation_id: str = "",
1032
- ) -> Optional[Dict[str, Any]]:
1033
- """
1034
- Generate a valid (json-compatible) dict for all the `dto_class` properties.
1035
- """
1036
-
1037
- def get_constrained_values(property_name: str) -> List[Any]:
1038
- relations = dto_class.get_relations()
1039
- values_list = [
1040
- c.values
1041
- for c in relations
1042
- if (
1043
- isinstance(c, PropertyValueConstraint)
1044
- and c.property_name == property_name
1045
- )
1046
- ]
1047
- # values should be empty or contain 1 list of allowed values
1048
- return values_list.pop() if values_list else []
1049
-
1050
- def get_dependent_id(
1051
- property_name: str, operation_id: str
1052
- ) -> Optional[Union[str, int, float]]:
1053
- relations = dto_class.get_relations()
1054
- # multiple get paths are possible based on the operation being performed
1055
- id_get_paths = [
1056
- (d.get_path, d.operation_id)
1057
- for d in relations
1058
- if (isinstance(d, IdDependency) and d.property_name == property_name)
1059
- ]
1060
- if not id_get_paths:
1061
- return None
1062
- if len(id_get_paths) == 1:
1063
- id_get_path, _ = id_get_paths.pop()
1064
- else:
1065
- try:
1066
- [id_get_path] = [
1067
- path
1068
- for path, operation in id_get_paths
1069
- if operation == operation_id
1070
- ]
1071
- # There could be multiple get_paths, but not one for the current operation
1072
- except ValueError:
1073
- return None
1074
- valid_id = self.get_valid_id_for_endpoint(
1075
- endpoint=id_get_path, method="get"
1076
- )
1077
- logger.debug(f"get_dependent_id for {id_get_path} returned {valid_id}")
1078
- return valid_id
1079
-
1080
- json_data: Dict[str, Any] = {}
1081
-
1082
- for property_name in schema.get("properties", []):
1083
- properties_schema = schema["properties"][property_name]
1084
-
1085
- property_type = properties_schema.get("type")
1086
- if not property_type:
1087
- selected_type_schema = choice(properties_schema["types"])
1088
- property_type = selected_type_schema["type"]
1089
- if properties_schema.get("readOnly", False):
1090
- continue
1091
- if constrained_values := get_constrained_values(property_name):
1092
- # do not add properties that are configured to be ignored
1093
- if IGNORE in constrained_values:
1094
- continue
1095
- json_data[property_name] = choice(constrained_values)
1096
- continue
1097
- if (
1098
- dependent_id := get_dependent_id(
1099
- property_name=property_name, operation_id=operation_id
1100
- )
1101
- ) is not None:
1102
- json_data[property_name] = dependent_id
1103
- continue
1104
- if property_type == "object":
1105
- object_data = self.get_json_data_for_dto_class(
1106
- schema=properties_schema,
1107
- dto_class=DefaultDto,
1108
- operation_id="",
1109
- )
1110
- json_data[property_name] = object_data
1111
- continue
1112
- if property_type == "array":
1113
- array_data = self.get_json_data_for_dto_class(
1114
- schema=properties_schema["items"],
1115
- dto_class=DefaultDto,
1116
- operation_id=operation_id,
1117
- )
1118
- json_data[property_name] = [array_data]
1119
- continue
1120
- json_data[property_name] = value_utils.get_valid_value(properties_schema)
1121
- return json_data
1122
-
1123
- @keyword
1124
- def get_invalidated_url(self, valid_url: str) -> Optional[str]:
1125
- """
1126
- Return an url with all the path parameters in the `valid_url` replaced by a
1127
- random UUID.
1128
-
1129
- Raises ValueError if the valid_url cannot be invalidated.
1130
- """
1131
- parameterized_endpoint = self.get_parameterized_endpoint_from_url(valid_url)
1132
- parameterized_url = self.base_url + parameterized_endpoint
1133
- valid_url_parts = list(reversed(valid_url.split("/")))
1134
- parameterized_parts = reversed(parameterized_url.split("/"))
1135
- for index, (parameterized_part, _) in enumerate(
1136
- zip(parameterized_parts, valid_url_parts)
1137
- ):
1138
- if parameterized_part.startswith("{") and parameterized_part.endswith("}"):
1139
- valid_url_parts[index] = uuid4().hex
1140
- valid_url_parts.reverse()
1141
- invalid_url = "/".join(valid_url_parts)
1142
- return invalid_url
1143
- raise ValueError(f"{parameterized_endpoint} could not be invalidated.")
1144
-
1145
- @keyword
1146
- def get_parameterized_endpoint_from_url(self, url: str) -> str:
1147
- """
1148
- Return the endpoint as found in the `paths` section based on the given `url`.
1149
- """
1150
- endpoint = url.replace(self.base_url, "")
1151
- endpoint_parts = endpoint.split("/")
1152
- # first part will be '' since an endpoint starts with /
1153
- endpoint_parts.pop(0)
1154
- parameterized_endpoint = self.get_parametrized_endpoint(endpoint=endpoint)
1155
- return parameterized_endpoint
1156
-
1157
- @keyword
1158
- def get_invalid_json_data(
1159
- self,
1160
- url: str,
1161
- method: str,
1162
- status_code: int,
1163
- request_data: RequestData,
1164
- ) -> Dict[str, Any]:
1165
- """
1166
- Return `json_data` based on the `dto` on the `request_data` that will cause
1167
- the provided `status_code` for the `method` operation on the `url`.
1168
-
1169
- > Note: applicable UniquePropertyValueConstraint and IdReference Relations are
1170
- considered before changes to `json_data` are made.
1171
- """
1172
- method = method.lower()
1173
- data_relations = request_data.dto.get_relations_for_error_code(status_code)
1174
- if not data_relations:
1175
- if not request_data.dto_schema:
1176
- raise ValueError(
1177
- "Failed to invalidate: no data_relations and empty schema."
1178
- )
1179
- json_data = request_data.dto.get_invalidated_data(
1180
- schema=request_data.dto_schema,
1181
- status_code=status_code,
1182
- invalid_property_default_code=self.invalid_property_default_response,
1183
- )
1184
- return json_data
1185
- resource_relation = choice(data_relations)
1186
- if isinstance(resource_relation, UniquePropertyValueConstraint):
1187
- json_data = run_keyword(
1188
- "get_json_data_with_conflict",
1189
- url,
1190
- method,
1191
- request_data.dto,
1192
- status_code,
1193
- )
1194
- elif isinstance(resource_relation, IdReference):
1195
- run_keyword("ensure_in_use", url, resource_relation)
1196
- json_data = request_data.dto.as_dict()
1197
- else:
1198
- json_data = request_data.dto.get_invalidated_data(
1199
- schema=request_data.dto_schema,
1200
- status_code=status_code,
1201
- invalid_property_default_code=self.invalid_property_default_response,
1202
- )
1203
- return json_data
1204
-
1205
- @keyword
1206
- def get_invalidated_parameters(
1207
- self,
1208
- status_code: int,
1209
- request_data: RequestData,
1210
- ) -> Tuple[Dict[str, Any], Dict[str, str]]:
1211
- """
1212
- Returns a version of `params, headers` as present on `request_data` that has
1213
- been modified to cause the provided `status_code`.
1214
- """
1215
- if not request_data.parameters:
1216
- raise ValueError("No params or headers to invalidate.")
1217
-
1218
- # ensure the status_code can be triggered
1219
- relations = request_data.dto.get_parameter_relations_for_error_code(status_code)
1220
- relations_for_status_code = [
1221
- r
1222
- for r in relations
1223
- if isinstance(r, PropertyValueConstraint)
1224
- and (
1225
- r.error_code == status_code or r.invalid_value_error_code == status_code
1226
- )
1227
- ]
1228
- parameters_to_ignore = {
1229
- r.property_name
1230
- for r in relations_for_status_code
1231
- if r.invalid_value_error_code == status_code and r.invalid_value == IGNORE
1232
- }
1233
- relation_property_names = {r.property_name for r in relations_for_status_code}
1234
- if not relation_property_names:
1235
- if status_code != self.invalid_property_default_response:
1236
- raise ValueError(
1237
- f"No relations to cause status_code {status_code} found."
1238
- )
1239
-
1240
- # ensure we're not modifying mutable properties
1241
- params = deepcopy(request_data.params)
1242
- headers = deepcopy(request_data.headers)
1243
-
1244
- if status_code == self.invalid_property_default_response:
1245
- # take the params and headers that can be invalidated based on data type
1246
- # and expand the set with properties that can be invalided by relations
1247
- parameter_names = set(request_data.params_that_can_be_invalidated).union(
1248
- request_data.headers_that_can_be_invalidated
1249
- )
1250
- parameter_names.update(relation_property_names)
1251
- if not parameter_names:
1252
- raise ValueError(
1253
- "None of the query parameters and headers can be invalidated."
1254
- )
1255
- else:
1256
- # non-default status_codes can only be the result of a Relation
1257
- parameter_names = relation_property_names
1258
-
1259
- # Dto mappings may contain generic mappings for properties that are not present
1260
- # in this specific schema
1261
- request_data_parameter_names = [p.get("name") for p in request_data.parameters]
1262
- additional_relation_property_names = {
1263
- n for n in relation_property_names if n not in request_data_parameter_names
1264
- }
1265
- if additional_relation_property_names:
1266
- logger.warning(
1267
- f"get_parameter_relations_for_error_code yielded properties that are "
1268
- f"not defined in the schema: {additional_relation_property_names}\n"
1269
- f"These properties will be ignored for parameter invalidation."
1270
- )
1271
- parameter_names = parameter_names - additional_relation_property_names
1272
-
1273
- if not parameter_names:
1274
- raise ValueError(
1275
- f"No parameter can be changed to cause status_code {status_code}."
1276
- )
1277
-
1278
- parameter_names = parameter_names - parameters_to_ignore
1279
- parameter_to_invalidate = choice(tuple(parameter_names))
1280
-
1281
- # check for invalid parameters in the provided request_data
1282
- try:
1283
- [parameter_data] = [
1284
- data
1285
- for data in request_data.parameters
1286
- if data["name"] == parameter_to_invalidate
1287
- ]
1288
- except Exception:
1289
- raise ValueError(
1290
- f"{parameter_to_invalidate} not found in provided parameters."
1291
- ) from None
1292
-
1293
- # get the invalid_value for the chosen parameter
1294
- try:
1295
- [invalid_value_for_error_code] = [
1296
- r.invalid_value
1297
- for r in relations_for_status_code
1298
- if r.property_name == parameter_to_invalidate
1299
- and r.invalid_value_error_code == status_code
1300
- ]
1301
- except ValueError:
1302
- invalid_value_for_error_code = NOT_SET
1303
-
1304
- # get the constraint values if available for the chosen parameter
1305
- try:
1306
- [values_from_constraint] = [
1307
- r.values
1308
- for r in relations_for_status_code
1309
- if r.property_name == parameter_to_invalidate
1310
- ]
1311
- except ValueError:
1312
- values_from_constraint = []
1313
-
1314
- # if the parameter was not provided, add it to params / headers
1315
- params, headers = self.ensure_parameter_in_parameters(
1316
- parameter_to_invalidate=parameter_to_invalidate,
1317
- params=params,
1318
- headers=headers,
1319
- parameter_data=parameter_data,
1320
- values_from_constraint=values_from_constraint,
1321
- )
1322
-
1323
- # determine the invalid_value
1324
- if invalid_value_for_error_code != NOT_SET:
1325
- invalid_value = invalid_value_for_error_code
1326
- else:
1327
- if parameter_to_invalidate in params.keys():
1328
- valid_value = params[parameter_to_invalidate]
1329
- else:
1330
- valid_value = headers[parameter_to_invalidate]
1331
-
1332
- value_schema = resolve_schema(parameter_data["schema"])
1333
- invalid_value = value_utils.get_invalid_value(
1334
- value_schema=value_schema,
1335
- current_value=valid_value,
1336
- values_from_constraint=values_from_constraint,
1337
- )
1338
- logger.debug(f"{parameter_to_invalidate} changed to {invalid_value}")
1339
-
1340
- # update the params / headers and return
1341
- if parameter_to_invalidate in params.keys():
1342
- params[parameter_to_invalidate] = invalid_value
1343
- else:
1344
- headers[parameter_to_invalidate] = invalid_value
1345
- return params, headers
1346
-
1347
- @staticmethod
1348
- def ensure_parameter_in_parameters(
1349
- parameter_to_invalidate: str,
1350
- params: Dict[str, Any],
1351
- headers: Dict[str, str],
1352
- parameter_data: Dict[str, Any],
1353
- values_from_constraint: List[Any],
1354
- ) -> Tuple[Dict[str, Any], Dict[str, str]]:
1355
- """
1356
- Returns the params, headers tuple with parameter_to_invalidate with a valid
1357
- value to params or headers if not originally present.
1358
- """
1359
- if (
1360
- parameter_to_invalidate not in params.keys()
1361
- and parameter_to_invalidate not in headers.keys()
1362
- ):
1363
- if values_from_constraint:
1364
- valid_value = choice(values_from_constraint)
1365
- else:
1366
- parameter_schema = resolve_schema(parameter_data["schema"])
1367
- valid_value = value_utils.get_valid_value(parameter_schema)
1368
- if (
1369
- parameter_data["in"] == "query"
1370
- and parameter_to_invalidate not in params.keys()
1371
- ):
1372
- params[parameter_to_invalidate] = valid_value
1373
- if (
1374
- parameter_data["in"] == "header"
1375
- and parameter_to_invalidate not in headers.keys()
1376
- ):
1377
- headers[parameter_to_invalidate] = valid_value
1378
- return params, headers
1379
-
1380
- @keyword
1381
- def ensure_in_use(self, url: str, resource_relation: IdReference) -> None:
1382
- """
1383
- Ensure that the (right-most) `id` of the resource referenced by the `url`
1384
- is used by the resource defined by the `resource_relation`.
1385
- """
1386
- resource_id = ""
1387
-
1388
- endpoint = url.replace(self.base_url, "")
1389
- endpoint_parts = endpoint.split("/")
1390
- parameterized_endpoint = self.get_parametrized_endpoint(endpoint=endpoint)
1391
- parameterized_endpoint_parts = parameterized_endpoint.split("/")
1392
- for part, param_part in zip(
1393
- reversed(endpoint_parts), reversed(parameterized_endpoint_parts)
1394
- ):
1395
- if param_part.endswith("}"):
1396
- resource_id = part
1397
- break
1398
- if not resource_id:
1399
- raise ValueError(f"The provided url ({url}) does not contain an id.")
1400
- request_data = self.get_request_data(
1401
- method="post", endpoint=resource_relation.post_path
1402
- )
1403
- json_data = request_data.dto.as_dict()
1404
- json_data[resource_relation.property_name] = resource_id
1405
- post_url: str = run_keyword(
1406
- "get_valid_url",
1407
- resource_relation.post_path,
1408
- "post",
1409
- )
1410
- response: Response = run_keyword(
1411
- "authorized_request",
1412
- post_url,
1413
- "post",
1414
- request_data.params,
1415
- request_data.headers,
1416
- json_data,
1417
- )
1418
- if not response.ok:
1419
- logger.debug(
1420
- f"POST on {post_url} with json {json_data} failed: {response.json()}"
1421
- )
1422
- response.raise_for_status()
1423
-
1424
- @keyword
1425
- def get_json_data_with_conflict(
1426
- self, url: str, method: str, dto: Dto, conflict_status_code: int
1427
- ) -> Dict[str, Any]:
1428
- """
1429
- Return `json_data` based on the `UniquePropertyValueConstraint` that must be
1430
- returned by the `get_relations` implementation on the `dto` for the given
1431
- `conflict_status_code`.
1432
- """
1433
- method = method.lower()
1434
- json_data = dto.as_dict()
1435
- unique_property_value_constraints = [
1436
- r
1437
- for r in dto.get_relations()
1438
- if isinstance(r, UniquePropertyValueConstraint)
1439
- ]
1440
- for relation in unique_property_value_constraints:
1441
- json_data[relation.property_name] = relation.value
1442
- # create a new resource that the original request will conflict with
1443
- if method in ["patch", "put"]:
1444
- post_url_parts = url.split("/")[:-1]
1445
- post_url = "/".join(post_url_parts)
1446
- # the PATCH or PUT may use a different dto than required for POST
1447
- # so a valid POST dto must be constructed
1448
- endpoint = post_url.replace(self.base_url, "")
1449
- request_data = self.get_request_data(endpoint=endpoint, method="post")
1450
- post_json = request_data.dto.as_dict()
1451
- for key in post_json.keys():
1452
- if key in json_data:
1453
- post_json[key] = json_data.get(key)
1454
- else:
1455
- post_url = url
1456
- post_json = json_data
1457
- endpoint = post_url.replace(self.base_url, "")
1458
- request_data = self.get_request_data(endpoint=endpoint, method="post")
1459
- response: Response = run_keyword(
1460
- "authorized_request",
1461
- post_url,
1462
- "post",
1463
- request_data.params,
1464
- request_data.headers,
1465
- post_json,
1466
- )
1467
- # conflicting resource may already exist
1468
- assert (
1469
- response.ok or response.status_code == conflict_status_code
1470
- ), f"get_json_data_with_conflict received {response.status_code}: {response.json()}"
1471
- return json_data
1472
- raise ValueError(
1473
- f"No UniquePropertyValueConstraint in the get_relations list on dto {dto}."
1474
- )
1475
-
1476
- @keyword
1477
- def authorized_request( # pylint: disable=too-many-arguments
1478
- self,
1479
- url: str,
1480
- method: str,
1481
- params: Optional[Dict[str, Any]] = None,
1482
- headers: Optional[Dict[str, str]] = None,
1483
- json_data: Optional[JSON] = None,
1484
- ) -> Response:
1485
- """
1486
- Perform a request using the security token or authentication set in the library.
1487
-
1488
- > Note: provided username / password or auth objects take precedence over token
1489
- based security
1490
- """
1491
- headers = headers if headers else {}
1492
- if self.extra_headers:
1493
- headers.update(self.extra_headers)
1494
- # if both an auth object and a token are available, auth takes precedence
1495
- if self.security_token and not self.auth:
1496
- security_header = {"Authorization": self.security_token}
1497
- headers.update(security_header)
1498
- headers = {k: str(v) for k, v in headers.items()}
1499
- response = self.session.request(
1500
- url=url,
1501
- method=method,
1502
- params=params,
1503
- headers=headers,
1504
- json=json_data,
1505
- cookies=self.cookies,
1506
- auth=self.auth,
1507
- proxies=self.proxies,
1508
- verify=self.verify,
1509
- cert=self.cert,
1510
- )
1511
- logger.debug(f"Response text: {response.text}")
1512
- return response
1
+ """
2
+ # OpenApiLibCore for Robot Framework
3
+
4
+ The OpenApiLibCore library is a utility library that is meant to simplify creation
5
+ of other Robot Framework libraries for API testing based on the information in
6
+ an OpenAPI document (also known as Swagger document).
7
+ This document explains how to use the OpenApiLibCore library.
8
+
9
+ My RoboCon 2022 talk about OpenApiDriver and OpenApiLibCore can be found
10
+ [here](https://www.youtube.com/watch?v=7YWZEHxk9Ps)
11
+
12
+ For more information about Robot Framework, see http://robotframework.org.
13
+
14
+ ---
15
+
16
+ > Note: OpenApiLibCore is still being developed so there are currently
17
+ restrictions / limitations that you may encounter when using this library to run
18
+ tests against an API. See [Limitations](#limitations) for details.
19
+
20
+ ---
21
+
22
+ ## Installation
23
+
24
+ If you already have Python >= 3.8 with pip installed, you can simply run:
25
+
26
+ `pip install --upgrade robotframework-openapi-libcore`
27
+
28
+ ---
29
+
30
+ ## OpenAPI (aka Swagger)
31
+
32
+ The OpenAPI Specification (OAS) defines a standard, language-agnostic interface
33
+ to RESTful APIs, see https://swagger.io/specification/
34
+
35
+ The OpenApiLibCore implements a number of Robot Framework keywords that make it
36
+ easy to interact with an OpenAPI implementation by using the information in the
37
+ openapi document (Swagger file), for examply by automatic generation of valid values
38
+ for requests based on the schema information in the document.
39
+
40
+ > Note: OpenApiLibCore is designed for APIs based on the OAS v3
41
+ The library has not been tested for APIs based on the OAS v2.
42
+
43
+ ---
44
+
45
+ ## Getting started
46
+
47
+ Before trying to use the keywords exposed by OpenApiLibCore on the target API
48
+ it's recommended to first ensure that the openapi document for the API is valid
49
+ under the OpenAPI Specification.
50
+
51
+ This can be done using the command line interface of a package that is installed as
52
+ a prerequisite for OpenApiLibCore.
53
+ Both a local openapi.json or openapi.yaml file or one hosted by the API server
54
+ can be checked using the `prance validate <reference_to_file>` shell command:
55
+
56
+ ```shell
57
+ prance validate --backend=openapi-spec-validator http://localhost:8000/openapi.json
58
+ Processing "http://localhost:8000/openapi.json"...
59
+ -> Resolving external references.
60
+ Validates OK as OpenAPI 3.0.2!
61
+
62
+ prance validate --backend=openapi-spec-validator /tests/files/petstore_openapi.yaml
63
+ Processing "/tests/files/petstore_openapi.yaml"...
64
+ -> Resolving external references.
65
+ Validates OK as OpenAPI 3.0.2!
66
+ ```
67
+
68
+ You'll have to change the url or file reference to the location of the openapi
69
+ document for your API.
70
+
71
+ > Note: Although recursion is technically allowed under the OAS, tool support is limited
72
+ and changing the OAS to not use recursion is recommended.
73
+ OpenApiLibCore has limited support for parsing OpenAPI documents with
74
+ recursion in them. See the `recursion_limit` and `recursion_default` parameters.
75
+
76
+ If the openapi document passes this validation, the next step is trying to do a test
77
+ run with a minimal test suite.
78
+ The example below can be used, with `source`, `origin` and 'endpoint' altered to
79
+ fit your situation.
80
+
81
+ ``` robotframework
82
+ *** Settings ***
83
+ Library OpenApiLibCore
84
+ ... source=http://localhost:8000/openapi.json
85
+ ... origin=http://localhost:8000
86
+
87
+ *** Test Cases ***
88
+ Getting Started
89
+ ${url}= Get Valid Url endpoint=/employees/{employee_id} method=get
90
+
91
+ ```
92
+
93
+ Running the above suite for the first time may result in an error / failed test.
94
+ You should look at the Robot Framework `log.html` to determine the reasons
95
+ for the failing tests.
96
+ Depending on the reasons for the failures, different solutions are possible.
97
+
98
+ Details about the OpenApiLibCore library parameters and keywords that you may need can be found
99
+ [here](https://marketsquare.github.io/robotframework-openapi-libcore/openapi_libcore.html).
100
+
101
+ The OpenApiLibCore also support handling of relations between resources within the scope
102
+ of the API being validated as well as handling dependencies on resources outside the
103
+ scope of the API. In addition there is support for handling restrictions on the values
104
+ of parameters and properties.
105
+
106
+ Details about the `mappings_path` variable usage can be found
107
+ [here](https://marketsquare.github.io/robotframework-openapi-libcore/advanced_use.html).
108
+
109
+ ---
110
+
111
+ ## Limitations
112
+
113
+ There are currently a number of limitations to supported API structures, supported
114
+ data types and properties. The following list details the most important ones:
115
+ - Only JSON request and response bodies are supported.
116
+ - No support for per-endpoint authorization levels.
117
+ - Parsing of OAS 3.1 documents is supported by the parsing tools, but runtime behavior is untested.
118
+
119
+ """
120
+
121
+ import json as _json
122
+ import re
123
+ import sys
124
+ from copy import deepcopy
125
+ from dataclasses import Field, dataclass, field, make_dataclass
126
+ from functools import cached_property
127
+ from itertools import zip_longest
128
+ from logging import getLogger
129
+ from pathlib import Path
130
+ from random import choice, sample
131
+ from typing import Any, Dict, List, Optional, Set, Tuple, Type, Union
132
+ from uuid import uuid4
133
+
134
+ from openapi_core import Config, OpenAPI, Spec
135
+ from openapi_core.contrib.requests import (
136
+ RequestsOpenAPIRequest,
137
+ RequestsOpenAPIResponse,
138
+ )
139
+ from prance import ResolvingParser, ValidationError
140
+ from prance.util.url import ResolutionError
141
+ from requests import Response, Session
142
+ from requests.auth import AuthBase, HTTPBasicAuth
143
+ from requests.cookies import RequestsCookieJar as CookieJar
144
+ from robot.api.deco import keyword, library
145
+ from robot.libraries.BuiltIn import BuiltIn
146
+
147
+ from OpenApiLibCore import value_utils
148
+ from OpenApiLibCore.dto_base import (
149
+ NOT_SET,
150
+ Dto,
151
+ IdDependency,
152
+ IdReference,
153
+ PathPropertiesConstraint,
154
+ PropertyValueConstraint,
155
+ Relation,
156
+ UniquePropertyValueConstraint,
157
+ resolve_schema,
158
+ )
159
+ from OpenApiLibCore.dto_utils import (
160
+ DEFAULT_ID_PROPERTY_NAME,
161
+ DefaultDto,
162
+ get_dto_class,
163
+ get_id_property_name,
164
+ )
165
+ from OpenApiLibCore.oas_cache import PARSER_CACHE
166
+ from OpenApiLibCore.value_utils import FAKE, IGNORE, JSON
167
+
168
+ run_keyword = BuiltIn().run_keyword
169
+
170
+ logger = getLogger(__name__)
171
+
172
+
173
+ def get_safe_key(key: str) -> str:
174
+ """
175
+ Helper function to convert a valid JSON property name to a string that can be used
176
+ as a Python variable or function / method name.
177
+ """
178
+ key = key.replace("-", "_")
179
+ key = key.replace("@", "_")
180
+ if key[0].isdigit():
181
+ key = f"_{key}"
182
+ return key
183
+
184
+
185
+ @dataclass
186
+ class RequestValues:
187
+ """Helper class to hold parameter values needed to make a request."""
188
+
189
+ url: str
190
+ method: str
191
+ params: Optional[Dict[str, Any]]
192
+ headers: Optional[Dict[str, str]]
193
+ json_data: Optional[Dict[str, Any]]
194
+
195
+
196
+ @dataclass
197
+ class RequestData:
198
+ """Helper class to manage parameters used when making requests."""
199
+
200
+ dto: Union[Dto, DefaultDto] = field(default_factory=DefaultDto)
201
+ dto_schema: Dict[str, Any] = field(default_factory=dict)
202
+ parameters: List[Dict[str, Any]] = field(default_factory=list)
203
+ params: Dict[str, Any] = field(default_factory=dict)
204
+ headers: Dict[str, Any] = field(default_factory=dict)
205
+ has_body: bool = True
206
+
207
+ def __post_init__(self) -> None:
208
+ # prevent modification by reference
209
+ self.dto_schema = deepcopy(self.dto_schema)
210
+ self.parameters = deepcopy(self.parameters)
211
+ self.params = deepcopy(self.params)
212
+ self.headers = deepcopy(self.headers)
213
+
214
+ @property
215
+ def has_optional_properties(self) -> bool:
216
+ """Whether or not the dto data (json data) contains optional properties."""
217
+
218
+ def is_required_property(property_name: str) -> bool:
219
+ return property_name in self.dto_schema.get("required", [])
220
+
221
+ properties = (self.dto.as_dict()).keys()
222
+ return not all(map(is_required_property, properties))
223
+
224
+ @property
225
+ def has_optional_params(self) -> bool:
226
+ """Whether or not any of the query parameters are optional."""
227
+
228
+ def is_optional_param(query_param: str) -> bool:
229
+ optional_params = [
230
+ p.get("name")
231
+ for p in self.parameters
232
+ if p.get("in") == "query" and not p.get("required")
233
+ ]
234
+ return query_param in optional_params
235
+
236
+ return any(map(is_optional_param, self.params))
237
+
238
+ @cached_property
239
+ def params_that_can_be_invalidated(self) -> Set[str]:
240
+ """
241
+ The query parameters that can be invalidated by violating data
242
+ restrictions, data type or by not providing them in a request.
243
+ """
244
+ result = set()
245
+ params = [h for h in self.parameters if h.get("in") == "query"]
246
+ for param in params:
247
+ # required params can be omitted to invalidate a request
248
+ if param["required"]:
249
+ result.add(param["name"])
250
+ continue
251
+
252
+ schema = resolve_schema(param["schema"])
253
+ if schema.get("type", None):
254
+ param_types = [schema]
255
+ else:
256
+ param_types = schema["types"]
257
+ for param_type in param_types:
258
+ # any basic non-string type except "null" can be invalidated by
259
+ # replacing it with a string
260
+ if param_type["type"] not in ["string", "array", "object", "null"]:
261
+ result.add(param["name"])
262
+ continue
263
+ # enums, strings and arrays with boundaries can be invalidated
264
+ if set(param_type.keys()).intersection(
265
+ {
266
+ "enum",
267
+ "minLength",
268
+ "maxLength",
269
+ "minItems",
270
+ "maxItems",
271
+ }
272
+ ):
273
+ result.add(param["name"])
274
+ continue
275
+ # an array of basic non-string type can be invalidated by replacing the
276
+ # items in the array with strings
277
+ if param_type["type"] == "array" and param_type["items"][
278
+ "type"
279
+ ] not in [
280
+ "string",
281
+ "array",
282
+ "object",
283
+ "null",
284
+ ]:
285
+ result.add(param["name"])
286
+ return result
287
+
288
+ @property
289
+ def has_optional_headers(self) -> bool:
290
+ """Whether or not any of the headers are optional."""
291
+
292
+ def is_optional_header(header: str) -> bool:
293
+ optional_headers = [
294
+ p.get("name")
295
+ for p in self.parameters
296
+ if p.get("in") == "header" and not p.get("required")
297
+ ]
298
+ return header in optional_headers
299
+
300
+ return any(map(is_optional_header, self.headers))
301
+
302
+ @cached_property
303
+ def headers_that_can_be_invalidated(self) -> Set[str]:
304
+ """
305
+ The header parameters that can be invalidated by violating data
306
+ restrictions or by not providing them in a request.
307
+ """
308
+ result = set()
309
+ headers = [h for h in self.parameters if h.get("in") == "header"]
310
+ for header in headers:
311
+ # required headers can be omitted to invalidate a request
312
+ if header["required"]:
313
+ result.add(header["name"])
314
+ continue
315
+
316
+ schema = resolve_schema(header["schema"])
317
+ if schema.get("type", None):
318
+ header_types = [schema]
319
+ else:
320
+ header_types = schema["types"]
321
+ for header_type in header_types:
322
+ # any basic non-string type except "null" can be invalidated by
323
+ # replacing it with a string
324
+ if header_type["type"] not in ["string", "array", "object", "null"]:
325
+ result.add(header["name"])
326
+ continue
327
+ # enums, strings and arrays with boundaries can be invalidated
328
+ if set(header_type.keys()).intersection(
329
+ {
330
+ "enum",
331
+ "minLength",
332
+ "maxLength",
333
+ "minItems",
334
+ "maxItems",
335
+ }
336
+ ):
337
+ result.add(header["name"])
338
+ continue
339
+ # an array of basic non-string type can be invalidated by replacing the
340
+ # items in the array with strings
341
+ if header_type["type"] == "array" and header_type["items"][
342
+ "type"
343
+ ] not in [
344
+ "string",
345
+ "array",
346
+ "object",
347
+ "null",
348
+ ]:
349
+ result.add(header["name"])
350
+ return result
351
+
352
+ def get_required_properties_dict(self) -> Dict[str, Any]:
353
+ """Get the json-compatible dto data containing only the required properties."""
354
+ required_properties = self.dto_schema.get("required", [])
355
+ required_properties_dict: Dict[str, Any] = {}
356
+ for key, value in (self.dto.as_dict()).items():
357
+ if key in required_properties:
358
+ required_properties_dict[key] = value
359
+ return required_properties_dict
360
+
361
+ def get_minimal_body_dict(self) -> Dict[str, Any]:
362
+ required_properties_dict = self.get_required_properties_dict()
363
+
364
+ min_properties = self.dto_schema.get("minProperties", 0)
365
+ number_of_optional_properties_to_add = min_properties - len(
366
+ required_properties_dict
367
+ )
368
+
369
+ if number_of_optional_properties_to_add < 1:
370
+ return required_properties_dict
371
+
372
+ optional_properties_dict = {
373
+ k: v
374
+ for k, v in self.dto.as_dict().items()
375
+ if k not in required_properties_dict
376
+ }
377
+ optional_properties_to_keep = sample(
378
+ sorted(optional_properties_dict), number_of_optional_properties_to_add
379
+ )
380
+ optional_properties_dict = {
381
+ k: v
382
+ for k, v in optional_properties_dict.items()
383
+ if k in optional_properties_to_keep
384
+ }
385
+
386
+ return {**required_properties_dict, **optional_properties_dict}
387
+
388
+ def get_required_params(self) -> Dict[str, str]:
389
+ """Get the params dict containing only the required query parameters."""
390
+ required_parameters = [
391
+ p.get("name") for p in self.parameters if p.get("required")
392
+ ]
393
+ return {k: v for k, v in self.params.items() if k in required_parameters}
394
+
395
+ def get_required_headers(self) -> Dict[str, str]:
396
+ """Get the headers dict containing only the required headers."""
397
+ required_parameters = [
398
+ p.get("name") for p in self.parameters if p.get("required")
399
+ ]
400
+ return {k: v for k, v in self.headers.items() if k in required_parameters}
401
+
402
+
403
+ @library(scope="TEST SUITE", doc_format="ROBOT")
404
+ class OpenApiLibCore: # pylint: disable=too-many-instance-attributes
405
+ """
406
+ Main class providing the keywords and core logic to interact with an OpenAPI server.
407
+
408
+ Visit the [https://github.com/MarketSquare/robotframework-openapi-libcore | library page]
409
+ for an introduction.
410
+ """
411
+
412
+ def __init__( # pylint: disable=too-many-arguments, too-many-locals, dangerous-default-value
413
+ self,
414
+ source: str,
415
+ origin: str = "",
416
+ base_path: str = "",
417
+ mappings_path: Union[str, Path] = "",
418
+ invalid_property_default_response: int = 422,
419
+ default_id_property_name: str = "id",
420
+ faker_locale: Optional[Union[str, List[str]]] = None,
421
+ recursion_limit: int = 1,
422
+ recursion_default: Any = {},
423
+ username: str = "",
424
+ password: str = "",
425
+ security_token: str = "",
426
+ auth: Optional[AuthBase] = None,
427
+ cert: Optional[Union[str, Tuple[str, str]]] = None,
428
+ verify_tls: Optional[Union[bool, str]] = True,
429
+ extra_headers: Optional[Dict[str, str]] = None,
430
+ cookies: Optional[Union[Dict[str, str], CookieJar]] = None,
431
+ proxies: Optional[Dict[str, str]] = None,
432
+ ) -> None:
433
+ """
434
+ == Base parameters ==
435
+
436
+ === source ===
437
+ An absolute path to an openapi.json or openapi.yaml file or an url to such a file.
438
+
439
+ === origin ===
440
+ The server (and port) of the target server. E.g. ``https://localhost:8000``
441
+
442
+ === base_path ===
443
+ The routing between ``origin`` and the endpoints as found in the ``paths``
444
+ section in the openapi document.
445
+ E.g. ``/petshop/v2``.
446
+
447
+ == API-specific configurations ==
448
+
449
+ === mappings_path ===
450
+ See [https://marketsquare.github.io/robotframework-openapi-libcore/advanced_use.html | this page]
451
+ for an in-depth explanation.
452
+
453
+ === invalid_property_default_response ===
454
+ The default response code for requests with a JSON body that does not comply
455
+ with the schema.
456
+ Example: a value outside the specified range or a string value
457
+ for a property defined as integer in the schema.
458
+
459
+ === default_id_property_name ===
460
+ The default name for the property that identifies a resource (i.e. a unique
461
+ entity) within the API.
462
+ The default value for this property name is ``id``.
463
+ If the target API uses a different name for all the resources within the API,
464
+ you can configure it globally using this property.
465
+
466
+ If different property names are used for the unique identifier for different
467
+ types of resources, an ``ID_MAPPING`` can be implemented using the ``mappings_path``.
468
+
469
+ === faker_locale ===
470
+ A locale string or list of locale strings to pass to the Faker library to be
471
+ used in generation of string data for supported format types.
472
+
473
+ == Parsing parameters ==
474
+
475
+ === recursion_limit ===
476
+ The recursion depth to which to fully parse recursive references before the
477
+ `recursion_default` is used to end the recursion.
478
+
479
+ === recursion_default ===
480
+ The value that is used instead of the referenced schema when the
481
+ `recursion_limit` has been reached.
482
+ The default `{}` represents an empty object in JSON.
483
+ Depending on schema definitions, this may cause schema validation errors.
484
+ If this is the case, 'None' (``${NONE}`` in Robot Framework) or an empty list
485
+ can be tried as an alternative.
486
+
487
+ == Security-related parameters ==
488
+ _Note: these parameters are equivalent to those in the ``requests`` library._
489
+
490
+ === username ===
491
+ The username to be used for Basic Authentication.
492
+
493
+ === password ===
494
+ The password to be used for Basic Authentication.
495
+
496
+ === security_token ===
497
+ The token to be used for token based security using the ``Authorization`` header.
498
+
499
+ === auth ===
500
+ A [https://requests.readthedocs.io/en/latest/api/#authentication | requests ``AuthBase`` instance]
501
+ to be used for authentication instead of the ``username`` and ``password``.
502
+
503
+ === cert ===
504
+ The SSL certificate to use with all requests.
505
+ If string: the path to ssl client cert file (.pem).
506
+ If tuple: the ('cert', 'key') pair.
507
+
508
+ === verify_tls ===
509
+ Whether or not to verify the TLS / SSL certificate of the server.
510
+ If boolean: whether or not to verify the server TLS certificate.
511
+ If string: path to a CA bundle to use for verification.
512
+
513
+ === extra_headers ===
514
+ A dictionary with extra / custom headers that will be send with every request.
515
+ This parameter can be used to send headers that are not documented in the
516
+ openapi document or to provide an API-key.
517
+
518
+ === cookies ===
519
+ A dictionary or
520
+ [https://docs.python.org/3/library/http.cookiejar.html#http.cookiejar.CookieJar | CookieJar object]
521
+ to send with all requests.
522
+
523
+ === proxies ===
524
+ A dictionary of 'protocol': 'proxy url' to use for all requests.
525
+ """
526
+ self._source = source
527
+ self._origin = origin
528
+ self._base_path = base_path
529
+ self._recursion_limit = recursion_limit
530
+ self._recursion_default = recursion_default
531
+ self.session = Session()
532
+ # only username and password, security_token or auth object should be provided
533
+ # if multiple are provided, username and password take precedence
534
+ self.security_token = security_token
535
+ self.auth = auth
536
+ if username and password:
537
+ self.auth = HTTPBasicAuth(username, password)
538
+ # Robot Framework does not allow users to create tuples and requests
539
+ # does not accept lists, so perform the conversion here
540
+ if isinstance(cert, list):
541
+ cert = tuple(cert)
542
+ self.cert = cert
543
+ self.verify = verify_tls
544
+ self.extra_headers = extra_headers
545
+ self.cookies = cookies
546
+ self.proxies = proxies
547
+ self.invalid_property_default_response = invalid_property_default_response
548
+ if mappings_path and str(mappings_path) != ".":
549
+ mappings_path = Path(mappings_path)
550
+ if not mappings_path.is_file():
551
+ logger.warning(
552
+ f"mappings_path '{mappings_path}' is not a Python module."
553
+ )
554
+ # intermediate variable to ensure path.append is possible so we'll never
555
+ # path.pop a location that we didn't append
556
+ mappings_folder = str(mappings_path.parent)
557
+ sys.path.append(mappings_folder)
558
+ mappings_module_name = mappings_path.stem
559
+ self.get_dto_class = get_dto_class(
560
+ mappings_module_name=mappings_module_name
561
+ )
562
+ self.get_id_property_name = get_id_property_name(
563
+ mappings_module_name=mappings_module_name
564
+ )
565
+ sys.path.pop()
566
+ else:
567
+ self.get_dto_class = get_dto_class(mappings_module_name="no mapping")
568
+ self.get_id_property_name = get_id_property_name(
569
+ mappings_module_name="no mapping"
570
+ )
571
+ if faker_locale:
572
+ FAKE.set_locale(locale=faker_locale)
573
+ # update the globally available DEFAULT_ID_PROPERTY_NAME to the provided value
574
+ DEFAULT_ID_PROPERTY_NAME.id_property_name = default_id_property_name
575
+
576
+ @property
577
+ def origin(self) -> str:
578
+ return self._origin
579
+
580
+ @keyword
581
+ def set_origin(self, origin: str) -> None:
582
+ """
583
+ Update the `origin` after the library is imported.
584
+
585
+ This can be done during the `Suite setup` when using DataDriver in situations
586
+ where the OpenAPI document is available on disk but the target host address is
587
+ not known before the test starts.
588
+
589
+ In combination with OpenApiLibCore, the `origin` can be used at any point to
590
+ target another server that hosts an API that complies to the same OAS.
591
+ """
592
+ self._origin = origin
593
+
594
+ @property
595
+ def base_url(self) -> str:
596
+ return f"{self.origin}{self._base_path}"
597
+
598
+ @cached_property
599
+ def validation_spec(self) -> Spec:
600
+ return Spec.from_dict(self.openapi_spec)
601
+
602
+ @property
603
+ def openapi_spec(self) -> Dict[str, Any]:
604
+ """Return a deepcopy of the parsed openapi document."""
605
+ # protect the parsed openapi spec from being mutated by reference
606
+ return deepcopy(self._openapi_spec)
607
+
608
+ @cached_property
609
+ def _openapi_spec(self) -> Dict[str, Any]:
610
+ parser = self._load_parser()
611
+ return parser.specification
612
+
613
+ def read_paths(self) -> Dict[str, Any]:
614
+ return self.openapi_spec["paths"]
615
+
616
+ def _load_parser(self) -> ResolvingParser:
617
+ try:
618
+
619
+ def recursion_limit_handler(
620
+ limit: int, refstring: str, recursions: Any
621
+ ) -> Any:
622
+ return self._recursion_default
623
+
624
+ # Since parsing of the OAS and creating the Spec can take a long time,
625
+ # they are cached. This is done by storing them in an imported module that
626
+ # will have a global scope due to how the Python import system works. This
627
+ # ensures that in a Suite of Suites where multiple Suites use the same
628
+ # `source`, that OAS is only parsed / loaded once.
629
+ parser = PARSER_CACHE.get(self._source, None)
630
+ if parser is None:
631
+ parser = ResolvingParser(
632
+ self._source,
633
+ backend="openapi-spec-validator",
634
+ recursion_limit=self._recursion_limit,
635
+ recursion_limit_handler=recursion_limit_handler,
636
+ )
637
+
638
+ if parser.specification is None: # pragma: no cover
639
+ BuiltIn().fatal_error(
640
+ "Source was loaded, but no specification was present after parsing."
641
+ )
642
+
643
+ PARSER_CACHE[self._source] = parser
644
+
645
+ return parser
646
+
647
+ except ResolutionError as exception:
648
+ BuiltIn().fatal_error(
649
+ f"ResolutionError while trying to load openapi spec: {exception}"
650
+ )
651
+ except ValidationError as exception:
652
+ BuiltIn().fatal_error(
653
+ f"ValidationError while trying to load openapi spec: {exception}"
654
+ )
655
+
656
+ def validate_response_vs_spec(
657
+ self, request: RequestsOpenAPIRequest, response: RequestsOpenAPIResponse
658
+ ) -> None:
659
+ """
660
+ Validate the reponse for a given request against the OpenAPI Spec that is
661
+ loaded during library initialization.
662
+ """
663
+ if response.content_type == "application/json":
664
+ config = None
665
+ else:
666
+ extra_deserializer = {response.content_type: _json.loads}
667
+ config = Config(extra_media_type_deserializers=extra_deserializer)
668
+
669
+ OpenAPI(spec=self.validation_spec, config=config).validate_response(
670
+ request, response
671
+ )
672
+
673
+ @keyword
674
+ def get_valid_url(self, endpoint: str, method: str) -> str:
675
+ """
676
+ This keyword returns a valid url for the given `endpoint` and `method`.
677
+
678
+ If the `endpoint` contains path parameters the Get Valid Id For Endpoint
679
+ keyword will be executed to retrieve valid ids for the path parameters.
680
+
681
+ > Note: if valid ids cannot be retrieved within the scope of the API, the
682
+ `PathPropertiesConstraint` Relation can be used. More information can be found
683
+ [https://marketsquare.github.io/robotframework-openapi-libcore/advanced_use.html | here].
684
+ """
685
+ method = method.lower()
686
+ try:
687
+ # endpoint can be partially resolved or provided by a PathPropertiesConstraint
688
+ parametrized_endpoint = self.get_parametrized_endpoint(endpoint=endpoint)
689
+ _ = self.openapi_spec["paths"][parametrized_endpoint]
690
+ except KeyError:
691
+ raise ValueError(
692
+ f"{endpoint} not found in paths section of the OpenAPI document."
693
+ ) from None
694
+ dto_class = self.get_dto_class(endpoint=endpoint, method=method)
695
+ relations = dto_class.get_relations()
696
+ paths = [p.path for p in relations if isinstance(p, PathPropertiesConstraint)]
697
+ if paths:
698
+ url = f"{self.base_url}{choice(paths)}"
699
+ return url
700
+ endpoint_parts = list(endpoint.split("/"))
701
+ for index, part in enumerate(endpoint_parts):
702
+ if part.startswith("{") and part.endswith("}"):
703
+ type_endpoint_parts = endpoint_parts[slice(index)]
704
+ type_endpoint = "/".join(type_endpoint_parts)
705
+ existing_id: Union[str, int, float] = run_keyword(
706
+ "get_valid_id_for_endpoint", type_endpoint, method
707
+ )
708
+ endpoint_parts[index] = str(existing_id)
709
+ resolved_endpoint = "/".join(endpoint_parts)
710
+ url = f"{self.base_url}{resolved_endpoint}"
711
+ return url
712
+
713
+ @keyword
714
+ def get_valid_id_for_endpoint(
715
+ self, endpoint: str, method: str
716
+ ) -> Union[str, int, float]:
717
+ """
718
+ Support keyword that returns the `id` for an existing resource at `endpoint`.
719
+
720
+ To prevent resource conflicts with other test cases, a new resource is created
721
+ (POST) if possible.
722
+ """
723
+
724
+ def dummy_transformer(
725
+ valid_id: Union[str, int, float]
726
+ ) -> Union[str, int, float]:
727
+ return valid_id
728
+
729
+ method = method.lower()
730
+ url: str = run_keyword("get_valid_url", endpoint, method)
731
+ # Try to create a new resource to prevent conflicts caused by
732
+ # operations performed on the same resource by other test cases
733
+ request_data = self.get_request_data(endpoint=endpoint, method="post")
734
+
735
+ response: Response = run_keyword(
736
+ "authorized_request",
737
+ url,
738
+ "post",
739
+ request_data.get_required_params(),
740
+ request_data.get_required_headers(),
741
+ request_data.get_required_properties_dict(),
742
+ )
743
+
744
+ # determine the id property name for this path and whether or not a transformer is used
745
+ mapping = self.get_id_property_name(endpoint=endpoint)
746
+ if isinstance(mapping, str):
747
+ id_property = mapping
748
+ # set the transformer to a dummy callable that returns the original value so
749
+ # the transformer can be applied on any returned id
750
+ id_transformer = dummy_transformer
751
+ else:
752
+ id_property, id_transformer = mapping
753
+
754
+ if not response.ok:
755
+ # If a new resource cannot be created using POST, try to retrieve a
756
+ # valid id using a GET request.
757
+ try:
758
+ valid_id = choice(run_keyword("get_ids_from_url", url))
759
+ return id_transformer(valid_id)
760
+ except Exception as exception:
761
+ raise AssertionError(
762
+ f"Failed to get a valid id using GET on {url}"
763
+ ) from exception
764
+
765
+ response_data = response.json()
766
+ if prepared_body := response.request.body:
767
+ if isinstance(prepared_body, bytes):
768
+ send_json = _json.loads(prepared_body.decode("UTF-8"))
769
+ else:
770
+ send_json = _json.loads(prepared_body)
771
+ else:
772
+ send_json = None
773
+
774
+ # no support for retrieving an id from an array returned on a POST request
775
+ if isinstance(response_data, list):
776
+ raise NotImplementedError(
777
+ f"Unexpected response body for POST request: expected an object but "
778
+ f"received an array ({response_data})"
779
+ )
780
+
781
+ # POST on /resource_type/{id}/array_item/ will return the updated {id} resource
782
+ # instead of a newly created resource. In this case, the send_json must be
783
+ # in the array of the 'array_item' property on {id}
784
+ send_path: str = response.request.path_url
785
+ response_href: Optional[str] = response_data.get("href", None)
786
+ if response_href and (send_path not in response_href) and send_json:
787
+ try:
788
+ property_to_check = send_path.replace(response_href, "")[1:]
789
+ item_list: List[Dict[str, Any]] = response_data[property_to_check]
790
+ # Use the (mandatory) id to get the POSTed resource from the list
791
+ [valid_id] = [
792
+ item[id_property]
793
+ for item in item_list
794
+ if item[id_property] == send_json[id_property]
795
+ ]
796
+ except Exception as exception:
797
+ raise AssertionError(
798
+ f"Failed to get a valid id from {response_href}"
799
+ ) from exception
800
+ else:
801
+ try:
802
+ valid_id = response_data[id_property]
803
+ except KeyError:
804
+ raise AssertionError(
805
+ f"Failed to get a valid id from {response_data}"
806
+ ) from None
807
+ return id_transformer(valid_id)
808
+
809
+ @keyword
810
+ def get_ids_from_url(self, url: str) -> List[str]:
811
+ """
812
+ Perform a GET request on the `url` and return the list of resource
813
+ `ids` from the response.
814
+ """
815
+ endpoint = self.get_parameterized_endpoint_from_url(url)
816
+ request_data = self.get_request_data(endpoint=endpoint, method="get")
817
+ response = run_keyword(
818
+ "authorized_request",
819
+ url,
820
+ "get",
821
+ request_data.get_required_params(),
822
+ request_data.get_required_headers(),
823
+ )
824
+ response.raise_for_status()
825
+ response_data: Union[Dict[str, Any], List[Dict[str, Any]]] = response.json()
826
+
827
+ # determine the property name to use
828
+ mapping = self.get_id_property_name(endpoint=endpoint)
829
+ if isinstance(mapping, str):
830
+ id_property = mapping
831
+ else:
832
+ id_property, _ = mapping
833
+
834
+ if isinstance(response_data, list):
835
+ valid_ids: List[str] = [item[id_property] for item in response_data]
836
+ return valid_ids
837
+ # if the response is an object (dict), check if it's hal+json
838
+ if embedded := response_data.get("_embedded"):
839
+ # there should be 1 item in the dict that has a value that's a list
840
+ for value in embedded.values():
841
+ if isinstance(value, list):
842
+ valid_ids = [item[id_property] for item in value]
843
+ return valid_ids
844
+ if (valid_id := response_data.get(id_property)) is not None:
845
+ return [valid_id]
846
+ valid_ids = [item[id_property] for item in response_data["items"]]
847
+ return valid_ids
848
+
849
+ @keyword
850
+ def get_request_data(self, endpoint: str, method: str) -> RequestData:
851
+ """Return an object with valid request data for body, headers and query params."""
852
+ method = method.lower()
853
+ dto_cls_name = self._get_dto_cls_name(endpoint=endpoint, method=method)
854
+ # The endpoint can contain already resolved Ids that have to be matched
855
+ # against the parametrized endpoints in the paths section.
856
+ spec_endpoint = self.get_parametrized_endpoint(endpoint)
857
+ dto_class = self.get_dto_class(endpoint=spec_endpoint, method=method)
858
+ try:
859
+ method_spec = self.openapi_spec["paths"][spec_endpoint][method]
860
+ except KeyError:
861
+ logger.info(
862
+ f"method '{method}' not supported on '{spec_endpoint}, using empty spec."
863
+ )
864
+ method_spec = {}
865
+
866
+ parameters, params, headers = self.get_request_parameters(
867
+ dto_class=dto_class, method_spec=method_spec
868
+ )
869
+ if (body_spec := method_spec.get("requestBody", None)) is None:
870
+ if dto_class == DefaultDto:
871
+ dto_instance: Dto = DefaultDto()
872
+ else:
873
+ dto_class = make_dataclass(
874
+ cls_name=method_spec.get("operationId", dto_cls_name),
875
+ fields=[],
876
+ bases=(dto_class,),
877
+ )
878
+ dto_instance = dto_class()
879
+ return RequestData(
880
+ dto=dto_instance,
881
+ parameters=parameters,
882
+ params=params,
883
+ headers=headers,
884
+ has_body=False,
885
+ )
886
+ content_schema = resolve_schema(self.get_content_schema(body_spec))
887
+ headers.update({"content-type": self.get_content_type(body_spec)})
888
+ dto_data = self.get_json_data_for_dto_class(
889
+ schema=content_schema,
890
+ dto_class=dto_class,
891
+ operation_id=method_spec.get("operationId", ""),
892
+ )
893
+ if dto_data is None:
894
+ dto_instance = DefaultDto()
895
+ else:
896
+ fields = self.get_fields_from_dto_data(content_schema, dto_data)
897
+ dto_class = make_dataclass(
898
+ cls_name=method_spec.get("operationId", dto_cls_name),
899
+ fields=fields,
900
+ bases=(dto_class,),
901
+ )
902
+ dto_data = {get_safe_key(key): value for key, value in dto_data.items()}
903
+ dto_instance = dto_class(**dto_data)
904
+ return RequestData(
905
+ dto=dto_instance,
906
+ dto_schema=content_schema,
907
+ parameters=parameters,
908
+ params=params,
909
+ headers=headers,
910
+ )
911
+
912
+ @staticmethod
913
+ def _get_dto_cls_name(endpoint: str, method: str) -> str:
914
+ method = method.capitalize()
915
+ path = endpoint.translate({ord(i): None for i in "{}"})
916
+ path_parts = path.split("/")
917
+ path_parts = [p.capitalize() for p in path_parts]
918
+ result = "".join([method, *path_parts])
919
+ return result
920
+
921
+ @staticmethod
922
+ def get_fields_from_dto_data(
923
+ content_schema: Dict[str, Any], dto_data: Dict[str, Any]
924
+ ):
925
+ # FIXME: annotation is not Pyhon 3.8-compatible
926
+ # ) -> List[Union[str, Tuple[str, Type[Any]], Tuple[str, Type[Any], Field[Any]]]]:
927
+ """Get a dataclasses fields list based on the content_schema and dto_data."""
928
+ fields: List[
929
+ Union[str, Tuple[str, Type[Any]], Tuple[str, Type[Any], Field[Any]]]
930
+ ] = []
931
+ for key, value in dto_data.items():
932
+ required_properties = content_schema.get("required", [])
933
+ safe_key = get_safe_key(key)
934
+ metadata = {"original_property_name": key}
935
+ if key in required_properties:
936
+ # The fields list is used to create a dataclass, so non-default fields
937
+ # must go before fields with a default
938
+ fields.insert(0, (safe_key, type(value), field(metadata=metadata)))
939
+ else:
940
+ fields.append((safe_key, type(value), field(default=None, metadata=metadata))) # type: ignore[arg-type]
941
+ return fields
942
+
943
+ def get_request_parameters(
944
+ self, dto_class: Union[Dto, Type[Dto]], method_spec: Dict[str, Any]
945
+ ) -> Tuple[List[Dict[str, Any]], Dict[str, Any], Dict[str, str]]:
946
+ """Get the methods parameter spec and params and headers with valid data."""
947
+ parameters = method_spec.get("parameters", [])
948
+ parameter_relations = dto_class.get_parameter_relations()
949
+ query_params = [p for p in parameters if p.get("in") == "query"]
950
+ header_params = [p for p in parameters if p.get("in") == "header"]
951
+ params = self.get_parameter_data(query_params, parameter_relations)
952
+ headers = self.get_parameter_data(header_params, parameter_relations)
953
+ return parameters, params, headers
954
+
955
+ @classmethod
956
+ def get_content_schema(cls, body_spec: Dict[str, Any]) -> Dict[str, Any]:
957
+ """Get the content schema from the requestBody spec."""
958
+ content_type = cls.get_content_type(body_spec)
959
+ content_schema = body_spec["content"][content_type]["schema"]
960
+ return resolve_schema(content_schema)
961
+
962
+ @staticmethod
963
+ def get_content_type(body_spec: Dict[str, Any]) -> str:
964
+ """Get and validate the first supported content type from the requested body spec
965
+
966
+ Should be application/json like content type,
967
+ e.g "application/json;charset=utf-8" or "application/merge-patch+json"
968
+ """
969
+ content_types: List[str] = body_spec["content"].keys()
970
+ json_regex = r"application/([a-z\-]+\+)?json(;\s?charset=(.+))?"
971
+ for content_type in content_types:
972
+ if re.search(json_regex, content_type):
973
+ return content_type
974
+
975
+ # At present no supported for other types.
976
+ raise NotImplementedError(
977
+ f"Only content types like 'application/json' are supported. "
978
+ f"Content types definded in the spec are '{content_types}'."
979
+ )
980
+
981
+ def get_parametrized_endpoint(self, endpoint: str) -> str:
982
+ """
983
+ Get the parametrized endpoint as found in the `paths` section of the openapi
984
+ document from a (partially) resolved endpoint.
985
+ """
986
+
987
+ def match_parts(parts: List[str], spec_parts: List[str]) -> bool:
988
+ for part, spec_part in zip_longest(parts, spec_parts, fillvalue="Filler"):
989
+ if part == "Filler" or spec_part == "Filler":
990
+ return False
991
+ if part != spec_part and not spec_part.startswith("{"):
992
+ return False
993
+ return True
994
+
995
+ endpoint_parts = endpoint.split("/")
996
+ # if the last part is empty, the path has a trailing `/` that
997
+ # should be ignored during matching
998
+ if endpoint_parts[-1] == "":
999
+ _ = endpoint_parts.pop(-1)
1000
+
1001
+ spec_endpoints: List[str] = {**self.openapi_spec}["paths"].keys()
1002
+
1003
+ candidates: List[str] = []
1004
+
1005
+ for spec_endpoint in spec_endpoints:
1006
+ spec_endpoint_parts = spec_endpoint.split("/")
1007
+ # ignore trailing `/` the same way as for endpoint_parts
1008
+ if spec_endpoint_parts[-1] == "":
1009
+ _ = spec_endpoint_parts.pop(-1)
1010
+ if match_parts(endpoint_parts, spec_endpoint_parts):
1011
+ candidates.append(spec_endpoint)
1012
+
1013
+ if not candidates:
1014
+ raise ValueError(
1015
+ f"{endpoint} not found in paths section of the OpenAPI document."
1016
+ )
1017
+
1018
+ if len(candidates) == 1:
1019
+ return candidates[0]
1020
+ # Multiple matches can happen in APIs with overloaded endpoints, e.g.
1021
+ # /users/me
1022
+ # /users/${user_id}
1023
+ # In this case, find the closest (or exact) match
1024
+ exact_match = [c for c in candidates if c == endpoint]
1025
+ if exact_match:
1026
+ return exact_match[0]
1027
+ # TODO: Implement a decision mechanism when real-world examples become available
1028
+ # In the face of ambiguity, refuse the temptation to guess.
1029
+ raise ValueError(f"{endpoint} matched to multiple paths: {candidates}")
1030
+
1031
+ @staticmethod
1032
+ def get_parameter_data(
1033
+ parameters: List[Dict[str, Any]],
1034
+ parameter_relations: List[Relation],
1035
+ ) -> Dict[str, str]:
1036
+ """Generate a valid list of key-value pairs for all parameters."""
1037
+ result: Dict[str, str] = {}
1038
+ value: Any = None
1039
+ for parameter in parameters:
1040
+ parameter_name = parameter["name"]
1041
+ parameter_schema = resolve_schema(parameter["schema"])
1042
+ relations = [
1043
+ r for r in parameter_relations if r.property_name == parameter_name
1044
+ ]
1045
+ if constrained_values := [
1046
+ r.values for r in relations if isinstance(r, PropertyValueConstraint)
1047
+ ]:
1048
+ value = choice(*constrained_values)
1049
+ if value is IGNORE:
1050
+ continue
1051
+ result[parameter_name] = value
1052
+ continue
1053
+ value = value_utils.get_valid_value(parameter_schema)
1054
+ result[parameter_name] = value
1055
+ return result
1056
+
1057
+ @keyword
1058
+ def get_json_data_for_dto_class(
1059
+ self,
1060
+ schema: Dict[str, Any],
1061
+ dto_class: Union[Dto, Type[Dto]],
1062
+ operation_id: str = "",
1063
+ ) -> Optional[Dict[str, Any]]:
1064
+ """
1065
+ Generate a valid (json-compatible) dict for all the `dto_class` properties.
1066
+ """
1067
+
1068
+ def get_constrained_values(property_name: str) -> List[Any]:
1069
+ relations = dto_class.get_relations()
1070
+ values_list = [
1071
+ c.values
1072
+ for c in relations
1073
+ if (
1074
+ isinstance(c, PropertyValueConstraint)
1075
+ and c.property_name == property_name
1076
+ )
1077
+ ]
1078
+ # values should be empty or contain 1 list of allowed values
1079
+ return values_list.pop() if values_list else []
1080
+
1081
+ def get_dependent_id(
1082
+ property_name: str, operation_id: str
1083
+ ) -> Optional[Union[str, int, float]]:
1084
+ relations = dto_class.get_relations()
1085
+ # multiple get paths are possible based on the operation being performed
1086
+ id_get_paths = [
1087
+ (d.get_path, d.operation_id)
1088
+ for d in relations
1089
+ if (isinstance(d, IdDependency) and d.property_name == property_name)
1090
+ ]
1091
+ if not id_get_paths:
1092
+ return None
1093
+ if len(id_get_paths) == 1:
1094
+ id_get_path, _ = id_get_paths.pop()
1095
+ else:
1096
+ try:
1097
+ [id_get_path] = [
1098
+ path
1099
+ for path, operation in id_get_paths
1100
+ if operation == operation_id
1101
+ ]
1102
+ # There could be multiple get_paths, but not one for the current operation
1103
+ except ValueError:
1104
+ return None
1105
+ valid_id = self.get_valid_id_for_endpoint(
1106
+ endpoint=id_get_path, method="get"
1107
+ )
1108
+ logger.debug(f"get_dependent_id for {id_get_path} returned {valid_id}")
1109
+ return valid_id
1110
+
1111
+ json_data: Dict[str, Any] = {}
1112
+
1113
+ property_names = []
1114
+ for property_name in schema.get("properties", []):
1115
+ if constrained_values := get_constrained_values(property_name):
1116
+ # do not add properties that are configured to be ignored
1117
+ if IGNORE in constrained_values:
1118
+ continue
1119
+ property_names.append(property_name)
1120
+
1121
+ max_properties = schema.get("maxProperties")
1122
+ if max_properties and len(property_names) > max_properties:
1123
+ required_properties = schema.get("required", [])
1124
+ number_of_optional_properties = max_properties - len(required_properties)
1125
+ optional_properties = [
1126
+ name for name in property_names if name not in required_properties
1127
+ ]
1128
+ selected_optional_properties = sample(
1129
+ optional_properties, number_of_optional_properties
1130
+ )
1131
+ property_names = required_properties + selected_optional_properties
1132
+
1133
+ for property_name in property_names:
1134
+ properties_schema = schema["properties"][property_name]
1135
+
1136
+ property_type = properties_schema.get("type")
1137
+ if property_type is None:
1138
+ property_types = properties_schema.get("types")
1139
+ if property_types is None:
1140
+ if properties_schema.get("properties") is not None:
1141
+ nested_data = self.get_json_data_for_dto_class(
1142
+ schema=properties_schema,
1143
+ dto_class=DefaultDto,
1144
+ )
1145
+ json_data[property_name] = nested_data
1146
+ continue
1147
+ selected_type_schema = choice(property_types)
1148
+ property_type = selected_type_schema["type"]
1149
+ if properties_schema.get("readOnly", False):
1150
+ continue
1151
+ if constrained_values := get_constrained_values(property_name):
1152
+ json_data[property_name] = choice(constrained_values)
1153
+ continue
1154
+ if (
1155
+ dependent_id := get_dependent_id(
1156
+ property_name=property_name, operation_id=operation_id
1157
+ )
1158
+ ) is not None:
1159
+ json_data[property_name] = dependent_id
1160
+ continue
1161
+ if property_type == "object":
1162
+ object_data = self.get_json_data_for_dto_class(
1163
+ schema=properties_schema,
1164
+ dto_class=DefaultDto,
1165
+ operation_id="",
1166
+ )
1167
+ json_data[property_name] = object_data
1168
+ continue
1169
+ if property_type == "array":
1170
+ array_data = self.get_json_data_for_dto_class(
1171
+ schema=properties_schema["items"],
1172
+ dto_class=DefaultDto,
1173
+ operation_id=operation_id,
1174
+ )
1175
+ json_data[property_name] = [array_data]
1176
+ continue
1177
+ json_data[property_name] = value_utils.get_valid_value(properties_schema)
1178
+
1179
+ return json_data
1180
+
1181
+ @keyword
1182
+ def get_invalidated_url(self, valid_url: str) -> Optional[str]:
1183
+ """
1184
+ Return an url with all the path parameters in the `valid_url` replaced by a
1185
+ random UUID.
1186
+
1187
+ Raises ValueError if the valid_url cannot be invalidated.
1188
+ """
1189
+ parameterized_endpoint = self.get_parameterized_endpoint_from_url(valid_url)
1190
+ parameterized_url = self.base_url + parameterized_endpoint
1191
+ valid_url_parts = list(reversed(valid_url.split("/")))
1192
+ parameterized_parts = reversed(parameterized_url.split("/"))
1193
+ for index, (parameterized_part, _) in enumerate(
1194
+ zip(parameterized_parts, valid_url_parts)
1195
+ ):
1196
+ if parameterized_part.startswith("{") and parameterized_part.endswith("}"):
1197
+ valid_url_parts[index] = uuid4().hex
1198
+ valid_url_parts.reverse()
1199
+ invalid_url = "/".join(valid_url_parts)
1200
+ return invalid_url
1201
+ raise ValueError(f"{parameterized_endpoint} could not be invalidated.")
1202
+
1203
+ @keyword
1204
+ def get_parameterized_endpoint_from_url(self, url: str) -> str:
1205
+ """
1206
+ Return the endpoint as found in the `paths` section based on the given `url`.
1207
+ """
1208
+ endpoint = url.replace(self.base_url, "")
1209
+ endpoint_parts = endpoint.split("/")
1210
+ # first part will be '' since an endpoint starts with /
1211
+ endpoint_parts.pop(0)
1212
+ parameterized_endpoint = self.get_parametrized_endpoint(endpoint=endpoint)
1213
+ return parameterized_endpoint
1214
+
1215
+ @keyword
1216
+ def get_invalid_json_data(
1217
+ self,
1218
+ url: str,
1219
+ method: str,
1220
+ status_code: int,
1221
+ request_data: RequestData,
1222
+ ) -> Dict[str, Any]:
1223
+ """
1224
+ Return `json_data` based on the `dto` on the `request_data` that will cause
1225
+ the provided `status_code` for the `method` operation on the `url`.
1226
+
1227
+ > Note: applicable UniquePropertyValueConstraint and IdReference Relations are
1228
+ considered before changes to `json_data` are made.
1229
+ """
1230
+ method = method.lower()
1231
+ data_relations = request_data.dto.get_relations_for_error_code(status_code)
1232
+ if not data_relations:
1233
+ if not request_data.dto_schema:
1234
+ raise ValueError(
1235
+ "Failed to invalidate: no data_relations and empty schema."
1236
+ )
1237
+ json_data = request_data.dto.get_invalidated_data(
1238
+ schema=request_data.dto_schema,
1239
+ status_code=status_code,
1240
+ invalid_property_default_code=self.invalid_property_default_response,
1241
+ )
1242
+ return json_data
1243
+ resource_relation = choice(data_relations)
1244
+ if isinstance(resource_relation, UniquePropertyValueConstraint):
1245
+ json_data = run_keyword(
1246
+ "get_json_data_with_conflict",
1247
+ url,
1248
+ method,
1249
+ request_data.dto,
1250
+ status_code,
1251
+ )
1252
+ elif isinstance(resource_relation, IdReference):
1253
+ run_keyword("ensure_in_use", url, resource_relation)
1254
+ json_data = request_data.dto.as_dict()
1255
+ else:
1256
+ json_data = request_data.dto.get_invalidated_data(
1257
+ schema=request_data.dto_schema,
1258
+ status_code=status_code,
1259
+ invalid_property_default_code=self.invalid_property_default_response,
1260
+ )
1261
+ return json_data
1262
+
1263
+ @keyword
1264
+ def get_invalidated_parameters(
1265
+ self,
1266
+ status_code: int,
1267
+ request_data: RequestData,
1268
+ ) -> Tuple[Dict[str, Any], Dict[str, str]]:
1269
+ """
1270
+ Returns a version of `params, headers` as present on `request_data` that has
1271
+ been modified to cause the provided `status_code`.
1272
+ """
1273
+ if not request_data.parameters:
1274
+ raise ValueError("No params or headers to invalidate.")
1275
+
1276
+ # ensure the status_code can be triggered
1277
+ relations = request_data.dto.get_parameter_relations_for_error_code(status_code)
1278
+ relations_for_status_code = [
1279
+ r
1280
+ for r in relations
1281
+ if isinstance(r, PropertyValueConstraint)
1282
+ and (
1283
+ r.error_code == status_code or r.invalid_value_error_code == status_code
1284
+ )
1285
+ ]
1286
+ parameters_to_ignore = {
1287
+ r.property_name
1288
+ for r in relations_for_status_code
1289
+ if r.invalid_value_error_code == status_code and r.invalid_value == IGNORE
1290
+ }
1291
+ relation_property_names = {r.property_name for r in relations_for_status_code}
1292
+ if not relation_property_names:
1293
+ if status_code != self.invalid_property_default_response:
1294
+ raise ValueError(
1295
+ f"No relations to cause status_code {status_code} found."
1296
+ )
1297
+
1298
+ # ensure we're not modifying mutable properties
1299
+ params = deepcopy(request_data.params)
1300
+ headers = deepcopy(request_data.headers)
1301
+
1302
+ if status_code == self.invalid_property_default_response:
1303
+ # take the params and headers that can be invalidated based on data type
1304
+ # and expand the set with properties that can be invalided by relations
1305
+ parameter_names = set(request_data.params_that_can_be_invalidated).union(
1306
+ request_data.headers_that_can_be_invalidated
1307
+ )
1308
+ parameter_names.update(relation_property_names)
1309
+ if not parameter_names:
1310
+ raise ValueError(
1311
+ "None of the query parameters and headers can be invalidated."
1312
+ )
1313
+ else:
1314
+ # non-default status_codes can only be the result of a Relation
1315
+ parameter_names = relation_property_names
1316
+
1317
+ # Dto mappings may contain generic mappings for properties that are not present
1318
+ # in this specific schema
1319
+ request_data_parameter_names = [p.get("name") for p in request_data.parameters]
1320
+ additional_relation_property_names = {
1321
+ n for n in relation_property_names if n not in request_data_parameter_names
1322
+ }
1323
+ if additional_relation_property_names:
1324
+ logger.warning(
1325
+ f"get_parameter_relations_for_error_code yielded properties that are "
1326
+ f"not defined in the schema: {additional_relation_property_names}\n"
1327
+ f"These properties will be ignored for parameter invalidation."
1328
+ )
1329
+ parameter_names = parameter_names - additional_relation_property_names
1330
+
1331
+ if not parameter_names:
1332
+ raise ValueError(
1333
+ f"No parameter can be changed to cause status_code {status_code}."
1334
+ )
1335
+
1336
+ parameter_names = parameter_names - parameters_to_ignore
1337
+ parameter_to_invalidate = choice(tuple(parameter_names))
1338
+
1339
+ # check for invalid parameters in the provided request_data
1340
+ try:
1341
+ [parameter_data] = [
1342
+ data
1343
+ for data in request_data.parameters
1344
+ if data["name"] == parameter_to_invalidate
1345
+ ]
1346
+ except Exception:
1347
+ raise ValueError(
1348
+ f"{parameter_to_invalidate} not found in provided parameters."
1349
+ ) from None
1350
+
1351
+ # get the invalid_value for the chosen parameter
1352
+ try:
1353
+ [invalid_value_for_error_code] = [
1354
+ r.invalid_value
1355
+ for r in relations_for_status_code
1356
+ if r.property_name == parameter_to_invalidate
1357
+ and r.invalid_value_error_code == status_code
1358
+ ]
1359
+ except ValueError:
1360
+ invalid_value_for_error_code = NOT_SET
1361
+
1362
+ # get the constraint values if available for the chosen parameter
1363
+ try:
1364
+ [values_from_constraint] = [
1365
+ r.values
1366
+ for r in relations_for_status_code
1367
+ if r.property_name == parameter_to_invalidate
1368
+ ]
1369
+ except ValueError:
1370
+ values_from_constraint = []
1371
+
1372
+ # if the parameter was not provided, add it to params / headers
1373
+ params, headers = self.ensure_parameter_in_parameters(
1374
+ parameter_to_invalidate=parameter_to_invalidate,
1375
+ params=params,
1376
+ headers=headers,
1377
+ parameter_data=parameter_data,
1378
+ values_from_constraint=values_from_constraint,
1379
+ )
1380
+
1381
+ # determine the invalid_value
1382
+ if invalid_value_for_error_code != NOT_SET:
1383
+ invalid_value = invalid_value_for_error_code
1384
+ else:
1385
+ if parameter_to_invalidate in params.keys():
1386
+ valid_value = params[parameter_to_invalidate]
1387
+ else:
1388
+ valid_value = headers[parameter_to_invalidate]
1389
+
1390
+ value_schema = resolve_schema(parameter_data["schema"])
1391
+ invalid_value = value_utils.get_invalid_value(
1392
+ value_schema=value_schema,
1393
+ current_value=valid_value,
1394
+ values_from_constraint=values_from_constraint,
1395
+ )
1396
+ logger.debug(f"{parameter_to_invalidate} changed to {invalid_value}")
1397
+
1398
+ # update the params / headers and return
1399
+ if parameter_to_invalidate in params.keys():
1400
+ params[parameter_to_invalidate] = invalid_value
1401
+ else:
1402
+ headers[parameter_to_invalidate] = invalid_value
1403
+ return params, headers
1404
+
1405
+ @staticmethod
1406
+ def ensure_parameter_in_parameters(
1407
+ parameter_to_invalidate: str,
1408
+ params: Dict[str, Any],
1409
+ headers: Dict[str, str],
1410
+ parameter_data: Dict[str, Any],
1411
+ values_from_constraint: List[Any],
1412
+ ) -> Tuple[Dict[str, Any], Dict[str, str]]:
1413
+ """
1414
+ Returns the params, headers tuple with parameter_to_invalidate with a valid
1415
+ value to params or headers if not originally present.
1416
+ """
1417
+ if (
1418
+ parameter_to_invalidate not in params.keys()
1419
+ and parameter_to_invalidate not in headers.keys()
1420
+ ):
1421
+ if values_from_constraint:
1422
+ valid_value = choice(values_from_constraint)
1423
+ else:
1424
+ parameter_schema = resolve_schema(parameter_data["schema"])
1425
+ valid_value = value_utils.get_valid_value(parameter_schema)
1426
+ if (
1427
+ parameter_data["in"] == "query"
1428
+ and parameter_to_invalidate not in params.keys()
1429
+ ):
1430
+ params[parameter_to_invalidate] = valid_value
1431
+ if (
1432
+ parameter_data["in"] == "header"
1433
+ and parameter_to_invalidate not in headers.keys()
1434
+ ):
1435
+ headers[parameter_to_invalidate] = valid_value
1436
+ return params, headers
1437
+
1438
+ @keyword
1439
+ def ensure_in_use(self, url: str, resource_relation: IdReference) -> None:
1440
+ """
1441
+ Ensure that the (right-most) `id` of the resource referenced by the `url`
1442
+ is used by the resource defined by the `resource_relation`.
1443
+ """
1444
+ resource_id = ""
1445
+
1446
+ endpoint = url.replace(self.base_url, "")
1447
+ endpoint_parts = endpoint.split("/")
1448
+ parameterized_endpoint = self.get_parametrized_endpoint(endpoint=endpoint)
1449
+ parameterized_endpoint_parts = parameterized_endpoint.split("/")
1450
+ for part, param_part in zip(
1451
+ reversed(endpoint_parts), reversed(parameterized_endpoint_parts)
1452
+ ):
1453
+ if param_part.endswith("}"):
1454
+ resource_id = part
1455
+ break
1456
+ if not resource_id:
1457
+ raise ValueError(f"The provided url ({url}) does not contain an id.")
1458
+ request_data = self.get_request_data(
1459
+ method="post", endpoint=resource_relation.post_path
1460
+ )
1461
+ json_data = request_data.dto.as_dict()
1462
+ json_data[resource_relation.property_name] = resource_id
1463
+ post_url: str = run_keyword(
1464
+ "get_valid_url",
1465
+ resource_relation.post_path,
1466
+ "post",
1467
+ )
1468
+ response: Response = run_keyword(
1469
+ "authorized_request",
1470
+ post_url,
1471
+ "post",
1472
+ request_data.params,
1473
+ request_data.headers,
1474
+ json_data,
1475
+ )
1476
+ if not response.ok:
1477
+ logger.debug(
1478
+ f"POST on {post_url} with json {json_data} failed: {response.json()}"
1479
+ )
1480
+ response.raise_for_status()
1481
+
1482
+ @keyword
1483
+ def get_json_data_with_conflict(
1484
+ self, url: str, method: str, dto: Dto, conflict_status_code: int
1485
+ ) -> Dict[str, Any]:
1486
+ """
1487
+ Return `json_data` based on the `UniquePropertyValueConstraint` that must be
1488
+ returned by the `get_relations` implementation on the `dto` for the given
1489
+ `conflict_status_code`.
1490
+ """
1491
+ method = method.lower()
1492
+ json_data = dto.as_dict()
1493
+ unique_property_value_constraints = [
1494
+ r
1495
+ for r in dto.get_relations()
1496
+ if isinstance(r, UniquePropertyValueConstraint)
1497
+ ]
1498
+ for relation in unique_property_value_constraints:
1499
+ json_data[relation.property_name] = relation.value
1500
+ # create a new resource that the original request will conflict with
1501
+ if method in ["patch", "put"]:
1502
+ post_url_parts = url.split("/")[:-1]
1503
+ post_url = "/".join(post_url_parts)
1504
+ # the PATCH or PUT may use a different dto than required for POST
1505
+ # so a valid POST dto must be constructed
1506
+ endpoint = post_url.replace(self.base_url, "")
1507
+ request_data = self.get_request_data(endpoint=endpoint, method="post")
1508
+ post_json = request_data.dto.as_dict()
1509
+ for key in post_json.keys():
1510
+ if key in json_data:
1511
+ post_json[key] = json_data.get(key)
1512
+ else:
1513
+ post_url = url
1514
+ post_json = json_data
1515
+ endpoint = post_url.replace(self.base_url, "")
1516
+ request_data = self.get_request_data(endpoint=endpoint, method="post")
1517
+ response: Response = run_keyword(
1518
+ "authorized_request",
1519
+ post_url,
1520
+ "post",
1521
+ request_data.params,
1522
+ request_data.headers,
1523
+ post_json,
1524
+ )
1525
+ # conflicting resource may already exist
1526
+ assert (
1527
+ response.ok or response.status_code == conflict_status_code
1528
+ ), f"get_json_data_with_conflict received {response.status_code}: {response.json()}"
1529
+ return json_data
1530
+ raise ValueError(
1531
+ f"No UniquePropertyValueConstraint in the get_relations list on dto {dto}."
1532
+ )
1533
+
1534
+ @keyword
1535
+ def authorized_request( # pylint: disable=too-many-arguments
1536
+ self,
1537
+ url: str,
1538
+ method: str,
1539
+ params: Optional[Dict[str, Any]] = None,
1540
+ headers: Optional[Dict[str, str]] = None,
1541
+ json_data: Optional[JSON] = None,
1542
+ ) -> Response:
1543
+ """
1544
+ Perform a request using the security token or authentication set in the library.
1545
+
1546
+ > Note: provided username / password or auth objects take precedence over token
1547
+ based security
1548
+ """
1549
+ headers = headers if headers else {}
1550
+ if self.extra_headers:
1551
+ headers.update(self.extra_headers)
1552
+ # if both an auth object and a token are available, auth takes precedence
1553
+ if self.security_token and not self.auth:
1554
+ security_header = {"Authorization": self.security_token}
1555
+ headers.update(security_header)
1556
+ headers = {k: str(v) for k, v in headers.items()}
1557
+ response = self.session.request(
1558
+ url=url,
1559
+ method=method,
1560
+ params=params,
1561
+ headers=headers,
1562
+ json=json_data,
1563
+ cookies=self.cookies,
1564
+ auth=self.auth,
1565
+ proxies=self.proxies,
1566
+ verify=self.verify,
1567
+ cert=self.cert,
1568
+ )
1569
+ logger.debug(f"Response text: {response.text}")
1570
+ return response