tracdap-ext-http 0.10.0.dev14__tar.gz → 0.10.0.dev15__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- tracdap_ext_http-0.10.0.dev15/PKG-INFO +172 -0
- tracdap_ext_http-0.10.0.dev15/README.md +142 -0
- {tracdap_ext_http-0.10.0.dev14 → tracdap_ext_http-0.10.0.dev15}/pyproject.toml +3 -5
- tracdap_ext_http-0.10.0.dev15/requirements.txt +2 -0
- {tracdap_ext_http-0.10.0.dev14 → tracdap_ext_http-0.10.0.dev15}/src/tracdap/ext/http/__init__.py +1 -1
- tracdap_ext_http-0.10.0.dev15/src/tracdap/ext/http/http_plugin.py +133 -0
- tracdap_ext_http-0.10.0.dev15/src/tracdap_ext_http.egg-info/PKG-INFO +172 -0
- {tracdap_ext_http-0.10.0.dev14 → tracdap_ext_http-0.10.0.dev15}/src/tracdap_ext_http.egg-info/SOURCES.txt +1 -0
- tracdap_ext_http-0.10.0.dev15/src/tracdap_ext_http.egg-info/requires.txt +1 -0
- tracdap_ext_http-0.10.0.dev14/PKG-INFO +0 -37
- tracdap_ext_http-0.10.0.dev14/README.md +0 -6
- tracdap_ext_http-0.10.0.dev14/requirements.txt +0 -0
- tracdap_ext_http-0.10.0.dev14/src/tracdap/ext/http/http_plugin.py +0 -42
- tracdap_ext_http-0.10.0.dev14/src/tracdap_ext_http.egg-info/PKG-INFO +0 -37
- {tracdap_ext_http-0.10.0.dev14 → tracdap_ext_http-0.10.0.dev15}/LICENSE +0 -0
- {tracdap_ext_http-0.10.0.dev14 → tracdap_ext_http-0.10.0.dev15}/setup.cfg +0 -0
- {tracdap_ext_http-0.10.0.dev14 → tracdap_ext_http-0.10.0.dev15}/src/tracdap_ext_http.egg-info/dependency_links.txt +0 -0
- {tracdap_ext_http-0.10.0.dev14 → tracdap_ext_http-0.10.0.dev15}/src/tracdap_ext_http.egg-info/top_level.txt +0 -0
@@ -0,0 +1,172 @@
|
|
1
|
+
Metadata-Version: 2.4
|
2
|
+
Name: tracdap-ext-http
|
3
|
+
Version: 0.10.0.dev15
|
4
|
+
Summary: An extension for TRAC D.A.P. that lets models connect to HTTP endpoints
|
5
|
+
Author-email: Martin Traverse <martin@fintrac.co.uk>
|
6
|
+
License-Expression: Apache-2.0
|
7
|
+
Project-URL: Homepage, https://tracdap.finos.org/
|
8
|
+
Project-URL: Documentation, https://tracdap.readthedocs.io/
|
9
|
+
Project-URL: Source Code, https://github.com/finos/tracdap
|
10
|
+
Project-URL: Issue Tracker, https://github.com/finos/tracdap/issues
|
11
|
+
Classifier: Operating System :: OS Independent
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
13
|
+
Classifier: Programming Language :: Python :: 3.10
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
16
|
+
Classifier: Programming Language :: Python :: 3.13
|
17
|
+
Classifier: Intended Audience :: Developers
|
18
|
+
Classifier: Intended Audience :: End Users/Desktop
|
19
|
+
Classifier: Intended Audience :: Financial and Insurance Industry
|
20
|
+
Classifier: Topic :: Office/Business
|
21
|
+
Classifier: Topic :: Office/Business :: Financial
|
22
|
+
Classifier: Topic :: Software Development
|
23
|
+
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
|
24
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
25
|
+
Requires-Python: >=3.10
|
26
|
+
Description-Content-Type: text/markdown
|
27
|
+
License-File: LICENSE
|
28
|
+
Requires-Dist: tracdap-runtime>=0.10.0-dev.15
|
29
|
+
Dynamic: license-file
|
30
|
+
|
31
|
+
<h1 align="center">
|
32
|
+
|
33
|
+

|
34
|
+
|
35
|
+
</h1>
|
36
|
+
|
37
|
+
<p align="center">
|
38
|
+
<a href="https://pypi.org/project/tracdap-ext-http"><img alt="PyPI Version" src="https://img.shields.io/pypi/v/tracdap-ext-http.svg?maxAge=86400" /></a>
|
39
|
+
<a href="https://pypi.org/project/tracdap-ext-http"><img alt="Python Versions" src="https://img.shields.io/pypi/pyversions/tracdap-ext-http.svg?maxAge=86400" /></a>
|
40
|
+
<a href="https://github.com/finos/tracdap/actions/workflows/packaging.yaml?query=branch%3Amain"><img alt="Packaging status" src="https://github.com/finos/tracdap/actions/workflows/packaging.yaml/badge.svg?branch:main&workflow:CI" /></a>
|
41
|
+
<a href="https://github.com/finos/tracdap/actions/workflows/compliance.yaml?query=branch%3Amain"><img alt="Compliance status" src="https://github.com/finos/tracdap/actions/workflows/compliance.yaml/badge.svg?branch:main&workflow:CI" /></a>
|
42
|
+
<a href="https://community.finos.org/docs/governance/software-projects/stages/incubating/"><img alt="FINOS - Incubating" src="https://cdn.jsdelivr.net/gh/finos/contrib-toolbox@master/images/badge-incubating.svg" /></a>
|
43
|
+
</p>
|
44
|
+
|
45
|
+
|
46
|
+
# HTTP Extension for the TRAC Model Runtime
|
47
|
+
|
48
|
+
This extension allows TRAC models to make calls to external systems using HTTP and HTTPS.
|
49
|
+
|
50
|
+
- Make HTTP endpoints available to models, to use directly in model code
|
51
|
+
- Connection settings managed by TRAC for both local and deployed models
|
52
|
+
- Supports http.client and urllib3
|
53
|
+
- Supports
|
54
|
+
|
55
|
+
Models that make external calls are not considered repeatable,
|
56
|
+
and will be flagged as non-repeatable when they run on the TRAC platform.
|
57
|
+
|
58
|
+
This extension is a pre-release and will be finalized inTRAC 0.10.
|
59
|
+
|
60
|
+
|
61
|
+
## Installing
|
62
|
+
|
63
|
+
The HTTP extension can be installed with [pip](https://pip.pypa.io):
|
64
|
+
|
65
|
+
```shell
|
66
|
+
$ pip install tracdap-ext-http
|
67
|
+
```
|
68
|
+
|
69
|
+
The package has the following dependencies:
|
70
|
+
|
71
|
+
- tracdap-runtime (version 0.10.0-beta1 or later)
|
72
|
+
- urllib3 (optional, version 2.x)
|
73
|
+
|
74
|
+
|
75
|
+
## Using the http.client API
|
76
|
+
|
77
|
+
Here is a minimum working example of a TRAC model using the http.client API:
|
78
|
+
|
79
|
+
```python
|
80
|
+
import tracdap.rt.api as trac
|
81
|
+
import http.client as hc
|
82
|
+
|
83
|
+
class TestModel(trac.TracModel):
|
84
|
+
|
85
|
+
# ... define parameters, inputs and outputs
|
86
|
+
|
87
|
+
def define_resources(self):
|
88
|
+
|
89
|
+
return {
|
90
|
+
"github": trac.define_external_system("http", hc.HTTPSConnection)
|
91
|
+
}
|
92
|
+
|
93
|
+
def run_model(self, ctx: trac.TracContext):
|
94
|
+
|
95
|
+
with ctx.get_external_system("github", hc.HTTPSConnection) as github:
|
96
|
+
|
97
|
+
github.connect()
|
98
|
+
github.request("GET", "finos/tracdap/refs/heads/main/README.md")
|
99
|
+
|
100
|
+
response = github.getresponse()
|
101
|
+
response_text = response.read().decode("utf-8")
|
102
|
+
first_line = response_text.splitlines()[0]
|
103
|
+
|
104
|
+
ctx.log.info(first_line)
|
105
|
+
|
106
|
+
if __name__ == '__main__':
|
107
|
+
import tracdap.rt.launch as launch
|
108
|
+
launch.launch_model(TestModel, "config/job_config.yaml", "config/sys_config.yaml")
|
109
|
+
```
|
110
|
+
|
111
|
+
To make this example work, you will need to add ``github`` as a resource in the system config file:
|
112
|
+
|
113
|
+
```yaml
|
114
|
+
resources:
|
115
|
+
|
116
|
+
github:
|
117
|
+
resourceType: EXTERNAL_SYSTEM
|
118
|
+
protocol: http
|
119
|
+
properties:
|
120
|
+
host: raw.githubusercontent.com
|
121
|
+
port: 443
|
122
|
+
tls: true
|
123
|
+
```
|
124
|
+
|
125
|
+
The following configuration properties are supported:
|
126
|
+
|
127
|
+
- host, string, required
|
128
|
+
- port, int, optional
|
129
|
+
- tls, bool, default = true
|
130
|
+
- timeout, float, optional
|
131
|
+
|
132
|
+
Models using the client type ``HTTPSConnection`` will only work if tls = true is set in the configuration.
|
133
|
+
Models requesting ``HTTPConnection`` will work with tls = true or tls = false,
|
134
|
+
and will receive an ``HTTPSConnection`` if tls = true.
|
135
|
+
|
136
|
+
|
137
|
+
## Using the urllib3 API
|
138
|
+
|
139
|
+
Here is a minimum working example of a TRAC model using the urllib3 API.
|
140
|
+
In order to use this API, the urllib3 package must be installed.
|
141
|
+
|
142
|
+
```python
|
143
|
+
import tracdap.rt.api as trac
|
144
|
+
import urllib3
|
145
|
+
|
146
|
+
class TestModel(trac.TracModel):
|
147
|
+
|
148
|
+
# ... define parameters, inputs and outputs
|
149
|
+
|
150
|
+
def define_resources(self):
|
151
|
+
|
152
|
+
return {
|
153
|
+
"github": trac.define_external_system("http", urllib3.HTTPSConnectionPool)
|
154
|
+
}
|
155
|
+
|
156
|
+
def run_model(self, ctx: trac.TracContext):
|
157
|
+
|
158
|
+
with ctx.get_external_system("github", urllib3.HTTPSConnectionPool, timeout=10.0) as github:
|
159
|
+
|
160
|
+
response = github.request("GET", "/finos/tracdap/refs/heads/main/README.md")
|
161
|
+
|
162
|
+
response_text = response.data.decode("utf-8")
|
163
|
+
first_line = response_text.splitlines()[0]
|
164
|
+
|
165
|
+
ctx.log.info(first_line)
|
166
|
+
|
167
|
+
if __name__ == '__main__':
|
168
|
+
import tracdap.rt.launch as launch
|
169
|
+
launch.launch_model(TestModel, "config/job_config.yaml", "config/sys_config.yaml")
|
170
|
+
```
|
171
|
+
|
172
|
+
The resource configuration for the urllib3 API is identical to the http.client API.
|
@@ -0,0 +1,142 @@
|
|
1
|
+
<h1 align="center">
|
2
|
+
|
3
|
+

|
4
|
+
|
5
|
+
</h1>
|
6
|
+
|
7
|
+
<p align="center">
|
8
|
+
<a href="https://pypi.org/project/tracdap-ext-http"><img alt="PyPI Version" src="https://img.shields.io/pypi/v/tracdap-ext-http.svg?maxAge=86400" /></a>
|
9
|
+
<a href="https://pypi.org/project/tracdap-ext-http"><img alt="Python Versions" src="https://img.shields.io/pypi/pyversions/tracdap-ext-http.svg?maxAge=86400" /></a>
|
10
|
+
<a href="https://github.com/finos/tracdap/actions/workflows/packaging.yaml?query=branch%3Amain"><img alt="Packaging status" src="https://github.com/finos/tracdap/actions/workflows/packaging.yaml/badge.svg?branch:main&workflow:CI" /></a>
|
11
|
+
<a href="https://github.com/finos/tracdap/actions/workflows/compliance.yaml?query=branch%3Amain"><img alt="Compliance status" src="https://github.com/finos/tracdap/actions/workflows/compliance.yaml/badge.svg?branch:main&workflow:CI" /></a>
|
12
|
+
<a href="https://community.finos.org/docs/governance/software-projects/stages/incubating/"><img alt="FINOS - Incubating" src="https://cdn.jsdelivr.net/gh/finos/contrib-toolbox@master/images/badge-incubating.svg" /></a>
|
13
|
+
</p>
|
14
|
+
|
15
|
+
|
16
|
+
# HTTP Extension for the TRAC Model Runtime
|
17
|
+
|
18
|
+
This extension allows TRAC models to make calls to external systems using HTTP and HTTPS.
|
19
|
+
|
20
|
+
- Make HTTP endpoints available to models, to use directly in model code
|
21
|
+
- Connection settings managed by TRAC for both local and deployed models
|
22
|
+
- Supports http.client and urllib3
|
23
|
+
- Supports
|
24
|
+
|
25
|
+
Models that make external calls are not considered repeatable,
|
26
|
+
and will be flagged as non-repeatable when they run on the TRAC platform.
|
27
|
+
|
28
|
+
This extension is a pre-release and will be finalized inTRAC 0.10.
|
29
|
+
|
30
|
+
|
31
|
+
## Installing
|
32
|
+
|
33
|
+
The HTTP extension can be installed with [pip](https://pip.pypa.io):
|
34
|
+
|
35
|
+
```shell
|
36
|
+
$ pip install tracdap-ext-http
|
37
|
+
```
|
38
|
+
|
39
|
+
The package has the following dependencies:
|
40
|
+
|
41
|
+
- tracdap-runtime (version 0.10.0-beta1 or later)
|
42
|
+
- urllib3 (optional, version 2.x)
|
43
|
+
|
44
|
+
|
45
|
+
## Using the http.client API
|
46
|
+
|
47
|
+
Here is a minimum working example of a TRAC model using the http.client API:
|
48
|
+
|
49
|
+
```python
|
50
|
+
import tracdap.rt.api as trac
|
51
|
+
import http.client as hc
|
52
|
+
|
53
|
+
class TestModel(trac.TracModel):
|
54
|
+
|
55
|
+
# ... define parameters, inputs and outputs
|
56
|
+
|
57
|
+
def define_resources(self):
|
58
|
+
|
59
|
+
return {
|
60
|
+
"github": trac.define_external_system("http", hc.HTTPSConnection)
|
61
|
+
}
|
62
|
+
|
63
|
+
def run_model(self, ctx: trac.TracContext):
|
64
|
+
|
65
|
+
with ctx.get_external_system("github", hc.HTTPSConnection) as github:
|
66
|
+
|
67
|
+
github.connect()
|
68
|
+
github.request("GET", "finos/tracdap/refs/heads/main/README.md")
|
69
|
+
|
70
|
+
response = github.getresponse()
|
71
|
+
response_text = response.read().decode("utf-8")
|
72
|
+
first_line = response_text.splitlines()[0]
|
73
|
+
|
74
|
+
ctx.log.info(first_line)
|
75
|
+
|
76
|
+
if __name__ == '__main__':
|
77
|
+
import tracdap.rt.launch as launch
|
78
|
+
launch.launch_model(TestModel, "config/job_config.yaml", "config/sys_config.yaml")
|
79
|
+
```
|
80
|
+
|
81
|
+
To make this example work, you will need to add ``github`` as a resource in the system config file:
|
82
|
+
|
83
|
+
```yaml
|
84
|
+
resources:
|
85
|
+
|
86
|
+
github:
|
87
|
+
resourceType: EXTERNAL_SYSTEM
|
88
|
+
protocol: http
|
89
|
+
properties:
|
90
|
+
host: raw.githubusercontent.com
|
91
|
+
port: 443
|
92
|
+
tls: true
|
93
|
+
```
|
94
|
+
|
95
|
+
The following configuration properties are supported:
|
96
|
+
|
97
|
+
- host, string, required
|
98
|
+
- port, int, optional
|
99
|
+
- tls, bool, default = true
|
100
|
+
- timeout, float, optional
|
101
|
+
|
102
|
+
Models using the client type ``HTTPSConnection`` will only work if tls = true is set in the configuration.
|
103
|
+
Models requesting ``HTTPConnection`` will work with tls = true or tls = false,
|
104
|
+
and will receive an ``HTTPSConnection`` if tls = true.
|
105
|
+
|
106
|
+
|
107
|
+
## Using the urllib3 API
|
108
|
+
|
109
|
+
Here is a minimum working example of a TRAC model using the urllib3 API.
|
110
|
+
In order to use this API, the urllib3 package must be installed.
|
111
|
+
|
112
|
+
```python
|
113
|
+
import tracdap.rt.api as trac
|
114
|
+
import urllib3
|
115
|
+
|
116
|
+
class TestModel(trac.TracModel):
|
117
|
+
|
118
|
+
# ... define parameters, inputs and outputs
|
119
|
+
|
120
|
+
def define_resources(self):
|
121
|
+
|
122
|
+
return {
|
123
|
+
"github": trac.define_external_system("http", urllib3.HTTPSConnectionPool)
|
124
|
+
}
|
125
|
+
|
126
|
+
def run_model(self, ctx: trac.TracContext):
|
127
|
+
|
128
|
+
with ctx.get_external_system("github", urllib3.HTTPSConnectionPool, timeout=10.0) as github:
|
129
|
+
|
130
|
+
response = github.request("GET", "/finos/tracdap/refs/heads/main/README.md")
|
131
|
+
|
132
|
+
response_text = response.data.decode("utf-8")
|
133
|
+
first_line = response_text.splitlines()[0]
|
134
|
+
|
135
|
+
ctx.log.info(first_line)
|
136
|
+
|
137
|
+
if __name__ == '__main__':
|
138
|
+
import tracdap.rt.launch as launch
|
139
|
+
launch.launch_model(TestModel, "config/job_config.yaml", "config/sys_config.yaml")
|
140
|
+
```
|
141
|
+
|
142
|
+
The resource configuration for the urllib3 API is identical to the http.client API.
|
@@ -23,12 +23,10 @@ authors = [
|
|
23
23
|
classifiers = [
|
24
24
|
"Operating System :: OS Independent",
|
25
25
|
"Programming Language :: Python :: 3",
|
26
|
-
"Programming Language :: Python :: 3.9",
|
27
26
|
"Programming Language :: Python :: 3.10",
|
28
27
|
"Programming Language :: Python :: 3.11",
|
29
28
|
"Programming Language :: Python :: 3.12",
|
30
29
|
"Programming Language :: Python :: 3.13",
|
31
|
-
"Framework :: Trac",
|
32
30
|
"Intended Audience :: Developers",
|
33
31
|
"Intended Audience :: End Users/Desktop",
|
34
32
|
"Intended Audience :: Financial and Insurance Industry",
|
@@ -39,15 +37,15 @@ classifiers = [
|
|
39
37
|
"Topic :: Software Development :: Libraries :: Python Modules"
|
40
38
|
]
|
41
39
|
|
42
|
-
requires-python = ">= 3.
|
40
|
+
requires-python = ">= 3.10"
|
43
41
|
|
44
42
|
|
45
43
|
[project.urls]
|
46
44
|
|
47
45
|
Homepage = "https://tracdap.finos.org/"
|
48
46
|
Documentation = "https://tracdap.readthedocs.io/"
|
49
|
-
"Source Code" = "https://github.com/
|
50
|
-
"Issue Tracker" = "https://github.com/
|
47
|
+
"Source Code" = "https://github.com/finos/tracdap"
|
48
|
+
"Issue Tracker" = "https://github.com/finos/tracdap/issues"
|
51
49
|
|
52
50
|
[tool.setuptools.packages.find]
|
53
51
|
|
@@ -0,0 +1,133 @@
|
|
1
|
+
# Licensed to the Fintech Open Source Foundation (FINOS) under one or
|
2
|
+
# more contributor license agreements. See the NOTICE file distributed
|
3
|
+
# with this work for additional information regarding copyright ownership.
|
4
|
+
# FINOS licenses this file to you under the Apache License, Version 2.0
|
5
|
+
# (the "License"); you may not use this file except in compliance with the
|
6
|
+
# License. You may obtain a copy of the License at
|
7
|
+
#
|
8
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
+
#
|
10
|
+
# Unless required by applicable law or agreed to in writing, software
|
11
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
+
# See the License for the specific language governing permissions and
|
14
|
+
# limitations under the License.
|
15
|
+
|
16
|
+
import http.client as _hc
|
17
|
+
|
18
|
+
try:
|
19
|
+
import urllib3 as _ul3 # noqa
|
20
|
+
except ModuleNotFoundError:
|
21
|
+
_ul3 = None
|
22
|
+
|
23
|
+
import tracdap.rt.config as _cfg
|
24
|
+
import tracdap.rt.exceptions as _ex
|
25
|
+
import tracdap.rt.ext.external as _external
|
26
|
+
import tracdap.rt.ext.plugins as _plugins
|
27
|
+
import tracdap.rt.ext.util as _util
|
28
|
+
|
29
|
+
|
30
|
+
class HttpPlugin(_external.IExternalSystem):
|
31
|
+
|
32
|
+
HOST_KEY = "host"
|
33
|
+
PORT_KEY = "port"
|
34
|
+
TLS_KEY = "tls"
|
35
|
+
TIMEOUT_KEY = "timeout"
|
36
|
+
|
37
|
+
def __init__(self, resource_name: str, config: _cfg.PluginConfig):
|
38
|
+
|
39
|
+
self.__resource_name = resource_name
|
40
|
+
|
41
|
+
self.__host = _util.read_plugin_config(config, self.HOST_KEY)
|
42
|
+
self.__port = _util.read_plugin_config(config, self.PORT_KEY, optional=True, convert=int)
|
43
|
+
self.__tls = _util.read_plugin_config(config, self.TLS_KEY, default=True, convert=bool)
|
44
|
+
self.__timeout = _util.read_plugin_config(config, self.TIMEOUT_KEY, optional=True, convert=int)
|
45
|
+
|
46
|
+
def supported_types(self) -> list[type]:
|
47
|
+
|
48
|
+
supported_types: list[type] = list()
|
49
|
+
|
50
|
+
if _hc:
|
51
|
+
supported_types.append(_hc.HTTPConnection)
|
52
|
+
supported_types.append(_hc.HTTPSConnection)
|
53
|
+
|
54
|
+
if _ul3:
|
55
|
+
supported_types.append(_ul3.HTTPConnectionPool)
|
56
|
+
supported_types.append(_ul3.HTTPSConnectionPool)
|
57
|
+
|
58
|
+
return supported_types
|
59
|
+
|
60
|
+
def supported_args(self) -> dict[str, type] | None:
|
61
|
+
|
62
|
+
return {
|
63
|
+
"timeout": float
|
64
|
+
}
|
65
|
+
|
66
|
+
def create_client(self, client_type: type, **client_args) -> object:
|
67
|
+
|
68
|
+
if client_type == _hc.HTTPConnection:
|
69
|
+
if self.__tls:
|
70
|
+
return self._create_client_hc_https(**client_args)
|
71
|
+
else:
|
72
|
+
return self._create_client_hc_http(**client_args)
|
73
|
+
|
74
|
+
if client_type == _hc.HTTPSConnection:
|
75
|
+
if self.__tls:
|
76
|
+
return self._create_client_hc_https(**client_args)
|
77
|
+
else:
|
78
|
+
raise self._error_tls_not_enabled()
|
79
|
+
|
80
|
+
if _ul3 and client_type == _ul3.HTTPConnectionPool:
|
81
|
+
if self.__tls:
|
82
|
+
return self._create_client_ul3_https(**client_args)
|
83
|
+
else:
|
84
|
+
return self._create_client_ul3_http(**client_args)
|
85
|
+
|
86
|
+
if _ul3 and client_type == _ul3.HTTPSConnectionPool:
|
87
|
+
if self.__tls:
|
88
|
+
return self._create_client_ul3_https(**client_args)
|
89
|
+
else:
|
90
|
+
raise self._error_tls_not_enabled()
|
91
|
+
|
92
|
+
raise _ex.EPluginNotAvailable(f"Client type [{client_type.__qualname__}] is not available in {self.__class__.__name__}")
|
93
|
+
|
94
|
+
def _create_client_hc_http(self, **client_args):
|
95
|
+
hc_args = self._build_common_args(**client_args)
|
96
|
+
return _hc.HTTPSConnection(self.__host, self.__port, **hc_args)
|
97
|
+
|
98
|
+
def _create_client_hc_https(self, **client_args):
|
99
|
+
hc_args = self._build_common_args(**client_args)
|
100
|
+
return _hc.HTTPSConnection(self.__host, self.__port, **hc_args)
|
101
|
+
|
102
|
+
def _create_client_ul3_http(self, **client_args):
|
103
|
+
ul3_args = self._build_common_args(**client_args)
|
104
|
+
return _ul3.HTTPSConnectionPool(self.__host, self.__port, **ul3_args)
|
105
|
+
|
106
|
+
def _create_client_ul3_https(self, **client_args):
|
107
|
+
ul3_args = self._build_common_args(**client_args)
|
108
|
+
return _ul3.HTTPSConnectionPool(self.__host, self.__port, **ul3_args)
|
109
|
+
|
110
|
+
def _build_common_args(self, **client_args):
|
111
|
+
|
112
|
+
args = dict()
|
113
|
+
|
114
|
+
if self.TIMEOUT_KEY in client_args and self.__timeout is not None:
|
115
|
+
args[self.TIMEOUT_KEY] = min(client_args[self.TIMEOUT_KEY], self.__timeout)
|
116
|
+
elif self.TIMEOUT_KEY in client_args:
|
117
|
+
args[self.TIMEOUT_KEY] = client_args[self.TIMEOUT_KEY]
|
118
|
+
elif self.__timeout is not None:
|
119
|
+
args[self.TIMEOUT_KEY] = self.__timeout
|
120
|
+
|
121
|
+
return args
|
122
|
+
|
123
|
+
def _error_tls_not_enabled(self):
|
124
|
+
detail = f"The resource [{self.__resource_name }] does not have TLS enabled"
|
125
|
+
return _ex.ERuntimeValidation(f"Cannot create HTTPS connection: {detail}")
|
126
|
+
|
127
|
+
def close_client(self, client: object):
|
128
|
+
client.close() # noqa
|
129
|
+
|
130
|
+
|
131
|
+
_plugins.PluginManager.register_plugin(
|
132
|
+
_external.IExternalSystem, HttpPlugin,
|
133
|
+
protocols=["http", "https"])
|
@@ -0,0 +1,172 @@
|
|
1
|
+
Metadata-Version: 2.4
|
2
|
+
Name: tracdap-ext-http
|
3
|
+
Version: 0.10.0.dev15
|
4
|
+
Summary: An extension for TRAC D.A.P. that lets models connect to HTTP endpoints
|
5
|
+
Author-email: Martin Traverse <martin@fintrac.co.uk>
|
6
|
+
License-Expression: Apache-2.0
|
7
|
+
Project-URL: Homepage, https://tracdap.finos.org/
|
8
|
+
Project-URL: Documentation, https://tracdap.readthedocs.io/
|
9
|
+
Project-URL: Source Code, https://github.com/finos/tracdap
|
10
|
+
Project-URL: Issue Tracker, https://github.com/finos/tracdap/issues
|
11
|
+
Classifier: Operating System :: OS Independent
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
13
|
+
Classifier: Programming Language :: Python :: 3.10
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
16
|
+
Classifier: Programming Language :: Python :: 3.13
|
17
|
+
Classifier: Intended Audience :: Developers
|
18
|
+
Classifier: Intended Audience :: End Users/Desktop
|
19
|
+
Classifier: Intended Audience :: Financial and Insurance Industry
|
20
|
+
Classifier: Topic :: Office/Business
|
21
|
+
Classifier: Topic :: Office/Business :: Financial
|
22
|
+
Classifier: Topic :: Software Development
|
23
|
+
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
|
24
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
25
|
+
Requires-Python: >=3.10
|
26
|
+
Description-Content-Type: text/markdown
|
27
|
+
License-File: LICENSE
|
28
|
+
Requires-Dist: tracdap-runtime>=0.10.0-dev.15
|
29
|
+
Dynamic: license-file
|
30
|
+
|
31
|
+
<h1 align="center">
|
32
|
+
|
33
|
+

|
34
|
+
|
35
|
+
</h1>
|
36
|
+
|
37
|
+
<p align="center">
|
38
|
+
<a href="https://pypi.org/project/tracdap-ext-http"><img alt="PyPI Version" src="https://img.shields.io/pypi/v/tracdap-ext-http.svg?maxAge=86400" /></a>
|
39
|
+
<a href="https://pypi.org/project/tracdap-ext-http"><img alt="Python Versions" src="https://img.shields.io/pypi/pyversions/tracdap-ext-http.svg?maxAge=86400" /></a>
|
40
|
+
<a href="https://github.com/finos/tracdap/actions/workflows/packaging.yaml?query=branch%3Amain"><img alt="Packaging status" src="https://github.com/finos/tracdap/actions/workflows/packaging.yaml/badge.svg?branch:main&workflow:CI" /></a>
|
41
|
+
<a href="https://github.com/finos/tracdap/actions/workflows/compliance.yaml?query=branch%3Amain"><img alt="Compliance status" src="https://github.com/finos/tracdap/actions/workflows/compliance.yaml/badge.svg?branch:main&workflow:CI" /></a>
|
42
|
+
<a href="https://community.finos.org/docs/governance/software-projects/stages/incubating/"><img alt="FINOS - Incubating" src="https://cdn.jsdelivr.net/gh/finos/contrib-toolbox@master/images/badge-incubating.svg" /></a>
|
43
|
+
</p>
|
44
|
+
|
45
|
+
|
46
|
+
# HTTP Extension for the TRAC Model Runtime
|
47
|
+
|
48
|
+
This extension allows TRAC models to make calls to external systems using HTTP and HTTPS.
|
49
|
+
|
50
|
+
- Make HTTP endpoints available to models, to use directly in model code
|
51
|
+
- Connection settings managed by TRAC for both local and deployed models
|
52
|
+
- Supports http.client and urllib3
|
53
|
+
- Supports
|
54
|
+
|
55
|
+
Models that make external calls are not considered repeatable,
|
56
|
+
and will be flagged as non-repeatable when they run on the TRAC platform.
|
57
|
+
|
58
|
+
This extension is a pre-release and will be finalized inTRAC 0.10.
|
59
|
+
|
60
|
+
|
61
|
+
## Installing
|
62
|
+
|
63
|
+
The HTTP extension can be installed with [pip](https://pip.pypa.io):
|
64
|
+
|
65
|
+
```shell
|
66
|
+
$ pip install tracdap-ext-http
|
67
|
+
```
|
68
|
+
|
69
|
+
The package has the following dependencies:
|
70
|
+
|
71
|
+
- tracdap-runtime (version 0.10.0-beta1 or later)
|
72
|
+
- urllib3 (optional, version 2.x)
|
73
|
+
|
74
|
+
|
75
|
+
## Using the http.client API
|
76
|
+
|
77
|
+
Here is a minimum working example of a TRAC model using the http.client API:
|
78
|
+
|
79
|
+
```python
|
80
|
+
import tracdap.rt.api as trac
|
81
|
+
import http.client as hc
|
82
|
+
|
83
|
+
class TestModel(trac.TracModel):
|
84
|
+
|
85
|
+
# ... define parameters, inputs and outputs
|
86
|
+
|
87
|
+
def define_resources(self):
|
88
|
+
|
89
|
+
return {
|
90
|
+
"github": trac.define_external_system("http", hc.HTTPSConnection)
|
91
|
+
}
|
92
|
+
|
93
|
+
def run_model(self, ctx: trac.TracContext):
|
94
|
+
|
95
|
+
with ctx.get_external_system("github", hc.HTTPSConnection) as github:
|
96
|
+
|
97
|
+
github.connect()
|
98
|
+
github.request("GET", "finos/tracdap/refs/heads/main/README.md")
|
99
|
+
|
100
|
+
response = github.getresponse()
|
101
|
+
response_text = response.read().decode("utf-8")
|
102
|
+
first_line = response_text.splitlines()[0]
|
103
|
+
|
104
|
+
ctx.log.info(first_line)
|
105
|
+
|
106
|
+
if __name__ == '__main__':
|
107
|
+
import tracdap.rt.launch as launch
|
108
|
+
launch.launch_model(TestModel, "config/job_config.yaml", "config/sys_config.yaml")
|
109
|
+
```
|
110
|
+
|
111
|
+
To make this example work, you will need to add ``github`` as a resource in the system config file:
|
112
|
+
|
113
|
+
```yaml
|
114
|
+
resources:
|
115
|
+
|
116
|
+
github:
|
117
|
+
resourceType: EXTERNAL_SYSTEM
|
118
|
+
protocol: http
|
119
|
+
properties:
|
120
|
+
host: raw.githubusercontent.com
|
121
|
+
port: 443
|
122
|
+
tls: true
|
123
|
+
```
|
124
|
+
|
125
|
+
The following configuration properties are supported:
|
126
|
+
|
127
|
+
- host, string, required
|
128
|
+
- port, int, optional
|
129
|
+
- tls, bool, default = true
|
130
|
+
- timeout, float, optional
|
131
|
+
|
132
|
+
Models using the client type ``HTTPSConnection`` will only work if tls = true is set in the configuration.
|
133
|
+
Models requesting ``HTTPConnection`` will work with tls = true or tls = false,
|
134
|
+
and will receive an ``HTTPSConnection`` if tls = true.
|
135
|
+
|
136
|
+
|
137
|
+
## Using the urllib3 API
|
138
|
+
|
139
|
+
Here is a minimum working example of a TRAC model using the urllib3 API.
|
140
|
+
In order to use this API, the urllib3 package must be installed.
|
141
|
+
|
142
|
+
```python
|
143
|
+
import tracdap.rt.api as trac
|
144
|
+
import urllib3
|
145
|
+
|
146
|
+
class TestModel(trac.TracModel):
|
147
|
+
|
148
|
+
# ... define parameters, inputs and outputs
|
149
|
+
|
150
|
+
def define_resources(self):
|
151
|
+
|
152
|
+
return {
|
153
|
+
"github": trac.define_external_system("http", urllib3.HTTPSConnectionPool)
|
154
|
+
}
|
155
|
+
|
156
|
+
def run_model(self, ctx: trac.TracContext):
|
157
|
+
|
158
|
+
with ctx.get_external_system("github", urllib3.HTTPSConnectionPool, timeout=10.0) as github:
|
159
|
+
|
160
|
+
response = github.request("GET", "/finos/tracdap/refs/heads/main/README.md")
|
161
|
+
|
162
|
+
response_text = response.data.decode("utf-8")
|
163
|
+
first_line = response_text.splitlines()[0]
|
164
|
+
|
165
|
+
ctx.log.info(first_line)
|
166
|
+
|
167
|
+
if __name__ == '__main__':
|
168
|
+
import tracdap.rt.launch as launch
|
169
|
+
launch.launch_model(TestModel, "config/job_config.yaml", "config/sys_config.yaml")
|
170
|
+
```
|
171
|
+
|
172
|
+
The resource configuration for the urllib3 API is identical to the http.client API.
|
@@ -0,0 +1 @@
|
|
1
|
+
tracdap-runtime>=0.10.0-dev.15
|
@@ -1,37 +0,0 @@
|
|
1
|
-
Metadata-Version: 2.4
|
2
|
-
Name: tracdap-ext-http
|
3
|
-
Version: 0.10.0.dev14
|
4
|
-
Summary: An extension for TRAC D.A.P. that lets models connect to HTTP endpoints
|
5
|
-
Author-email: Martin Traverse <martin@fintrac.co.uk>
|
6
|
-
License-Expression: Apache-2.0
|
7
|
-
Project-URL: Homepage, https://tracdap.finos.org/
|
8
|
-
Project-URL: Documentation, https://tracdap.readthedocs.io/
|
9
|
-
Project-URL: Source Code, https://github.com/martin-traverse/tracdap
|
10
|
-
Project-URL: Issue Tracker, https://github.com/martin-traverse/tracdap/issues
|
11
|
-
Classifier: Operating System :: OS Independent
|
12
|
-
Classifier: Programming Language :: Python :: 3
|
13
|
-
Classifier: Programming Language :: Python :: 3.9
|
14
|
-
Classifier: Programming Language :: Python :: 3.10
|
15
|
-
Classifier: Programming Language :: Python :: 3.11
|
16
|
-
Classifier: Programming Language :: Python :: 3.12
|
17
|
-
Classifier: Programming Language :: Python :: 3.13
|
18
|
-
Classifier: Framework :: Trac
|
19
|
-
Classifier: Intended Audience :: Developers
|
20
|
-
Classifier: Intended Audience :: End Users/Desktop
|
21
|
-
Classifier: Intended Audience :: Financial and Insurance Industry
|
22
|
-
Classifier: Topic :: Office/Business
|
23
|
-
Classifier: Topic :: Office/Business :: Financial
|
24
|
-
Classifier: Topic :: Software Development
|
25
|
-
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
|
26
|
-
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
27
|
-
Requires-Python: >=3.9
|
28
|
-
Description-Content-Type: text/markdown
|
29
|
-
License-File: LICENSE
|
30
|
-
Dynamic: license-file
|
31
|
-
|
32
|
-
# HTTP Extension for the TRAC Model Runtime
|
33
|
-
|
34
|
-
An extension for TRAC D.A.P. that lets models connect to HTTP endpoints.
|
35
|
-
The extension depends on the tracdap-runtime package with the same version.
|
36
|
-
|
37
|
-
This extension is a pre-release, full documentation will be provided TRAC 0.10.
|
@@ -1,6 +0,0 @@
|
|
1
|
-
# HTTP Extension for the TRAC Model Runtime
|
2
|
-
|
3
|
-
An extension for TRAC D.A.P. that lets models connect to HTTP endpoints.
|
4
|
-
The extension depends on the tracdap-runtime package with the same version.
|
5
|
-
|
6
|
-
This extension is a pre-release, full documentation will be provided TRAC 0.10.
|
File without changes
|
@@ -1,42 +0,0 @@
|
|
1
|
-
# Licensed to the Fintech Open Source Foundation (FINOS) under one or
|
2
|
-
# more contributor license agreements. See the NOTICE file distributed
|
3
|
-
# with this work for additional information regarding copyright ownership.
|
4
|
-
# FINOS licenses this file to you under the Apache License, Version 2.0
|
5
|
-
# (the "License"); you may not use this file except in compliance with the
|
6
|
-
# License. You may obtain a copy of the License at
|
7
|
-
#
|
8
|
-
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
-
#
|
10
|
-
# Unless required by applicable law or agreed to in writing, software
|
11
|
-
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
-
# See the License for the specific language governing permissions and
|
14
|
-
# limitations under the License.
|
15
|
-
|
16
|
-
import typing as _tp
|
17
|
-
|
18
|
-
import tracdap.rt.ext.external as _external
|
19
|
-
import tracdap.rt.ext.plugins as _plugins
|
20
|
-
|
21
|
-
|
22
|
-
class HttpPlugin(_external.IExternalSystem):
|
23
|
-
|
24
|
-
def __init__(self, properties: _tp.Dict[str, str]):
|
25
|
-
self.__properties = properties
|
26
|
-
|
27
|
-
def supported_types(self) -> _tp.List[type]:
|
28
|
-
pass
|
29
|
-
|
30
|
-
def supported_args(self) -> _tp.Optional[_tp.Dict[str, type]]:
|
31
|
-
pass
|
32
|
-
|
33
|
-
def create_client(self, client_type: type, **client_args) -> _tp.Any:
|
34
|
-
pass
|
35
|
-
|
36
|
-
def close_client(self, client: _tp.Any):
|
37
|
-
pass
|
38
|
-
|
39
|
-
|
40
|
-
_plugins.PluginManager.register_plugin(
|
41
|
-
_external.IExternalSystem, HttpPlugin,
|
42
|
-
protocols=["http", "https"])
|
@@ -1,37 +0,0 @@
|
|
1
|
-
Metadata-Version: 2.4
|
2
|
-
Name: tracdap-ext-http
|
3
|
-
Version: 0.10.0.dev14
|
4
|
-
Summary: An extension for TRAC D.A.P. that lets models connect to HTTP endpoints
|
5
|
-
Author-email: Martin Traverse <martin@fintrac.co.uk>
|
6
|
-
License-Expression: Apache-2.0
|
7
|
-
Project-URL: Homepage, https://tracdap.finos.org/
|
8
|
-
Project-URL: Documentation, https://tracdap.readthedocs.io/
|
9
|
-
Project-URL: Source Code, https://github.com/martin-traverse/tracdap
|
10
|
-
Project-URL: Issue Tracker, https://github.com/martin-traverse/tracdap/issues
|
11
|
-
Classifier: Operating System :: OS Independent
|
12
|
-
Classifier: Programming Language :: Python :: 3
|
13
|
-
Classifier: Programming Language :: Python :: 3.9
|
14
|
-
Classifier: Programming Language :: Python :: 3.10
|
15
|
-
Classifier: Programming Language :: Python :: 3.11
|
16
|
-
Classifier: Programming Language :: Python :: 3.12
|
17
|
-
Classifier: Programming Language :: Python :: 3.13
|
18
|
-
Classifier: Framework :: Trac
|
19
|
-
Classifier: Intended Audience :: Developers
|
20
|
-
Classifier: Intended Audience :: End Users/Desktop
|
21
|
-
Classifier: Intended Audience :: Financial and Insurance Industry
|
22
|
-
Classifier: Topic :: Office/Business
|
23
|
-
Classifier: Topic :: Office/Business :: Financial
|
24
|
-
Classifier: Topic :: Software Development
|
25
|
-
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
|
26
|
-
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
27
|
-
Requires-Python: >=3.9
|
28
|
-
Description-Content-Type: text/markdown
|
29
|
-
License-File: LICENSE
|
30
|
-
Dynamic: license-file
|
31
|
-
|
32
|
-
# HTTP Extension for the TRAC Model Runtime
|
33
|
-
|
34
|
-
An extension for TRAC D.A.P. that lets models connect to HTTP endpoints.
|
35
|
-
The extension depends on the tracdap-runtime package with the same version.
|
36
|
-
|
37
|
-
This extension is a pre-release, full documentation will be provided TRAC 0.10.
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|