pulsar-client 1.7.0 → 1.8.0-rc.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.asf.yaml +8 -1
- package/.github/PULL_REQUEST_TEMPLATE.md +85 -0
- package/.github/workflows/ci-build-release-napi.yml +213 -0
- package/.github/workflows/ci-pr-validation.yml +233 -0
- package/README.md +86 -106
- package/binding.gyp +34 -43
- package/{pulsar-test-service-stop.sh → build-support/dep-version.py} +4 -6
- package/build-support/download-release-artifacts.py +74 -0
- package/build-support/generate-source-archive.sh +28 -0
- package/{pulsar-test-service-start.sh → build-support/pulsar-test-container-start.sh} +11 -21
- package/build-support/pulsar-test-service-start.sh +37 -0
- package/build-support/pulsar-test-service-stop.sh +32 -0
- package/{.github/workflows/nodejs.yml → build-support/sign-files.sh} +13 -12
- package/build-support/stage-release.sh +44 -0
- package/dependencies.yaml +28 -0
- package/docs/release-process.md +271 -0
- package/examples/consumer.js +1 -1
- package/examples/consumer_listener.js +1 -1
- package/examples/consumer_tls_auth.js +1 -1
- package/examples/custom_logger.js +60 -0
- package/examples/encryption-consumer.js +1 -1
- package/examples/encryption-producer.js +1 -1
- package/examples/producer.js +1 -1
- package/examples/producer_tls_auth.js +1 -1
- package/examples/reader.js +1 -1
- package/examples/reader_listener.js +1 -1
- package/index.d.ts +7 -0
- package/index.js +2 -1
- package/package.json +17 -8
- package/pkg/linux/Dockerfile_linux_glibc +35 -0
- package/pkg/linux/Dockerfile_linux_musl +32 -0
- package/pkg/linux/build-napi-inside-docker.sh +31 -0
- package/pkg/linux/download-cpp-client.sh +65 -0
- package/pkg/load_test.js +34 -0
- package/pkg/mac/build-cpp-deps-lib.sh +186 -0
- package/pkg/mac/build-cpp-lib.sh +48 -0
- package/pkg/mac/common.sh +37 -0
- package/pkg/windows/download-cpp-client.bat +12 -0
- package/pulsar-client-cpp.txt +2 -0
- package/src/AuthenticationAthenz.js +1 -1
- package/src/AuthenticationOauth2.js +1 -1
- package/src/AuthenticationTls.js +1 -1
- package/src/AuthenticationToken.js +1 -1
- package/src/Consumer.cc +72 -4
- package/src/Consumer.h +2 -0
- package/src/ConsumerConfig.cc +21 -0
- package/src/ProducerConfig.cc +6 -0
- package/src/Reader.cc +51 -1
- package/src/Reader.h +2 -0
- package/src/pulsar-binding.js +26 -0
- package/tests/conf/standalone.conf +6 -0
- package/tests/docker-load-test.sh +35 -0
- package/tests/end_to_end.test.js +212 -0
- package/{run-unit-tests.sh → tests/load-test.sh} +18 -19
- package/{docker-tests.sh → tests/run-unit-tests.sh} +10 -13
- package/pulsar-version.txt +0 -1
package/README.md
CHANGED
|
@@ -23,155 +23,135 @@
|
|
|
23
23
|
|
|
24
24
|
The Pulsar Node.js client can be used to create Pulsar producers and consumers in Node.js.
|
|
25
25
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
Pulsar Node.js client library is based on the C++ client library. Follow the instructions for
|
|
29
|
-
[C++ library](https://pulsar.apache.org/docs/en/client-libraries-cpp/) for installing the binaries through
|
|
30
|
-
[RPM](https://pulsar.apache.org/docs/en/client-libraries-cpp/#rpm),
|
|
31
|
-
[Deb](https://pulsar.apache.org/docs/en/client-libraries-cpp/#deb) or
|
|
32
|
-
[Homebrew packages](https://pulsar.apache.org/docs/en/client-libraries-cpp/#macos).
|
|
33
|
-
|
|
34
|
-
(Note: you will need to install not only the pulsar-client library but also the pulsar-client-dev library)
|
|
35
|
-
|
|
36
|
-
Also, this library works only in Node.js 10.x or later because it uses the
|
|
26
|
+
This library works only in Node.js 10.x or later because it uses the
|
|
37
27
|
[node-addon-api](https://github.com/nodejs/node-addon-api) module to wrap the C++ library.
|
|
38
28
|
|
|
39
|
-
##
|
|
40
|
-
|
|
41
|
-
Compatibility between each version of the Node.js client and the C++ client is as follows:
|
|
42
|
-
|
|
43
|
-
| Node.js client | C++ client |
|
|
44
|
-
|----------------|-----------------|
|
|
45
|
-
| 1.0.x | 2.3.0 or later |
|
|
46
|
-
| 1.1.x | 2.4.0 or later |
|
|
47
|
-
| 1.2.x | 2.5.0 or later |
|
|
48
|
-
| 1.3.x | 2.7.0 or later |
|
|
49
|
-
| 1.4.x - 1.6.x | 2.8.0 or later |
|
|
50
|
-
| 1.7.x | 2.10.1 or later |
|
|
29
|
+
## Getting Started
|
|
51
30
|
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
31
|
+
> **Note**
|
|
32
|
+
>
|
|
33
|
+
> These instructions are only available for versions after 1.8.0. For versions previous to 1.8.0, you need to install the C++ client first. Please switch to the corresponding version branch of this repo to read the specific instructions.
|
|
34
|
+
>
|
|
35
|
+
> To run the examples, skip this section.
|
|
57
36
|
|
|
58
|
-
|
|
37
|
+
To use the Pulsar Node.js client in your project, run:
|
|
59
38
|
|
|
60
39
|
```shell
|
|
61
|
-
|
|
62
|
-
-A x64 \
|
|
63
|
-
-DBUILD_PYTHON_WRAPPER=OFF -DBUILD_TESTS=OFF \
|
|
64
|
-
-DVCPKG_TRIPLET=x64-windows \
|
|
65
|
-
-DCMAKE_BUILD_TYPE=Release \
|
|
66
|
-
-S .
|
|
67
|
-
cmake --config Release
|
|
40
|
+
npm install pulsar-client
|
|
68
41
|
```
|
|
69
42
|
|
|
70
|
-
|
|
71
|
-
2. Set the variable `PULSAR_CPP_DIR` with the `pulsar-client-cpp` path in a Windows command tool.
|
|
43
|
+
or
|
|
72
44
|
|
|
73
45
|
```shell
|
|
74
|
-
|
|
75
|
-
set PULSAR_CPP_DIR=C:\pulsar\pulsar-client-cpp
|
|
46
|
+
yarn add pulsar-client
|
|
76
47
|
```
|
|
77
48
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
```
|
|
81
|
-
|
|
49
|
+
Then you can run the following simple end-to-end example:
|
|
50
|
+
|
|
51
|
+
```javascript
|
|
52
|
+
const Pulsar = require('pulsar-client');
|
|
53
|
+
|
|
54
|
+
(async () => {
|
|
55
|
+
// Create a client
|
|
56
|
+
const client = new Pulsar.Client({
|
|
57
|
+
serviceUrl: 'pulsar://localhost:6650'
|
|
58
|
+
});
|
|
59
|
+
|
|
60
|
+
// Create a producer
|
|
61
|
+
const producer = await client.createProducer({
|
|
62
|
+
topic: 'persistent://public/default/my-topic',
|
|
63
|
+
});
|
|
64
|
+
|
|
65
|
+
// Create a consumer
|
|
66
|
+
const consumer = await client.subscribe({
|
|
67
|
+
topic: 'persistent://public/default/my-topic',
|
|
68
|
+
subscription: 'sub1'
|
|
69
|
+
});
|
|
70
|
+
|
|
71
|
+
// Send a message
|
|
72
|
+
producer.send({
|
|
73
|
+
data: Buffer.from("hello")
|
|
74
|
+
});
|
|
75
|
+
|
|
76
|
+
// Receive the message
|
|
77
|
+
const msg = await consumer.receive();
|
|
78
|
+
console.log(msg.getData().toString());
|
|
79
|
+
consumer.acknowledge(msg);
|
|
80
|
+
|
|
81
|
+
await producer.close();
|
|
82
|
+
await consumer.close();
|
|
83
|
+
await client.close();
|
|
84
|
+
})();
|
|
82
85
|
```
|
|
83
86
|
|
|
84
|
-
|
|
87
|
+
You should find the output as:
|
|
85
88
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
```shell
|
|
89
|
-
brew install libpulsar
|
|
89
|
+
```
|
|
90
|
+
hello
|
|
90
91
|
```
|
|
91
92
|
|
|
92
|
-
|
|
93
|
+
You can see more examples in the [examples](./examples) directory. However, since these examples might use an API that was not released yet, you need to build this module. See the next section.
|
|
93
94
|
|
|
94
|
-
|
|
95
|
-
brew info libpulsar
|
|
96
|
-
```
|
|
95
|
+
## How to build
|
|
97
96
|
|
|
97
|
+
> **Note**
|
|
98
|
+
>
|
|
99
|
+
> Build from source code requires the Node.js version greater than 16.18
|
|
98
100
|
|
|
99
|
-
|
|
101
|
+
First, clone the repository.
|
|
100
102
|
|
|
101
103
|
```shell
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
export PULSAR_CPP_DIR=/usr/local/Cellar/libpulsar/2.9.1_1
|
|
105
|
-
|
|
106
|
-
## Apple Silicon and Homebrew since 3.0.0
|
|
107
|
-
## cf. https://brew.sh/2021/02/05/homebrew-3.0.0/
|
|
108
|
-
export PULSAR_CPP_DIR=/opt/homebrew/Cellar/libpulsar/2.9.1_1
|
|
104
|
+
git clone https://github.com/apache/pulsar-client-node.git
|
|
105
|
+
cd pulsar-client-node
|
|
109
106
|
```
|
|
110
107
|
|
|
111
|
-
|
|
108
|
+
Since this client is a [C++ addon](https://nodejs.org/api/addons.html#c-addons) that depends on the [Pulsar C++ client](https://github.com/apache/pulsar-client-cpp), you need to install the C++ client first. You need to ensure there is a C++ compiler that supports C++11 installed in your system.
|
|
112
109
|
|
|
113
|
-
|
|
114
|
-
```
|
|
115
|
-
# rpm
|
|
116
|
-
$ yum install gcc-c++ make
|
|
117
|
-
|
|
118
|
-
# debian
|
|
119
|
-
$ apt-get install g++ make
|
|
120
|
-
```
|
|
110
|
+
- Install C++ client on Linux:
|
|
121
111
|
|
|
122
|
-
|
|
112
|
+
```shell
|
|
113
|
+
pkg/linux/download-cpp-client.sh
|
|
123
114
|
```
|
|
124
|
-
# Set the version of Pulsar C++ client to install
|
|
125
|
-
$ PULSAR_CPP_CLIENT_VERSION=2.9.1
|
|
126
|
-
|
|
127
|
-
# rpm
|
|
128
|
-
$ wget https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_CPP_CLIENT_VERSION}/RPMS/apache-pulsar-client-${PULSAR_CPP_CLIENT_VERSION}-1.x86_64.rpm
|
|
129
|
-
$ wget https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_CPP_CLIENT_VERSION}/RPMS/apache-pulsar-client-devel-${PULSAR_CPP_CLIENT_VERSION}-1.x86_64.rpm
|
|
130
115
|
|
|
131
|
-
|
|
132
|
-
$ wget https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_CPP_CLIENT_VERSION}/DEB/apache-pulsar-client.deb
|
|
133
|
-
$ wget https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_CPP_CLIENT_VERSION}/DEB/apache-pulsar-client-dev.deb
|
|
134
|
-
```
|
|
116
|
+
- Install C++ client on Windows:
|
|
135
117
|
|
|
136
|
-
3. Install the Pulsar C++ client.
|
|
137
118
|
```shell
|
|
138
|
-
|
|
139
|
-
$ rpm -ivh apache-pulsar-client*.rpm
|
|
140
|
-
|
|
141
|
-
# debian
|
|
142
|
-
$ apt install ./apache-pulsar-client*.deb
|
|
119
|
+
pkg\windows\download-cpp-client.bat
|
|
143
120
|
```
|
|
144
121
|
|
|
145
|
-
|
|
146
|
-
### Install pulsar-client to your project
|
|
122
|
+
- Install C++ client on macOS:
|
|
147
123
|
|
|
148
124
|
```shell
|
|
149
|
-
|
|
125
|
+
pkg/mac/build-cpp-deps-lib.sh
|
|
126
|
+
pkg/mac/build-cpp-lib.sh
|
|
150
127
|
```
|
|
151
128
|
|
|
152
|
-
|
|
129
|
+
After the C++ client is installed, run the following command to build this C++ addon.
|
|
153
130
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
131
|
+
```shell
|
|
132
|
+
npm install
|
|
133
|
+
```
|
|
157
134
|
|
|
158
|
-
|
|
135
|
+
To verify it has been installed successfully, you can run an example like:
|
|
159
136
|
|
|
160
137
|
```shell
|
|
161
|
-
|
|
162
|
-
$ cd pulsar-client-node
|
|
163
|
-
$ npm install
|
|
138
|
+
node examples/producer
|
|
164
139
|
```
|
|
165
140
|
|
|
166
|
-
|
|
167
|
-
>
|
|
168
|
-
> If you build `pulsar-client-node on` windows, you need to set the variable `PULSAR_CPP_DIR` first, then install npm (run the command `npm install`) in a Windows command-line tool.
|
|
169
|
-
|
|
170
|
-
### Rebuild Pulsar client library:
|
|
141
|
+
You should find the output as:
|
|
171
142
|
|
|
172
|
-
```
|
|
173
|
-
|
|
143
|
+
```
|
|
144
|
+
Sent message: my-message-0
|
|
145
|
+
Sent message: my-message-1
|
|
146
|
+
Sent message: my-message-2
|
|
147
|
+
Sent message: my-message-3
|
|
148
|
+
Sent message: my-message-4
|
|
149
|
+
Sent message: my-message-5
|
|
150
|
+
Sent message: my-message-6
|
|
151
|
+
Sent message: my-message-7
|
|
152
|
+
Sent message: my-message-8
|
|
153
|
+
Sent message: my-message-9
|
|
174
154
|
```
|
|
175
155
|
|
|
176
156
|
## Documentation
|
|
177
|
-
* Please see https://pulsar.apache.org/docs/
|
|
157
|
+
* Please see https://pulsar.apache.org/docs/client-libraries-node/ for more details about the Pulsar Node.js client.
|
package/binding.gyp
CHANGED
|
@@ -18,22 +18,9 @@
|
|
|
18
18
|
#
|
|
19
19
|
|
|
20
20
|
{
|
|
21
|
-
'conditions': [
|
|
22
|
-
['OS=="win"', {
|
|
23
|
-
'variables': {
|
|
24
|
-
'pulsar_cpp_dir%': '<!(echo %PULSAR_CPP_DIR%)',
|
|
25
|
-
'os_arch%': '<!(echo %OS_ARCH%)',
|
|
26
|
-
}
|
|
27
|
-
}],
|
|
28
|
-
['OS=="mac"', {
|
|
29
|
-
'variables': {
|
|
30
|
-
'pulsar_cpp_dir': '<!(echo $PULSAR_CPP_DIR)'
|
|
31
|
-
}
|
|
32
|
-
}]
|
|
33
|
-
],
|
|
34
21
|
"targets": [
|
|
35
22
|
{
|
|
36
|
-
"target_name": "
|
|
23
|
+
"target_name": "pulsar",
|
|
37
24
|
"cflags_cc": ["-std=gnu++11"],
|
|
38
25
|
"cflags!": ["-fno-exceptions"],
|
|
39
26
|
"cflags_cc!": ["-fno-exceptions", "-std=gnu++14", "-std=gnu++17"],
|
|
@@ -60,21 +47,27 @@
|
|
|
60
47
|
['OS=="mac"', {
|
|
61
48
|
'xcode_settings': {
|
|
62
49
|
'GCC_ENABLE_CPP_EXCEPTIONS': 'YES',
|
|
63
|
-
'
|
|
50
|
+
'GCC_ENABLE_CPP_RTTI': 'YES',
|
|
51
|
+
'MACOSX_DEPLOYMENT_TARGET': '11.0',
|
|
52
|
+
'CLANG_CXX_LANGUAGE_STANDARD': 'gnu++11',
|
|
53
|
+
'OTHER_CFLAGS': [
|
|
54
|
+
"-fPIC",
|
|
55
|
+
]
|
|
64
56
|
},
|
|
57
|
+
"dependencies": [
|
|
58
|
+
"<!@(node -p \"require('node-addon-api').gyp\")"
|
|
59
|
+
],
|
|
65
60
|
"include_dirs": [
|
|
66
|
-
"
|
|
61
|
+
"pkg/mac/build-pulsar/install/include"
|
|
67
62
|
],
|
|
68
63
|
"libraries": [
|
|
69
|
-
"
|
|
70
|
-
],
|
|
71
|
-
"dependencies": [
|
|
72
|
-
"<!@(node -p \"require('node-addon-api').gyp\")"
|
|
64
|
+
"../pkg/mac/build-pulsar/install/lib/libpulsarwithdeps.a"
|
|
73
65
|
],
|
|
74
66
|
}],
|
|
75
67
|
['OS=="win"', {
|
|
76
68
|
"defines": [
|
|
77
|
-
"_HAS_EXCEPTIONS=1"
|
|
69
|
+
"_HAS_EXCEPTIONS=1",
|
|
70
|
+
"PULSAR_STATIC"
|
|
78
71
|
],
|
|
79
72
|
"msvs_settings": {
|
|
80
73
|
"VCCLCompilerTool": {
|
|
@@ -82,40 +75,38 @@
|
|
|
82
75
|
},
|
|
83
76
|
},
|
|
84
77
|
"include_dirs": [
|
|
85
|
-
"
|
|
78
|
+
"pkg\\windows\\pulsar-cpp\\include",
|
|
86
79
|
],
|
|
87
80
|
"libraries": [
|
|
88
|
-
"-
|
|
81
|
+
"..\\pkg\\windows\\pulsar-cpp\\lib\\pulsarWithDeps.lib"
|
|
89
82
|
],
|
|
90
83
|
"dependencies": [
|
|
91
84
|
"<!(node -p \"require('node-addon-api').gyp\")"
|
|
92
|
-
],
|
|
93
|
-
"copies": [
|
|
94
|
-
{
|
|
95
|
-
"destination": "<(PRODUCT_DIR)",
|
|
96
|
-
"files": [
|
|
97
|
-
"<(pulsar_cpp_dir)\\lib\Release\pulsar.dll",
|
|
98
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\libcurl.dll",
|
|
99
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\libprotobuf.dll",
|
|
100
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\libssl-1_1-x64.dll",
|
|
101
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\libcrypto-1_1-x64.dll",
|
|
102
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\dl.dll",
|
|
103
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\snappy.dll",
|
|
104
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\zlib1.dll",
|
|
105
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\zstd.dll",
|
|
106
|
-
]
|
|
107
|
-
}
|
|
108
85
|
]
|
|
109
86
|
}],
|
|
110
|
-
['OS
|
|
111
|
-
"libraries": [
|
|
112
|
-
"-lpulsar",
|
|
113
|
-
],
|
|
87
|
+
['OS=="linux"', {
|
|
114
88
|
"dependencies": [
|
|
115
89
|
"<!@(node -p \"require('node-addon-api').gyp\")"
|
|
116
90
|
],
|
|
91
|
+
"include_dirs": [
|
|
92
|
+
"pkg/linux/pulsar-cpp/include"
|
|
93
|
+
],
|
|
94
|
+
"libraries": [
|
|
95
|
+
"../pkg/linux/pulsar-cpp/lib/libpulsarwithdeps.a"
|
|
96
|
+
],
|
|
117
97
|
}]
|
|
118
98
|
]
|
|
99
|
+
},
|
|
100
|
+
{
|
|
101
|
+
"target_name": "action_after_build",
|
|
102
|
+
"type": "none",
|
|
103
|
+
"dependencies": [ "<(module_name)" ],
|
|
104
|
+
"copies": [
|
|
105
|
+
{
|
|
106
|
+
"files": [ "<(PRODUCT_DIR)/<(module_name).node" ],
|
|
107
|
+
"destination": "<(module_path)"
|
|
108
|
+
}
|
|
109
|
+
]
|
|
119
110
|
}
|
|
120
111
|
]
|
|
121
112
|
}
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#!/bin/
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
2
|
#
|
|
3
3
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
4
|
# or more contributor license agreements. See the NOTICE file
|
|
@@ -18,9 +18,7 @@
|
|
|
18
18
|
# under the License.
|
|
19
19
|
#
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
import yaml, sys
|
|
22
22
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
bin/pulsar-daemon stop standalone
|
|
23
|
+
deps = yaml.safe_load(open("dependencies.yaml"))
|
|
24
|
+
print(deps[sys.argv[1]])
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
|
|
21
|
+
import sys, json, urllib.request, os, shutil, zipfile, tempfile
|
|
22
|
+
from pathlib import Path
|
|
23
|
+
|
|
24
|
+
if len(sys.argv) != 3:
|
|
25
|
+
print("Usage: ")
|
|
26
|
+
print(" %s $WORKFLOW_RUN_ID $DEST_PATH" % sys.argv[0])
|
|
27
|
+
sys.exit(-1)
|
|
28
|
+
|
|
29
|
+
if 'GITHUB_TOKEN' not in os.environ:
|
|
30
|
+
print('You should have a GITHUB_TOKEN environment variable')
|
|
31
|
+
sys.exit(-1)
|
|
32
|
+
|
|
33
|
+
GITHUB_TOKEN = os.environ['GITHUB_TOKEN']
|
|
34
|
+
|
|
35
|
+
ACCEPT_HEADER = 'application/vnd.github+json'
|
|
36
|
+
LIST_URL = 'https://api.github.com/repos/apache/pulsar-client-node/actions/runs/%d/artifacts'
|
|
37
|
+
|
|
38
|
+
workflow_run_id = int(sys.argv[1])
|
|
39
|
+
dest_path = sys.argv[2]
|
|
40
|
+
|
|
41
|
+
workflow_run_url = LIST_URL % workflow_run_id
|
|
42
|
+
request = urllib.request.Request(workflow_run_url,
|
|
43
|
+
headers={'Accept': ACCEPT_HEADER, 'Authorization': 'Bearer ' + GITHUB_TOKEN})
|
|
44
|
+
with urllib.request.urlopen(request) as response:
|
|
45
|
+
data = json.loads(response.read().decode("utf-8"))
|
|
46
|
+
for artifact in data['artifacts']:
|
|
47
|
+
name = artifact['name']
|
|
48
|
+
url = artifact['archive_download_url']
|
|
49
|
+
|
|
50
|
+
print('Downloading %s from %s' % (name, url))
|
|
51
|
+
artifact_request = urllib.request.Request(url,
|
|
52
|
+
headers={'Authorization': 'Bearer ' + GITHUB_TOKEN})
|
|
53
|
+
with urllib.request.urlopen(artifact_request) as response:
|
|
54
|
+
tmp_zip = tempfile.NamedTemporaryFile(delete=False)
|
|
55
|
+
try:
|
|
56
|
+
#
|
|
57
|
+
shutil.copyfileobj(response, tmp_zip)
|
|
58
|
+
tmp_zip.close()
|
|
59
|
+
|
|
60
|
+
dest_dir = os.path.join(dest_path, name)
|
|
61
|
+
Path(dest_dir).mkdir(parents=True, exist_ok=True)
|
|
62
|
+
with zipfile.ZipFile(tmp_zip.name, 'r') as z:
|
|
63
|
+
z.extractall(dest_dir)
|
|
64
|
+
finally:
|
|
65
|
+
os.unlink(tmp_zip.name)
|
|
66
|
+
|
|
67
|
+
for root, dirs, files in os.walk(dest_path, topdown=False):
|
|
68
|
+
for name in files:
|
|
69
|
+
shutil.move(os.path.join(root, name), dest_path)
|
|
70
|
+
if not os.listdir(root):
|
|
71
|
+
os.rmdir(root)
|
|
72
|
+
|
|
73
|
+
|
|
74
|
+
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
ROOT_DIR=$(git rev-parse --show-toplevel)
|
|
21
|
+
|
|
22
|
+
VERSION=$(npm version | grep pulsar-client | awk -F ':' '{print $2}' | sed "s?'??g" | sed "s?,??g" | sed "s? ??g")
|
|
23
|
+
|
|
24
|
+
NAME=apache-pulsar-client-node-$VERSION
|
|
25
|
+
|
|
26
|
+
OUT_DIR=${1:-.}
|
|
27
|
+
|
|
28
|
+
git archive --format=tar.gz --prefix ${NAME}/ -o ${OUT_DIR}/${NAME}.tar.gz HEAD
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#!/bin/bash
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
2
|
#
|
|
3
3
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
4
|
# or more contributor license agreements. See the NOTICE file
|
|
@@ -18,28 +18,18 @@
|
|
|
18
18
|
# under the License.
|
|
19
19
|
#
|
|
20
20
|
|
|
21
|
-
set -e
|
|
21
|
+
set -e -x
|
|
22
22
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
rm -rf $PULSAR_DIR
|
|
31
|
-
curl -L --create-dir "https://archive.apache.org/dist/pulsar/pulsar-${VERSION}/${PKG}" -o $PULSAR_DIR/$PKG
|
|
32
|
-
tar xfz $PULSAR_DIR/$PKG -C $PULSAR_DIR --strip-components 1
|
|
33
|
-
|
|
34
|
-
DATA_DIR=/tmp/pulsar-test-data
|
|
35
|
-
rm -rf $DATA_DIR
|
|
36
|
-
mkdir -p $DATA_DIR
|
|
37
|
-
|
|
38
|
-
export PULSAR_STANDALONE_CONF=$ROOT_DIR/tests/conf/standalone.conf
|
|
39
|
-
$PULSAR_DIR/bin/pulsar-daemon start standalone \
|
|
23
|
+
export PULSAR_STANDALONE_CONF=test-conf/standalone.conf
|
|
24
|
+
# There is an issue when starting the pulsar standalone with other metadata store: https://github.com/apache/pulsar/issues/17984
|
|
25
|
+
# We need to use Zookeeper here. Otherwise the `Message Chunking` test will not pass.
|
|
26
|
+
# We can remove this line after the pulsar release a new version contains this fix: https://github.com/apache/pulsar/pull/18126
|
|
27
|
+
export PULSAR_STANDALONE_USE_ZOOKEEPER=1
|
|
28
|
+
bin/pulsar-daemon start standalone \
|
|
40
29
|
--no-functions-worker --no-stream-storage \
|
|
41
|
-
--
|
|
42
|
-
--bookkeeper-dir $DATA_DIR/bookkeeper
|
|
30
|
+
--bookkeeper-dir data/bookkeeper
|
|
43
31
|
|
|
44
32
|
echo "-- Wait for Pulsar service to be ready"
|
|
45
33
|
until curl http://localhost:8080/metrics > /dev/null 2>&1 ; do sleep 1; done
|
|
34
|
+
|
|
35
|
+
echo "-- Ready to start tests"
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
|
|
21
|
+
set -e
|
|
22
|
+
|
|
23
|
+
SRC_DIR=$(git rev-parse --show-toplevel)
|
|
24
|
+
cd $SRC_DIR
|
|
25
|
+
|
|
26
|
+
build-support/pulsar-test-service-stop.sh
|
|
27
|
+
|
|
28
|
+
CONTAINER_ID=$(docker run -i -p 8080:8080 -p 6650:6650 -p 8443:8443 -p 6651:6651 --rm --detach apachepulsar/pulsar:2.10.2 sleep 3600)
|
|
29
|
+
|
|
30
|
+
echo $CONTAINER_ID >.tests-container-id.txt
|
|
31
|
+
|
|
32
|
+
docker cp $SRC_DIR/tests/conf $CONTAINER_ID:/pulsar/test-conf
|
|
33
|
+
docker cp $SRC_DIR/build-support/pulsar-test-container-start.sh $CONTAINER_ID:pulsar-test-container-start.sh
|
|
34
|
+
|
|
35
|
+
docker exec -i $CONTAINER_ID /pulsar-test-container-start.sh
|
|
36
|
+
|
|
37
|
+
echo "-- Ready to start tests"
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
|
|
21
|
+
set -e
|
|
22
|
+
|
|
23
|
+
SRC_DIR=$(git rev-parse --show-toplevel)
|
|
24
|
+
cd $SRC_DIR
|
|
25
|
+
|
|
26
|
+
CONTAINER_ID_PATH=".tests-container-id.txt"
|
|
27
|
+
|
|
28
|
+
if [ -f ${CONTAINER_ID_PATH} ]; then
|
|
29
|
+
CONTAINER_ID=$(cat $CONTAINER_ID_PATH)
|
|
30
|
+
docker kill $CONTAINER_ID || true
|
|
31
|
+
rm .tests-container-id.txt
|
|
32
|
+
fi
|
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
1
2
|
#
|
|
2
3
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
3
4
|
# or more contributor license agreements. See the NOTICE file
|
|
@@ -16,16 +17,16 @@
|
|
|
16
17
|
# specific language governing permissions and limitations
|
|
17
18
|
# under the License.
|
|
18
19
|
#
|
|
19
|
-
name: Node.js
|
|
20
|
-
on: [pull_request]
|
|
21
|
-
jobs:
|
|
22
|
-
build:
|
|
23
|
-
name: Build
|
|
24
|
-
runs-on: ubuntu-latest
|
|
25
|
-
steps:
|
|
26
|
-
- name: Check out code into the Node.js module directory
|
|
27
|
-
uses: actions/checkout@v2
|
|
28
20
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
21
|
+
set -e
|
|
22
|
+
|
|
23
|
+
FILES=$*
|
|
24
|
+
|
|
25
|
+
for FILE in $FILES
|
|
26
|
+
do
|
|
27
|
+
echo "Signing $FILE"
|
|
28
|
+
gpg --armor --output $FILE.asc --detach-sig $FILE
|
|
29
|
+
|
|
30
|
+
# SHA-512 signature
|
|
31
|
+
shasum -a 512 $FILE > $FILE.sha512
|
|
32
|
+
done
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
|
|
21
|
+
set -e -x
|
|
22
|
+
|
|
23
|
+
if [ $# -neq 2 ]; then
|
|
24
|
+
echo "Usage: $0 \$DEST_PATH \$WORKFLOW_ID"
|
|
25
|
+
exit 1
|
|
26
|
+
fi
|
|
27
|
+
|
|
28
|
+
DEST_PATH=$(readlink -f $1)
|
|
29
|
+
WORKFLOW_ID=$2
|
|
30
|
+
|
|
31
|
+
pushd $(dirname "$0")
|
|
32
|
+
PULSAR_NODE_PATH=$(git rev-parse --show-toplevel)
|
|
33
|
+
popd
|
|
34
|
+
|
|
35
|
+
mkdir -p $DEST_PATH
|
|
36
|
+
|
|
37
|
+
cd $PULSAR_NODE_PATH
|
|
38
|
+
|
|
39
|
+
build-support/download-release-artifacts.py $WORKFLOW_ID $DEST_PATH
|
|
40
|
+
build-support/generate-source-archive.sh $DEST_PATH
|
|
41
|
+
|
|
42
|
+
# Sign all files
|
|
43
|
+
cd $DEST_PATH
|
|
44
|
+
find . -type f | xargs $PULSAR_NODE_PATH/build-support/sign-files.sh
|