pulsar-client 1.7.0 → 1.8.0-rc1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.asf.yaml +8 -1
- package/.github/PULL_REQUEST_TEMPLATE.md +65 -0
- package/.github/workflows/ci-build-release-napi.yml +195 -0
- package/.github/workflows/ci-pr-validation.yml +225 -0
- package/README.md +83 -107
- package/binding.gyp +28 -44
- package/{pulsar-test-service-stop.sh → build-support/dep-version.py} +4 -6
- package/build-support/download-release-artifacts.py +74 -0
- package/build-support/generate-source-archive.sh +28 -0
- package/build-support/install-cpp-client.sh +66 -0
- package/{pulsar-test-service-start.sh → build-support/pulsar-test-container-start.sh} +11 -21
- package/build-support/pulsar-test-service-start.sh +37 -0
- package/build-support/pulsar-test-service-stop.sh +32 -0
- package/{.github/workflows/nodejs.yml → build-support/sign-files.sh} +13 -12
- package/build-support/stage-release.sh +44 -0
- package/dependencies.yaml +28 -0
- package/docs/release-process.md +242 -0
- package/examples/consumer.js +1 -1
- package/examples/consumer_listener.js +1 -1
- package/examples/consumer_tls_auth.js +1 -1
- package/examples/custom_logger.js +60 -0
- package/examples/encryption-consumer.js +1 -1
- package/examples/encryption-producer.js +1 -1
- package/examples/producer.js +1 -1
- package/examples/producer_tls_auth.js +1 -1
- package/examples/reader.js +1 -1
- package/examples/reader_listener.js +1 -1
- package/index.d.ts +7 -0
- package/index.js +2 -1
- package/package.json +15 -7
- package/pkg/build-napi-inside-docker.sh +31 -0
- package/pkg/linux_glibc/Dockerfile +33 -0
- package/pkg/linux_musl/Dockerfile +32 -0
- package/pkg/load_test.js +30 -0
- package/pkg/mac/build-cpp-deps-lib.sh +186 -0
- package/pkg/mac/build-cpp-lib.sh +51 -0
- package/pkg/mac/common.sh +37 -0
- package/pkg/windows/download-cpp-client.bat +12 -0
- package/pulsar-client-cpp.txt +2 -0
- package/src/AuthenticationAthenz.js +1 -1
- package/src/AuthenticationOauth2.js +1 -1
- package/src/AuthenticationTls.js +1 -1
- package/src/AuthenticationToken.js +1 -1
- package/src/Consumer.cc +72 -4
- package/src/Consumer.h +2 -0
- package/src/ConsumerConfig.cc +21 -0
- package/src/ProducerConfig.cc +6 -0
- package/src/Reader.cc +51 -1
- package/src/Reader.h +2 -0
- package/src/pulsar-binding.js +8 -0
- package/tests/conf/standalone.conf +6 -0
- package/tests/end_to_end.test.js +212 -0
- package/{docker-tests.sh → tests/run-unit-tests.sh} +10 -13
- package/pulsar-version.txt +0 -1
- package/run-unit-tests.sh +0 -44
package/README.md
CHANGED
|
@@ -23,155 +23,131 @@
|
|
|
23
23
|
|
|
24
24
|
The Pulsar Node.js client can be used to create Pulsar producers and consumers in Node.js.
|
|
25
25
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
Pulsar Node.js client library is based on the C++ client library. Follow the instructions for
|
|
29
|
-
[C++ library](https://pulsar.apache.org/docs/en/client-libraries-cpp/) for installing the binaries through
|
|
30
|
-
[RPM](https://pulsar.apache.org/docs/en/client-libraries-cpp/#rpm),
|
|
31
|
-
[Deb](https://pulsar.apache.org/docs/en/client-libraries-cpp/#deb) or
|
|
32
|
-
[Homebrew packages](https://pulsar.apache.org/docs/en/client-libraries-cpp/#macos).
|
|
33
|
-
|
|
34
|
-
(Note: you will need to install not only the pulsar-client library but also the pulsar-client-dev library)
|
|
35
|
-
|
|
36
|
-
Also, this library works only in Node.js 10.x or later because it uses the
|
|
26
|
+
This library works only in Node.js 10.x or later because it uses the
|
|
37
27
|
[node-addon-api](https://github.com/nodejs/node-addon-api) module to wrap the C++ library.
|
|
38
28
|
|
|
39
|
-
##
|
|
40
|
-
|
|
41
|
-
Compatibility between each version of the Node.js client and the C++ client is as follows:
|
|
42
|
-
|
|
43
|
-
| Node.js client | C++ client |
|
|
44
|
-
|----------------|-----------------|
|
|
45
|
-
| 1.0.x | 2.3.0 or later |
|
|
46
|
-
| 1.1.x | 2.4.0 or later |
|
|
47
|
-
| 1.2.x | 2.5.0 or later |
|
|
48
|
-
| 1.3.x | 2.7.0 or later |
|
|
49
|
-
| 1.4.x - 1.6.x | 2.8.0 or later |
|
|
50
|
-
| 1.7.x | 2.10.1 or later |
|
|
29
|
+
## Getting Started
|
|
51
30
|
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
31
|
+
> **Note**
|
|
32
|
+
>
|
|
33
|
+
> These instructions are only available for versions after 1.8.0. For versions previous to 1.8.0, you need to install the C++ client first. Please switch to the corresponding version branch of this repo to read the specific instructions.
|
|
34
|
+
>
|
|
35
|
+
> To run the examples, skip this section.
|
|
57
36
|
|
|
58
|
-
|
|
37
|
+
To use the Pulsar Node.js client in your project, run:
|
|
59
38
|
|
|
60
39
|
```shell
|
|
61
|
-
|
|
62
|
-
-A x64 \
|
|
63
|
-
-DBUILD_PYTHON_WRAPPER=OFF -DBUILD_TESTS=OFF \
|
|
64
|
-
-DVCPKG_TRIPLET=x64-windows \
|
|
65
|
-
-DCMAKE_BUILD_TYPE=Release \
|
|
66
|
-
-S .
|
|
67
|
-
cmake --config Release
|
|
40
|
+
npm install pulsar-client
|
|
68
41
|
```
|
|
69
42
|
|
|
70
|
-
|
|
71
|
-
2. Set the variable `PULSAR_CPP_DIR` with the `pulsar-client-cpp` path in a Windows command tool.
|
|
43
|
+
or
|
|
72
44
|
|
|
73
45
|
```shell
|
|
74
|
-
|
|
75
|
-
set PULSAR_CPP_DIR=C:\pulsar\pulsar-client-cpp
|
|
46
|
+
yarn add pulsar-client
|
|
76
47
|
```
|
|
77
48
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
```
|
|
81
|
-
|
|
49
|
+
Then you can run the following simple end-to-end example:
|
|
50
|
+
|
|
51
|
+
```javascript
|
|
52
|
+
const Pulsar = require('pulsar-client');
|
|
53
|
+
|
|
54
|
+
(async () => {
|
|
55
|
+
// Create a client
|
|
56
|
+
const client = new Pulsar.Client({
|
|
57
|
+
serviceUrl: 'pulsar://localhost:6650'
|
|
58
|
+
});
|
|
59
|
+
|
|
60
|
+
// Create a producer
|
|
61
|
+
const producer = await client.createProducer({
|
|
62
|
+
topic: 'persistent://public/default/my-topic',
|
|
63
|
+
});
|
|
64
|
+
|
|
65
|
+
// Create a consumer
|
|
66
|
+
const consumer = await client.subscribe({
|
|
67
|
+
topic: 'persistent://public/default/my-topic',
|
|
68
|
+
subscription: 'sub1'
|
|
69
|
+
});
|
|
70
|
+
|
|
71
|
+
// Send a message
|
|
72
|
+
producer.send({
|
|
73
|
+
data: Buffer.from("hello")
|
|
74
|
+
});
|
|
75
|
+
|
|
76
|
+
// Receive the message
|
|
77
|
+
const msg = await consumer.receive();
|
|
78
|
+
console.log(msg.getData().toString());
|
|
79
|
+
consumer.acknowledge(msg);
|
|
80
|
+
|
|
81
|
+
await producer.close();
|
|
82
|
+
await consumer.close();
|
|
83
|
+
await client.close();
|
|
84
|
+
})();
|
|
82
85
|
```
|
|
83
86
|
|
|
84
|
-
|
|
87
|
+
You should find the output as:
|
|
85
88
|
|
|
86
|
-
1. Install the Pulsar C++ client on mac.
|
|
87
|
-
|
|
88
|
-
```shell
|
|
89
|
-
brew install libpulsar
|
|
90
89
|
```
|
|
91
|
-
|
|
92
|
-
2. Get the installation path of libpulsar
|
|
93
|
-
|
|
94
|
-
```shell
|
|
95
|
-
brew info libpulsar
|
|
90
|
+
hello
|
|
96
91
|
```
|
|
97
92
|
|
|
93
|
+
You can see more examples in the [examples](./examples) directory. However, since these examples might use an API that was not released yet, you need to build this module. See the next section.
|
|
94
|
+
|
|
95
|
+
## How to build
|
|
98
96
|
|
|
99
|
-
|
|
97
|
+
First, clone the repository.
|
|
100
98
|
|
|
101
99
|
```shell
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
export PULSAR_CPP_DIR=/usr/local/Cellar/libpulsar/2.9.1_1
|
|
105
|
-
|
|
106
|
-
## Apple Silicon and Homebrew since 3.0.0
|
|
107
|
-
## cf. https://brew.sh/2021/02/05/homebrew-3.0.0/
|
|
108
|
-
export PULSAR_CPP_DIR=/opt/homebrew/Cellar/libpulsar/2.9.1_1
|
|
100
|
+
git clone https://github.com/apache/pulsar-client-node.git
|
|
101
|
+
cd pulsar-client-node
|
|
109
102
|
```
|
|
110
103
|
|
|
111
|
-
|
|
104
|
+
Since this client is a [C++ addon](https://nodejs.org/api/addons.html#c-addons) that depends on the [Pulsar C++ client](https://github.com/apache/pulsar-client-cpp), you need to install the C++ client first. You need to ensure there is a C++ compiler that supports C++11 installed in your system.
|
|
112
105
|
|
|
113
|
-
|
|
114
|
-
```
|
|
115
|
-
# rpm
|
|
116
|
-
$ yum install gcc-c++ make
|
|
117
|
-
|
|
118
|
-
# debian
|
|
119
|
-
$ apt-get install g++ make
|
|
120
|
-
```
|
|
106
|
+
- Install C++ client on Linux:
|
|
121
107
|
|
|
122
|
-
|
|
108
|
+
```shell
|
|
109
|
+
build-support/install-cpp-client.sh
|
|
123
110
|
```
|
|
124
|
-
# Set the version of Pulsar C++ client to install
|
|
125
|
-
$ PULSAR_CPP_CLIENT_VERSION=2.9.1
|
|
126
|
-
|
|
127
|
-
# rpm
|
|
128
|
-
$ wget https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_CPP_CLIENT_VERSION}/RPMS/apache-pulsar-client-${PULSAR_CPP_CLIENT_VERSION}-1.x86_64.rpm
|
|
129
|
-
$ wget https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_CPP_CLIENT_VERSION}/RPMS/apache-pulsar-client-devel-${PULSAR_CPP_CLIENT_VERSION}-1.x86_64.rpm
|
|
130
111
|
|
|
131
|
-
|
|
132
|
-
$ wget https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_CPP_CLIENT_VERSION}/DEB/apache-pulsar-client.deb
|
|
133
|
-
$ wget https://archive.apache.org/dist/pulsar/pulsar-${PULSAR_CPP_CLIENT_VERSION}/DEB/apache-pulsar-client-dev.deb
|
|
134
|
-
```
|
|
112
|
+
- Install C++ client on Windows:
|
|
135
113
|
|
|
136
|
-
3. Install the Pulsar C++ client.
|
|
137
114
|
```shell
|
|
138
|
-
|
|
139
|
-
$ rpm -ivh apache-pulsar-client*.rpm
|
|
140
|
-
|
|
141
|
-
# debian
|
|
142
|
-
$ apt install ./apache-pulsar-client*.deb
|
|
115
|
+
pkg\windows\download-cpp-client.bat
|
|
143
116
|
```
|
|
144
117
|
|
|
145
|
-
|
|
146
|
-
### Install pulsar-client to your project
|
|
118
|
+
- Install C++ client on macOS:
|
|
147
119
|
|
|
148
120
|
```shell
|
|
149
|
-
|
|
121
|
+
pkg/mac/build-cpp-deps-lib.sh
|
|
122
|
+
pkg/mac/build-cpp-lib.sh
|
|
150
123
|
```
|
|
151
124
|
|
|
152
|
-
|
|
125
|
+
After the C++ client is installed, run the following command to build this C++ addon.
|
|
153
126
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
127
|
+
```shell
|
|
128
|
+
npm install
|
|
129
|
+
```
|
|
157
130
|
|
|
158
|
-
|
|
131
|
+
To verify it has been installed successfully, you can run an example like:
|
|
159
132
|
|
|
160
133
|
```shell
|
|
161
|
-
|
|
162
|
-
$ cd pulsar-client-node
|
|
163
|
-
$ npm install
|
|
134
|
+
node examples/producer
|
|
164
135
|
```
|
|
165
136
|
|
|
166
|
-
|
|
167
|
-
>
|
|
168
|
-
> If you build `pulsar-client-node on` windows, you need to set the variable `PULSAR_CPP_DIR` first, then install npm (run the command `npm install`) in a Windows command-line tool.
|
|
169
|
-
|
|
170
|
-
### Rebuild Pulsar client library:
|
|
137
|
+
You should find the output as:
|
|
171
138
|
|
|
172
|
-
```
|
|
173
|
-
|
|
139
|
+
```
|
|
140
|
+
Sent message: my-message-0
|
|
141
|
+
Sent message: my-message-1
|
|
142
|
+
Sent message: my-message-2
|
|
143
|
+
Sent message: my-message-3
|
|
144
|
+
Sent message: my-message-4
|
|
145
|
+
Sent message: my-message-5
|
|
146
|
+
Sent message: my-message-6
|
|
147
|
+
Sent message: my-message-7
|
|
148
|
+
Sent message: my-message-8
|
|
149
|
+
Sent message: my-message-9
|
|
174
150
|
```
|
|
175
151
|
|
|
176
152
|
## Documentation
|
|
177
|
-
* Please see https://pulsar.apache.org/docs/
|
|
153
|
+
* Please see https://pulsar.apache.org/docs/client-libraries-node/ for more details about the Pulsar Node.js client.
|
package/binding.gyp
CHANGED
|
@@ -18,19 +18,6 @@
|
|
|
18
18
|
#
|
|
19
19
|
|
|
20
20
|
{
|
|
21
|
-
'conditions': [
|
|
22
|
-
['OS=="win"', {
|
|
23
|
-
'variables': {
|
|
24
|
-
'pulsar_cpp_dir%': '<!(echo %PULSAR_CPP_DIR%)',
|
|
25
|
-
'os_arch%': '<!(echo %OS_ARCH%)',
|
|
26
|
-
}
|
|
27
|
-
}],
|
|
28
|
-
['OS=="mac"', {
|
|
29
|
-
'variables': {
|
|
30
|
-
'pulsar_cpp_dir': '<!(echo $PULSAR_CPP_DIR)'
|
|
31
|
-
}
|
|
32
|
-
}]
|
|
33
|
-
],
|
|
34
21
|
"targets": [
|
|
35
22
|
{
|
|
36
23
|
"target_name": "Pulsar",
|
|
@@ -60,21 +47,24 @@
|
|
|
60
47
|
['OS=="mac"', {
|
|
61
48
|
'xcode_settings': {
|
|
62
49
|
'GCC_ENABLE_CPP_EXCEPTIONS': 'YES',
|
|
63
|
-
'
|
|
50
|
+
'GCC_ENABLE_CPP_RTTI': 'YES',
|
|
51
|
+
'MACOSX_DEPLOYMENT_TARGET': '11.0',
|
|
52
|
+
'CLANG_CXX_LANGUAGE_STANDARD': 'gnu++11',
|
|
53
|
+
'OTHER_CFLAGS': [
|
|
54
|
+
"-fPIC",
|
|
55
|
+
]
|
|
64
56
|
},
|
|
65
|
-
"include_dirs": [
|
|
66
|
-
"<(pulsar_cpp_dir)/include",
|
|
67
|
-
],
|
|
68
|
-
"libraries": [
|
|
69
|
-
"<(pulsar_cpp_dir)/lib/libpulsar.dylib"
|
|
70
|
-
],
|
|
71
57
|
"dependencies": [
|
|
72
58
|
"<!@(node -p \"require('node-addon-api').gyp\")"
|
|
73
59
|
],
|
|
60
|
+
"include_dirs": [
|
|
61
|
+
"pkg/mac/build-pulsar/install/include"
|
|
62
|
+
],
|
|
74
63
|
}],
|
|
75
64
|
['OS=="win"', {
|
|
76
65
|
"defines": [
|
|
77
|
-
"_HAS_EXCEPTIONS=1"
|
|
66
|
+
"_HAS_EXCEPTIONS=1",
|
|
67
|
+
"PULSAR_STATIC"
|
|
78
68
|
],
|
|
79
69
|
"msvs_settings": {
|
|
80
70
|
"VCCLCompilerTool": {
|
|
@@ -82,40 +72,34 @@
|
|
|
82
72
|
},
|
|
83
73
|
},
|
|
84
74
|
"include_dirs": [
|
|
85
|
-
"
|
|
75
|
+
"pkg\\windows\\pulsar-cpp\\include",
|
|
86
76
|
],
|
|
87
77
|
"libraries": [
|
|
88
|
-
"-
|
|
78
|
+
"..\\pkg\\windows\\pulsar-cpp\\lib\\pulsarWithDeps.lib"
|
|
89
79
|
],
|
|
90
80
|
"dependencies": [
|
|
91
81
|
"<!(node -p \"require('node-addon-api').gyp\")"
|
|
92
|
-
],
|
|
93
|
-
"copies": [
|
|
94
|
-
{
|
|
95
|
-
"destination": "<(PRODUCT_DIR)",
|
|
96
|
-
"files": [
|
|
97
|
-
"<(pulsar_cpp_dir)\\lib\Release\pulsar.dll",
|
|
98
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\libcurl.dll",
|
|
99
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\libprotobuf.dll",
|
|
100
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\libssl-1_1-x64.dll",
|
|
101
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\libcrypto-1_1-x64.dll",
|
|
102
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\dl.dll",
|
|
103
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\snappy.dll",
|
|
104
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\zlib1.dll",
|
|
105
|
-
"<(pulsar_cpp_dir)\\vcpkg_installed\\<(os_arch)\\bin\zstd.dll",
|
|
106
|
-
]
|
|
107
|
-
}
|
|
108
82
|
]
|
|
109
|
-
}
|
|
110
|
-
['OS!="mac" and OS!="win"', {
|
|
111
|
-
"libraries": [
|
|
112
|
-
"-lpulsar",
|
|
113
|
-
],
|
|
83
|
+
}, { # 'OS!="win"'
|
|
114
84
|
"dependencies": [
|
|
115
85
|
"<!@(node -p \"require('node-addon-api').gyp\")"
|
|
116
86
|
],
|
|
87
|
+
"libraries": [
|
|
88
|
+
"../pkg/lib/libpulsarwithdeps.a"
|
|
89
|
+
],
|
|
117
90
|
}]
|
|
118
91
|
]
|
|
92
|
+
},
|
|
93
|
+
{
|
|
94
|
+
"target_name": "action_after_build",
|
|
95
|
+
"type": "none",
|
|
96
|
+
"dependencies": [ "<(module_name)" ],
|
|
97
|
+
"copies": [
|
|
98
|
+
{
|
|
99
|
+
"files": [ "<(PRODUCT_DIR)/<(module_name).node" ],
|
|
100
|
+
"destination": "<(module_path)"
|
|
101
|
+
}
|
|
102
|
+
]
|
|
119
103
|
}
|
|
120
104
|
]
|
|
121
105
|
}
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#!/bin/
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
2
|
#
|
|
3
3
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
4
|
# or more contributor license agreements. See the NOTICE file
|
|
@@ -18,9 +18,7 @@
|
|
|
18
18
|
# under the License.
|
|
19
19
|
#
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
import yaml, sys
|
|
22
22
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
bin/pulsar-daemon stop standalone
|
|
23
|
+
deps = yaml.safe_load(open("dependencies.yaml"))
|
|
24
|
+
print(deps[sys.argv[1]])
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
|
|
21
|
+
import sys, json, urllib.request, os, shutil, zipfile, tempfile
|
|
22
|
+
from pathlib import Path
|
|
23
|
+
|
|
24
|
+
if len(sys.argv) != 3:
|
|
25
|
+
print("Usage: ")
|
|
26
|
+
print(" %s $WORKFLOW_RUN_ID $DEST_PATH" % sys.argv[0])
|
|
27
|
+
sys.exit(-1)
|
|
28
|
+
|
|
29
|
+
if 'GITHUB_TOKEN' not in os.environ:
|
|
30
|
+
print('You should have a GITHUB_TOKEN environment variable')
|
|
31
|
+
sys.exit(-1)
|
|
32
|
+
|
|
33
|
+
GITHUB_TOKEN = os.environ['GITHUB_TOKEN']
|
|
34
|
+
|
|
35
|
+
ACCEPT_HEADER = 'application/vnd.github+json'
|
|
36
|
+
LIST_URL = 'https://api.github.com/repos/apache/pulsar-client-node/actions/runs/%d/artifacts'
|
|
37
|
+
|
|
38
|
+
workflow_run_id = int(sys.argv[1])
|
|
39
|
+
dest_path = sys.argv[2]
|
|
40
|
+
|
|
41
|
+
workflow_run_url = LIST_URL % workflow_run_id
|
|
42
|
+
request = urllib.request.Request(workflow_run_url,
|
|
43
|
+
headers={'Accept': ACCEPT_HEADER, 'Authorization': 'Bearer ' + GITHUB_TOKEN})
|
|
44
|
+
with urllib.request.urlopen(request) as response:
|
|
45
|
+
data = json.loads(response.read().decode("utf-8"))
|
|
46
|
+
for artifact in data['artifacts']:
|
|
47
|
+
name = artifact['name']
|
|
48
|
+
url = artifact['archive_download_url']
|
|
49
|
+
|
|
50
|
+
print('Downloading %s from %s' % (name, url))
|
|
51
|
+
artifact_request = urllib.request.Request(url,
|
|
52
|
+
headers={'Authorization': 'Bearer ' + GITHUB_TOKEN})
|
|
53
|
+
with urllib.request.urlopen(artifact_request) as response:
|
|
54
|
+
tmp_zip = tempfile.NamedTemporaryFile(delete=False)
|
|
55
|
+
try:
|
|
56
|
+
#
|
|
57
|
+
shutil.copyfileobj(response, tmp_zip)
|
|
58
|
+
tmp_zip.close()
|
|
59
|
+
|
|
60
|
+
dest_dir = os.path.join(dest_path, name)
|
|
61
|
+
Path(dest_dir).mkdir(parents=True, exist_ok=True)
|
|
62
|
+
with zipfile.ZipFile(tmp_zip.name, 'r') as z:
|
|
63
|
+
z.extractall(dest_dir)
|
|
64
|
+
finally:
|
|
65
|
+
os.unlink(tmp_zip.name)
|
|
66
|
+
|
|
67
|
+
for root, dirs, files in os.walk(dest_path, topdown=False):
|
|
68
|
+
for name in files:
|
|
69
|
+
shutil.move(os.path.join(root, name), dest_path)
|
|
70
|
+
if not os.listdir(root):
|
|
71
|
+
os.rmdir(root)
|
|
72
|
+
|
|
73
|
+
|
|
74
|
+
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
ROOT_DIR=$(git rev-parse --show-toplevel)
|
|
21
|
+
|
|
22
|
+
VERSION=$(npm version | grep pulsar-client | awk -F ':' '{print $2}' | sed "s?'??g" | sed "s?,??g" | sed "s? ??g")
|
|
23
|
+
|
|
24
|
+
NAME=apache-pulsar-client-node-$VERSION
|
|
25
|
+
|
|
26
|
+
OUT_DIR=${1:-.}
|
|
27
|
+
|
|
28
|
+
git archive --format=tar.gz --prefix ${NAME}/ -o ${OUT_DIR}/${NAME}.tar.gz HEAD
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
|
|
21
|
+
set -e -x
|
|
22
|
+
|
|
23
|
+
ROOT_DIR=`cd $(dirname $0) && cd .. && pwd`
|
|
24
|
+
source $ROOT_DIR/pulsar-client-cpp.txt
|
|
25
|
+
|
|
26
|
+
if [ $USER != "root" ]; then
|
|
27
|
+
SUDO="sudo"
|
|
28
|
+
fi
|
|
29
|
+
|
|
30
|
+
# Get the flavor of Linux
|
|
31
|
+
export $(cat /etc/*-release | grep "^ID=")
|
|
32
|
+
|
|
33
|
+
cd /tmp
|
|
34
|
+
|
|
35
|
+
UNAME_ARCH=$(uname -m)
|
|
36
|
+
if [ $UNAME_ARCH == 'aarch64' ]; then
|
|
37
|
+
PLATFORM=arm64
|
|
38
|
+
else
|
|
39
|
+
PLATFORM=x86_64
|
|
40
|
+
fi
|
|
41
|
+
|
|
42
|
+
if [ $ID == 'ubuntu' -o $ID == 'debian' ]; then
|
|
43
|
+
curl -L -O ${CPP_CLIENT_BASE_URL}/deb-${PLATFORM}/apache-pulsar-client.deb
|
|
44
|
+
curl -L -O ${CPP_CLIENT_BASE_URL}/deb-${PLATFORM}/apache-pulsar-client-dev.deb
|
|
45
|
+
$SUDO apt install -y /tmp/*.deb
|
|
46
|
+
|
|
47
|
+
elif [ $ID == 'alpine' ]; then
|
|
48
|
+
curl -L -O ${CPP_CLIENT_BASE_URL}/apk-${PLATFORM}/${UNAME_ARCH}/apache-pulsar-client-${CPP_CLIENT_VERSION}-r0.apk
|
|
49
|
+
curl -L -O ${CPP_CLIENT_BASE_URL}/apk-${PLATFORM}/${UNAME_ARCH}/apache-pulsar-client-dev-${CPP_CLIENT_VERSION}-r0.apk
|
|
50
|
+
$SUDO apk add --allow-untrusted /tmp/*.apk
|
|
51
|
+
|
|
52
|
+
elif [ $ID == '"centos"' ]; then
|
|
53
|
+
curl -L -O ${CPP_CLIENT_BASE_URL}/rpm-${PLATFORM}/${UNAME_ARCH}/apache-pulsar-client-${CPP_CLIENT_VERSION}-1.${UNAME_ARCH}.rpm
|
|
54
|
+
curl -L -O ${CPP_CLIENT_BASE_URL}/rpm-${PLATFORM}/${UNAME_ARCH}/apache-pulsar-client-devel-${CPP_CLIENT_VERSION}-1.${UNAME_ARCH}.rpm
|
|
55
|
+
$SUDO rpm -i /tmp/*.rpm
|
|
56
|
+
|
|
57
|
+
else
|
|
58
|
+
echo "Unknown Linux distribution: '$ID'"
|
|
59
|
+
exit 1
|
|
60
|
+
fi
|
|
61
|
+
|
|
62
|
+
mkdir -p $ROOT_DIR/pkg/lib/
|
|
63
|
+
cp /usr/lib/libpulsarwithdeps.a $ROOT_DIR/pkg/lib/
|
|
64
|
+
|
|
65
|
+
|
|
66
|
+
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#!/bin/bash
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
2
|
#
|
|
3
3
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
4
|
# or more contributor license agreements. See the NOTICE file
|
|
@@ -18,28 +18,18 @@
|
|
|
18
18
|
# under the License.
|
|
19
19
|
#
|
|
20
20
|
|
|
21
|
-
set -e
|
|
21
|
+
set -e -x
|
|
22
22
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
rm -rf $PULSAR_DIR
|
|
31
|
-
curl -L --create-dir "https://archive.apache.org/dist/pulsar/pulsar-${VERSION}/${PKG}" -o $PULSAR_DIR/$PKG
|
|
32
|
-
tar xfz $PULSAR_DIR/$PKG -C $PULSAR_DIR --strip-components 1
|
|
33
|
-
|
|
34
|
-
DATA_DIR=/tmp/pulsar-test-data
|
|
35
|
-
rm -rf $DATA_DIR
|
|
36
|
-
mkdir -p $DATA_DIR
|
|
37
|
-
|
|
38
|
-
export PULSAR_STANDALONE_CONF=$ROOT_DIR/tests/conf/standalone.conf
|
|
39
|
-
$PULSAR_DIR/bin/pulsar-daemon start standalone \
|
|
23
|
+
export PULSAR_STANDALONE_CONF=test-conf/standalone.conf
|
|
24
|
+
# There is an issue when starting the pulsar standalone with other metadata store: https://github.com/apache/pulsar/issues/17984
|
|
25
|
+
# We need to use Zookeeper here. Otherwise the `Message Chunking` test will not pass.
|
|
26
|
+
# We can remove this line after the pulsar release a new version contains this fix: https://github.com/apache/pulsar/pull/18126
|
|
27
|
+
export PULSAR_STANDALONE_USE_ZOOKEEPER=1
|
|
28
|
+
bin/pulsar-daemon start standalone \
|
|
40
29
|
--no-functions-worker --no-stream-storage \
|
|
41
|
-
--
|
|
42
|
-
--bookkeeper-dir $DATA_DIR/bookkeeper
|
|
30
|
+
--bookkeeper-dir data/bookkeeper
|
|
43
31
|
|
|
44
32
|
echo "-- Wait for Pulsar service to be ready"
|
|
45
33
|
until curl http://localhost:8080/metrics > /dev/null 2>&1 ; do sleep 1; done
|
|
34
|
+
|
|
35
|
+
echo "-- Ready to start tests"
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
|
|
21
|
+
set -e
|
|
22
|
+
|
|
23
|
+
SRC_DIR=$(git rev-parse --show-toplevel)
|
|
24
|
+
cd $SRC_DIR
|
|
25
|
+
|
|
26
|
+
build-support/pulsar-test-service-stop.sh
|
|
27
|
+
|
|
28
|
+
CONTAINER_ID=$(docker run -i -p 8080:8080 -p 6650:6650 -p 8443:8443 -p 6651:6651 --rm --detach apachepulsar/pulsar:2.10.2 sleep 3600)
|
|
29
|
+
|
|
30
|
+
echo $CONTAINER_ID >.tests-container-id.txt
|
|
31
|
+
|
|
32
|
+
docker cp $SRC_DIR/tests/conf $CONTAINER_ID:/pulsar/test-conf
|
|
33
|
+
docker cp $SRC_DIR/build-support/pulsar-test-container-start.sh $CONTAINER_ID:pulsar-test-container-start.sh
|
|
34
|
+
|
|
35
|
+
docker exec -i $CONTAINER_ID /pulsar-test-container-start.sh
|
|
36
|
+
|
|
37
|
+
echo "-- Ready to start tests"
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
|
4
|
+
# or more contributor license agreements. See the NOTICE file
|
|
5
|
+
# distributed with this work for additional information
|
|
6
|
+
# regarding copyright ownership. The ASF licenses this file
|
|
7
|
+
# to you under the Apache License, Version 2.0 (the
|
|
8
|
+
# "License"); you may not use this file except in compliance
|
|
9
|
+
# with the License. You may obtain a copy of the License at
|
|
10
|
+
#
|
|
11
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
#
|
|
13
|
+
# Unless required by applicable law or agreed to in writing,
|
|
14
|
+
# software distributed under the License is distributed on an
|
|
15
|
+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
16
|
+
# KIND, either express or implied. See the License for the
|
|
17
|
+
# specific language governing permissions and limitations
|
|
18
|
+
# under the License.
|
|
19
|
+
#
|
|
20
|
+
|
|
21
|
+
set -e
|
|
22
|
+
|
|
23
|
+
SRC_DIR=$(git rev-parse --show-toplevel)
|
|
24
|
+
cd $SRC_DIR
|
|
25
|
+
|
|
26
|
+
CONTAINER_ID_PATH=".tests-container-id.txt"
|
|
27
|
+
|
|
28
|
+
if [ -f ${CONTAINER_ID_PATH} ]; then
|
|
29
|
+
CONTAINER_ID=$(cat $CONTAINER_ID_PATH)
|
|
30
|
+
docker kill $CONTAINER_ID || true
|
|
31
|
+
rm .tests-container-id.txt
|
|
32
|
+
fi
|