fluent-plugin-s3 1.5.1 → 1.7.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/ISSUE_TEMPLATE/bug_report.yaml +72 -0
- data/.github/ISSUE_TEMPLATE/config.yml +5 -0
- data/.github/ISSUE_TEMPLATE/feature_request.yaml +38 -0
- data/.github/workflows/linux.yml +5 -3
- data/.github/workflows/stale-actions.yml +22 -0
- data/ChangeLog +15 -0
- data/README.md +13 -781
- data/VERSION +1 -1
- data/docs/credentials.md +171 -0
- data/docs/howto.md +92 -0
- data/docs/input.md +98 -0
- data/docs/output.md +453 -0
- data/docs/v0.12.md +52 -0
- data/fluent-plugin-s3.gemspec +3 -0
- data/lib/fluent/plugin/in_s3.rb +26 -1
- data/lib/fluent/plugin/out_s3.rb +12 -3
- data/lib/fluent/plugin/s3_compressor_parquet.rb +83 -0
- data/test/test_in_s3.rb +108 -5
- data/test/test_out_s3.rb +167 -118
- metadata +28 -7
- data/.travis.yml +0 -24
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: db01fa7706627c40baf9e272644c4449c47e4adac7c67ae336099c1c65021e04
|
|
4
|
+
data.tar.gz: d74f9c06bcce4606b62d06ba416f934824213e42b508cf49dc093392c34692fe
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 920c6ddc28b300bf9b00c2a21005d4adfeefc2e6c59ddb9afffedb2fcd45145baf70e2f7ec01997c12380d7b31def2289ece915a3099d0045b086314c914a555
|
|
7
|
+
data.tar.gz: eb84c4fc47ad9840fb375d65f532f7f5df701ce733b8959c6e037ffd45010548555ae5695b9e0839d4af60967ec791b5cb4cd049e103653b7b441a4f2bf54209
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
name: Bug Report
|
|
2
|
+
description: Create a report with a procedure for reproducing the bug
|
|
3
|
+
body:
|
|
4
|
+
- type: markdown
|
|
5
|
+
attributes:
|
|
6
|
+
value: |
|
|
7
|
+
Check [README](https://github.com/fluent/fluent-plugin-s3#readme) first and here is the list to help us investigate the problem.
|
|
8
|
+
- type: textarea
|
|
9
|
+
id: description
|
|
10
|
+
attributes:
|
|
11
|
+
label: Describe the bug
|
|
12
|
+
description: A clear and concise description of what the bug is
|
|
13
|
+
validations:
|
|
14
|
+
required: true
|
|
15
|
+
- type: textarea
|
|
16
|
+
id: reproduce
|
|
17
|
+
attributes:
|
|
18
|
+
label: To Reproduce
|
|
19
|
+
description: Steps to reproduce the behavior
|
|
20
|
+
validations:
|
|
21
|
+
required: true
|
|
22
|
+
- type: textarea
|
|
23
|
+
id: expected
|
|
24
|
+
attributes:
|
|
25
|
+
label: Expected behavior
|
|
26
|
+
description: A clear and concise description of what you expected to happen
|
|
27
|
+
validations:
|
|
28
|
+
required: true
|
|
29
|
+
- type: textarea
|
|
30
|
+
id: environment
|
|
31
|
+
attributes:
|
|
32
|
+
label: Your Environment
|
|
33
|
+
description: |
|
|
34
|
+
- Fluentd or td-agent version: `fluentd --version` or `td-agent --version`
|
|
35
|
+
- Operating system: `cat /etc/os-release`
|
|
36
|
+
- Kernel version: `uname -r`
|
|
37
|
+
|
|
38
|
+
Tip: If you hit the problem with older fluentd version, try latest version first.
|
|
39
|
+
value: |
|
|
40
|
+
- Fluentd version:
|
|
41
|
+
- TD Agent version:
|
|
42
|
+
- fluent-plugin-s3 version:
|
|
43
|
+
- aws-sdk-s3 version:
|
|
44
|
+
- aws-sdk-sqs version:
|
|
45
|
+
- Operating system:
|
|
46
|
+
- Kernel version:
|
|
47
|
+
render: markdown
|
|
48
|
+
validations:
|
|
49
|
+
required: true
|
|
50
|
+
- type: textarea
|
|
51
|
+
id: configuration
|
|
52
|
+
attributes:
|
|
53
|
+
label: Your Configuration
|
|
54
|
+
description: |
|
|
55
|
+
Write your configuration here. Minimum reproducible fluentd.conf is recommended.
|
|
56
|
+
validations:
|
|
57
|
+
required: true
|
|
58
|
+
- type: textarea
|
|
59
|
+
id: logs
|
|
60
|
+
attributes:
|
|
61
|
+
label: Your Error Log
|
|
62
|
+
description: Write your ALL error log here
|
|
63
|
+
render: shell
|
|
64
|
+
validations:
|
|
65
|
+
required: true
|
|
66
|
+
- type: textarea
|
|
67
|
+
id: addtional-context
|
|
68
|
+
attributes:
|
|
69
|
+
label: Additional context
|
|
70
|
+
description: Add any other context about the problem here.
|
|
71
|
+
validations:
|
|
72
|
+
required: false
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
name: Feature request
|
|
2
|
+
description: Suggest an idea for this project
|
|
3
|
+
body:
|
|
4
|
+
- type: markdown
|
|
5
|
+
attributes:
|
|
6
|
+
value: |
|
|
7
|
+
Check [README.md](https://github.com/fluent/fluent-plugin-s3/blob/master/README.md) first and here is the list to help us investigate the problem.
|
|
8
|
+
- type: textarea
|
|
9
|
+
id: description
|
|
10
|
+
attributes:
|
|
11
|
+
label: Is your feature request related to a problem? Please describe.
|
|
12
|
+
description: |
|
|
13
|
+
A clear and concise description of what the problem is.
|
|
14
|
+
Ex. I'm always frustrated when [...]
|
|
15
|
+
validations:
|
|
16
|
+
required: true
|
|
17
|
+
- type: textarea
|
|
18
|
+
id: solution
|
|
19
|
+
attributes:
|
|
20
|
+
label: Describe the solution you'd like
|
|
21
|
+
description: A clear and concise description of what you want to happen.
|
|
22
|
+
validations:
|
|
23
|
+
required: true
|
|
24
|
+
- type: textarea
|
|
25
|
+
id: alternative
|
|
26
|
+
attributes:
|
|
27
|
+
label: Describe alternatives you've considered
|
|
28
|
+
description: A clear and concise description of any alternative solutions or features you've considered.
|
|
29
|
+
validations:
|
|
30
|
+
required: true
|
|
31
|
+
- type: textarea
|
|
32
|
+
id: addtional-context
|
|
33
|
+
attributes:
|
|
34
|
+
label: Additional context
|
|
35
|
+
description: Add any other context or screenshots about the feature request here.
|
|
36
|
+
validations:
|
|
37
|
+
required: false
|
|
38
|
+
|
data/.github/workflows/linux.yml
CHANGED
|
@@ -1,14 +1,16 @@
|
|
|
1
1
|
name: linux
|
|
2
2
|
on:
|
|
3
|
-
|
|
4
|
-
|
|
3
|
+
push:
|
|
4
|
+
branches: [master]
|
|
5
|
+
pull_request:
|
|
6
|
+
branches: [master]
|
|
5
7
|
jobs:
|
|
6
8
|
build:
|
|
7
9
|
runs-on: ${{ matrix.os }}
|
|
8
10
|
strategy:
|
|
9
11
|
fail-fast: false
|
|
10
12
|
matrix:
|
|
11
|
-
ruby: [ '
|
|
13
|
+
ruby: [ '3.1', '3.0', '2.7' ]
|
|
12
14
|
os:
|
|
13
15
|
- ubuntu-latest
|
|
14
16
|
name: Ruby ${{ matrix.ruby }} unit testing on ${{ matrix.os }}
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
name: "Mark or close stale issues and PRs"
|
|
2
|
+
on:
|
|
3
|
+
schedule:
|
|
4
|
+
- cron: "00 10 * * *"
|
|
5
|
+
|
|
6
|
+
jobs:
|
|
7
|
+
stale:
|
|
8
|
+
runs-on: ubuntu-latest
|
|
9
|
+
steps:
|
|
10
|
+
- uses: actions/stale@v3
|
|
11
|
+
with:
|
|
12
|
+
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
|
13
|
+
days-before-stale: 90
|
|
14
|
+
days-before-close: 30
|
|
15
|
+
stale-issue-message: "This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days"
|
|
16
|
+
stale-pr-message: "This PR has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this PR will be closed in 30 days"
|
|
17
|
+
close-issue-message: "This issue was automatically closed because of stale in 30 days"
|
|
18
|
+
close-pr-message: "This PR was automatically closed because of stale in 30 days"
|
|
19
|
+
stale-pr-label: "stale"
|
|
20
|
+
stale-issue-label: "stale"
|
|
21
|
+
exempt-issue-labels: "bug,help wanted,enhancement"
|
|
22
|
+
exempt-pr-labels: "bug,help wanted,enhancement"
|
data/ChangeLog
CHANGED
|
@@ -1,3 +1,18 @@
|
|
|
1
|
+
Release 1.7.0 - 2022/06/14
|
|
2
|
+
|
|
3
|
+
* in_s3: Allow multi workers
|
|
4
|
+
* in_s3: Support alternative AWS key ID and secret for SQS
|
|
5
|
+
* out_s3: Add warning for multi workers
|
|
6
|
+
* out_s3: Support object tagging
|
|
7
|
+
|
|
8
|
+
Release 1.6.1 - 2021/08/19
|
|
9
|
+
|
|
10
|
+
* in_s3/out_s3: Don't raise error when s3_endpoint is used for VPC endpoint (GitHub: #384)
|
|
11
|
+
|
|
12
|
+
Release 1.6.0 - 2021/04/08
|
|
13
|
+
|
|
14
|
+
* out_s3: Add support for Parquet compressor. Use `<compress>` section to configure columnify command behavior.
|
|
15
|
+
|
|
1
16
|
Release 1.5.1 - 2021/02/16
|
|
2
17
|
|
|
3
18
|
* out_s3: Fix assume_role_credentials regression
|