sidekiq 5.2.9 → 6.4.1

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (106) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +318 -1
  3. data/LICENSE +3 -3
  4. data/README.md +23 -34
  5. data/bin/sidekiq +27 -3
  6. data/bin/sidekiqload +67 -61
  7. data/bin/sidekiqmon +8 -0
  8. data/lib/generators/sidekiq/job_generator.rb +57 -0
  9. data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb} +2 -2
  10. data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb} +1 -1
  11. data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb} +1 -1
  12. data/lib/sidekiq/api.rb +335 -267
  13. data/lib/sidekiq/cli.rb +164 -182
  14. data/lib/sidekiq/client.rb +58 -61
  15. data/lib/sidekiq/delay.rb +7 -6
  16. data/lib/sidekiq/exception_handler.rb +10 -12
  17. data/lib/sidekiq/extensions/action_mailer.rb +13 -22
  18. data/lib/sidekiq/extensions/active_record.rb +13 -10
  19. data/lib/sidekiq/extensions/class_methods.rb +14 -11
  20. data/lib/sidekiq/extensions/generic_proxy.rb +6 -4
  21. data/lib/sidekiq/fetch.rb +40 -32
  22. data/lib/sidekiq/job.rb +13 -0
  23. data/lib/sidekiq/job_logger.rb +33 -7
  24. data/lib/sidekiq/job_retry.rb +70 -71
  25. data/lib/sidekiq/job_util.rb +65 -0
  26. data/lib/sidekiq/launcher.rb +161 -71
  27. data/lib/sidekiq/logger.rb +170 -0
  28. data/lib/sidekiq/manager.rb +17 -21
  29. data/lib/sidekiq/middleware/chain.rb +20 -8
  30. data/lib/sidekiq/middleware/current_attributes.rb +57 -0
  31. data/lib/sidekiq/middleware/i18n.rb +5 -7
  32. data/lib/sidekiq/monitor.rb +133 -0
  33. data/lib/sidekiq/paginator.rb +20 -16
  34. data/lib/sidekiq/processor.rb +71 -70
  35. data/lib/sidekiq/rails.rb +40 -37
  36. data/lib/sidekiq/redis_connection.rb +48 -48
  37. data/lib/sidekiq/scheduled.rb +62 -28
  38. data/lib/sidekiq/sd_notify.rb +149 -0
  39. data/lib/sidekiq/systemd.rb +24 -0
  40. data/lib/sidekiq/testing/inline.rb +2 -1
  41. data/lib/sidekiq/testing.rb +36 -27
  42. data/lib/sidekiq/util.rb +57 -15
  43. data/lib/sidekiq/version.rb +2 -1
  44. data/lib/sidekiq/web/action.rb +15 -11
  45. data/lib/sidekiq/web/application.rb +88 -75
  46. data/lib/sidekiq/web/csrf_protection.rb +180 -0
  47. data/lib/sidekiq/web/helpers.rb +109 -92
  48. data/lib/sidekiq/web/router.rb +23 -19
  49. data/lib/sidekiq/web.rb +61 -105
  50. data/lib/sidekiq/worker.rb +247 -105
  51. data/lib/sidekiq.rb +77 -44
  52. data/sidekiq.gemspec +23 -16
  53. data/web/assets/images/apple-touch-icon.png +0 -0
  54. data/web/assets/javascripts/application.js +83 -64
  55. data/web/assets/javascripts/dashboard.js +54 -73
  56. data/web/assets/stylesheets/application-dark.css +143 -0
  57. data/web/assets/stylesheets/application-rtl.css +0 -4
  58. data/web/assets/stylesheets/application.css +45 -232
  59. data/web/locales/ar.yml +8 -2
  60. data/web/locales/de.yml +14 -2
  61. data/web/locales/en.yml +6 -1
  62. data/web/locales/es.yml +18 -2
  63. data/web/locales/fr.yml +10 -3
  64. data/web/locales/ja.yml +7 -1
  65. data/web/locales/lt.yml +83 -0
  66. data/web/locales/pl.yml +4 -4
  67. data/web/locales/ru.yml +4 -0
  68. data/web/locales/vi.yml +83 -0
  69. data/web/views/_footer.erb +1 -1
  70. data/web/views/_job_info.erb +3 -2
  71. data/web/views/_poll_link.erb +2 -5
  72. data/web/views/_summary.erb +7 -7
  73. data/web/views/busy.erb +54 -20
  74. data/web/views/dashboard.erb +22 -14
  75. data/web/views/dead.erb +3 -3
  76. data/web/views/layout.erb +3 -1
  77. data/web/views/morgue.erb +9 -6
  78. data/web/views/queue.erb +19 -10
  79. data/web/views/queues.erb +10 -2
  80. data/web/views/retries.erb +11 -8
  81. data/web/views/retry.erb +3 -3
  82. data/web/views/scheduled.erb +5 -2
  83. metadata +34 -64
  84. data/.circleci/config.yml +0 -61
  85. data/.github/contributing.md +0 -32
  86. data/.github/issue_template.md +0 -11
  87. data/.gitignore +0 -15
  88. data/.travis.yml +0 -11
  89. data/3.0-Upgrade.md +0 -70
  90. data/4.0-Upgrade.md +0 -53
  91. data/5.0-Upgrade.md +0 -56
  92. data/COMM-LICENSE +0 -97
  93. data/Ent-Changes.md +0 -238
  94. data/Gemfile +0 -23
  95. data/Pro-2.0-Upgrade.md +0 -138
  96. data/Pro-3.0-Upgrade.md +0 -44
  97. data/Pro-4.0-Upgrade.md +0 -35
  98. data/Pro-Changes.md +0 -759
  99. data/Rakefile +0 -9
  100. data/bin/sidekiqctl +0 -20
  101. data/code_of_conduct.md +0 -50
  102. data/lib/generators/sidekiq/worker_generator.rb +0 -49
  103. data/lib/sidekiq/core_ext.rb +0 -1
  104. data/lib/sidekiq/ctl.rb +0 -221
  105. data/lib/sidekiq/logging.rb +0 -122
  106. data/lib/sidekiq/middleware/server/active_record.rb +0 -23
data/4.0-Upgrade.md DELETED
@@ -1,53 +0,0 @@
1
- # Welcome to Sidekiq 4.0!
2
-
3
- Sidekiq 4.0 contains a redesigned, more efficient core with less overhead per job.
4
- See my blog for [an overview of Sidekiq 4's higher performance](http://www.mikeperham.com/2015/10/14/optimizing-sidekiq/).
5
-
6
- ## What's New
7
-
8
- * Sidekiq no longer uses Celluloid. If your application code uses Celluloid,
9
- you will need to pull it in yourself.
10
-
11
- * `redis-namespace` has been removed from Sidekiq's gem dependencies. If
12
- you want to use namespacing ([and I strongly urge you not to](http://www.mikeperham.com/2015/09/24/storing-data-with-redis/)), you'll need to add the gem to your Gemfile:
13
- ```ruby
14
- gem 'redis-namespace'
15
- ```
16
-
17
- * **Redis 2.8.0 or greater is required.** Redis 2.8 was released two years
18
- ago and contains **many** useful features which Sidekiq couldn't
19
- leverage until now. **Redis 3.0.3 or greater is recommended** for large
20
- scale use [#2431](https://github.com/mperham/sidekiq/issues/2431).
21
-
22
- * Jobs are now fetched from Redis in parallel, making Sidekiq more
23
- resilient to high network latency. This means that Sidekiq requires
24
- more Redis connections per process. You must have a minimum of
25
- `concurrency + 2` connections in your pool or Sidekiq will exit.
26
- When in doubt, let Sidekiq size the connection pool for you.
27
-
28
- * Worker data is no longer updated in real-time but rather upon every
29
- heartbeat. Don't expect the `Sidekiq::Workers` API to be millisecond-precise.
30
-
31
- * There's a new testing API based off the `Sidekiq::Queues` namespace. All
32
- assertions made against the Worker class still work as expected.
33
- ```ruby
34
- assert_equal 0, Sidekiq::Queues["default"].size
35
- HardWorker.perform_async("log")
36
- assert_equal 1, Sidekiq::Queues["default"].size
37
- assert_equal "log", Sidekiq::Queues["default"].first['args'][0]
38
- Sidekiq::Queues.clear_all
39
- ```
40
-
41
- ## Upgrade
42
-
43
- First, make sure you are using Redis 2.8 or greater. Next:
44
-
45
- * Upgrade to the latest Sidekiq 3.x.
46
- ```ruby
47
- gem 'sidekiq', '< 4'
48
- ```
49
- * Fix any deprecation warnings you see.
50
- * Upgrade to 4.x.
51
- ```ruby
52
- gem 'sidekiq', '< 5'
53
- ```
data/5.0-Upgrade.md DELETED
@@ -1,56 +0,0 @@
1
- # Welcome to Sidekiq 5.0!
2
-
3
- Sidekiq 5.0 contains a reworked job dispatch and execution core to integrate
4
- better with the new Rails 5.0 Executor. It also drops support for older
5
- versions of Ruby and Rails and adds support for RTL languages in the Web UI.
6
-
7
- ## What's New
8
-
9
- * Integrate job logging and retry logic directly in with the job
10
- execution logic in Sidekiq::Processor. Previously this logic was
11
- defined as middleware. In Rails 5.0, ActiveSupport::Executor handles ActiveRecord
12
- connection management, job callbacks, development mode class loading,
13
- etc. Because of its extensive responsibilities, the Executor can't be
14
- integrated as Sidekiq middleware; the logging/retry logic had to be pulled out
15
- too. Sidekiq 4.2 had a hack to make it work but this redesign provides
16
- a cleaner integration. [#3235]
17
- * The Delayed Extensions `delay`, `delay_in` and `delay_until` APIs are
18
- no longer available by default. The extensions allow you to marshal
19
- job arguments as YAML, leading to cases where job payloads could be many
20
- 100s of KB or larger if not careful, leading to Redis networking
21
- timeouts or other problems. As noted in the Best Practices wiki page,
22
- Sidekiq is designed for jobs with small, simple arguments.
23
-
24
- Add this line to your initializer to re-enable them and get the old behavior:
25
- ```ruby
26
- Sidekiq::Extensions.enable_delay!
27
- ```
28
- The old `Sidekiq.remove_delay!` API has been removed as it is now the default. [#3299]
29
- * Sidekiq's quiet signal is now `TSTP` (think of it as **T**hread
30
- **ST**o**P**) instead of USR1 as USR1 is not available on JRuby.
31
- USR1 will continue to be supported in Sidekiq 5.x for backwards
32
- compatibility and will be removed in Sidekiq 6.x. [#3302]
33
- * The Web UI is now bi-directional - it can render either LTR
34
- (left-to-right) or RTL languages. With this change, **Farsi, Arabic,
35
- Hebrew and Urdu** are officially supported. [#3381]
36
- * Jobs which can't be parsed due to invalid JSON are now pushed
37
- immediately to the Dead set since they require manual intervention and
38
- will never execute successfully as is. The Web UI has been updated to
39
- more gracefully display these jobs. [#3296]
40
- * **Rails 3.2** is no longer supported.
41
- * **Ruby 2.0 and Ruby 2.1** are no longer supported. Ruby 2.2.2+ is required.
42
-
43
- ## Upgrade
44
-
45
- As always, please upgrade Sidekiq **one major version at a time**.
46
- If you are already running Sidekiq 4.x, then:
47
-
48
- * Upgrade to the latest Sidekiq 4.x.
49
- ```ruby
50
- gem 'sidekiq', '< 5'
51
- ```
52
- * Fix any deprecation warnings you see.
53
- * Upgrade to 5.x.
54
- ```ruby
55
- gem 'sidekiq', '< 6'
56
- ```
data/COMM-LICENSE DELETED
@@ -1,97 +0,0 @@
1
- END-USER LICENSE AGREEMENT
2
-
3
- ------------------------------------------------------------------------------
4
-
5
- IMPORTANT: THIS SOFTWARE END-USER LICENSE AGREEMENT ("EULA") IS A LEGAL AGREEMENT (“Agreement”) BETWEEN YOU (THE CUSTOMER, EITHER AS AN INDIVIDUAL OR, IF PURCHASED OR OTHERWISE ACQUIRED BY OR FOR AN ENTITY, AS AN ENTITY) AND CONTRIBUTED SYSTEMS. READ IT CAREFULLY BEFORE COMPLETING THE INSTALLATION PROCESS AND USING SIDEKIQ PRO AND RELATED SOFTWARE COMPONENTS (“SOFTWARE”). IT PROVIDES A LICENSE TO USE THE SOFTWARE AND CONTAINS WARRANTY INFORMATION AND LIABILITY DISCLAIMERS. BY INSTALLING AND USING THE SOFTWARE, YOU ARE CONFIRMING YOUR ACCEPTANCE OF THE SOFTWARE AND AGREEING TO BECOME BOUND BY THE TERMS OF THIS AGREEMENT.
6
-
7
- ------------------------------------------------------------------------------
8
-
9
- In order to use the Software under this Agreement, you must receive a “Source URL” at the time of purchase, in accordance with the scope of use and other terms specified for each type of Software and as set forth in this Section 1 of this Agreement.
10
-
11
- 1. License Grant
12
-
13
- 1.1 General Use. This Agreement grants you a non-exclusive, non-transferable, limited license to the use rights for the Software, without the right to grant sublicenses, subject to the terms and conditions in this Agreement. The Software is licensed, not sold.
14
-
15
- 1.2 Unlimited Organization License. If you purchased an Organization License (included with the Sidekiq Pro Software), you may install the Software on an unlimited number of Hosts. “Host” means any physical or virtual machine which is controlled by you. You may also run an unlimited number of Workers. “Worker” means a thread within a Sidekiq server process which executes jobs. You may concurrently run the software on an unlimited number of Hosts, with each host running an unlimited number of Workers.
16
-
17
- 1.3 Limited Enterprise License. If you purchased a Limited License for the Sidekiq Enterprise Software, you may install the Software on an unlimited number of Hosts. “Host” means any physical or virtual machine which is controlled by you. The aggregate number of Workers run by the hosts must not exceed the maximum number of Workers authorized at the time of purchase. “Worker” means a thread within a Sidekiq server process which executes jobs. In order to run additional Workers, you must purchase an additional allowance from Contributed Systems.
18
-
19
- 1.4 Enterprise Site License. If you purchased a Site License for the Sidekiq Enterprise Software, you may install the Software on an unlimited number of Hosts. “Host” means any physical or virtual machine which is controlled by you. You may also run an unlimited number of Workers. “Worker” means a thread within a Sidekiq server process which executes jobs. You may concurrently run the software on an unlimited number of Hosts, with each host running an unlimited number of Workers.
20
-
21
- 1.5 Appliance License. If you purchased an Appliance License, you may distribute the Software in any applications, frameworks, or elements (collectively referred to as an “Application” or “Applications”) that you develop using the Software in accordance with this EULA, provided that such distribution does not violate the restrictions set forth in section 3 of this EULA. You must not remove, obscure or interfere with any copyright, acknowledgment, attribution, trademark, warning or disclaimer statement affixed to, incorporated in or otherwise applied in connection with the Software. You are required to ensure that the Software is not reused by or with any applications other than those with which you distribute it as permitted herein. For example, if You install the Software on a customer's server, that customer is not permitted to use the Software independently of your Application. You must inform Contributed Systems of your knowledge of any infringing use of the Software by any of your customers. You are liable for compliance by those third parties with the terms and conditions of this EULA. You will not owe Contributed Systems any royalties for your distribution of the Software in accordance with this EULA.
22
-
23
- 1.6 Archive Copies. You are entitled to make a reasonable amount of copies of the Software for archival purposes. Each copy must reproduce all copyright and other proprietary rights notices on or in the Software Product.
24
-
25
- 1.7 Electronic Delivery. All Software and license documentation shall be delivered by electronic means unless otherwise specified on the applicable invoice or at the time of purchase. Software shall be deemed delivered when it is made available for download by you (“Delivery”).
26
-
27
- 2. Modifications. Contributed Systems shall provide you with source code so that you can create Modifications of the original software. “Modification” means: (a) any addition to or deletion from the contents of a file included in the original Software or previous Modifications created by You, or (b) any new file that contains any part of the original Software or previous Modifications. While you retain all rights to any original work authored by you as part of the Modifications, We continue to own all copyright and other intellectual property rights in the Software.
28
-
29
- 3. Restricted Uses.
30
-
31
- 3.1 You shall not (and shall not allow any third party to): (a) decompile, disassemble, or otherwise reverse engineer the Software or attempt to reconstruct or discover any source code, underlying ideas, algorithms, file formats or programming interfaces of the Software by any means whatsoever (except and only to the extent that applicable law prohibits or restricts reverse engineering restrictions); (b) distribute, sell, sublicense, rent, lease or use the Software for time sharing, hosting, service provider or like purposes, except as expressly permitted under this Agreement; (c) redistribute the Software or Modifications other than by including the Software or a portion thereof within your own product, which must have substantially different functionality than the Software or Modifications and must not allow any third party to use the Software or Modifications, or any portions thereof, for software development or application development purposes; (d) redistribute the Software as part of a product, "appliance" or "virtual server"; (e) redistribute the Software on any server which is not directly under your control; (f) remove any product identification, proprietary, copyright or other notices contained in the Software; (g) modify any part of the Software, create a derivative work of any part of the Software (except as permitted in Section 4), or incorporate the Software, except to the extent expressly authorized in writing by Contributed Systems; (h) publicly disseminate performance information or analysis (including, without limitation, benchmarks) from any source relating to the Software; (i) utilize any equipment, device, software, or other means designed to circumvent or remove any form of Source URL or copy protection used by Contributed Systems in connection with the Software, or use the Software together with any authorization code, Source URL, serial number, or other copy protection device not supplied by Contributed Systems; (j) use the Software to develop a product which is competitive with any Contributed Systems product offerings; or (k) use unauthorized Source URLS or keycode(s) or distribute or publish Source URLs or keycode(s), except as may be expressly permitted by Contributed Systems in writing. If your unique Source URL is ever published, Contributed Systems reserves the right to terminate your access without notice.
32
-
33
- 3.2 UNDER NO CIRCUMSTANCES MAY YOU USE THE SOFTWARE AS PART OF A PRODUCT OR SERVICE THAT PROVIDES SIMILAR FUNCTIONALITY TO THE SOFTWARE ITSELF.
34
-
35
- The Open Source version of the Software (“LGPL Version”) is licensed
36
- under the terms of the GNU Lesser General Public License version 3.0
37
- (“LGPL”) and not under this EULA.
38
-
39
- 4. Ownership. Notwithstanding anything to the contrary contained herein, except for the limited license rights expressly provided herein, Contributed Systems and its suppliers have and will retain all rights, title and interest (including, without limitation, all patent, copyright, trademark, trade secret and other intellectual property rights) in and to the Software and all copies, modifications and derivative works thereof (including any changes which incorporate any of your ideas, feedback or suggestions). You acknowledge that you are obtaining only a limited license right to the Software, and that irrespective of any use of the words “purchase”, “sale” or like terms hereunder no ownership rights are being conveyed to you under this Agreement or otherwise.
40
-
41
- 5. Fees and Payment. The Software license fees will be due and payable in full as set forth in the applicable invoice or at the time of purchase. If the Software does not function properly within two weeks of purchase, please contact us within those two weeks for a refund. You shall be responsible for all taxes, withholdings, duties and levies arising from the order (excluding taxes based on the net income of Contributed Systems).
42
-
43
- 6. Support, Maintenance and Services. Subject to the terms and conditions of this Agreement, as set forth in your invoice, and as set forth on the Sidekiq Pro support page (https://github.com/mperham/sidekiq/wiki/Commercial-Support), support and maintenance services may be included with the purchase of your license subscription.
44
-
45
- 7. Term of Agreement.
46
-
47
- 7.1 Term. This Agreement is effective as of the Delivery of the Software and expires at such time as all license and service subscriptions hereunder have expired in accordance with their own terms (the “Term”). For clarification, the term of your license under this Agreement may be perpetual, limited for Evaluation Version, or designated as a fixed-term license in the Invoice, and shall be specified at your time of purchase. Either party may terminate this Agreement (including all related Invoices) if the other party: (a) fails to cure any material breach of this Agreement within thirty (30) days after written notice of such breach, provided that Contributed Systems may terminate this Agreement immediately upon any breach of Section 3 or if you exceed any other restrictions contained in Section 1, unless otherwise specified in this agreement; (b) ceases operation without a successor; or (c) seeks protection under any bankruptcy, receivership, trust deed, creditors arrangement, composition or comparable proceeding, or if any such proceeding is instituted against such party (and not dismissed within sixty (60) days)). Termination is not an exclusive remedy and the exercise by either party of any remedy under this Agreement will be without prejudice to any other remedies it may have under this Agreement, by law, or otherwise.
48
-
49
- 7.2 Termination. Upon any termination of this Agreement, you shall cease any and all use of any Software and destroy all copies thereof.
50
-
51
- 7.3 Expiration of License. Upon the expiration of any term under this Agreement, (a) all Software updates and services pursuant to the license shall cease, (b) you may only continue to run existing installations of the Software, (c) you may not install the Software on any additional Hosts, and (d) any new installation of the Software shall require the purchase of a new license subscription from Contributed Systems.
52
-
53
- 8. Disclaimer of Warranties. The Software is provided "as is," with all faults, defects and errors, and without warranty of any kind. Contributed Systems does not warrant that the Software will be free of bugs, errors, viruses or other defects, and Contributed Systems shall have no liability of any kind for the use of or inability to use the Software, the Software content or any associated service, and you acknowledge that it is not technically practicable for Contributed Systems to do so.
54
- To the maximum extent permitted by applicable law, Contributed Systems disclaims all warranties, express, implied, arising by law or otherwise, regarding the Software, the Software content and their respective performance or suitability for your intended use, including without limitation any implied warranty of merchantability, fitness for a particular purpose.
55
-
56
- 9. Limitation of Liability.
57
-
58
- In no event will Contributed Systems be liable for any direct, indirect, consequential, incidental, special, exemplary, or punitive damages or liabilities whatsoever arising from or relating to the Software, the Software content or this Agreement, whether based on contract, tort (including negligence), strict liability or other theory, even if Contributed Systems has been advised of the possibility of such damages.
59
-
60
- In no event will Contributed Systems' liability exceed the Software license price as indicated in the invoice. The existence of more than one claim will not enlarge or extend this limit.
61
-
62
- 10. Remedies. Your exclusive remedy and Contributed Systems' entire liability for breach of this Agreement shall be limited, at Contributed Systems' sole and exclusive discretion, to (a) replacement of any defective software or documentation; or (b) refund of the license fee paid to Contributed Systems, payable in accordance with Contributed Systems' refund policy.
63
-
64
- 11. Acknowledgements.
65
-
66
- 11.1 Consent to the Use of Data. You agree that Contributed Systems and its affiliates may collect and use technical information gathered as part of the product support services. Contributed Systems may use this information solely to improve products and services and will not disclose this information in a form that personally identifies you.
67
-
68
- 11.2 Verification. We or a certified auditor acting on our behalf, may, upon its reasonable request and at its expense, audit you with respect to the use of the Software. Such audit may be conducted by mail, electronic means or through an in-person visit to your place of business. Any such in-person audit shall be conducted during regular business hours at your facilities and shall not unreasonably interfere with your business activities. We shall not remove, copy, or redistribute any electronic material during the course of an audit. If an audit reveals that you are using the Software in a way that is in material violation of the terms of the EULA, then you shall pay our reasonable costs of conducting the audit. In the case of a material violation, you agree to pay Us any amounts owing that are attributable to the unauthorized use. In the alternative, We reserve the right, at our sole option, to terminate the licenses for the Software.
69
-
70
- 11.3 Government End Users. If the Software and related documentation are supplied to or purchased by or on behalf of the United States Government, then the Software is deemed to be "commercial software" as that term is used in the Federal Acquisition Regulation system. Rights of the United States shall not exceed the minimum rights set forth in FAR 52.227-19 for "restricted computer software". All other terms and conditions of this Agreement apply.
71
-
72
- 12. Third Party Software. Examples included in Software may provide links to third party libraries or code (collectively “Third Party Software”) to implement various functions. Third Party Software does not comprise part of the Software. In some cases, access to Third Party Software may be included along with the Software delivery as a convenience for demonstration purposes. Such source code and libraries may be included in the “…/examples” source tree delivered with the Software and do not comprise the Software. Licensee acknowledges (1) that some part of Third Party Software may require additional licensing of copyright and patents from the owners of such, and (2) that distribution of any of the Software referencing or including any portion of a Third Party Software may require appropriate licensing from such third parties.
73
-
74
-
75
- 13. Miscellaneous
76
-
77
- 13.1 Entire Agreement. This Agreement sets forth our entire agreement with respect to the Software and the subject matter hereof and supersedes all prior and contemporaneous understandings and agreements whether written or oral.
78
-
79
- 13.2 Amendment. Contributed Systems reserves the right, in its sole discretion, to amend this Agreement from time. Amendments to this Agreement can be located at: https://github.com/mperham/sidekiq/blob/master/COMM-LICENSE.
80
-
81
- 13.3 Assignment. You may not assign this Agreement or any of its rights under this Agreement without the prior written consent of Contributed Systems and any attempted assignment without such consent shall be void.
82
-
83
- 13.4 Export Compliance. You agree to comply with all applicable laws and regulations, including laws, regulations, orders or other restrictions on export, re-export or redistribution of software.
84
-
85
- 13.5 Indemnification. You agree to defend, indemnify, and hold harmless Contributed Systems from and against any lawsuits, claims, losses, damages, fines and expenses (including attorneys' fees and costs) arising out of your use of the Software or breach of this Agreement.
86
-
87
- 13.6 Governing Law. This Agreement is governed by the laws of the State of Oregon and the United States without regard to conflicts of laws provisions thereof, and without regard to the United Nations Convention on the International Sale of Goods or the Uniform Computer Information Transactions Act, as currently enacted by any jurisdiction or as may be codified or amended from time to time by any jurisdiction. The jurisdiction and venue for actions related to the subject matter hereof shall be the state of Oregon and United States federal courts located in Portland, Oregon, and both parties hereby submit to the personal jurisdiction of such courts.
88
-
89
- 13.7 Attorneys' Fees and Costs. The prevailing party in any action to enforce this Agreement will be entitled to recover its attorneys' fees and costs in connection with such action.
90
-
91
- 13.8 Severability. If any provision of this Agreement is held by a court of competent jurisdiction to be invalid, illegal, or unenforceable, the remainder of this Agreement will remain in full force and effect.
92
-
93
- 13.9 Waiver. Failure or neglect by either party to enforce at any time any of the provisions of this licence Agreement shall not be construed or deemed to be a waiver of that party's rights under this Agreement.
94
-
95
- 13.10 Headings. The headings of sections and paragraphs of this Agreement are for convenience of reference only and are not intended to restrict, affect or be of any weight in the interpretation or construction of the provisions of such sections or paragraphs.
96
-
97
- 14. Contact Information. If you have any questions about this EULA, or if you want to contact Contributed Systems for any reason, please direct correspondence to info@contribsys.com.
data/Ent-Changes.md DELETED
@@ -1,238 +0,0 @@
1
- # Sidekiq Enterprise Changelog
2
-
3
- [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
4
-
5
- Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
6
-
7
- 1.8.1
8
- -------------
9
-
10
- - Fix excessive lock reclaims with concurrent limiter [#4105]
11
- - Add ES translations, see issues [#3949](https://github.com/mperham/sidekiq/issues/3949) and [#3951](https://github.com/mperham/sidekiq/issues/3951) to add your own language.
12
-
13
- 1.8.0
14
- -------------
15
-
16
- - Require Sidekiq Pro 4.0 and Sidekiq 5.2.
17
- - Refactor historical metrics API to use revamped Statsd support in Sidekiq Pro
18
- - Add a gauge to historical metrics for `default` queue latency [#4079]
19
-
20
- 1.7.2
21
- -------------
22
-
23
- - Add PT and JA translations
24
- - Fix elapsed time calculations to use monotonic clock [#4000, sj26]
25
- - Fix edge case where flapping leadership would cause old periodic
26
- jobs to be fired once [#3974]
27
- - Add support for sidekiqswarm memory monitoring on FreeBSD [#3884]
28
-
29
- 1.7.1
30
- -------------
31
-
32
- - Fix Lua error in concurrent rate limiter under heavy contention
33
- - Remove superfluous `freeze` calls on Strings [#3759]
34
-
35
- 1.7.0
36
- -------------
37
-
38
- - **NEW FEATURE** [Rolling restarts](https://github.com/mperham/sidekiq/wiki/Ent-Rolling-Restarts) - great for long running jobs!
39
- - Adjust middleware so unique jobs that don't push aren't registered in a Batch [#3662]
40
- - Add new unlimited rate limiter, useful for testing [#3743]
41
- ```ruby
42
- limiter = Sidekiq::Limiter.unlimited(...any args...)
43
- ```
44
-
45
- 1.6.1
46
- -------------
47
-
48
- - Fix crash in rate limiter middleware when used with custom exceptions [#3604]
49
-
50
- 1.6.0
51
- -------------
52
-
53
- - Show process "leader" tag on Busy page, requires Sidekiq 5.0.2 [#2867]
54
- - Capture custom metrics with the `save_history` API. [#2815]
55
- - Implement new `unique_until: 'start'` policy option. [#3471]
56
-
57
- 1.5.4
58
- -------------
59
-
60
- - Fix broken Cron page in Web UI [#3458]
61
-
62
- 1.5.3
63
- -------------
64
-
65
- - Remove dependency on the algorithms gem [#3446]
66
- - Allow user to specify max memory in megabytes with SIDEKIQ\_MAXMEM\_MB [#3451]
67
- - Implement logic to detect app startup failure, sidekiqswarm will exit
68
- rather than try to restart the app forever [#3450]
69
- - Another fix for doubly-encrypted arguments [#3368]
70
-
71
- 1.5.2
72
- -------------
73
-
74
- - Fix encrypted arguments double-encrypted by retry or rate limiting [#3368]
75
- - Fix leak in concurrent rate limiter, run this in Rails console to clean up existing data [#3323]
76
- ```ruby
77
- expiry = 1.month.to_i; Sidekiq::Limiter.redis { |c| c.scan_each(match: "lmtr-cfree-*") { |key| c.expire(key, expiry) } }
78
- ```
79
-
80
- 1.5.1
81
- -------------
82
-
83
- - Fix issue with census startup when not using Bundler configuration for
84
- source credentials.
85
-
86
- 1.5.0
87
- -------------
88
-
89
- - Add new web authorization API [#3251]
90
- - Update all sidekiqswarm env vars to use SIDEKIQ\_ prefix [#3218]
91
- - Add census reporting, the leader will ping contribsys nightly with aggregate usage metrics
92
-
93
- 1.4.0
94
- -------------
95
-
96
- - No functional changes, require latest Sidekiq and Sidekiq Pro versions
97
-
98
- 1.3.2
99
- -------------
100
-
101
- - Upgrade encryption to use OpenSSL's more secure GCM mode. [#3060]
102
-
103
- 1.3.1
104
- -------------
105
-
106
- - Fix multi-process memory monitoring on CentOS 6.x [#3063]
107
- - Polish the new encryption feature a bit.
108
-
109
- 1.3.0
110
- -------------
111
-
112
- - **BETA** [New encryption feature](https://github.com/mperham/sidekiq/wiki/Ent-Encryption)
113
- which automatically encrypts the last argument of a Worker, aka the secret bag.
114
-
115
- 1.2.4
116
- -------------
117
-
118
- - Fix issue causing some minutely jobs to execute every other minute.
119
- - Log a warning if slow periodic processing causes us to miss a clock tick.
120
-
121
- 1.2.3
122
- -------------
123
-
124
- - Periodic jobs could stop executing until process restart if Redis goes down [#3047]
125
-
126
- 1.2.2
127
- -------------
128
-
129
- - Add API to check if a unique lock is present. See [#2932] for details.
130
- - Tune concurrent limiters to minimize thread thrashing under heavy contention. [#2944]
131
- - Add option for tuning which Bundler groups get preloaded with `sidekiqswarm` [#3025]
132
- ```
133
- SIDEKIQ_PRELOAD=default,production bin/sidekiqswarm ...
134
- # Use an empty value for maximum application compatibility
135
- SIDEKIQ_PRELOAD= bin/sidekiqswarm ...
136
- ```
137
-
138
- 1.2.1
139
- -------------
140
-
141
- - Multi-Process mode can now monitor the RSS memory of children and
142
- restart any that grow too large. To limit children to 1GB each:
143
- ```
144
- MAXMEM_KB=1048576 COUNT=2 bundle exec sidekiqswarm ...
145
- ```
146
-
147
- 1.2.0
148
- -------------
149
-
150
- - **NEW FEATURE** Multi-process mode! Sidekiq Enterprise can now fork multiple worker
151
- processes, enabling significant memory savings. See the [wiki
152
- documentation](https://github.com/mperham/sidekiq/wiki/Ent-Multi-Process) for details.
153
-
154
-
155
- 0.7.10
156
- -------------
157
-
158
- - More precise gemspec dependency versioning
159
-
160
- 1.1.0
161
- -------------
162
-
163
- - **NEW FEATURE** Historical queue metrics, [documented in the wiki](https://github.com/mperham/sidekiq/wiki/Ent-Historical-Metrics) [#2719]
164
-
165
- 0.7.9, 1.0.2
166
- -------------
167
-
168
- - Window limiters can now accept arbitrary window sizes [#2686]
169
- - Fix race condition in window limiters leading to non-stop OverLimit [#2704]
170
- - Fix invalid overage counts when nesting concurrent limiters
171
-
172
- 1.0.1
173
- ----------
174
-
175
- - Fix crash in periodic subsystem when a follower shuts down, thanks
176
- to @justinko for reporting.
177
-
178
- 1.0.0
179
- ----------
180
-
181
- - Enterprise 1.x targets Sidekiq 4.x.
182
- - Rewrite several features to remove Celluloid dependency. No
183
- functional changes.
184
-
185
- 0.7.8
186
- ----------
187
-
188
- - Fix `unique_for: false` [#2658]
189
-
190
-
191
- 0.7.7
192
- ----------
193
-
194
- - Enterprise 0.x targets Sidekiq 3.x.
195
- - Fix racy shutdown event which could lead to disappearing periodic
196
- jobs, requires Sidekiq >= 3.5.3.
197
- - Add new :leader event which is fired when a process gains leadership.
198
-
199
- 0.7.6
200
- ----------
201
-
202
- - Redesign how overrated jobs are rescheduled to avoid creating new
203
- jobs. [#2619]
204
-
205
- 0.7.5
206
- ----------
207
-
208
- - Fix dynamic creation of concurrent limiters [#2617]
209
-
210
- 0.7.4
211
- ----------
212
- - Add additional check to prevent duplicate periodic job creation
213
- - Allow user-specified TTLs for rate limiters [#2607]
214
- - Paginate rate limiter index page [#2606]
215
-
216
- 0.7.3
217
- ----------
218
-
219
- - Rework `Sidekiq::Limiter` redis handling to match global redis handling.
220
- - Allow user to customize rate limit backoff logic and handle custom
221
- rate limit errors.
222
- - Fix scalability issue with Limiter index page.
223
-
224
- 0.7.2
225
- ----------
226
-
227
- - Fix typo which prevented limiters with '0' in their names.
228
-
229
- 0.7.1
230
- ----------
231
-
232
- - Fix issue where unique scheduled jobs can't be enqueued upon schedule
233
- due to the existing unique lock. [#2499]
234
-
235
- 0.7.0
236
- ----------
237
-
238
- Initial release.
data/Gemfile DELETED
@@ -1,23 +0,0 @@
1
- source 'https://rubygems.org'
2
-
3
- gemspec
4
-
5
- gem 'rake'
6
- gem 'redis-namespace'
7
- gem 'rails', '~> 5.2'
8
- gem 'sqlite3', '~> 1.3.6', platforms: :ruby
9
- gem 'activerecord-jdbcsqlite3-adapter', platforms: :jruby
10
-
11
- group :test do
12
- gem 'minitest'
13
- gem 'simplecov'
14
- end
15
-
16
- group :development, :test do
17
- gem 'pry-byebug', platforms: :mri
18
- end
19
-
20
- group :load_test do
21
- gem 'hiredis'
22
- gem 'toxiproxy'
23
- end
data/Pro-2.0-Upgrade.md DELETED
@@ -1,138 +0,0 @@
1
- # Upgrading to Sidekiq Pro 2.0
2
-
3
- Sidekiq Pro 2.0 allows nested batches for more complex job workflows
4
- and provides a new reliable scheduler which uses Lua to guarantee
5
- atomicity and much higher performance.
6
-
7
- It also removes deprecated APIs, changes the batch data format and
8
- how features are activated. Read carefully to ensure your upgrade goes
9
- smoothly.
10
-
11
- Sidekiq Pro 2.0 requires Sidekiq 3.3.2 or greater. Redis 2.8 is
12
- recommended; Redis 2.4 or 2.6 will work but some functionality will not be
13
- available.
14
-
15
- **Note that you CANNOT go back to Pro 1.x once you've created batches
16
- with 2.x. The new batches will not process correctly with 1.x.**
17
-
18
- **If you are on a version of Sidekiq Pro <1.5, you should upgrade to the
19
- latest 1.x version and run it for a week before upgrading to 2.0.**
20
-
21
- ## Nested Batches
22
-
23
- Batches can now be nested within the `jobs` method.
24
- This feature enables Sidekiq Pro to handle workflow processing of any size
25
- and complexity!
26
-
27
- ```ruby
28
- a = Sidekiq::Batch.new
29
- a.on(:success, SomeCallback)
30
- a.jobs do
31
- SomeWork.perform_async
32
-
33
- b = Sidekiq::Batch.new
34
- b.on(:success, MyCallback)
35
- b.jobs do
36
- OtherWork.perform_async
37
- end
38
- end
39
- ```
40
-
41
- Parent batch callbacks are not processed until all child batch callbacks have
42
- run successfully. In the example above, `MyCallback` will always fire
43
- before `SomeCallback` because `b` is considered a child of `a`.
44
-
45
- Of course you can dynamically add child batches while a batch job is executing.
46
-
47
- ```ruby
48
- def perform(*args)
49
- do_something(args)
50
-
51
- if more_work?
52
- # Sidekiq::Worker#batch returns the Batch this job is part of.
53
- batch.jobs do
54
- b = Sidekiq::Batch.new
55
- b.on(:success, MyCallback)
56
- b.jobs do
57
- OtherWork.perform_async
58
- end
59
- end
60
- end
61
- end
62
- ```
63
-
64
- More context: [#1485]
65
-
66
- ## Batch Data
67
-
68
- The batch data model was overhauled. Batch data should take
69
- significantly less space in Redis now. A simple benchmark shows 25%
70
- savings but real world savings should be even greater.
71
-
72
- * Batch 2.x BIDs are 14 character URL-safe Base64-encoded strings, e.g.
73
- "vTF1-9QvLPnREQ". Batch 1.x BIDs were 16 character hex-encoded
74
- strings, e.g. "4a3fc67d30370edf".
75
- * In 1.x, batch data was not removed until it naturally expired in Redis.
76
- In 2.x, all data for a batch is removed from Redis once the batch has
77
- run any success callbacks.
78
- * Because of the former point, batch expiry is no longer a concern.
79
- Batch expiry is hardcoded to 30 days and is no longer user-tunable.
80
- * Failed batch jobs no longer automatically store any associated
81
- backtrace in Redis.
82
-
83
- **There's no data migration required. Sidekiq Pro 2.0 transparently handles
84
- both old and new format.**
85
-
86
- More context: [#2130]
87
-
88
- ## Reliability
89
-
90
- 2.0 brings a new reliable scheduler which uses Lua inside Redis so enqueuing
91
- scheduled jobs is atomic. Benchmarks show it 50x faster when enqueuing
92
- lots of jobs.
93
-
94
- **Two caveats**:
95
- - Client-side middleware is not executed
96
- for each job when enqueued with the reliable scheduler. No Sidekiq or
97
- Sidekiq Pro functionality is affected by this change but some 3rd party
98
- plugins might be.
99
- - The Lua script used inside the reliable scheduler is not safe for use
100
- with Redis Cluster or other multi-master Redis solutions.
101
- It is safe to use with Redis Sentinel or a typical master/slave replication setup.
102
-
103
- **You no longer require anything to use the Reliability features.**
104
-
105
- * Activate reliable fetch and/or the new reliable scheduler:
106
- ```ruby
107
- Sidekiq.configure_server do |config|
108
- config.reliable_fetch!
109
- config.reliable_scheduler!
110
- end
111
- ```
112
- * Activate reliable push:
113
- ```ruby
114
- Sidekiq::Client.reliable_push!
115
- ```
116
-
117
- More context: [#2130]
118
-
119
- ## Other Changes
120
-
121
- * You must require `sidekiq/pro/notifications` if you want to use the
122
- existing notification schemes. I don't recommend using them as the
123
- newer-style `Sidekiq::Batch#on` method is simpler and more flexible.
124
- * Several classes have been renamed. Generally these classes are ones
125
- you should not need to require/use in your own code, e.g. the Batch
126
- middleware.
127
- * You can add `attr_accessor :jid` to a Batch callback class and Sidekiq
128
- Pro will set it to the jid of the callback job. [#2178]
129
- * There's now an official API to iterate all known Batches [#2191]
130
- ```ruby
131
- Sidekiq::BatchSet.new.each {|status| p status.bid }
132
- ```
133
- * The Web UI now shows the Sidekiq Pro version in the footer. [#1991]
134
-
135
- ## Thanks
136
-
137
- Adam Prescott, Luke van der Hoeven and Jon Hyman all provided valuable
138
- feedback during the release process. Thank you guys!
data/Pro-3.0-Upgrade.md DELETED
@@ -1,44 +0,0 @@
1
- # Welcome to Sidekiq Pro 3.0!
2
-
3
- Sidekiq Pro 3.0 is designed to work with Sidekiq 4.0.
4
-
5
- ## What's New
6
-
7
- * **Redis 2.8.0 or greater is required.** Redis 2.8 was released two years
8
- ago and contains **many** useful features which Sidekiq couldn't
9
- leverage until now. **Redis 3.0.3 or greater is recommended** for large
10
- scale use.
11
-
12
- * Sidekiq Pro no longer uses Celluloid. If your application code uses Celluloid,
13
- you will need to pull it in yourself.
14
-
15
- * Pausing and unpausing queues is now instantaneous, no more polling!
16
-
17
- * Reliable fetch has been re-implemented due to the fetch changes in
18
- Sidekiq 4.0.
19
-
20
- * Support for platforms without persistent hostnames. Since the reliable\_fetch
21
- algorithm requires a persistent hostname, an alternative reliability
22
- algorithm is now available for platforms like Heroku and Docker:
23
- ```ruby
24
- Sidekiq.configure_server do |config|
25
- config.timed_fetch!
26
- end
27
- ```
28
- The wiki contains [much more detail about each reliability option](https://github.com/mperham/sidekiq/wiki/Pro-Reliability-Server).
29
-
30
- * The old 'sidekiq/notifications' features have been removed.
31
-
32
- ## Upgrade
33
-
34
- First, make sure you are using Redis 2.8 or greater. Next:
35
-
36
- * Upgrade to the latest Sidekiq Pro 2.x.
37
- ```ruby
38
- gem 'sidekiq-pro', '< 3'
39
- ```
40
- * Fix any deprecation warnings you see.
41
- * Upgrade to 3.x.
42
- ```ruby
43
- gem 'sidekiq-pro', '< 4'
44
- ```