ac-ratelimiter 1.0.10 → 2.0.0-beta.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/.eslintrc.js +3 -2
  2. package/.github/workflows/node.js.yml +8 -2
  3. package/README.md +59 -16
  4. package/index.js +175 -156
  5. package/package.json +7 -7
  6. package/test/test.js +324 -219
package/.eslintrc.js CHANGED
@@ -17,10 +17,11 @@ const config = {
17
17
  },
18
18
  globals: {
19
19
  describe: true,
20
- it: true
20
+ it: true,
21
+ after: true
21
22
  },
22
23
  'parserOptions': {
23
- 'ecmaVersion': 2018
24
+ 'ecmaVersion': 2022
24
25
  },
25
26
  }
26
27
 
package/.github/workflows/node.js.yml CHANGED
@@ -5,9 +5,9 @@ name: Node.js CI
5
5
 
6
6
  on:
7
7
  push:
8
- branches: [ master ]
8
+ branches: [ master, develop ]
9
9
  pull_request:
10
- branches: [ master ]
10
+ branches: [ master, develop ]
11
11
 
12
12
  jobs:
13
13
  build:
@@ -18,6 +18,7 @@ jobs:
18
18
  matrix:
19
19
  node-version: [16.x, 18.x]
20
20
  # See supported Node.js release schedule at https://nodejs.org/en/about/releases/
21
+ redis-version: [6]
21
22
 
22
23
  steps:
23
24
  - uses: actions/checkout@v3
@@ -26,5 +27,10 @@ jobs:
26
27
  with:
27
28
  node-version: ${{ matrix.node-version }}
28
29
 
30
+ - name: Start Redis
31
+ uses: supercharge/redis-github-action@1.4.0
32
+ with:
33
+ redis-version: ${{ matrix.redis-version }}
34
+
29
35
  - run: yarn install
30
36
  - run: yarn run test
package/README.md CHANGED
@@ -1,12 +1,18 @@
1
1
  # ac-ratelimiter
2
2
 
3
- This tool provides rate-limiter based on Redis and ExpressJs.
3
+ This tool provides rate-limiter that can be used as middleware with ExpressJs.
4
+
5
+ For huge production load it is recommended that you use it with Redis. However, for smaller applications you can use the build in Node Cache.
4
6
 
5
7
  [![Node.js CI](https://github.com/AdmiralCloud/ac-ratelimiter/actions/workflows/node.js.yml/badge.svg)](https://github.com/AdmiralCloud/ac-ratelimiter/actions/workflows/node.js.yml)
6
8
 
7
- ## Usage
9
+ ## Breaking changes for version 2
10
+ Version 2 is a complete re-write of this module. It is now a class and uses async/await.
8
11
 
9
12
  ```
13
+ // Migration example
14
+
15
+ // Version 1
10
16
  const acrl = require('ac-ratelimiter')
11
17
 
12
18
  const init = {
@@ -26,37 +32,73 @@ acrl.limiter(req, {}, err => {
26
32
  return res.json({ status: _.get(err, 'status') })
27
33
  })
28
34
 
35
+
36
+ // Version 2
37
+ const acrl = require('ac-ratelimiter')
38
+
39
+ const init = {
40
+ routes: [
41
+ { route: 'user/find', throttleLimit: 1, limit: 2, expires: 3, delay: 250 },
42
+ ],
43
+ redis: REDIS INSTANCE
44
+ logger: winston.log INSTANCE
45
+ }
46
+
47
+ const rateLimiter = new acrl(init)
48
+
49
+ try {
50
+ await rateLimiter.limiter(req)
51
+ }
52
+ catch(e) {
53
+ // e.status === 900 => throttling is active
54
+ // e.status === 429 => limiter is active
55
+ }
56
+
57
+ ```
58
+
59
+ ## Usage
60
+
61
+ ### Without any dependencies
62
+ This example initiates the rate limiter with NodeCache (instead of Redis) and console.log (instead of Winston). Default limits are 150 requests within 3 seconds. Starting at 50 request, requests will be throttled by 250ms.
63
+
64
+ ```
65
+ const acrl = require('ac-ratelimiter')
66
+
67
+ const rateLimiter = new acrl()
68
+
69
+ try {
70
+ await rateLimiter.limiter(req)
71
+ }
72
+ catch(e) {
73
+ // e.status === 900 => throttling is active
74
+ // e.status === 429 => limiter is active
75
+ }
76
+
29
77
  ```
30
78
 
31
79
  ## Prerequisites
32
80
 
33
81
  ### Init
34
- The ac-ratelimiter uses Redis as storage for the rate-limiter keys. You can also write your own, memory-based, backend. See the test for an example.
82
+ The ac-ratelimiter can use Redis as storage for the rate-limiter keys. By default and to run out-of-the-box it uses Node Cache.
35
83
 
36
84
  Additionally, for logging purposes, we use Winston. But you can also use any other logger that provides logging for "warn" and "error".
37
85
 
38
86
  Last but not least, provide an array of objects with rate limiter instructions. Each object has the following properties:
39
- Property | Type | Example | Remarks
87
+ Property | Type | Defaults | Remarks
40
88
  ---|---|---|---|
41
- route | string | user/find | A combination of controller and action (express) or any other identifier you can provide
42
- throttleLimit | integer | 20 | Number of calls before throttling starts
89
+ routes | string | | A combination of controller and action (express) or any other identifier you can provide
90
+ throttleLimit | 50 | 20 | Number of calls before throttling starts
43
91
  delay | integer | 250 | Number of milliseconds a throttle request is delayed (on purpose)
44
- limit | integer | 100 | Number of calls before the limiter kicks in
92
+ limit | integer | 150 | Number of calls before the limiter kicks in
45
93
  expires | integer | 3 | Number of seconds before the rate-limiter resets
46
94
 
47
95
 
48
- The init function can take additional parameters like
49
- + environment
50
- + debugMode
51
96
 
52
97
  ### RateLimiter
53
98
  The actual rateLimiter function takes two arguments, the Express request object (req) and an options object with the following optional properties:
54
99
 
55
100
  Property | Type | Example | Remarks
56
101
  ---|---|---|---|
57
- knownIP | Object | { name: 'AdmiralCloud' } | Display known IPs with their name when limiter is active
58
- link | String | myLink | Display and locks the limiter to a given link (as identifier)
59
- token | String | myToken | Locks the limiter to a given token/user session
60
102
  name | String | myName | Identifier for the route - falls back to controller/action
61
103
  redisKey | String | myKey | Optional RedisKey to use for rate limiter
62
104
  fallbackRoute | String | fbroute | Optional fallback route identifier
@@ -73,12 +115,13 @@ Both values might be retrieved prior to the rate limiter so there is no need to
73
115
 
74
116
  # Links
75
117
  - [Website](https://www.admiralcloud.com/)
76
- - [Twitter (@admiralcloud)](https://twitter.com/admiralcloud)
77
118
  - [Facebook](https://www.facebook.com/MediaAssetManagement/)
78
119
 
79
120
  # Run tests
80
- Call yarn run test
121
+ ```
122
+ yarn run test
123
+ ```
81
124
 
82
125
  ## License
83
126
 
84
- [MIT License](https://opensource.org/licenses/MIT) Copyright © 2009-present, AdmiralCloud, Mark Poepping
127
+ [MIT License](https://opensource.org/licenses/MIT) Copyright © 2009-present, AdmiralCloud AG, Mark Poepping
package/index.js CHANGED
@@ -1,188 +1,207 @@
1
- /**
2
- * Copyright mmpro film- und medienproduktion GmbH and other Node contributors
3
- *
4
- */
5
-
6
- const async = require('async');
7
- const _ = require('lodash');
1
+ const { setTimeout } = require('timers/promises')
8
2
 
3
+ const NodeCache = require('node-cache')
9
4
  const acts = require('ac-ip')
10
5
 
11
- const ratelimiter = () => {
12
-
13
- let environment = process.env.NODE_ENV || 'development'
14
- let routes = []
15
- let redis
16
- let logger
17
- let debugMode
18
- let knownIPs
19
- let ignorePrivateIps = true
20
-
21
- const init = (options) => {
22
- if (_.get(options, 'environment')) environment = _.get(options, 'environment')
23
- if (_.get(options, 'routes')) routes = _.get(options, 'routes')
24
- if (_.get(options, 'debugMode')) debugMode = _.get(options, 'debugMode')
25
- if (_.get(options, 'knownIPs')) knownIPs = _.get(options, 'knownIPs')
26
- if (_.has(options, 'ignorePrivateIps')) ignorePrivateIps = _.get(options, 'ignorePrivateIps')
27
-
28
- if (!_.get(options, 'redis')) {
29
- throw new Error('Redis instance is required')
6
+ class ACError extends Error {
7
+ constructor(message, options = {}) {
8
+ super(message)
9
+
10
+ if (Error.captureStackTrace) {
11
+ Error.captureStackTrace(this, ACError)
12
+ }
13
+ // info
14
+ this.code = options.code || -1
15
+ this.errorMessage = message
16
+ // show other properties of options object
17
+ for (const [key, value] of Object.entries(options)) {
18
+ this[key] = value;
19
+ }
20
+ }
21
+ }
22
+
23
+ class RateLimiter {
24
+ constructor({ redisInstance, logger = console, routes = [], knownIPs = [], ignorePrivateIps } = {}) {
25
+ // make sure only one instance exists!
26
+ if (RateLimiter._instance) {
27
+ return RateLimiter._instance
30
28
  }
31
- redis = _.get(options, 'redis')
32
- if (!_.get(options, 'logger')) {
33
- throw new Error('Logger instance is required')
29
+ RateLimiter._instance = this;
30
+
31
+ this.environment = process.env.NODE_ENV || 'development'
32
+ if(redisInstance) {
33
+ this.redisInstance = redisInstance
34
+ }
35
+ else {
36
+ this.cache = new NodeCache()
34
37
  }
35
- logger = _.get(options, 'logger')
38
+
39
+ this.limits = {
40
+ expires: 3,
41
+ limit: 150,
42
+ throttleLimit: 50,
43
+ delay: 250
44
+ }
45
+
46
+ this.logger = logger
47
+ this.routes = routes
48
+ this.knownIPs = knownIPs
49
+ this.ignorePrivateIps = ignorePrivateIps
36
50
  }
37
51
 
38
- const prepareRedisKey = (params) => {
39
- const ip = _.get(params, 'ip')
40
- const controller = _.get(params, 'controller', 'controller')
41
- const action = _.get(params, 'action', 'action')
42
- const clientId = _.get(params, 'clientId', 'clientId')
43
- const identifier = _.get(params, 'identifier')
44
-
45
- let redisKey = _.get(params, 'redisKey', (environment + ':rateLimiter:' + clientId + ':' + ip + ':' + controller + ':' + action))
46
- if (identifier) redisKey += ':' + identifier
47
- return redisKey
52
+ whichStorage() {
53
+ if (this.redisInstance) return 'Redis'
54
+ else return 'NodeCache'
48
55
  }
49
56
 
50
- const limiter = (req, options, cb) => {
51
- const ip = _.get(req, 'determinedIP') || acts.determineIP(req)
52
- if (ignorePrivateIps && acts.isPrivate(ip)) return cb(null)
53
- const controller = _.get(req, 'options.controller')
54
- const action = _.get(req, 'options.action')
55
- const clientId = _.get(options, 'clientId')
56
- const route = _.get(options, 'name') || `${controller}/${action}`
57
- const knownIP = _.find(knownIPs, { ip })
58
- const identifier = _.get(options, 'identifier')
59
- // obscure identifier for logging
60
- const logIdentifier = _.isString(identifier) && identifier.replace(/(\w{1,4})-(\w{1,4})/g, 'xxxx')
61
-
62
- const redisKey = prepareRedisKey({
57
+ // private
58
+ prepareRedisKey({ ip, controller = 'controller', action = 'action', clientId = 'clientId', identifier = 'identifier', redisKey }) {
59
+ let rateLimiterKey = redisKey || (this.environment + ':rateLimiter:' + clientId + ':' + ip + ':' + controller + ':' + action)
60
+ if (identifier) rateLimiterKey += ':' + identifier
61
+ return rateLimiterKey
62
+ }
63
+
64
+ async limiter(req, {
65
+ ip = req?.determinedIP || acts.determineIP(req),
66
+ clientId = 'clientId',
67
+ identifier = 'identifier',
68
+ redisKey,
69
+ expires,
70
+ limit,
71
+ throttleLimit,
72
+ delay,
73
+ fallbackRoute = 'default',
74
+ name,
75
+ debugMode,
76
+ rateLimitCounter
77
+ }) {
78
+
79
+ if (this.ignorePrivateIps && acts.isPrivate(ip)) return
80
+
81
+ const logIdentifier = typeof identifier === 'string' && identifier.replace(/(\w{1,4})-(\w{1,4})/g, 'xxxx')
82
+ const knownIP = this.knownIPs.find(({ knownIP }) => knownIP === ip)
83
+
84
+ const controller = req?.options?.controller || 'controller'
85
+ const action = req?.options?.action || 'action'
86
+ const currentRoute = name || `${controller}/${action}`
87
+
88
+
89
+ const rateLimiterKey = this.prepareRedisKey({
63
90
  ip,
64
91
  controller,
65
92
  action,
66
93
  clientId,
67
94
  identifier,
68
- redisKey: _.get(options, 'redisKey')
95
+ redisKey
69
96
  })
70
97
 
71
- const fallbackRoute = _.get(options, 'fallbackRoute', 'default')
72
- let settings = routes.find(item => {
73
- if (ip && clientId && route && item.ip === ip && item.clientId === clientId && item.route === route ) return item
74
- else if (ip && route && item.ip === ip && item.route === route && !item.clientId ) return item
75
- else if (clientId && route && item.clientId === clientId && item.route === route && !item.ip) return item
76
- else if (route && item.route === route && !item.clientId && !item.ip) return item
98
+ const rateLogger = ({ type, rateLimitCounter, currentLimit }) => {
99
+ this.logger.warn('-'.repeat(80))
100
+ if (debugMode) this.logger.warn('DEBUG MODE - DEBUG MODE - DEBUG MODE')
101
+ this.logger.warn('%s | %s | %s | %s | Counter %s/%s', 'ACRateLimiter'.padEnd(15), type.padEnd(12), currentRoute.padEnd(32), (ip + ' ' + (knownIP?.name || '')).padEnd(16), rateLimitCounter, currentLimit)
102
+ if (logIdentifier) this.logger.warn('%s | Identifier: %s', ' '.padEnd(15), logIdentifier)
103
+ }
104
+
105
+ let settings = this.routes.find(item => {
106
+ if (ip && clientId && currentRoute && item.ip === ip && item.clientId === clientId && item.route === currentRoute ) return item
107
+ else if (ip && currentRoute && item.ip === ip && item.route === currentRoute && !item.clientId ) return item
108
+ else if (clientId && currentRoute && item.clientId === clientId && item.route === currentRoute && !item.ip) return item
109
+ else if (currentRoute && item.route === currentRoute && !item.clientId && !item.ip) return item
77
110
  })
78
111
  if (!settings) {
79
112
  // check fallback route
80
- settings = routes.find(item => {
113
+ settings = this.routes.find(item => {
81
114
  if (item.route === fallbackRoute && !item.clientId && !item.ip) return item
82
115
  })
83
116
  }
84
- const expires = _.get(options, 'expires', _.get(settings, 'expires', 3))
85
- const limit = _.get(options, 'limit', _.get(settings, 'limit', 150))
86
- const throttleLimit = _.get(options, 'throttleLimit', _.get(settings, 'throttleLimit', 50)) // optional
87
- const delay = _.get(options, 'delay', _.get(settings, 'delay', 250)) // delay in ms which kicks in if throttle if active
88
-
89
- let rateLimitCounter
90
-
91
- const rateLogger = (params) => {
92
- const type = _.get(params, 'type')
93
- logger.warn(_.repeat('-', 80))
94
- if (debugMode) logger.warn('DEBUG MODE - DEBUG MODE - DEBUG MODE')
95
- logger.warn('%s | %s | %s | %s | Counter %s/%s', _.padEnd('ACRateLimiter', 15), _.padEnd(type, 12), _.padEnd(route, 32), _.padEnd((ip + ' ' + _.get(knownIP, 'name', '')), 16), rateLimitCounter, limit)
96
- if (logIdentifier) logger.warn('%s | Identifier: %s', _.padEnd(' ', 15), logIdentifier)
117
+
118
+ let current = {
119
+ expires,
120
+ limit,
121
+ throttleLimit,
122
+ delay
97
123
  }
98
-
99
-
100
- async.series({
101
- increaseCounter: (done) => {
102
- redis.incr(redisKey, (err, result) => {
103
- if (err) {
104
- logger.error('ACRateLimiter | Redis failed %j', err)
105
- // silently ignore
106
- return done()
107
- }
108
- rateLimitCounter = result
109
- return done()
110
- })
111
- },
112
- processLimits: (done) => {
113
- if (rateLimitCounter === 1 && rateLimitCounter < limit) {
114
- // key has never been set before - set expire time and return
115
- redis.expire(redisKey, expires, done)
116
- }
117
- else if (rateLimitCounter > limit) {
118
- // only log every 10th entry
119
- if (rateLimitCounter === limit || rateLimitCounter % 10 === 0) {
120
- rateLogger({ type: 'Blocking' })
121
- }
122
- // debug mode shows the rate limiting but does not limit
123
- if (debugMode) return done()
124
- return done({ message: 'tooManyRequestsFromThisIP', status: 429, logging: false, counter: rateLimitCounter, additionalInfo: { expires } })
125
- }
126
- else if (throttleLimit && rateLimitCounter > limit * 0.9) {
127
- // at 90 percent delay with expire time, to avoid limit kicking in
128
- rateLogger({ type: 'Final Throttling' })
129
- _.delay(() => {
130
- // do not "taint" process time when deliberately throttline
131
- if (req._startTime) req._startTime += expires * 1000
132
-
133
- return done({ message: 'finalThrottlingActive_requestsIsDelayed', status: 900, additionalInfo: { expires } })
134
- }, expires * 1000)
135
- }
136
- else if (throttleLimit && rateLimitCounter > throttleLimit) {
137
- // log the first throttling and every 50th entry
138
- if (rateLimitCounter === (throttleLimit + 1) || rateLimitCounter % 50 === 0) {
139
- rateLogger({ type: 'Throttling' })
140
- }
141
- _.delay(() => {
142
- // do not "taint" process time when deliberately throttline
143
- if (req._startTime) req._startTime += delay
144
-
145
- return done({ message: 'throttlingActive_requestsIsDelayed', status: 900, additionalInfo: { expires } })
146
- }, delay)
147
- }
148
- else {
149
- return done()
150
- }
124
+
125
+ const props = ['expires', 'limit', 'throttleLimit', 'delay']
126
+ props.forEach(prop => {
127
+ if (!Number.isFinite(current[prop])) {
128
+ // use from setting
129
+ if (Number.isFinite(settings?.[prop])) current[prop] = settings[prop]
130
+ else current[prop] = this.limits[prop]
151
131
  }
152
- }, err => {
153
- return cb(err, { ip, controller, action, counter: rateLimitCounter, knownIPName: _.get(knownIP, 'name', '-'), identifier: logIdentifier })
154
132
  })
155
- }
156
133
 
157
- const resetLimiter = (params, cb) => {
158
- const ip = _.get(params, 'ip')
159
- const controller = _.get(params, 'controller')
160
- const action = _.get(params, 'action')
161
- const identifier = _.get(params, 'identifier')
162
-
163
- let redisKey = environment + ':rateLimiter:'
164
- if (ip) redisKey += ip + ':'
165
- if (controller) redisKey += controller + ':'
166
- if (action) redisKey += action + ':'
167
- if (identifier) redisKey += ':' + identifier
168
- redisKey += '*'
169
- redis.keys(redisKey, (err, keys) => {
170
- if (err) return cb(err)
171
- const multi = redis.multi()
172
- _.forEach(keys, key => {
173
- multi.del(key)
174
- })
175
- multi.exec(cb)
176
- })
134
+ if (rateLimitCounter) {
135
+ // if rateLimitCounter is sent with the request, then don't fetch it again
136
+ }
137
+ else if (this.redisInstance) {
138
+ // use Redis instance for rate limiting
139
+ rateLimitCounter = await this.redisInstance.incr(rateLimiterKey)
140
+ if (rateLimitCounter === 1 && rateLimitCounter < current.limit) {
141
+ // key has never been set before - set expire time and return
142
+ await this.redisInstance.expire(redisKey, expires)
143
+ }
144
+ }
145
+ else {
146
+ // use Node cache (memory) for rate limiting - please see README before using in production!
147
+ rateLimitCounter = this.cache.get(rateLimiterKey)
148
+ let ts = this.cache.getTtl(rateLimiterKey) // ts in ms when the key will expire
149
+ rateLimitCounter = rateLimitCounter + 1 || 1
150
+ if (ts === undefined || ts === 0) {
151
+ // first entry - set expiration
152
+ this.cache.set(rateLimiterKey, rateLimitCounter, current?.expires )
153
+ }
154
+ else {
155
+ // update but use the existing timestamp
156
+ const newTTL = ts - new Date().getTime()
157
+ this.cache.set(rateLimiterKey, rateLimitCounter, Math.ceil(newTTL/1000))
158
+ }
159
+ }
160
+
161
+ if (debugMode) {
162
+ console.log('Route %s | Current Counter %s | Throttle %s | Limit %s | Delay %s | Expires %s', currentRoute, rateLimitCounter, current.throttleLimit, current.limit, current.delay, current.expires)
163
+ }
164
+
165
+ if (rateLimitCounter > current.limit) {
166
+ // only log every 10th entry
167
+ if (rateLimitCounter === current.limit || rateLimitCounter % 10 === 0) {
168
+ rateLogger({ type: 'Blocking', rateLimitCounter, currentLimit: current.limit })
169
+ }
170
+ throw new ACError('tooManyRequestsFromThisIP', { status: 429, logging: false, counter: rateLimitCounter, additionalInfo: { expires: current.expires } })
171
+
172
+ }
173
+ else if (current.throttleLimit && rateLimitCounter > current.limit * 0.9) {
174
+ // at 90 percent delay with expire time, to avoid limit kicking in
175
+ rateLogger({ type: 'Final Throttling', rateLimitCounter, currentLimit: current.limit })
176
+ await setTimeout(current.expires * 1000)
177
+ // do not "taint" process time when deliberately throttline
178
+ if (req._startTime) req._startTime += current.expires * 1000
179
+ throw new ACError('finalThrottlingActive_requestsIsDelayed', { status: 900, additionalInfo: { counter: rateLimitCounter, expires: current.expires } })
180
+ }
181
+ else if (current.throttleLimit && rateLimitCounter > current.throttleLimit) {
182
+ // log the first throttling and every 50th entry
183
+ if (rateLimitCounter === (throttleLimit + 1) || rateLimitCounter % 50 === 0) {
184
+ rateLogger({ type: 'Throttling', rateLimitCounter, currentLimit: current.limit })
185
+ }
186
+ await setTimeout(current.delay)
187
+ // do not "taint" process time when deliberately throttline
188
+ if (req._startTime) req._startTime += current.delay
189
+ throw new ACError('throttlingActive_requestsIsDelayed', { status: 900, additionalInfo: { counter: rateLimitCounter, expires: current.expires } })
190
+ }
177
191
  }
178
192
 
193
+ async updateLimiter({ routes, knownIPs, ignorePrivateIps }) {
194
+ if (Array.isArray(routes)) this.routes = routes
195
+ if (Array.isArray(knownIPs)) this.knownIPs = knownIPs
196
+ if (typeof ignorePrivateIps === 'boolean') this.ignorePrivateIps = ignorePrivateIps
197
+ }
179
198
 
180
- return {
181
- init,
182
- limiter,
183
- prepareRedisKey,
184
- resetLimiter
199
+ async resetLimiter() {
200
+ // reset completely
201
+ this.cache.flushAll()
185
202
  }
186
203
 
204
+
187
205
  }
188
- module.exports = ratelimiter()
206
+
207
+ module.exports = RateLimiter
package/package.json CHANGED
@@ -1,8 +1,8 @@
1
1
  {
2
2
  "name": "ac-ratelimiter",
3
3
  "description": "Simple ratelimiter for express",
4
- "version": "1.0.10",
5
- "author": "Mark Poepping",
4
+ "version": "2.0.0-beta.1",
5
+ "author": "Mark Poepping (www.admiralcloud.com)",
6
6
  "repository": {
7
7
  "type": "git",
8
8
  "url": "git://github.com/admiralcloud/ac-ratelimiter"
@@ -10,13 +10,13 @@
10
10
  "homepage": "https://www.admiralcloud.com",
11
11
  "dependencies": {
12
12
  "ac-ip": "^3.0.1",
13
- "async": "^3.2.4",
14
- "lodash": "^4.17.21"
13
+ "node-cache": "^5.1.2"
15
14
  },
16
15
  "devDependencies": {
17
- "ac-semantic-release": "^0.3.4",
16
+ "ac-semantic-release": "^0.3.5",
18
17
  "chai": "^4.3.7",
19
- "eslint": "8.31.0",
18
+ "eslint": "8.35.0",
19
+ "ioredis": "^5.3.1",
20
20
  "mocha": "^10.2.0"
21
21
  },
22
22
  "scripts": {
@@ -25,6 +25,6 @@
25
25
  "test-jenkins": "JUNIT_REPORT_PATH=./report.xml mocha --slow 1000 --colors --reporter mocha-jenkins-reporter --reporter-options junit_report_name='RATELIMITER'"
26
26
  },
27
27
  "engines": {
28
- "node": ">=10.0.0"
28
+ "node": ">=16.0.0"
29
29
  }
30
30
  }
package/test/test.js CHANGED
@@ -1,298 +1,403 @@
1
- const _ = require('lodash')
2
- const { expect } = require('chai');
1
+ const { expect } = require('chai')
2
+ const { setTimeout } = require('timers/promises')
3
3
 
4
- const ratelimiter = require('../index')
4
+ const ratelimiterModule = require('../index')
5
5
 
6
- const redisStoreSimulator = {}
7
-
8
- const init = {
9
- routes: [
10
- { route: 'user/find', throttleLimit: 1, limit: 2, expires: 3, delay: 250 },
11
- ],
12
- redis: {
13
- incr: (key, cb) => {
14
- redisStoreSimulator[key] = redisStoreSimulator[key] || 0
15
- redisStoreSimulator[key] += 1
16
- //console.log('Redis incr', key, redisStoreSimulator[key])
17
- return cb(null, redisStoreSimulator[key])
18
- },
19
- expire: (key, expires, cb) => {
20
- //console.log('Redis expires', key)
21
- setTimeout(() => {
22
- _.unset(redisStoreSimulator, key)
23
- }, expires * 1000)
24
- return cb()
25
- }
26
- },
27
- logger: {
28
- warn: () => {
29
- // just for simulation
30
- }
31
- }
32
- }
6
+ const Redis = require('ioredis')
7
+ const redis = new Redis()
33
8
 
34
9
  let req = {
35
10
  options: {
36
11
  controller: 'user',
37
- action: 'find'
12
+ action: 'find',
38
13
  },
39
14
  determinedIP: '1.2.3.4'
40
15
  }
41
16
 
17
+
18
+ let initOptions = {
19
+ routes: [
20
+ { route: 'user/find', throttleLimit: 1, limit: 2, expires: 3, delay: 250 },
21
+ ]
22
+ }
42
23
  let options = {}
43
24
 
44
- describe('Test section #1', function () {
45
- describe('RATE LIMITER TEST', function() {
46
- this.timeout(5000)
47
- it('init tests', done => {
48
- ratelimiter.init(init)
49
- return done()
50
- })
25
+ const ratelimiter = new ratelimiterModule(initOptions)
26
+ const ratelimiterRedis = new ratelimiterModule({ redisInstance: redis, routes: initOptions.routes })
27
+
51
28
 
52
- it('should not trigger', done => {
53
- ratelimiter.limiter(req, options, (err) => {
54
- expect(err).eql(null)
55
- return done()
29
+ describe('Use NodeCache', () => {
30
+
31
+ describe('Test section #1', function () {
32
+ describe('RATE LIMITER TEST', function() {
33
+ this.timeout(5000)
34
+
35
+ it('Reset Limiter', async() => {
36
+ await ratelimiter.resetLimiter()
56
37
  })
57
- })
58
- it('should still not trigger', done => {
59
- ratelimiter.limiter(req, options, (err) => {
60
- expect(err).to.have.property('status', 900)
61
- return done()
38
+
39
+ it('should not trigger - req #1', async() => {
40
+ let result = await ratelimiter.limiter(req, options)
41
+ expect(result).eql(undefined)
62
42
  })
63
- })
64
- it('should still not trigger as only throttling is active', done => {
65
- ratelimiter.limiter(req, options, (err) => {
66
- expect(err).eql(null)
67
- return done()
43
+ it('should still not trigger - req #2 - but throw 900', async() => {
44
+ try {
45
+ await ratelimiter.limiter(req, options)
46
+ }
47
+ catch(e) {
48
+ expect(e).to.have.property('message', 'finalThrottlingActive_requestsIsDelayed')
49
+ expect(e).to.have.property('status', 900)
50
+ }
51
+ })
52
+ it('should not trigger - req #3 - rate limiter is reset', async() => {
53
+ let result = await ratelimiter.limiter(req, options)
54
+ expect(result).eql(undefined)
68
55
  })
69
56
  })
70
57
  })
71
- })
72
58
 
73
59
 
74
- describe('Test section #2 - immediate limiter', function () {
75
- describe('RATE LIMITER TEST', function() {
76
- this.timeout(5000)
77
- it('init tests', done => {
78
- req.determinedIP = '4.1.4.1'
79
- init.routes = [
80
- { route: 'user/find', limit: 0, expires: 3, delay: 250 },
81
- ]
82
- ratelimiter.init(init)
83
- return done()
84
- })
60
+ describe('Test section #2 - immediate limiter', function () {
61
+ describe('RATE LIMITER TEST', function() {
62
+ this.timeout(5000)
63
+
64
+ it('Reset Limiter', async() => {
65
+ await ratelimiter.resetLimiter()
66
+ })
85
67
 
86
- it('should trigger immediately', done => {
87
- ratelimiter.limiter(req, options, (err) => {
88
- expect(err).to.have.property('status', 429)
89
- return done()
68
+ it('Update settings', async() => {
69
+ await ratelimiter.updateLimiter({
70
+ routes: [
71
+ { route: 'user/find', limit: 0, expires: 3, delay: 250 },
72
+ ]
73
+ })
74
+ })
75
+
76
+ it('should trigger immediately', async() => {
77
+ req.determinedIP = '4.1.4.1'
78
+ try {
79
+ await ratelimiter.limiter(req, options)
80
+ }
81
+ catch(e) {
82
+ expect(e).to.be.an('error')
83
+ expect(e).to.have.property('message', 'tooManyRequestsFromThisIP')
84
+ expect(e).to.have.property('status', 429)
85
+ }
90
86
  })
91
87
  })
92
88
  })
93
- })
94
89
 
95
90
 
96
- describe('Test section #3 - no throttling', function () {
97
- describe('RATE LIMITER TEST', function() {
98
- this.timeout(5000)
99
- it('init tests', done => {
100
- req.determinedIP = '2.3.4.1'
101
- init.routes = [
102
- { route: 'user/find', throttleLimit: 0, limit: 2, expires: 3, delay: 0 },
103
- ]
104
- ratelimiter.init(init)
105
- return done()
106
- })
91
+ describe('Test section #3 - no throttling', function () {
92
+ describe('RATE LIMITER TEST', function() {
93
+ this.timeout(5000)
94
+ it('Reset Limiter', async() => {
95
+ await ratelimiter.resetLimiter()
96
+ })
107
97
 
108
- it('should not trigger', done => {
109
- ratelimiter.limiter(req, options, (err) => {
110
- expect(err).eql(null)
111
- return done()
98
+ it('Update settings', async() => {
99
+ await ratelimiter.updateLimiter({
100
+ routes: [
101
+ { route: 'user/find', throttleLimit: 0, limit: 2, expires: 3, delay: 0 },
102
+ ]
103
+ })
112
104
  })
113
- })
114
- it('should still not trigger', done => {
115
- ratelimiter.limiter(req, options, (err) => {
116
- expect(err).eql(null)
117
- return done()
105
+
106
+ it('should not trigger req #1', async() => {
107
+ req.determinedIP = '2.3.4.1'
108
+ let result = await ratelimiter.limiter(req, options)
109
+ expect(result).eql(undefined)
118
110
  })
119
- })
120
- it('should trigger the limiter', done => {
121
- ratelimiter.limiter(req, options, (err) => {
122
- expect(err).to.have.property('status', 429)
123
- return done()
111
+ it('should still not trigger - delayed req #2', async() => {
112
+ try {
113
+ await ratelimiter.limiter(req, options)
114
+ }
115
+ catch(e) {
116
+ expect(e).to.have.property('message', 'finalThrottlingActive_requestsIsDelayed')
117
+ expect(e).to.have.property('status', 900)
118
+ }
119
+ })
120
+ it('should trigger the limiter', async() => {
121
+ try {
122
+ await ratelimiter.limiter(req, options)
123
+ }
124
+ catch(e) {
125
+ expect(e).to.have.property('message', 'tooManyRequestsFromThisIP')
126
+ expect(e).to.have.property('status', 429)
127
+ }
124
128
  })
125
129
  })
126
130
  })
127
- })
128
131
 
129
- describe('Test section #4 - routes with clientId', function() {
130
- describe('RATE LIMITER TEST', function() {
131
- this.timeout(5000)
132
- let req = {
133
- options: {
134
- controller: 'customer',
135
- action: 'find'
136
- },
137
- determinedIP: '1.2.3.4'
138
- }
139
- let options = {
140
- clientId: 'abc'
141
- }
142
- it('init tests', done => {
143
- init.routes = [
144
- { route: 'customer/find', clientId: 'abc', throttleLimit: 1, limit: 2, expires: 3, delay: 250 },
145
- { route: 'customer/find', throttleLimit: 3, limit: 10, expires: 3, delay: 250 },
146
- ]
147
- ratelimiter.init(init)
148
- return done()
149
- })
132
+ describe('Test section #4 - routes with clientId', function() {
133
+ describe('RATE LIMITER TEST', function() {
134
+ this.timeout(5000)
135
+ let req = {
136
+ options: {
137
+ controller: 'customer',
138
+ action: 'find'
139
+ },
140
+ determinedIP: '1.2.3.4'
141
+ }
142
+ let options = {
143
+ clientId: 'abc',
144
+ routes: [
145
+ { route: 'customer/find', clientId: 'abc', throttleLimit: 1, limit: 2, expires: 3, delay: 250 },
146
+ { route: 'customer/find', throttleLimit: 3, limit: 10, expires: 3, delay: 250 },
147
+ ]
148
+ }
150
149
 
151
- it('should not trigger', done => {
152
- ratelimiter.limiter(req, options, (err) => {
153
- expect(err).eql(null)
154
- return done()
150
+ it('Reset Limiter', async() => {
151
+ await ratelimiter.resetLimiter()
155
152
  })
156
- })
157
- it('should still not trigger', done => {
158
- ratelimiter.limiter(req, options, (err) => {
159
- expect(err).to.have.property('status', 900)
160
- return done()
153
+
154
+ it('should not trigger - req #1', async() => {
155
+ let result = await ratelimiter.limiter(req, options)
156
+ expect(result).eql(undefined)
161
157
  })
162
- })
163
- it('should still not trigger as only throttling is active', done => {
164
- ratelimiter.limiter(req, options, (err) => {
165
- expect(err).eql(null)
166
- return done()
158
+ it('should still not trigger but delay - req #2', async() => {
159
+ try {
160
+ await ratelimiter.limiter(req, options)
161
+ }
162
+ catch(e) {
163
+ expect(e).to.have.property('message', 'finalThrottlingActive_requestsIsDelayed')
164
+ expect(e).to.have.property('status', 900)
165
+ }
166
+ })
167
+ it('should still not trigger as ratelimiter is reset', async() => {
168
+ let result = await ratelimiter.limiter(req, options)
169
+ expect(result).eql(undefined)
167
170
  })
168
- })
169
171
 
170
- it('Now make request without clientId - should throttler after 3 requests', done => {
171
- options = {}
172
- return done()
173
- })
172
+ it('Now make request without clientId - should throttler after 3 requests', async() => {
173
+ options = {
174
+ routes: [
175
+ { route: 'customer/find', clientId: 'abc', throttleLimit: 1, limit: 2, expires: 3, delay: 250 },
176
+ { route: 'customer/find', throttleLimit: 3, limit: 10, expires: 3, delay: 250 },
177
+ ]
178
+ }
179
+ })
174
180
 
175
- it('should not trigger #1', done => {
176
- ratelimiter.limiter(req, options, (err) => {
177
- expect(err).eql(null)
178
- return done()
181
+ it('should not trigger #1', async() => {
182
+ let result = await ratelimiter.limiter(req, options)
183
+ expect(result).eql(undefined)
179
184
  })
180
- })
181
185
 
182
- it('should not trigger #2', done => {
183
- ratelimiter.limiter(req, options, (err) => {
184
- expect(err).eql(null)
185
- return done()
186
+ it('should not trigger #2', async() => {
187
+ let result = await ratelimiter.limiter(req, options)
188
+ expect(result).eql(undefined)
186
189
  })
187
- })
188
190
 
189
- it('should not trigger #3', done => {
190
- ratelimiter.limiter(req, options, (err) => {
191
- expect(err).eql(null)
192
- return done()
191
+ it('should not trigger #3', async() => {
192
+ let result = await ratelimiter.limiter(req, options)
193
+ expect(result).eql(undefined)
193
194
  })
194
- })
195
195
 
196
- it('should still not trigger', done => {
197
- ratelimiter.limiter(req, options, (err) => {
198
- expect(err).to.have.property('status', 900)
199
- return done()
196
+ it('should delay request #4', async() => {
197
+ try {
198
+ await ratelimiter.limiter(req, options)
199
+ }
200
+ catch(e) {
201
+ expect(e).to.have.property('message', 'throttlingActive_requestsIsDelayed')
202
+ expect(e).to.have.property('status', 900)
203
+ }
200
204
  })
201
205
  })
202
206
  })
203
- })
204
207
 
208
+ describe('Test section #5 - no limiting', function () {
209
+ describe('RATE LIMITER TEST', function() {
210
+ this.timeout(5000)
211
+ let req = {
212
+ options: {
213
+ controller: 'search',
214
+ action: 'search'
215
+ },
216
+ determinedIP: '1.2.3.4'
217
+ }
218
+
219
+ it('Reset Limiter', async() => {
220
+ await ratelimiter.resetLimiter()
221
+ })
205
222
 
206
- describe('Test section #5 - no limiting', function () {
207
- describe('RATE LIMITER TEST', function() {
208
- this.timeout(5000)
209
- let req = {
210
- options: {
211
- controller: 'search',
212
- action: 'search'
213
- },
214
- determinedIP: '1.2.3.4'
215
- }
216
-
217
- it('init tests', done => {
218
- ratelimiter.init(init)
219
- return done()
220
- })
223
+ it('should not trigger #1', async() => {
224
+ let result = await ratelimiter.limiter(req, options)
225
+ expect(result).eql(undefined)
226
+ })
221
227
 
222
- it('should not trigger #1', done => {
223
- ratelimiter.limiter(req, options, (err) => {
224
- expect(err).eql(null)
225
- return done()
228
+ it('should not trigger #2', async() => {
229
+ let result = await ratelimiter.limiter(req, options)
230
+ expect(result).eql(undefined)
226
231
  })
227
- })
228
232
 
229
- it('should not trigger #2', done => {
230
- ratelimiter.limiter(req, options, (err) => {
231
- expect(err).eql(null)
232
- return done()
233
+ it('should not trigger #3', async() => {
234
+ let result = await ratelimiter.limiter(req, options)
235
+ expect(result).eql(undefined)
236
+ })
237
+
238
+ it('should not trigger #4', async() => {
239
+ let result = await ratelimiter.limiter(req, options)
240
+ expect(result).eql(undefined)
233
241
  })
234
- })
235
242
 
236
- it('should not trigger #3', done => {
237
- ratelimiter.limiter(req, options, (err) => {
238
- expect(err).eql(null)
239
- return done()
243
+ it('should not trigger #5', async() => {
244
+ let result = await ratelimiter.limiter(req, options)
245
+ expect(result).eql(undefined)
240
246
  })
241
247
  })
248
+ })
249
+
250
+ describe('Test section #6 - ignore ignorePrivateIps', function () {
251
+ describe('RATE LIMITER TEST - ignorePrivateIps', function() {
252
+ this.timeout(5000)
242
253
 
243
- it('should not trigger #4', done => {
244
- ratelimiter.limiter(req, options, (err) => {
245
- expect(err).eql(null)
246
- return done()
254
+ let req = {
255
+ options: {
256
+ controller: 'user',
257
+ action: 'find'
258
+ },
259
+ determinedIP: '4.1.4.1'
260
+ }
261
+
262
+ it('Reset Limiter', async() => {
263
+ await ratelimiter.resetLimiter()
264
+ })
265
+
266
+ it('Update settings', async() => {
267
+ await ratelimiter.updateLimiter({
268
+ routes: [
269
+ { route: 'user/find', limit: 0, expires: 3, delay: 250 }
270
+ ],
271
+ ignorePrivateIps: true
272
+ })
273
+ })
274
+
275
+ it('should trigger immediately', async() => {
276
+ try {
277
+ await ratelimiter.limiter(req, options)
278
+ }
279
+ catch(e) {
280
+ expect(e).to.have.property('message', 'tooManyRequestsFromThisIP')
281
+ expect(e).to.have.property('status', 429)
282
+ }
283
+ })
284
+
285
+ it('should not trigger', async() => {
286
+ req.determinedIP = '127.0.0.1'
287
+ let result = await ratelimiter.limiter(req, options)
288
+ expect(result).eql(undefined)
289
+ })
290
+
291
+ it('should not trigger - IPv6', async() => {
292
+ req.determinedIP = '::ffff:127.0.0.1'
293
+ let result = await ratelimiter.limiter(req, options)
294
+ expect(result).eql(undefined)
247
295
  })
248
296
  })
297
+ })
298
+
299
+ describe('Test section #7 - send rateLimitCounter with request', function() {
300
+ describe('RATE LIMITER TEST - send rateLimitCounter', function() {
301
+ this.timeout(5000)
249
302
 
250
- it('should not trigger #5', done => {
251
- ratelimiter.limiter(req, options, (err) => {
252
- expect(err).eql(null)
253
- return done()
303
+ it('Reset Limiter', async() => {
304
+ await ratelimiter.resetLimiter()
254
305
  })
306
+
307
+ it('Update settings', async() => {
308
+ await ratelimiter.updateLimiter({
309
+ routes: [
310
+ { route: 'user/find', limit: 2, expires: 3, delay: 250 }
311
+ ]
312
+ })
313
+ })
314
+
315
+ it('should not trigger', async() => {
316
+ let result = await ratelimiter.limiter(req, { rateLimitCounter: 0 })
317
+ expect(result).eql(undefined)
318
+ })
319
+
320
+ it('should trigger immediately', async() => {
321
+ try {
322
+ await ratelimiter.limiter(req, { rateLimitCounter: 100 })
323
+ }
324
+ catch(e) {
325
+ expect(e).to.have.property('message', 'tooManyRequestsFromThisIP')
326
+ expect(e).to.have.property('status', 429)
327
+ }
328
+ })
329
+
330
+
255
331
  })
256
332
  })
333
+
257
334
  })
258
335
 
336
+ describe('Use Redis', () => {
337
+
338
+ after(() => {
339
+ redis.quit()
340
+ })
259
341
 
260
- describe('Test section #6 - ignore local ips', function () {
261
- describe('RATE LIMITER TEST - LOCAL IPS', function() {
342
+ describe('Use Redis for ratelimiter', function() {
262
343
  this.timeout(5000)
263
344
 
264
- for (let i=0; i<1000; i++) {
265
- it('init tests', done => {
266
- req.determinedIP = '4.1.4.1'
267
- init.routes = [
268
- { route: 'user/find', limit: 0, expires: 3, delay: 250 },
269
- ]
270
- ratelimiter.init(init)
271
- return done()
345
+ it('Update settings', async() => {
346
+ await ratelimiterRedis.updateLimiter({
347
+ routes: [
348
+ { route: 'user/find', limit: 0, expires: 2, delay: 250 },
349
+ ],
272
350
  })
273
- }
351
+ })
274
352
 
275
- it('should trigger immediately', done => {
276
- ratelimiter.limiter(req, options, (err) => {
277
- expect(err).to.have.property('status', 429)
278
- return done()
279
- })
353
+ it('should trigger immediately', async() => {
354
+ req.determinedIP = '4.1.4.1'
355
+ try {
356
+ let x = await ratelimiterRedis.limiter(req, options)
357
+ console.log(304, x)
358
+ }
359
+ catch(e) {
360
+ expect(e).to.be.an('error')
361
+ expect(e).to.have.property('message', 'tooManyRequestsFromThisIP')
362
+ expect(e).to.have.property('status', 429)
363
+ }
280
364
  })
281
365
 
282
- it('should not trigger', done => {
283
- req.determinedIP = '127.0.0.1'
284
- ratelimiter.limiter(req, options, (err) => {
285
- expect(err).eql(null)
286
- return done()
287
- })
366
+ it('wait for rate limiter to reset', async() => {
367
+ await setTimeout(3000)
288
368
  })
289
369
 
290
- it('should not trigger - IPv6', done => {
291
- req.determinedIP = '::ffff:127.0.0.1'
292
- ratelimiter.limiter(req, options, (err) => {
293
- expect(err).eql(null)
294
- return done()
370
+ it('Update settings', async() => {
371
+ await ratelimiterRedis.updateLimiter({
372
+ routes: [
373
+ { route: 'user/find', throttleLimit: 1, limit: 2, expires: 3, delay: 250 },
374
+ ],
295
375
  })
296
376
  })
377
+
378
+ it('req #1 should not trigger', async() => {
379
+ let result = await ratelimiterRedis.limiter(req, options)
380
+ expect(result).eql(undefined)
381
+ })
382
+
383
+ it('req #2 should not trigger throttle warning', async() => {
384
+
385
+ try {
386
+ await ratelimiterRedis.limiter(req, options)
387
+ }
388
+ catch(e) {
389
+ expect(e).to.be.an('error')
390
+ expect(e).to.have.property('message', 'finalThrottlingActive_requestsIsDelayed')
391
+ expect(e).to.have.property('status', 900)
392
+ expect(e.additionalInfo).to.have.property('counter', 2)
393
+ }
394
+ })
395
+
396
+ it('req #3 should not trigger the limiter - the throttling has reset the limiter', async() => {
397
+ let result = await ratelimiterRedis.limiter(req, options)
398
+ expect(result).eql(undefined)
399
+ })
400
+
297
401
  })
402
+
298
403
  })