@appland/scanner 1.40.1 → 1.41.1

package/doc/rules/missingAuthentication.md

@@ -0,0 +1,59 @@
+---
+rule: missing-authentication
+name: Missing authentication
+title: Unauthenticated HTTP server request
+references:
+  CWE-306: https://cwe.mitre.org/data/definitions/306.html
+impactDomain: Security
+labels:
+  - public
+  - security.authentication
+scope: http_server_request
+---
+
+An HTTP server request is missing authentication. In this case, the request may be serving assets
+that should be protected by authentication - but no authentication is actually happening.
+
+### Rule logic
+
+This rule checks all HTTP server requests that satisfy the following conditions:
+
+- HTTP status code is `< 300`
+- Matches include and exclude lists of content type (by default, these are empty).
+
+For each matching request, any event that satisfies either of these conditions will satisfy the rule:
+
+1. Has label `public`.
+2. Has label `security.authentication`, and returns a truthy value.
+
+### Notes
+
+If a request does not require an authenticated user (e.g. because it contains completely public
+information), then this rule can be satisfied by calling any function labeled `public`.
+
+If the `security.authentication` event returns a falsey value (`false`, `null`, etc), then
+authentication is assumed to be denied, and the rule is not satisfied.
+
+### Resolution
+
+If the request is designed to be public, and the omission of authentication is intentionaly, modify
+the code so that it calls a function labeled `public`.
+
+Otherwise, modify the code so that it calls a function labeled `security.authentication` which
+returns a truthy result (for example, a User object).
+
+### Options
+
+- `includeContentTypes: ` [MatchPatternConfig](/docs/analysis/match-pattern-config.html)`[]`.
+  Default: empty - including all content types.
+- `excludeContentTypes: ` [MatchPatternConfig](/docs/analysis/match-pattern-config.html)`[]`.
+  Default: empty - excluding no content types.
+
+### Examples
+
+```yaml
+- rule: missingAuthentication
+  properties:
+    includeContentTypes:
+      - match: ^application/json
+```

package/doc/rules/missingContentType.md

@@ -0,0 +1,33 @@
+---
+rule: missing-content-type
+name: Missing content type
+title: HTTP server request without a Content-Type header
+impactDomain: Stability
+scope: http_server_request
+---
+
+Finds HTTP server requests that don't provide a `Content-Type` header in the response.
+
+### Rule logic
+
+Every HTTP server request is checked. If the status code indicates redirect or "No content", then
+the rule passes. Otherwise, a finding is issued if the response is missing a `Content-Type` header.
+
+### Notes
+
+When a response is missing the `Content-Type` header, it's unclear to clients how to handle the
+response data. Bugs are likely to result.
+
+### Resolution
+
+Provide a `Content-Type` in the response.
+
+### Options
+
+None
+
+### Examples
+
+```yaml
+- rule: missingContentType
+```

package/doc/rules/nPlusOneQuery.md

@@ -0,0 +1,52 @@
+---
+rule: n-plus-one-query
+name: N plus one query
+title: N plus 1 SQL query
+references:
+  CWE-1073: https://cwe.mitre.org/data/definitions/1073.html
+impactDomain: Performance
+scope: command
+---
+
+
+Finds occurrences of a query being repeated within a loop.
+
+### Rule logic
+
+Within each command, SQL queries are normalized, grouped, and counted.
+
+If the number of duplicate instances of a normalized query exceeds the threshold, it's reported as a
+finding.
+
+### Notes
+
+N plus one queries typically occur when a
+[data access object (DAO)](https://en.wikipedia.org/wiki/Data_access_object) has a one-to-many or
+many-to-many relationship, and the relationship is enumerated within a loop. The DAO will issue a
+separate query to fetch each related record. If the number of related objects is large, or if each
+one is fairly expensive to fetch, application performance suffers.
+
+### Resolution
+
+DAO libraries typically offer a feature called "eager loading". For example:
+
+- [ActiveRecord](https://news.learnenough.com/eager-loading)
+- [Django ORM](https://docs.djangoproject.com/en/4.0/topics/db/optimization/#retrieve-everything-at-once-if-you-know-you-will-need-it)
+- [Hibernate](https://docs.jboss.org/hibernate/orm/5.3/javadocs/org/hibernate/jpamodelgen/xml/jaxb/FetchType.html)
+
+Enable eager loading on the association in question to fetch the data efficiently, without repeated,
+identical queries.
+
+### Options
+
+- `warningLimit` Threshold for reporting a warning. Default: 5.
+- `errorLimit` Threshold for reporting an error. Default: 10.
+
+### Examples
+
+```yaml
+- rule: nPlusOneQuery
+  parameters:
+    warningLimit: 5
+    errorLimit: 10
+```

package/doc/rules/queryFromInvalidPackage.md

@@ -0,0 +1,45 @@
+---
+rule: query-from-invalid-package
+name: Query from invalid package
+title: Queries from invalid packages
+references:
+  CWE-1057: https://cwe.mitre.org/data/definitions/1057.html
+impactDomain: Maintainability
+---
+
+Ensures that SQL queries are made only from an approved list of packages. This helps to make the
+code more maintainable by encapsulating access to the database.
+
+### Notes
+
+When too many packages are performing direct SQL queries, it's very hard to keep the code modular.
+The result is a lack of coherent design and degredation in maintainability.
+
+### Rule logic
+
+All functions which make SQL queries are inspected. Each one must match one of the
+`allowedPackages`.
+
+### Resolution
+
+The code which is making the direct query should be refactored to use the database through one of
+the allowed packages.
+
+### Options
+
+- `allowedPackages: `[MatchPatternConfig](/docs/analysis/match-pattern-config.html)`[]`. Packages
+  which are allowed to make queries. Required.
+- `allowedQueries: `[MatchPatternConfig](/docs/analysis/match-pattern-config.html)`[]`. Queries which
+  are allowed from anywhere. Default:
+  `[/\bBEGIN\b/i, /\bCOMMIT\b/i, /\bROLLBACK\b/i, /\bRELEASE\b/i, /\bSAVEPOINT\b/i]`.
+
+### Examples
+
+```yaml
+- rule: queryFromInvalidPackage
+  properties:
+    callerPackages:
+      - equal: actionpack
+    calleePackage:
+      equal: app/controllers
+```

package/doc/rules/queryFromView.md

@@ -0,0 +1,42 @@
+---
+rule: query-from-view
+name: Query from view
+title: Queries from view
+references:
+  CWE-1057: https://cwe.mitre.org/data/definitions/1057.html
+impactDomain: Maintainability
+---
+
+Ensures that SQL queries are not performed directly from the view layer. This helps to make the code
+more maintainable by encapsulating access to the database.
+
+### Notes
+
+Performing SQL queries directly from the view layer introduces several maintainability concerns:
+
+1. View logic is tied to the server, and cannot be refactored to the client side.
+2. Database interactions are harder to test, because they depend on details of the view
+   implementation.
+3. Performance can be adversely and unexpectedly affected by minor changes to the view.
+
+### Rule logic
+
+Each query is tested to see if it has an ancestor event with the view label.
+
+### Resolution
+
+Data objects which are passed to the view layer for rendering should not have access to the
+database. Disable database access in some way, or transfer data from DAO objects into plain old
+structs.
+
+### Options
+
+- `forbiddenLabel`. Label which identifies the view layer. Default: `mvc.template`.
+
+### Examples
+
+```yaml
+- rule: queryFromView
+  properties:
+    forbiddenLabel: mvc.template
+```

package/doc/rules/rpcWithoutCircuitBreaker.md

@@ -0,0 +1,44 @@
+---
+rule: rpc-without-circuit-breaker
+name: RPC without circuit breaker
+title: RPC without circuit breaker
+impactDomain: Stability
+labels:
+  - rpc.circuit_breaker
+---
+
+Identifies HTTP client requests which do not utilize a
+[circuit breaker](https://martinfowler.com/bliki/CircuitBreaker.html).
+
+### Rule logic
+
+Each HTTP client request is expected to have a descendant labeled with the expected label.
+
+### Notes
+
+Use the circuit breaker pattern in microservices architecture to make system behavior more
+predictable when a service becomes overloaded or unavailable.
+
+### Resolution
+
+Utilize a circuit breaker library - your organization may have a specific preference.
+
+Some examples:
+
+- [Hystrix (Java)](https://github.com/Netflix/Hystrix/wiki/How-it-Works#CircuitBreaker)
+- [CircuitBox (Ruby)](https://github.com/yammer/circuitbox)
+- [Semian (Ruby)](https://github.com/Shopify/semian#circuit-breaker)
+- [pybreaker (Python)](https://github.com/danielfm/pybreaker)
+
+### Options
+
+- `expectedLabel`. Label which identifies the circuit breaker function. Default:
+  `rpc.circuit_breaker`.
+
+### Examples
+
+```yaml
+- rule: rpcWithoutCircuitBreaker
+  properties:
+    expectedLabel: rpc.circuit_breaker
+```

package/doc/rules/saveWithoutValidation.md

@@ -0,0 +1,33 @@
+---
+rule: save-without-validation
+name: Save without validation
+title: Save without validation
+references:
+  CWE-20: https://cwe.mitre.org/data/definitions/20.html
+impactDomain: Stability
+---
+
+Ensures that data saved by data access object is validated first.
+
+### Rule logic
+
+Finds events whose method name is `save` or `save!`. Then verifies that each of these events has a
+descendant whose method name is `valid?` or `validate!`.
+
+### Notes
+
+In a future revision, this rule will be refactored to use labels rather than method names.
+
+### Resolution
+
+Ensure that data is validated before being saved; for example, using a `before_save` hook.
+
+### Options
+
+None
+
+### Examples
+
+```yaml
+- rule: saveWithoutValidation
+```

package/doc/rules/secretInLog.md

@@ -0,0 +1,49 @@
+---
+rule: secret-in-log
+name: Secret in log
+title: Secret in log
+references:
+  CWE-532: https://cwe.mitre.org/data/definitions/532.html
+impactDomain: Security
+labels:
+  - secret
+  - log
+---
+
+Identifies when a known or assumed secret is written to a log. Logs are often transported into other
+systems that are treated with lesser security - such as backups. Therefore, secrets written into log
+files are more likely to be leaked or discovered by cyber-attackers.
+
+### Rule logic
+
+Operation of this rule depends on two labels: `secret` and `log`. A function labeled `secret` is
+assumed to return a secret value - this rule keeps a running tally of all the secrets created in an
+AppMap. When a function labeled `log` is called, the rule looks at the log message, and checks
+whether any of the known secrets are in it.
+
+The log message is also tested against a list of known regular expressions that are likely to match
+secrets.
+
+### Notes
+
+Be sure and apply the `secret` label to any function in your code base that generates a secret (such
+as encryption keys, user tokens, API keys, reset tokens, etc.).
+
+### Resolution
+
+If the log message is written by code that you control, the simplest resolution is to remove the log
+statement - or to remove the secret from the log message.
+
+If the log message is not controlled by you (it's generated by a framework), your choices are more
+limited. You could open a pull request with the framework, or you can change the log level to
+suppress the offending message.
+
+### Options
+
+None
+
+### Examples
+
+```yaml
+- rule: secretInLog
+```

package/doc/rules/slowFunctionCall.md

@@ -0,0 +1,39 @@
+---
+rule: slow-function-call
+name: Slow function call
+title: Slow function call
+impactDomain: Performance
+scope: root
+---
+
+Ensures that function elapsed time does not exceed a threshold.
+
+### Rule logic
+
+Checks all configured functions to see if the elapsed time exceeds the configured threshold.
+
+### Notes
+
+This rule is most useful when applied to code that is specifically designed to test application
+performance.
+
+### Resolution
+
+Optimize the function elapsed time using a profiler, SQL tuning, etc.
+
+### Options
+
+- `functions: `[MatchPatternConfig](/docs/analysis/match-pattern-config.html)`[]` list of functions
+  to check. Required.
+- `timeAllowed` max time (in seconds) allowed for the function.
+
+### Examples
+
+```yaml
+- rule: slowFunctionCall
+  properties:
+    timeAllowed: 0.25
+    functions:
+      - match: ^app/models
+      - match: ^app/jobs
+```

package/doc/rules/slowHttpServerRequest.md

@@ -0,0 +1,34 @@
+---
+rule: slow-http-server-request
+name: Slow HTTP server request
+title: Slow HTTP server request
+impactDomain: Performance
+scope: http_server_request
+---
+
+Ensures that HTTP server request elapsed time does not exceed a threshold.
+
+### Rule logic
+
+Checks HTTP server requests to see if the elapsed time exceeds the configured threshold.
+
+### Notes
+
+This rule is most useful when applied to code that is specifically designed to test application
+performance.
+
+### Resolution
+
+Optimize the request elapsed time using a profiler, SQL tuning, etc.
+
+### Options
+
+- `timeAllowed` max time (in seconds) allowed for the request.
+
+### Examples
+
+```yaml
+- rule: slowHttpServerRequest
+  properties:
+    timeAllowed: 0.25
+```

package/doc/rules/slowQuery.md

@@ -0,0 +1,33 @@
+---
+rule: slow-query
+name: Slow query
+title: Slow SQL query
+impactDomain: Performance
+---
+
+Ensures that SQL query elapsed time does not exceed a threshold.
+
+### Rule logic
+
+Checks all configured queries to see if the elapsed time exceeds the configured threshold.
+
+### Notes
+
+This rule is most useful when applied to code that is specifically designed to test application
+performance.
+
+### Resolution
+
+Optimize the elapsed time using SQL tuning.
+
+### Options
+
+- `timeAllowed` max time (in seconds) allowed for the query.
+
+### Examples
+
+```yaml
+- rule: slowQuery
+  properties:
+    timeAllowed: 0.25
+```

package/doc/rules/tooManyJoins.md

@@ -0,0 +1,40 @@
+---
+rule: too-many-joins
+name: Too many joins
+title: Too many joins
+references:
+  CWE-1049: https://cwe.mitre.org/data/definitions/1049.html
+impactDomain: Performance
+scope: command
+---
+
+Verifies that the number of joins in SQL queries does not exceed a threshold.
+
+### Rule logic
+
+Counts the number of joins in each SQL query. If the count exceeds the configured threshold, a
+finding is reported.
+
+### Notes
+
+Queries with too many joins often have poor or unpredictable performance.
+
+### Resolution
+
+Having multiple joins may be legitimate - but it is a situation that merits a closer look,
+especially when the query is generated by an ORM.
+
+Take a look at a query plan from a database that has a realistically high volume of data, and verify
+that the query can be executed efficiently.
+
+### Options
+
+- `warningLimit` the maximum number of joins allowed.
+
+### Examples
+
+```yaml
+- rule: tooManyJoins
+  properties:
+    warningLimit: 5
+```

package/doc/rules/tooManyUpdates.md

@@ -0,0 +1,46 @@
+---
+rule: too-many-updates
+name: Too many updates
+title: Too many SQL and RPC updates performed in one command
+references:
+  CWE-1048: https://cwe.mitre.org/data/definitions/1048.html
+impactDomain: Maintainability
+scope: command
+---
+
+Verifies that the number of SQL and RPC updates performed by a command does not exceed a threshold.
+
+### Rule logic
+
+Counts the number of SQL and RPC updates in each command. A SQL update is any `INSERT` or `UPDATE`
+query. An RPC update is an HTTP client request that uses `PUT`, `POST`, or `PATCH`.
+
+If the number of updates exceeds the threshold, a finding is reported.
+
+### Notes
+
+As a codebase evolves, sometimes a request can start to make more and more SQL and RPC updates.There
+are several negative repercussions of this:
+
+1. It's no longer clear to a developer what the primary responsibility of the command is.
+2. The command becomes more likely to fail - resulting in a rollback of all the updates, or possibly
+   leaving the system in an inconsistent state.
+3. The performance of the command degrades as it does more and more work.
+
+### Resolution
+
+Consider refactoring the command into multiple commands.
+
+Schedule a job to perform some of the work offline.
+
+### Options
+
+- `warningLimit` the maximum number of joins allowed.
+
+### Examples
+
+```yaml
+- rule: tooManyUpdates
+  properties:
+    warningLimit: 20
+```

package/doc/rules/unbatchedMaterializedQuery.md

@@ -0,0 +1,54 @@
+---
+rule: unbatched-materialized-query
+name: Unbatched materialized query
+title: Unbatched materialized SQL query
+references:
+  CWE-1049: https://cwe.mitre.org/data/definitions/1049.html
+impactDomain: Performance
+labels:
+  - dao.materialize
+scope: command
+---
+
+Finds large data sets that are queried from the database and loaded into memory.
+
+### Rule logic
+
+Examines all SQL SELECT queries (as opposed to insert/update). If the query satisfies any of the
+following conditions:
+
+- Has a LIMIT clause
+- Has a COUNT clause at the top level
+- Queries only for metadata (`sqlite_master` table)
+
+then the query is skipped.
+
+Otherwise, the rule checks to see if the query has an ancestor labeled `dao.materialize`. If so,
+it's emitted as a finding.
+
+### Notes
+
+Materializing large amounts of data code objects is a frequent cause of poor performance and memory
+exhaustion.
+
+A `COUNT` or `LIMIT` clause is a good indication that the code is taking steps to limit the amount
+of data that's loaded into memory.
+
+### Resolution
+
+If data is being loaded into memory from an un-LIMITed query:
+
+- Consider whether the data processing that's being performed in-memory can be performed in the
+  database - either via SQL or a stored procedure.
+- If the data is being loaded solely for presentation purposes, fetch data in batches - e.g. with a
+  pagination library.
+
+### Options
+
+None
+
+### Examples
+
+```yaml
+- rule: unbatchedMaterializedQuery
+```

package/doc/rules/updateInGetRequest.md

@@ -0,0 +1,44 @@
+---
+rule: update-in-get-request
+name: Update in get request
+title: Data update performed in GET or HEAD request
+impactDomain: Maintainability
+labels:
+  - audit
+scope: http_server_request
+---
+
+Finds SQL updates that are performed in an HTTP server `GET` request.
+
+### Rule logic
+
+Checks each HTTP server `GET` and `HEAD` request. Within each of these requests, checks for SQL
+queries that match the `queryInclude` option and don't match `queryExclude`. If any such queries
+exist, they are emitted as findings.
+
+### Notes
+
+Performing data updates in a `GET` request is anti-pattern, and counter to the intent of HTTP. For
+data modifications, use `PUT`, `POST`, or `PATCH`.
+
+Data updates which are used for the purposes of tracking user activity are fine in any type of
+request. Use the `queryExclude` option to allow these queries.
+
+### Resolution
+
+Perform data updates in `PUT`, `POST`, or `PATCH` requests.
+
+### Options
+
+- `queryInclude: RegExp[]`. Default: `[/\binsert\b/i, /\bupdate\b/i]`.
+- `queryExclude: RegExp[]`. Default: empty.
+
+### Examples
+
+```yaml
+- rule: updateInGetRequest
+  properties:
+    queryExclude:
+      - /\bINSERT\b/i
+      - /\bUPDATE\b/i
+```

package/package.json

@@ -1,10 +1,11 @@
 {
   "name": "@appland/scanner",
-  "version": "1.40.1",
+  "version": "1.41.1",
   "description": "",
   "bin": "built/cli.js",
   "files": [
-    "built"
+    "built",
+    "doc"
   ],
   "scripts": {
     "build": "mkdir -p built && cp -r src/sampleConfig built && tsc && yarn schema && yarn doc",