analytics-platform 0.3.4__tar.gz → 0.3.5__tar.gz

{analytics_platform-0.3.4 → analytics_platform-0.3.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: analytics-platform
-Version: 0.3.4
+Version: 0.3.5
 Summary: Firewall, error monitoring and server analytics for your Python server, run from your analytics dashboard.
 Author: steviec87
 License-Expression: MIT
@@ -106,6 +106,7 @@ Celery's `task_always_eager`.
   process of its own, so it sends after every job; only the worker checks
   in (Health → All systems), not a process per job.
 - Don't also wrap a task in `platform.job()`: it would be timed twice.
+- Each queue's backlog: [`platform.watch_queue(...)`](#job-queues).
 
 ## Modules
 
@@ -121,8 +122,8 @@ Celery's `task_always_eager`.
 | `maintenance` | A "back soon" page while maintenance mode is on in the dashboard | on |
 | `headers` | Adds the security headers you pick on the Security page; your own headers win | on |
 | `forms` | Checks forms before you accept them: `platform.check_form(request)` | on |
-| `performance` | Response times and errors of every route (Health → Server speed): Django, Flask and FastAPI route patterns, and the database queries each one ran (below) | on |
-| `crawlers` | Which search engines and AI bots fetch your pages (Health → Crawlers). Django counts them for you; elsewhere call `platform.record_crawler(...)` | on |
+| `performance` | Response times and errors of every route (Health → Server speed): Django, Flask and FastAPI route patterns, the database queries each one ran, the other services it waited on, and your job queues' backlog (below) | on |
+| `crawlers` | Which search engines and AI bots fetch your pages (Off-site → Crawlers). Django counts them for you; elsewhere call `platform.record_crawler(...)` | on |
 | `inventory` | The installed packages and Python version, once a day, so the Security check can flag known security holes (`{"host": ...}`: the server's name) | on |
 | `posture` | How the server is set up, once a day, for the Security check: Django's `DEBUG`, `SECRET_KEY`, `ALLOWED_HOSTS` and cookie settings (when it's a Django site), whether the app runs as root, code anyone on the machine can write, and files in the folders you serve to the web that shouldn't be public (a dump, a `.env`, a `.bak`). The dashboard then asks your site for each of those files and tells you whether it really is downloadable. Reads only, and sends no file's contents. `{"roots": ["public", {"dir": "uploads", "prefix": "files"}]}` (default: Django's `STATIC_ROOT` and `MEDIA_ROOT`, reported at their `STATIC_URL` / `MEDIA_URL`), `{"checks": fn}` for checks of your own (protocol: `sdk/protocols/posture.md`) | on |
 
@@ -275,15 +276,26 @@ Several services on one site: `Platform(..., server_name="worker")`.
 **Several apps on one site** (microservices, a worker, a Node service next to
 this one): install the package in each, with the same `site_id`, `secret` and
 `endpoint`, and a `server_name` of its own. Errors, Server speed, Crawlers and
-Security → Updates then tell the apps apart, and Health → All systems lists
+Health → Updates then tell the apps apart, and Health → All systems lists
 each with a light: the package checks in about once a minute, so an app that
 stops reads "Stopped" within ten minutes (`sdk/protocols/checkin.md`).
 Without one, each app is named
 after its machine: two apps on one machine are reported as one, and only the
 last package list sent is kept. A service only your other apps call takes
 `firewall=False, proxy=False, maintenance=False` (its requests all come from
-your own apps, so the rate limit would see one visitor); on a machine shared
-by several apps, keep `resources` on in only one of them.
+your own apps, so the rate limit would see one visitor).
+
+**Several apps on one machine**: leave `resources` on in all of them. The
+machine's CPU, memory and disks are shown once, and each app (each gunicorn
+or Celery worker process too) adds what is its own: its container's memory
+and CPU caps and how close it is to them, how often the cap throttled it,
+processes the kernel killed for memory, open files against the limit, the
+most requests in progress at once, and how long it has run. On Linux the
+machine's waiting time (`/proc/pressure`), connections refused before the
+app accepted them and the connection-tracking table come with them. Python
+sends no heap or garbage-collection figures. A figure the machine won't give
+(no cap, no PSI, not Linux) is left out, never sent as 0 (protocol:
+`sdk/protocols/resources.md`, "Limits").
 
 Options: `server_name`, `root` (traceback paths relative to it),
 `capture_uncaught`, `capture_logging`, `spool_dir` (where unsent errors wait;
@@ -304,6 +316,7 @@ read, so no data leaves it.
 | SQLAlchemy | nothing: every engine (Flask-SQLAlchemy too) |
 | pymongo | nothing, but **create the platform before your `MongoClient`**: pymongo only reports to clients made after the package is listening |
 | Any other DB-API driver (psycopg, psycopg2, mysqlclient, PyMySQL, sqlite3...) | `conn = platform.watch_database(psycopg.connect(...))`, and use `conn` as before |
+| ClickHouse: clickhouse-connect, clickhouse-driver | `client = platform.watch_database(clickhouse_connect.get_client(...))` (or a `clickhouse_driver.Client`), and use `client` as before |
 
 A query outside any request or job is counted as `JOB (background)`. Each
 query is counted once: when SQLAlchemy or Django runs it over a watched
@@ -311,6 +324,78 @@ connection, they count it and the connection doesn't. It is on with
 `performance`, and switched off with it in the dashboard; `queries=False`
 keeps Server speed and leaves queries out (protocol: `sdk/protocols/queries.md`).
 
+ClickHouse: clickhouse-connect's `query()` (and `query_df()`, `query_np()`,
+`query_arrow()`, `raw_query()`) reads the whole result before it returns, so
+its time is the result's too; `command()` and the inserts are timed until
+they answer. clickhouse-driver's `execute()` likewise. The streaming
+queries (`query_rows_stream()`, `execute_iter()`) aren't timed. Only the
+client you pass is wrapped; the library itself is left alone.
+
+## Outgoing calls
+
+Health → Server speed → Calls shows how long each route waited on other
+services (a payment provider, an email API, your own microservices) by the
+host it called (`api.stripe.com`), with how many calls failed. One line for
+each client, where you make it:
+
+```python
+session = platform.watch_http(requests.Session())     # requests
+client = platform.watch_http(httpx.Client())          # httpx; AsyncClient too
+
+# httpx, with calls that got no answer counted too:
+client = httpx.Client(transport=platform.watch_http(httpx.HTTPTransport()))
+```
+
+- **Only the host is sent.** A call's path, query string, headers and body
+  never leave your server: paths carry IDs and keys.
+- A call is timed from being sent to its response's headers arriving. A
+  call that got no answer (refused, timed out), or a 5xx, is a failed one; a
+  4xx is the other side's answer, not a failure. A call your code cancels
+  isn't counted.
+- requests: each adapter the session has mounted is wrapped, keeping its
+  retries and pool settings; mount your own adapters before watching.
+- httpx: `watch_http(client)` adds event hooks, which see every answered
+  call but not one that got no answer at all. A watched transport
+  (`AsyncHTTPTransport` for an `AsyncClient`) sees those too; with both on
+  one client a call is counted once.
+- urllib3 and `urllib.request` have no public hook, so their calls aren't
+  counted (and nothing of theirs is patched).
+- Calls are filed as queries are: under the request, the job, or
+  `JOB (background)`. The package's own calls to us are left out.
+- They ride on Server speed's switch; `calls=False` leaves them out
+  (protocol: `sdk/protocols/calls.md`).
+
+## Job queues
+
+Health → Server speed → Jobs shows each queue's backlog once a minute: jobs
+waiting, running, delayed and failed, how long the oldest has waited, and
+jobs added against jobs finished, so a queue falling behind shows before the
+emails go out an hour late.
+
+```python
+platform.watch_queue(celery_app)                 # Celery on a Redis broker; queues=["emails", ...] to name them
+platform.watch_queue(Queue("emails", connection=redis))   # RQ, in web servers and workers alike
+```
+
+- **Celery** has no backlog API, so the broker's own lists are read from
+  Redis (one `LLEN` per queue and priority). Celery can't say what is
+  running, delayed or failed, nor how long a message has waited, so those
+  are left out; each process counts the tasks it published and, in a
+  worker, the ones it finished (a retry isn't finished). With a RabbitMQ
+  broker nothing is sent from here: the server agent reads RabbitMQ's own
+  numbers.
+- **RQ**: the queue, its started, scheduled, deferred and failed job
+  registries, and when its first job was added. RQ has no hook for a job
+  being added; jobs finished are counted by `platform.rq_worker()`'s
+  worker, so watch the queue in the worker's code too.
+- Watching one queue in several processes is fine: the dashboard takes the
+  largest reading, never the sum. Its Redis is named by host, port and
+  database; never a password.
+- A reading that fails is a gap, not a zero. Readings run on a background
+  thread, never in a request.
+- They ride on Server speed's switch; `queues=False` leaves them out
+  (protocol: `sdk/protocols/queues.md`).
+
 ## Server analytics
 
 For what only your server knows for sure. Not pageviews: the tracker counts those.

{analytics_platform-0.3.4 → analytics_platform-0.3.5}/README.md

@@ -80,6 +80,7 @@ Celery's `task_always_eager`.
   process of its own, so it sends after every job; only the worker checks
   in (Health → All systems), not a process per job.
 - Don't also wrap a task in `platform.job()`: it would be timed twice.
+- Each queue's backlog: [`platform.watch_queue(...)`](#job-queues).
 
 ## Modules
 
@@ -95,8 +96,8 @@ Celery's `task_always_eager`.
 | `maintenance` | A "back soon" page while maintenance mode is on in the dashboard | on |
 | `headers` | Adds the security headers you pick on the Security page; your own headers win | on |
 | `forms` | Checks forms before you accept them: `platform.check_form(request)` | on |
-| `performance` | Response times and errors of every route (Health → Server speed): Django, Flask and FastAPI route patterns, and the database queries each one ran (below) | on |
-| `crawlers` | Which search engines and AI bots fetch your pages (Health → Crawlers). Django counts them for you; elsewhere call `platform.record_crawler(...)` | on |
+| `performance` | Response times and errors of every route (Health → Server speed): Django, Flask and FastAPI route patterns, the database queries each one ran, the other services it waited on, and your job queues' backlog (below) | on |
+| `crawlers` | Which search engines and AI bots fetch your pages (Off-site → Crawlers). Django counts them for you; elsewhere call `platform.record_crawler(...)` | on |
 | `inventory` | The installed packages and Python version, once a day, so the Security check can flag known security holes (`{"host": ...}`: the server's name) | on |
 | `posture` | How the server is set up, once a day, for the Security check: Django's `DEBUG`, `SECRET_KEY`, `ALLOWED_HOSTS` and cookie settings (when it's a Django site), whether the app runs as root, code anyone on the machine can write, and files in the folders you serve to the web that shouldn't be public (a dump, a `.env`, a `.bak`). The dashboard then asks your site for each of those files and tells you whether it really is downloadable. Reads only, and sends no file's contents. `{"roots": ["public", {"dir": "uploads", "prefix": "files"}]}` (default: Django's `STATIC_ROOT` and `MEDIA_ROOT`, reported at their `STATIC_URL` / `MEDIA_URL`), `{"checks": fn}` for checks of your own (protocol: `sdk/protocols/posture.md`) | on |
 
@@ -249,15 +250,26 @@ Several services on one site: `Platform(..., server_name="worker")`.
 **Several apps on one site** (microservices, a worker, a Node service next to
 this one): install the package in each, with the same `site_id`, `secret` and
 `endpoint`, and a `server_name` of its own. Errors, Server speed, Crawlers and
-Security → Updates then tell the apps apart, and Health → All systems lists
+Health → Updates then tell the apps apart, and Health → All systems lists
 each with a light: the package checks in about once a minute, so an app that
 stops reads "Stopped" within ten minutes (`sdk/protocols/checkin.md`).
 Without one, each app is named
 after its machine: two apps on one machine are reported as one, and only the
 last package list sent is kept. A service only your other apps call takes
 `firewall=False, proxy=False, maintenance=False` (its requests all come from
-your own apps, so the rate limit would see one visitor); on a machine shared
-by several apps, keep `resources` on in only one of them.
+your own apps, so the rate limit would see one visitor).
+
+**Several apps on one machine**: leave `resources` on in all of them. The
+machine's CPU, memory and disks are shown once, and each app (each gunicorn
+or Celery worker process too) adds what is its own: its container's memory
+and CPU caps and how close it is to them, how often the cap throttled it,
+processes the kernel killed for memory, open files against the limit, the
+most requests in progress at once, and how long it has run. On Linux the
+machine's waiting time (`/proc/pressure`), connections refused before the
+app accepted them and the connection-tracking table come with them. Python
+sends no heap or garbage-collection figures. A figure the machine won't give
+(no cap, no PSI, not Linux) is left out, never sent as 0 (protocol:
+`sdk/protocols/resources.md`, "Limits").
 
 Options: `server_name`, `root` (traceback paths relative to it),
 `capture_uncaught`, `capture_logging`, `spool_dir` (where unsent errors wait;
@@ -278,6 +290,7 @@ read, so no data leaves it.
 | SQLAlchemy | nothing: every engine (Flask-SQLAlchemy too) |
 | pymongo | nothing, but **create the platform before your `MongoClient`**: pymongo only reports to clients made after the package is listening |
 | Any other DB-API driver (psycopg, psycopg2, mysqlclient, PyMySQL, sqlite3...) | `conn = platform.watch_database(psycopg.connect(...))`, and use `conn` as before |
+| ClickHouse: clickhouse-connect, clickhouse-driver | `client = platform.watch_database(clickhouse_connect.get_client(...))` (or a `clickhouse_driver.Client`), and use `client` as before |
 
 A query outside any request or job is counted as `JOB (background)`. Each
 query is counted once: when SQLAlchemy or Django runs it over a watched
@@ -285,6 +298,78 @@ connection, they count it and the connection doesn't. It is on with
 `performance`, and switched off with it in the dashboard; `queries=False`
 keeps Server speed and leaves queries out (protocol: `sdk/protocols/queries.md`).
 
+ClickHouse: clickhouse-connect's `query()` (and `query_df()`, `query_np()`,
+`query_arrow()`, `raw_query()`) reads the whole result before it returns, so
+its time is the result's too; `command()` and the inserts are timed until
+they answer. clickhouse-driver's `execute()` likewise. The streaming
+queries (`query_rows_stream()`, `execute_iter()`) aren't timed. Only the
+client you pass is wrapped; the library itself is left alone.
+
+## Outgoing calls
+
+Health → Server speed → Calls shows how long each route waited on other
+services (a payment provider, an email API, your own microservices) by the
+host it called (`api.stripe.com`), with how many calls failed. One line for
+each client, where you make it:
+
+```python
+session = platform.watch_http(requests.Session())     # requests
+client = platform.watch_http(httpx.Client())          # httpx; AsyncClient too
+
+# httpx, with calls that got no answer counted too:
+client = httpx.Client(transport=platform.watch_http(httpx.HTTPTransport()))
+```
+
+- **Only the host is sent.** A call's path, query string, headers and body
+  never leave your server: paths carry IDs and keys.
+- A call is timed from being sent to its response's headers arriving. A
+  call that got no answer (refused, timed out), or a 5xx, is a failed one; a
+  4xx is the other side's answer, not a failure. A call your code cancels
+  isn't counted.
+- requests: each adapter the session has mounted is wrapped, keeping its
+  retries and pool settings; mount your own adapters before watching.
+- httpx: `watch_http(client)` adds event hooks, which see every answered
+  call but not one that got no answer at all. A watched transport
+  (`AsyncHTTPTransport` for an `AsyncClient`) sees those too; with both on
+  one client a call is counted once.
+- urllib3 and `urllib.request` have no public hook, so their calls aren't
+  counted (and nothing of theirs is patched).
+- Calls are filed as queries are: under the request, the job, or
+  `JOB (background)`. The package's own calls to us are left out.
+- They ride on Server speed's switch; `calls=False` leaves them out
+  (protocol: `sdk/protocols/calls.md`).
+
+## Job queues
+
+Health → Server speed → Jobs shows each queue's backlog once a minute: jobs
+waiting, running, delayed and failed, how long the oldest has waited, and
+jobs added against jobs finished, so a queue falling behind shows before the
+emails go out an hour late.
+
+```python
+platform.watch_queue(celery_app)                 # Celery on a Redis broker; queues=["emails", ...] to name them
+platform.watch_queue(Queue("emails", connection=redis))   # RQ, in web servers and workers alike
+```
+
+- **Celery** has no backlog API, so the broker's own lists are read from
+  Redis (one `LLEN` per queue and priority). Celery can't say what is
+  running, delayed or failed, nor how long a message has waited, so those
+  are left out; each process counts the tasks it published and, in a
+  worker, the ones it finished (a retry isn't finished). With a RabbitMQ
+  broker nothing is sent from here: the server agent reads RabbitMQ's own
+  numbers.
+- **RQ**: the queue, its started, scheduled, deferred and failed job
+  registries, and when its first job was added. RQ has no hook for a job
+  being added; jobs finished are counted by `platform.rq_worker()`'s
+  worker, so watch the queue in the worker's code too.
+- Watching one queue in several processes is fine: the dashboard takes the
+  largest reading, never the sum. Its Redis is named by host, port and
+  database; never a password.
+- A reading that fails is a gap, not a zero. Readings run on a background
+  thread, never in a request.
+- They ride on Server speed's switch; `queues=False` leaves them out
+  (protocol: `sdk/protocols/queues.md`).
+
 ## Server analytics
 
 For what only your server knows for sure. Not pageviews: the tracker counts those.

{analytics_platform-0.3.4 → analytics_platform-0.3.5}/analytics_platform/__init__.py

@@ -82,13 +82,15 @@ class Platform:
     "cloudflare"); debug (print send problems); per module True / False / {options}.
     client: the name this package goes by to the collector ("python", or
     "django", which the Django integration passes), so a request rule aimed
-    at one framework reaches it. queries=False keeps Server speed and leaves
-    database queries out of it."""
+    at one framework reaches it. queries=False, calls=False and queues=False
+    keep Server speed and leave database queries, outgoing calls or job
+    queues out of it."""
 
     def __init__(self, site_id, secret, endpoint, trust_proxy=None, debug=False, firewall=True, errors=True, analytics=True,
                  performance=True, crawlers=True, forms=True, headers=True, audit=True, flags=True, maintenance=True,
                  inventory=True,
-                 posture=True, resources=True, proxy=True, server_name=None, client="python", queries=True):
+                 posture=True, resources=True, proxy=True, server_name=None, client="python", queries=True,
+                 calls=True, queues=True):
         self.core = Core(site_id, secret, endpoint, debug, server_name=server_name, client=client)
         if not self.core.valid:
             print("[analytics-platform] site_id, secret and endpoint are required; nothing is sent and every request passes")
@@ -110,6 +112,8 @@ class Platform:
                 opts.setdefault("server_name", server_name)
             if name == "performance":
                 opts.setdefault("queries", queries)
+                opts.setdefault("calls", calls)
+                opts.setdefault("queues", queues)
             # Inventory and posture call it host, and are stored one per site
             # and host: without it two services on one machine overwrite each other's.
             if server_name and name in ("inventory", "posture"):
@@ -118,6 +122,9 @@ class Platform:
                 self._modules[name] = _load(name, self.core, opts)
             except Exception as error:
                 print("[analytics-platform] the %s module could not start and is off: %s" % (name, error))
+        # Requests in progress at once (the limits pass), from the request timing.
+        if "performance" in self._modules and "resources" in self._modules:
+            self._modules["resources"].inflight_peak = self._modules["performance"].take_inflight_peak
         self.core.start(list(self._modules))
         # At exit, what's waiting is sent within one deadline for every
         # module together (EXIT_DEADLINE): registered after the core's own
@@ -332,8 +339,9 @@ class Platform:
 
     def watch_database(self, connection, db=None):
         """A DB-API connection (psycopg, psycopg2, mysqlclient, PyMySQL,
-        sqlite3...) whose queries are timed for Server speed: use what it
-        returns in its place. db: "postgresql", "mysql"... when the driver
+        sqlite3...) or a ClickHouse client (clickhouse-connect,
+        clickhouse-driver) whose queries are timed for Server speed: use what
+        it returns in its place. db: "postgresql", "mysql"... when the driver
         doesn't say. Django, SQLAlchemy and pymongo need nothing of this."""
         queries = self._queries()
         if queries is None:
@@ -341,13 +349,43 @@ class Platform:
         from .queries import watch_database
         return watch_database(connection, db)
 
+    def watch_http(self, client):
+        """A requests Session, an httpx Client or AsyncClient, or an httpx
+        transport whose outgoing calls are timed for Server speed (by host
+        only): use what it returns. See analytics_platform.calls."""
+        if self._calls() is None:
+            return client
+        try:
+            from .calls import watch_http
+            return watch_http(client)
+        except Exception as error:
+            self.core.log("performance", "watch_http: its calls aren't timed:", error)
+            return client
+
+    def watch_queue(self, queue_or_app, queues=None):
+        """A Celery app (Redis broker; queues: the names to read, default
+        its configured ones) or an RQ Queue, read once a minute for Server
+        speed -> Jobs. Returns what it was given."""
+        watcher = self._queues_module()
+        return watcher.watch(queue_or_app, queues) if watcher is not None else queue_or_app
+
     def _queries(self):
         performance = self._modules.get("performance")
-        return performance.queries if performance else None
+        queries = performance.queries if performance else None
+        return queries if queries is not None and queries.has("queries") else None
+
+    def _calls(self):
+        performance = self._modules.get("performance")
+        queries = performance.queries if performance else None
+        return queries if queries is not None and queries.has("calls") else None
+
+    def _queues_module(self):
+        performance = self._modules.get("performance")
+        return performance.queues if performance else None
 
     def record_crawler(self, user_agent, path, status, started=None, size=0, ip=""):
         """Records a finished request when it came from a crawler or AI bot
-        (Health → Crawlers). started: time.perf_counter() when it arrived."""
+        (Off-site → Crawlers). started: time.perf_counter() when it arrived."""
         crawlers = self._modules.get("crawlers")
         if crawlers:
             try:
@@ -389,10 +427,12 @@ class Platform:
 
     # ---- Requests ----
 
-    def blocked(self, remote, headers, method, path):
+    def blocked(self, remote, headers, method, path, late=None):
         """(status, body, headers) when the firewall turns the request away,
         else None. headers: a dict with lowercase names. path: the request
-        target as sent, query string and all, not yet decoded (request_target)."""
+        target as sent, query string and all, not yet decoded (request_target).
+        late: a dict given "static" when the request is a static file let
+        through, for count_static() after the response."""
         firewall = self._modules.get("firewall")
         if not firewall:
             return None
@@ -403,6 +443,8 @@ class Platform:
             if decision["action"] == "block" and decision["rule"] == "challenge":
                 status, extra, body = firewall.challenge_page(ip, method)
                 return status, body, extra
+            if late is not None and decision.get("static") and decision["action"] != "block":
+                late["static"] = decision["static"]
         except Exception as error:
             self.core.log("firewall", "check failed, request passed:", error)
             return None
@@ -410,10 +452,24 @@ class Platform:
             return None
         extra = [("Content-Type", "text/plain; charset=utf-8"), ("Cache-Control", "no-store")]
         if decision["status"] == 429:
-            extra.append(("Retry-After", firewall.retry_after()))
-            return 429, b"Too Many Requests", extra
+            from .firewall import LIMIT_BODY
+            extra.append(("Retry-After", firewall.retry_after(decision["rule"])))
+            return 429, LIMIT_BODY if decision["rule"].startswith("limit:") else b"Too Many Requests", extra
         return 403, b"Forbidden", extra
 
+    def count_static(self, late, status, routed=False):
+        """After the response: a static-looking request blocked() let through
+        (late["static"]) counts toward the rate and the page limits after all
+        when it answered 400 or more or the router sent it to a page
+        (firewall.md, `rate`). Never raises."""
+        firewall = self._modules.get("firewall")
+        if not firewall or not late or not late.get("static"):
+            return
+        try:
+            firewall.count_static(late["static"], status, routed)
+        except Exception as error:
+            self.core.log("firewall", "could not count a static file:", error)
+
     def wsgi(self, app, timing=True):
         """WSGI middleware (Flask, Bottle, Pyramid, plain WSGI). timing=False
         leaves timing to the framework integration (Flask, which knows the route)."""
@@ -432,8 +488,9 @@ class Platform:
                 status, extra, body = answered
                 start_response("%d " % status, list(extra))
                 return [body]
+            late = {}
             blocked = platform.blocked(environ.get("REMOTE_ADDR", ""), headers, environ.get("REQUEST_METHOD", ""),
-                                       _wsgi_target(environ))
+                                       _wsgi_target(environ), late)
             if blocked:
                 status, body, extra = blocked
                 start_response("%d %s" % (status, _reason(status)), extra)
@@ -451,14 +508,31 @@ class Platform:
                     seen["status"] = int(str(status).split(" ", 1)[0])
                 except ValueError:
                     pass
+                seen["started"] = True
+                if late:
+                    seen["type"] = next((v for n, v in response_headers if str(n).lower() == "content-type"), "")
                 response_headers = list(response_headers) + platform.security_headers(n for n, _ in response_headers) + remember
                 return start_response(status, response_headers, exc_info) if exc_info else start_response(status, response_headers)
 
             trail = current_trail.set([])
             queries = _queries_start(platform)
+            inflight = _inflight_start(platform)
             try:
-                return app(environ, timed_start_response)
+                result = app(environ, timed_start_response)
+                # A static-looking request counts once its body is sent (the
+                # server closes the iterable): toward the rate and the page
+                # limits when it answered 400 or more, or Flask routed it to a
+                # view (init_flask leaves the word in the environ). Plain WSGI
+                # has no router to ask, so there only an error counts.
+                if not late or _nothing_to_count(environ, seen):
+                    # A file a page asked for, answered already: the server's
+                    # own file wrapper (sendfile) is left as it is.
+                    return result
+                return _Counted(result, lambda: platform.count_static(
+                    late, seen["status"], _wsgi_routed(environ, seen.get("type", ""))))
             except Exception as error:
+                if late:
+                    platform.count_static(late, 500)
                 platform.capture_exception(error, status=500, kind="error", request={
                     "method": environ.get("REQUEST_METHOD", ""), "path": environ.get("PATH_INFO", ""),
                     "session_id": headers.get(SESSION_HEADER, ""),
@@ -466,6 +540,7 @@ class Platform:
                 raise
             finally:
                 current_trail.reset(trail)
+                _inflight_end(inflight)
                 try:
                     if timing:
                         platform.time_request(environ.get("REQUEST_METHOD", ""), "", seen["status"], started, environ.get("PATH_INFO", ""))
@@ -505,6 +580,15 @@ class Platform:
 
         @app.after_request
         def _analytics_timed(response):
+            # The rule the request matched, for a static-looking request's
+            # count after the response (wsgi() above): Flask's own static
+            # endpoint (and a blueprint's) serves files.
+            try:
+                rule = getattr(request, "url_rule", None)
+                if rule is not None and rule.endpoint != "static" and not str(rule.endpoint).endswith(".static"):
+                    request.environ[_ROUTE_KEY] = rule.rule
+            except Exception:
+                pass
             started = getattr(g, "_analytics_started", None)
             if started is not None:
                 rule = getattr(request, "url_rule", None)
@@ -540,6 +624,20 @@ def _queries_start(platform):
     return current_queries.set(queries.request()) if queries else None
 
 
+def _inflight_start(platform):
+    """A request starts (not a job): counted as in progress until it ends."""
+    performance = (getattr(platform, "_modules", None) or {}).get("performance")
+    if performance is None:
+        return None
+    performance.request_started()
+    return performance
+
+
+def _inflight_end(performance):
+    if performance is not None:
+        performance.request_ended()
+
+
 def _queries_end(token):
     """The request or job has finished: queries after this are another's."""
     if token is not None:
@@ -650,6 +748,65 @@ class _Job:
         return run
 
 
+_ROUTE_KEY = "analytics_platform.route"
+
+
+def _wsgi_routed(environ, content_type):
+    """Whether the router sent a WSGI request to a page: the route a
+    framework integration left in the environ (init_flask), else none."""
+    from .firewall import routed_to_page
+    return routed_to_page(environ.get(_ROUTE_KEY), content_type)
+
+
+def _nothing_to_count(environ, seen):
+    """A WSGI response already started (its status known) that won't count
+    toward the rate and the page limits: under 400, and not routed to a page
+    (or a 304)."""
+    try:
+        if not seen.get("started"):
+            return False
+        status = seen["status"]
+        return status < 400 and (status == 304 or not _wsgi_routed(environ, seen.get("type", "")))
+    except Exception:
+        return False
+
+
+class _Counted:
+    """A WSGI response that runs `done` once the server has sent it (close),
+    passing the server's close() on. Only wraps a static-looking request."""
+
+    def __init__(self, result, done):
+        self._result = result
+        self._done = done
+
+    def __iter__(self):
+        return iter(self._result)
+
+    def close(self):
+        try:
+            close = getattr(self._result, "close", None)
+            if close is not None:
+                close()
+        finally:
+            done, self._done = self._done, None
+            if done:
+                done()
+
+
+def _asgi_routed(scope, content_type):
+    """Whether Starlette's router (FastAPI's too) sent the request to a
+    route: scope["route"] (FastAPI) or the endpoint Starlette left in the
+    scope. A StaticFiles mount serves files."""
+    from .firewall import routed_to_page
+    route = scope.get("route")
+    endpoint = scope.get("endpoint")
+    app = getattr(route, "app", None) if route is not None else endpoint
+    if type(app).__name__ == "StaticFiles" or type(endpoint).__name__ == "StaticFiles":
+        return False
+    pattern = getattr(route, "path", None) if route is not None else ("*" if endpoint is not None else None)
+    return routed_to_page(pattern, content_type)
+
+
 class PlatformMiddleware:
     """ASGI middleware: app.add_middleware(PlatformMiddleware, platform=platform)."""
 
@@ -695,7 +852,8 @@ class PlatformMiddleware:
             return
         # raw_path is the path as sent; servers that leave it out get path encoded again.
         target = request_target(scope.get("path", ""), scope.get("query_string", b""), scope.get("raw_path"))
-        blocked = self.platform.blocked(client[0] or "", headers, scope.get("method", ""), target)
+        late = {}
+        blocked = self.platform.blocked(client[0] or "", headers, scope.get("method", ""), target, late)
         if blocked:
             status, body, extra = blocked
             await send({"type": "http.response.start", "status": status,
@@ -716,6 +874,8 @@ class PlatformMiddleware:
         async def timed_send(message):
             if message.get("type") == "http.response.start":
                 seen["status"] = int(message.get("status") or 500)
+                if late:
+                    seen["type"] = next((v.decode("latin-1") for k, v in message.get("headers") or [] if k.lower() == b"content-type"), "")
                 present = [k.decode("latin-1") for k, _ in message.get("headers") or []]
                 extra = self.platform.security_headers(present)
                 if extra or remember:
@@ -731,8 +891,14 @@ class PlatformMiddleware:
         })
         trail = current_trail.set([])
         queries = _queries_start(self.platform)
+        inflight = _inflight_start(self.platform)
         try:
             await self.app(scope, receive, timed_send)
+        except asyncio.CancelledError:
+            # The visitor went away (a page left while its files load): no
+            # answer, and not the app's error.
+            seen["gone"] = True
+            raise
         except Exception as error:
             self.platform.capture_exception(error, status=500, kind="error", request={
                 "method": scope.get("method", ""), "path": scope.get("path", ""),
@@ -742,12 +908,21 @@ class PlatformMiddleware:
         finally:
             current_request.reset(token)
             current_trail.reset(trail)
+            _inflight_end(inflight)
             # The router sets scope["route"] (FastAPI, Starlette) on the way in.
             try:
                 self.platform.time_request(scope.get("method", ""), getattr(scope.get("route"), "path", "") or "",
                                            seen["status"], started, scope.get("path", ""))
             finally:
                 _queries_end(queries)
+                # A static-looking request, once its body is sent: counted when
+                # it answered 400 or more or the router sent it to a route.
+                if late and not (seen.get("gone") and "type" not in seen):
+                    try:
+                        routed = _asgi_routed(scope, seen.get("type", ""))
+                    except Exception:
+                        routed = False
+                    self.platform.count_static(late, seen["status"], routed)
 
 
 def create_platform(**options):