activity-frames 0.1.0__tar.gz

activity_frames-0.1.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,21 @@
+---
+name: Bug report
+about: Something compiled wrong or crashed
+title: ""
+labels: bug
+---
+
+**What happened:**
+
+**What you expected:**
+
+**Reproduce:**
+- Command or API call:
+- `aframes --version`:
+- OS and Python version:
+
+**Output** (redact anything sensitive - and note that `--include-text` is off by
+default, so paste only what you are comfortable sharing):
+
+```
+```

activity_frames-0.1.0/.github/ISSUE_TEMPLATE/parser_request.md

@@ -0,0 +1,20 @@
+---
+name: Site parser request
+about: Ask for (or propose) typing support for a website
+title: "Parser: <site>"
+labels: parser
+---
+
+**Site**: <e.g. news.ycombinator.com>
+
+**Example URLs and what they should become:**
+
+| URL | kind | entity |
+|-----|------|--------|
+| https://news.ycombinator.com/item?id=123 | post | 123 |
+| https://news.ycombinator.com/user?id=pg | profile | pg |
+
+**Notes** (path structure, query params that matter, anything tricky):
+
+Parsers are pure functions of the URL and take about 15 minutes to add - see
+[CONTRIBUTING.md](../../CONTRIBUTING.md). PRs very welcome.

activity_frames-0.1.0/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,7 @@
+## What this changes
+
+## Checklist
+- [ ] Output stays deterministic (no clock, network, randomness, or model calls)
+- [ ] New behavior has a test; `pytest -q` passes
+- [ ] New schema fields, if any, are measured - or live in the tier-2 inferred block with a confidence tag
+- [ ] No typed text or page content is emitted by default

activity_frames-0.1.0/.github/workflows/test.yml

@@ -0,0 +1,22 @@
+name: tests
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.9", "3.11", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: pip install -e ".[dev]"
+      - run: pytest -q

activity_frames-0.1.0/.gitignore

@@ -0,0 +1,11 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+venv/
+dist/
+build/
+.pytest_cache/
+.DS_Store
+*.sqlite
+*.db

activity_frames-0.1.0/ACKNOWLEDGMENTS.md

@@ -0,0 +1,8 @@
+# Acknowledgments
+
+The default capture engine provisioned by `aframes record` is a pinned
+build of screenpipe v0.3.324 (Mediar AI), released under the MIT
+license. It is downloaded on demand from the public npm registry and is
+not bundled with this package. The compiler itself is engine-agnostic:
+any capture system writing a compatible SQLite schema works via
+`$AFRAMES_DB`.

activity_frames-0.1.0/CHANGELOG.md

@@ -0,0 +1,39 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/), and versions follow semantic
+versioning. The document schema version is tracked separately in [SPEC.md](SPEC.md).
+
+## [0.1.0] - 2026-07-04
+
+Initial release.
+
+### Added
+- **Schema v1** ([SPEC.md](SPEC.md)): two-tier document format separating measured
+  fields from an optional confidence-tagged inferred tier, with coverage gaps,
+  blind spots, and evidence pointers as first-class elements.
+- **Deterministic compiler**: dwell-capped sessionization (per capture device,
+  so multi-monitor streams never fragment each other), session-gap detection,
+  flicker merging with interruption records, and single-assignment input
+  accounting across overlapping monitor segments.
+- **Enrichment library API** (`activity_frames.enrich`): nearest-frame app
+  attribution, coordinate-based click resolution against the recorded element
+  tree (with neighbor-frame rescue), and optional keyboard-layout decoding.
+- **Entity typing**: deterministic URL parsers for LinkedIn, GitHub, Google
+  (Search/Docs/Gmail/Maps/Meet/Calendar), YouTube, X, Instagram, Reddit, Luma,
+  Partiful, Product Hunt, Vercel, Supabase, Stripe, Discord, Notion, Figma,
+  Stack Overflow, Calendly, and AI-chat sites, plus a subdomain/path heuristic
+  layer (sign-in, dashboard, email, calendar, meeting) and a total generic
+  fallback.
+- **Built-in capture**: `aframes record` provisions and runs a pinned,
+  MIT-licensed screenpipe build, sha512-verified before first run (audio off
+  by default; `--status` checks that frames are actually flowing and points
+  at macOS permissions when they are not).
+- **MCP server**: zero-dependency stdio JSON-RPC server exposing `get_context`,
+  `get_activity`, `get_day_summary`, and `get_patterns`.
+- **CLI** (`aframes`): `record`, `today`, `day`, `context`, `apps`, `patterns`,
+  `mcp`, with JSON / YAML / Markdown / context-block output.
+- **Python API** (`ActivityLog`) and workflow-pattern detection.
+- Test suite (58 tests) and CI on macOS and Linux (Python 3.9, 3.11, 3.13).
+
+[0.1.0]: https://github.com/nossa-y/activity-frames/releases/tag/v0.1.0

activity_frames-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,55 @@
+# Contributing
+
+Thanks for helping build a shared, honest representation of computer activity for agents.
+
+## The easiest, highest-value contribution: a site parser
+
+44% of a real browser history is niche sites. Every site parser you add makes activity frames sharper for everyone. A parser is a pure function of the URL - no network, no state, reviewable in one sitting.
+
+1. Open `src/activity_frames/entities.py`.
+2. Add a function that takes `(domain, parts, q)` and returns a `PageRef` (or `None` to fall through). `parts` is the path split on `/`; `q` is the parsed query dict.
+3. Register it in `_SITE_PARSERS` by host.
+4. Add a test in `tests/test_entities.py`.
+
+```python
+def _hackernews(domain, parts, q):
+    if parts and parts[0] == "item":
+        return PageRef(kind="post", domain=domain, entity=q.get("id", [None])[0])
+    if parts and parts[0] == "user":
+        return PageRef(kind="profile", domain=domain, entity=q.get("id", [None])[0])
+    return None
+
+# in _SITE_PARSERS:
+"news.ycombinator.com": _hackernews,
+```
+
+### Rules for parsers
+
+- **Deterministic.** Output must be a pure function of the URL. No clocks, no randomness, no fetching.
+- **Total.** Return `None` for paths you do not handle so the fallback layers can type them; never raise.
+- **Honest kinds.** A `kind` describes what a page *is* (`profile`, `repo`, `event`), never what the user was *doing* there (`prospecting`, `researching`). Intent is the agent's job, not the parser's. See [SPEC.md](SPEC.md) section 7.
+- **No content.** Parse the URL only. Do not pull identifiers from page bodies.
+
+## Other contributions
+
+- **Capture sources.** The compiler is engine-agnostic. A new source needs an adapter that presents `frames`, `ui_events`, and `elements` tables (see `db.py` and [SPEC.md](SPEC.md) section 9).
+- **Bug fixes and robustness.** Edge cases in timezones, multi-monitor coordinates, and malformed capture data are always welcome.
+
+## Development
+
+```bash
+git clone https://github.com/nossa-y/activity-frames
+cd activity-frames
+pip install -e ".[dev]"
+pytest -q
+```
+
+Everything is standard-library Python (PyYAML is the only optional extra). Keep it that way where you can: zero required dependencies is a feature.
+
+## Ground rules
+
+- Determinism is the core promise. A change that makes output depend on wall-clock time, network, or a model will be declined.
+- Keep the measured tier measured. New fields must be derivable by code; anything inferred belongs in the tier-2 extension with a confidence tag.
+- Add a test with every behavior change.
+
+By contributing you agree your work is licensed under the project's MIT license.

activity_frames-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Nossa Iyamu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

activity_frames-0.1.0/PKG-INFO

@@ -0,0 +1,157 @@
+Metadata-Version: 2.4
+Name: activity-frames
+Version: 0.1.0
+Summary: Episodic memory for AI agents: compile raw screen capture into structured, deterministic activity frames.
+Project-URL: Homepage, https://github.com/nossa-y/activity-frames
+Project-URL: Repository, https://github.com/nossa-y/activity-frames
+Project-URL: Issues, https://github.com/nossa-y/activity-frames/issues
+Author: Nossa Iyamu
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agent-memory,ai-agents,context,episodic-memory,llm,mcp,screen-activity,screenpipe
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: pyyaml>=6.0; extra == 'dev'
+Provides-Extra: yaml
+Requires-Dist: pyyaml>=6.0; extra == 'yaml'
+Description-Content-Type: text/markdown
+
+# activity-frames
+
+[![tests](https://github.com/nossa-y/activity-frames/actions/workflows/test.yml/badge.svg)](https://github.com/nossa-y/activity-frames/actions/workflows/test.yml)
+[![PyPI](https://img.shields.io/pypi/v/activity-frames)](https://pypi.org/project/activity-frames/)
+[![Python](https://img.shields.io/pypi/pyversions/activity-frames)](https://pypi.org/project/activity-frames/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![MCP](https://img.shields.io/badge/MCP-server-6E56CF)](https://modelcontextprotocol.io)
+
+**Episodic memory for AI agents.**
+
+Your agent can read your code, search the web, and call APIs - but it has no idea what you have been doing for the last 8 hours. It starts every conversation blind.
+
+activity-frames gives your agent eyes. It records your screen locally, compiles what it sees into structured **activity frames** (bounded episodes of what you actually did), and serves them to any agent over MCP. No cloud, no LLM in the pipeline, no guessing.
+
+```bash
+pip install activity-frames
+aframes record      # start capturing (local, audio off by default)
+aframes context     # your last 2 hours, agent-ready
+```
+
+## What your agent sees
+
+Capture stores instants: thousands of snapshot rows a day, each one saying "at 22:53:05, Chrome showed linkedin.com/in/...". Useless to reason over.
+
+activity-frames compiles those instants into episodes:
+
+```yaml
+- id: f-0007
+  app: Google Chrome
+  site: linkedin.com
+  start: "20:24:04"
+  end: "20:42:11"
+  duration_min: 18.0
+  pages:
+    - {kind: people_search, entity: "cto paris", count: 2}
+    - {kind: profile, entity: najmuzzaman}
+    - {kind: company, entity: nexdotai}
+  input: {keys: 214, clicks: 31}
+  evidence: {frame_ids: "99871..100147"}
+```
+
+And into a compact context block for any system prompt:
+
+```
+USER ACTIVITY (2026-07-04, local time; measured from screen capture, no interpretation):
+coverage: 09:12-18:47, 342 active min, 11 apps
+away: 12:30-13:15 (45m)
+- 09:12-09:58 Cursor (46.2m): main.py - api
+- 10:01-10:44 Google Chrome/github.com (41.3m): pull_request:acme/api#412; code:acme/api
+- 20:24-20:42 Google Chrome/linkedin.com (18.0m): people_search:cto paris x2; profile:najmuzzaman; company:nexdotai
+```
+
+Drop that into a prompt and your agent knows your day. A full day compiles in under a second and costs zero tokens.
+
+## Episodic memory, done honestly
+
+Agent memory today means conversation memory: what you told the model. Episodic memory is what you actually *did* - and the hard part is representing it without lying.
+
+activity-frames enforces a two-tier contract ([SPEC.md](SPEC.md)):
+
+- **Tier 1, measured (this package):** everything is derivable by deterministic code from capture data. Sessions, durations, typed page entities, input volume, coverage gaps. Same input, same output, every time. There are no intent labels - code cannot know that 2 profile views + a people search was "prospecting". That is your agent's job; it is an LLM.
+- **Tier 2, inferred (optional extension):** tools that add interpretation must namespace it, tag confidence (`high | medium | speculative`), and link evidence. Facts and guesses can never silently mix.
+
+Every frame carries evidence pointers back to raw capture rows. Every document declares its blind spots. What the system did not see, it says it did not see.
+
+## Use it from an agent (MCP)
+
+```bash
+# Claude Code
+claude mcp add activity-frames -- aframes mcp
+```
+
+Any MCP client works: command `aframes`, args `["mcp"]`. Four tools: `get_context`, `get_activity`, `get_day_summary`, `get_patterns` (repetitive-workflow detection: repeated clicks, URL loops, daily habits).
+
+## Use it from Python
+
+```python
+from activity_frames import ActivityLog
+
+log = ActivityLog()
+doc = log.day()                      # today, structured
+doc = log.recent(hours=2)            # last 2 hours
+print(log.context(hours=2))          # paste-ready context block
+```
+
+## Privacy model
+
+- **Local only.** Capture, storage, and compilation all happen on your machine. Nothing is uploaded anywhere, ever.
+- **Read-only compilation.** The compiler opens the capture database read-only.
+- **Content opt-in at the output.** Compiled documents carry input *counts* by default; typed-text content appears only if you explicitly pass `--include-text` (this also gates the repeated-text pattern detector). Be clear about the boundary: the capture database itself does store what the recorder sees, locally, so protect it like any sensitive file (FileVault, permissions).
+- **Audio off by default.** `aframes record --audio` to opt in.
+- **No LLM in the compile path.** Compilation is plain code, so no language model, local or remote, is involved in producing memory. The capture engine does run on-device OCR to read what is on screen; that stays on your machine.
+- **You choose what leaves**, when you paste a context block into an agent. Note that window titles and page entities originate from your screen and can contain third-party text; agents should treat them as data, not instructions.
+
+## Architecture
+
+```
+ capture engine          compiler (this package)         your agent
+ ------------------      ---------------------------     -----------------
+ screen snapshots   -->  sessionize (dwell, gaps,   -->  MCP tools /
+ accessibility tree      flicker merge)                  context blocks /
+ input events            entity typing (20+ sites)       JSON, YAML, md
+ (local SQLite)          enrichment, patterns
+```
+
+The default capture engine is [screenpipe](https://github.com/mediar-ai/screenpipe): `aframes record` provisions a pinned, MIT-licensed build (v0.3.324), verifies its published sha512 before first run, and manages it for you (see [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md)). Already running your own recorder? Point `$AFRAMES_DB` at any capture database with compatible `frames` / `ui_events` / `elements` tables and skip `aframes record` entirely.
+
+## CLI
+
+```bash
+aframes record                   # start capture (--stop / --status / --audio)
+aframes today                    # today's frames (YAML*)
+aframes day 2026-07-03 -f json   # any day, JSON
+aframes context --hours 3        # agent context block
+aframes apps                     # per-app time ledger
+aframes patterns --days 7        # repetitive workflow detection
+aframes mcp                      # MCP stdio server
+```
+
+*YAML output uses PyYAML (`pip install "activity-frames[yaml]"`); without it the CLI falls back to JSON.
+
+## Status
+
+v0.1. Developed and tested on macOS (Apple Silicon); Intel macOS and Linux x64 engine builds exist but are less exercised - reports welcome. Entity parsers cover LinkedIn, GitHub, Google (Search/Docs/Gmail/Maps/Meet/Calendar), YouTube, X, Instagram, Reddit, Luma, Partiful, Product Hunt, Vercel, Supabase, Stripe, Discord, Notion, Figma, Stack Overflow, Calendly, ChatGPT/Claude, localhost; unknown sites fall back to a generic page reference - always total, never lossy. Issues and parser PRs welcome.
+
+Built by [Nossa Iyamu](https://github.com/nossa-y), maker of [Nocta](https://usenocta.com). MIT.

activity_frames-0.1.0/README.md

@@ -0,0 +1,126 @@
+# activity-frames
+
+[![tests](https://github.com/nossa-y/activity-frames/actions/workflows/test.yml/badge.svg)](https://github.com/nossa-y/activity-frames/actions/workflows/test.yml)
+[![PyPI](https://img.shields.io/pypi/v/activity-frames)](https://pypi.org/project/activity-frames/)
+[![Python](https://img.shields.io/pypi/pyversions/activity-frames)](https://pypi.org/project/activity-frames/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![MCP](https://img.shields.io/badge/MCP-server-6E56CF)](https://modelcontextprotocol.io)
+
+**Episodic memory for AI agents.**
+
+Your agent can read your code, search the web, and call APIs - but it has no idea what you have been doing for the last 8 hours. It starts every conversation blind.
+
+activity-frames gives your agent eyes. It records your screen locally, compiles what it sees into structured **activity frames** (bounded episodes of what you actually did), and serves them to any agent over MCP. No cloud, no LLM in the pipeline, no guessing.
+
+```bash
+pip install activity-frames
+aframes record      # start capturing (local, audio off by default)
+aframes context     # your last 2 hours, agent-ready
+```
+
+## What your agent sees
+
+Capture stores instants: thousands of snapshot rows a day, each one saying "at 22:53:05, Chrome showed linkedin.com/in/...". Useless to reason over.
+
+activity-frames compiles those instants into episodes:
+
+```yaml
+- id: f-0007
+  app: Google Chrome
+  site: linkedin.com
+  start: "20:24:04"
+  end: "20:42:11"
+  duration_min: 18.0
+  pages:
+    - {kind: people_search, entity: "cto paris", count: 2}
+    - {kind: profile, entity: najmuzzaman}
+    - {kind: company, entity: nexdotai}
+  input: {keys: 214, clicks: 31}
+  evidence: {frame_ids: "99871..100147"}
+```
+
+And into a compact context block for any system prompt:
+
+```
+USER ACTIVITY (2026-07-04, local time; measured from screen capture, no interpretation):
+coverage: 09:12-18:47, 342 active min, 11 apps
+away: 12:30-13:15 (45m)
+- 09:12-09:58 Cursor (46.2m): main.py - api
+- 10:01-10:44 Google Chrome/github.com (41.3m): pull_request:acme/api#412; code:acme/api
+- 20:24-20:42 Google Chrome/linkedin.com (18.0m): people_search:cto paris x2; profile:najmuzzaman; company:nexdotai
+```
+
+Drop that into a prompt and your agent knows your day. A full day compiles in under a second and costs zero tokens.
+
+## Episodic memory, done honestly
+
+Agent memory today means conversation memory: what you told the model. Episodic memory is what you actually *did* - and the hard part is representing it without lying.
+
+activity-frames enforces a two-tier contract ([SPEC.md](SPEC.md)):
+
+- **Tier 1, measured (this package):** everything is derivable by deterministic code from capture data. Sessions, durations, typed page entities, input volume, coverage gaps. Same input, same output, every time. There are no intent labels - code cannot know that 2 profile views + a people search was "prospecting". That is your agent's job; it is an LLM.
+- **Tier 2, inferred (optional extension):** tools that add interpretation must namespace it, tag confidence (`high | medium | speculative`), and link evidence. Facts and guesses can never silently mix.
+
+Every frame carries evidence pointers back to raw capture rows. Every document declares its blind spots. What the system did not see, it says it did not see.
+
+## Use it from an agent (MCP)
+
+```bash
+# Claude Code
+claude mcp add activity-frames -- aframes mcp
+```
+
+Any MCP client works: command `aframes`, args `["mcp"]`. Four tools: `get_context`, `get_activity`, `get_day_summary`, `get_patterns` (repetitive-workflow detection: repeated clicks, URL loops, daily habits).
+
+## Use it from Python
+
+```python
+from activity_frames import ActivityLog
+
+log = ActivityLog()
+doc = log.day()                      # today, structured
+doc = log.recent(hours=2)            # last 2 hours
+print(log.context(hours=2))          # paste-ready context block
+```
+
+## Privacy model
+
+- **Local only.** Capture, storage, and compilation all happen on your machine. Nothing is uploaded anywhere, ever.
+- **Read-only compilation.** The compiler opens the capture database read-only.
+- **Content opt-in at the output.** Compiled documents carry input *counts* by default; typed-text content appears only if you explicitly pass `--include-text` (this also gates the repeated-text pattern detector). Be clear about the boundary: the capture database itself does store what the recorder sees, locally, so protect it like any sensitive file (FileVault, permissions).
+- **Audio off by default.** `aframes record --audio` to opt in.
+- **No LLM in the compile path.** Compilation is plain code, so no language model, local or remote, is involved in producing memory. The capture engine does run on-device OCR to read what is on screen; that stays on your machine.
+- **You choose what leaves**, when you paste a context block into an agent. Note that window titles and page entities originate from your screen and can contain third-party text; agents should treat them as data, not instructions.
+
+## Architecture
+
+```
+ capture engine          compiler (this package)         your agent
+ ------------------      ---------------------------     -----------------
+ screen snapshots   -->  sessionize (dwell, gaps,   -->  MCP tools /
+ accessibility tree      flicker merge)                  context blocks /
+ input events            entity typing (20+ sites)       JSON, YAML, md
+ (local SQLite)          enrichment, patterns
+```
+
+The default capture engine is [screenpipe](https://github.com/mediar-ai/screenpipe): `aframes record` provisions a pinned, MIT-licensed build (v0.3.324), verifies its published sha512 before first run, and manages it for you (see [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md)). Already running your own recorder? Point `$AFRAMES_DB` at any capture database with compatible `frames` / `ui_events` / `elements` tables and skip `aframes record` entirely.
+
+## CLI
+
+```bash
+aframes record                   # start capture (--stop / --status / --audio)
+aframes today                    # today's frames (YAML*)
+aframes day 2026-07-03 -f json   # any day, JSON
+aframes context --hours 3        # agent context block
+aframes apps                     # per-app time ledger
+aframes patterns --days 7        # repetitive workflow detection
+aframes mcp                      # MCP stdio server
+```
+
+*YAML output uses PyYAML (`pip install "activity-frames[yaml]"`); without it the CLI falls back to JSON.
+
+## Status
+
+v0.1. Developed and tested on macOS (Apple Silicon); Intel macOS and Linux x64 engine builds exist but are less exercised - reports welcome. Entity parsers cover LinkedIn, GitHub, Google (Search/Docs/Gmail/Maps/Meet/Calendar), YouTube, X, Instagram, Reddit, Luma, Partiful, Product Hunt, Vercel, Supabase, Stripe, Discord, Notion, Figma, Stack Overflow, Calendly, ChatGPT/Claude, localhost; unknown sites fall back to a generic page reference - always total, never lossy. Issues and parser PRs welcome.
+
+Built by [Nossa Iyamu](https://github.com/nossa-y), maker of [Nocta](https://usenocta.com). MIT.

activity_frames-0.1.0/SPEC.md

@@ -0,0 +1,142 @@
+# Activity Frames Specification
+
+**Version:** 1 (`schema_version: 1`)
+**Status:** stable
+**Purpose:** a standard, deterministic representation of human computer activity for consumption by AI agents.
+
+## 1. Motivation
+
+Screen recorders capture instants: thousands of snapshot rows per day, each one saying "at time T, app A showed window W at URL U". Agents cannot reason over that efficiently. They need bounded episodes: "the user was in app A on site S from T1 to T2, looked at these pages, typed this much."
+
+This spec defines that episode format, and the rules that keep it trustworthy:
+
+1. **Measured, not guessed.** Every field at this tier is derivable by deterministic code from recorder data. No intent labels, no summaries, no model output.
+2. **Reproducible.** The same input database and window must always produce the same document.
+3. **Evidenced.** Every frame points back to the raw rows it was compiled from.
+4. **Honest about absence.** Coverage gaps and known blind spots are part of the document, not an implementation detail.
+
+## 2. Document structure
+
+A document is a single JSON/YAML object:
+
+```yaml
+schema_version: 1
+generated_at: "2026-07-04T21:14:03Z"     # UTC, ISO-8601
+source:
+  recorder: screenpipe                    # schema family of the capture DB
+window:
+  start_utc: "2026-07-04T07:00:00"
+  end_utc: "2026-07-05T07:00:00"
+  day: "2026-07-04"                       # present when the window is a local day
+coverage: { ... }                         # section 3
+frames: [ ... ]                           # section 4
+blind_spots: [ ... ]                      # section 6
+```
+
+All times inside `coverage` and `frames` are **local** wall-clock times (`HH:MM` or `HH:MM:SS`), because they describe a human's day. The `window` is UTC, because it describes a database query.
+
+## 3. Coverage
+
+What the recorder actually saw. Consumers must treat anything outside covered time as unknown, not as inactivity.
+
+```yaml
+coverage:
+  first_activity: "09:12"
+  last_activity: "18:47"
+  active_minutes: 342        # distinct minutes with at least one frame
+  span_minutes: 575          # last - first
+  coverage_pct: 59           # active / span, capped at 100
+  frames_analyzed: 4211      # raw snapshot count in the window
+  distinct_apps: 11
+  gaps:                      # periods >= 5 min with no capture
+    - {start: "12:30", end: "13:15", minutes: 45}
+```
+
+## 4. Frames
+
+A frame is one bounded stretch of attention in a single context. The context key is `(app, site)`: the site is the URL host for browser activity, absent otherwise.
+
+```yaml
+- id: f-0007                  # stable within the document
+  app: "Google Chrome"
+  site: "linkedin.com"        # omitted for non-browser apps
+  start: "20:24:04"           # local
+  end: "20:42:11"
+  duration_min: 18.0          # ACTIVE minutes (dwell-based, see 5.1)
+  wall_min: 21.5              # end - start; emitted when it differs by > 1 min
+  windows:                    # up to 3 most-seen window titles
+    - "Search | LinkedIn"
+  pages:                      # typed page references, browser frames only
+    - {kind: people_search, entity: "cto paris", count: 2}
+    - {kind: profile, entity: "najmuzzaman"}
+    - {kind: company, entity: "nexdotai"}
+  input:                      # omitted when empty
+    keys: 214                 # keystrokes + typed characters
+    clicks: 31
+    copies: 2
+  interruptions:              # brief context switches folded in (see 5.3)
+    - {app: "Slack", seconds: 12}       # site key present for browser flickers
+  evidence:
+    frame_ids: "99871..100147"   # raw row id range in the source DB
+```
+
+### 4.1 Page references
+
+A page reference types a URL into something an agent can read: `kind` (what sort of page), optional `entity` (the human-relevant identifier), optional `count` (views within the frame, emitted when > 1).
+
+Standard kinds include: `profile`, `company`, `feed`, `messaging`, `people_search`, `search`, `repo`, `pull_request`, `issue`, `code`, `doc`, `sheet`, `email`, `video`, `channel`, `post`, `question`, `booking`, `design`, `ai_chat`, `local_dev`, `notifications`, and the fallback `page`. Kinds are open: producers may add site-specific kinds, but must keep them deterministic functions of the URL.
+
+### 4.2 Input privacy rule
+
+Input **volume** (counts) is part of the standard document. Input **content** (typed text) must be excluded by default and only included on an explicit opt-in from the operator of the producing tool.
+
+## 5. Determinism rules
+
+Implementations must produce identical documents for identical inputs. The reference constants:
+
+### 5.1 Dwell
+Capture is event-driven: a frame is stored when the screen changes. A frame contributes `min(gap_to_next_frame, 90s)` of active time. The cap prevents a static screen (or an absent user) from earning unbounded credit. `duration_min` is the sum of dwells inside the frame.
+
+### 5.2 Session gap
+A gap larger than `300s` between consecutive raw frames closes the current activity frame and is a candidate coverage gap (reported when >= 5 min).
+
+### 5.3 Flicker merge
+The sequence A -> B -> A, where B lasts at most `20s` of wall time and no session gap intervenes on either side of B, collapses into a single A frame. B is recorded in `interruptions` with its measured seconds. B's time is **not** added to A's `duration_min`. Nothing is silently dropped.
+
+### 5.4 Minimum duration
+Producers may offer a `min_minutes` filter for consumer convenience. Filtered-out frames are omitted with the count disclosed in the document (`omitted: {below_min_minutes: N, min_minutes: X}`); they must never be silently merged into neighbors.
+
+
+### 5.5 Capture devices
+
+Each monitor records its own frame stream (`device_name`). Segmentation runs per device, so simultaneous monitors do not fragment each other's frames; the document lists all devices' frames sorted by start time, which means frames MAY overlap in time. Input events are assigned to exactly one containing segment (ties across devices resolved by the nearest captured frame), so input volume is never double-counted. A consequence to disclose: the same app visible on two monitors at once earns active time on both.
+
+## 6. Blind spots
+
+Every document carries a `blind_spots` list: plain-language statements of what the capture pipeline systematically cannot see (e.g. "browser URLs are only captured for browser apps"). Producers must not remove entries to make output look more complete.
+
+## 7. Tier 2: inferred fields (extension, not part of this package)
+
+Tools MAY extend documents with interpretation: task labels, project clusters, meeting summaries. Such fields are **inference**, and the spec requires them to be:
+
+- namespaced under `inferred` (at document or frame level),
+- tagged with `confidence: high | medium | speculative`,
+- tagged with `evidence`: which measured fields or raw rows support the inference.
+
+```yaml
+inferred:
+  - label: "LinkedIn prospecting"
+    frames: [f-0007, f-0009]
+    confidence: medium
+    evidence: "people_search + 2 profile views + 1 company page in 18 min"
+```
+
+A consumer must always be able to strip everything under `inferred` and be left with a purely measured document. This package emits tier 1 only.
+
+## 8. Versioning
+
+`schema_version` increments only on breaking changes to tier-1 field semantics. Additive fields do not bump the version. Consumers should ignore unknown fields.
+
+## 9. Sources
+
+The reference producer reads the local SQLite database written by the built-in capture engine (screenpipe, provisioned by `aframes record`). Required surface: a `frames` table (`id`, `timestamp` UTC ISO, `app_name`, `window_name`, `browser_url`, `focused`; `device_name` optional, treated as one stream when absent), and optionally `ui_events` (input volume) and `elements` (click resolution) - the compiler degrades gracefully when the optional tables are missing. Any capture system can be supported by a producer that satisfies sections 2 through 6; `source.recorder` identifies the schema family.