matrx-files 0.1.0__tar.gz

matrx_files-0.1.0/.gitignore

@@ -0,0 +1,271 @@
+*.pyc
+secrets/
+ignore/
+temp/
+logs/
+# The broad `logs/` rule above is for RUNTIME log output, but it also matched
+# the dashboard's SOURCE directory and silently swallowed an entire feature's
+# files (only the pre-existing index.tsx stayed tracked), breaking the prod
+# Docker build with "Could not resolve ./structured-tab". Re-include the source.
+!apps/dashboard/src/features/logs/
+!apps/dashboard/src/features/logs/**
+todo
+text_notes/
+aidream/secrets/2.env
+automation_matrix/matrix_processing/temp/*
+cd
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+.venv/
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+# The blanket lib/ rule above is from the standard Python .gitignore template
+# and was silently swallowing TS source under the SPA `src/lib/` folders.
+# Re-allow them explicitly so frontend builds don't ship without their lib layer.
+!apps/dashboard/src/lib/
+!apps/dashboard/src/lib/**
+!apps/workflow-studio/src/lib/
+!apps/workflow-studio/src/lib/**
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+ai/tests/clean_response.json
+ai/tests/cx_storage_response.json
+ai/tests/execution_test.py
+ai/tests/final_response.json
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.env_remote
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+.env.armanonly
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# random armani files
+/armani_dev/secrets/
+/armani/
+/_armani/
+
+
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+.idea/
+.vscode/
+/node_modules/
+
+# Frontend pnpm workspace (apps/) — node_modules at the workspace root and any
+# member, plus Vite caches and build output. The unified lockfile (apps/pnpm-lock.yaml)
+# IS committed; everything below is regenerated.
+node_modules/
+apps/**/.vite/
+apps/**/dist/
+.vite/
+
+dump.rdb
+
+frontend/
+
+# AME Temp Files and directory structure
+# Ignore all files in the temp directory and its subdirectories
+/temp/**/*
+/tmp/**/*
+
+# Allow .gitkeep files to retain directory structure
+!/temp/**/.gitkeep
+!/tmp/**/.gitkeep
+
+# Armani
+.history*
+.history/
+local_data/
+local_reports_data/
+webscraper/quick_scrapes/temp/
+automation_matrix/ai_apis/fireworks/_dev/*
+automation_matrix/ai_apis/fireworks/_dev/fireworks_sample.py
+*.pdf
+*.flac
+*.mp3
+*.wav
+miniconda.sh
+/database/python_sql/temp_data/
+ .history*
+ .history/
+.history/
+
+_dev/
+/_dev/
+requirements_filtered.txt
+
+# matrx-dev-tools backups
+.env-backups/
+# Matrx Ship config (contains API key)
+.matrx-ship.json
+
+# Matrx config (contains API keys)
+.matrx.json
+.matrx-tools.conf
+
+# Claude Code local worktrees and per-user settings
+.claude/worktrees/
+.claude/settings.local.json
+
+# Append-only snapshots from matrx_utils.update_history (unbounded; do not commit)
+common/utils/data_in_code/data_history.json
+packages/matrx-utils/matrx_utils/data_in_code/data_history.json
+
+# Tool-dispatch debug logs — one file per server start, never committed
+.matrx-debug/
+
+# macOS Finder metadata
+.DS_Store
+**/.DS_Store
+
+# Environment files
+.env
+.env.*
+*.env
+*.env.*
+
+# Keep safe templates trackable
+!.env.example
+!.env.sample
+!.env.template

matrx_files-0.1.0/CLAUDE.md

@@ -0,0 +1,47 @@
+# CLAUDE.md — matrx-files (development)
+
+**All things files.** Cloud storage (S3/Supabase/CDN), media management, conversion,
+PDF, sharing, analysis/OCR routing, and the package's own API layer. Carved out of
+`matrx_utils.file_handling` on 2026-07-12 as the FIRST instance of the
+**matrx-package-template** (`/Users/armanisadeghi/code/matrx-package-template`).
+
+## The law here: INDEPENDENCE.md
+
+Read [INDEPENDENCE.md](INDEPENDENCE.md) before changing anything. This package is
+**gate-enforced independent** — two gates run against it and both must stay green:
+
+```bash
+# fast AST gate (forbidden host imports, undeclared siblings)
+python /Users/armanisadeghi/code/matrx-package-template/scripts/check_imports.py packages/matrx-files
+# proof by execution: bare venv, install THIS package only (+ declared siblings), boot, smoke test
+/Users/armanisadeghi/code/matrx-package-template/scripts/independence_gate.sh \
+    packages/matrx-files packages/matrx-utils packages/matrx-connect packages/matrx-orm
+```
+
+- ❌ NEVER import `aidream` or any host module. Need a host capability → grow a
+  `configure()` kwarg (`matrx_files/_config.py`) with a standalone fallback.
+- ❌ NEVER add a dependency the bare-venv install can't resolve — the gate catches it
+  (it already caught rembg's fossil-metadata chain and the numba↔numpy cap; see
+  pyproject comments before "fixing" version pins).
+- ✅ Every new feature adds at least one bare-venv smoke test under `tests/`.
+- API surface lives in `matrx_files/api/` (router factories, caller-supplied auth dep);
+  `standalone/app.py` must mount everything a host can mount.
+
+## Consumer-side knowledge (NOT here)
+
+How aidream uses this package (MediaRef resolution rules, `/assets` presets, URL
+contract, storage_uri isolation) lives in the host docs: aidream root `CLAUDE.md`
+"Files" section + [matrx_files/FEATURE.md](matrx_files/FEATURE.md) +
+[matrx_files/MODULE_README.md](matrx_files/MODULE_README.md).
+
+## Status / roadmap
+
+Live work order: `docs/handoffs/matrx-files-split.md` (aidream repo). Phase 2 DONE
+2026-07-13: the package OWNS its DB truth — generated models in `matrx_files/db/`
+(`models_files.py` for files.*, `models_iam.py` borrowed iam.permissions,
+`_database="matrx_files"`), bound via `matrx_files.db.bind_to_host` (hosted alias)
+or `bootstrap_db` (standalone pool from `MATRX_FILES_POSTGRES_*`). ALL DB access
+goes through matrx-orm — the Supabase PostgREST client is gone from the DB layer
+(`cloud_sync/db.py` is the reference conversion; RPCs via `matrx_orm.call_function`).
+`cloud_sync/sql/` holds the package's own migrations (sanctioned SQL). Phase 3 =
+standalone microservice on EC2 us-east-1.

matrx_files-0.1.0/DEPLOY.md

@@ -0,0 +1,49 @@
+# Deploying the Matrx Files microservice (EC2 us-east-1)
+
+The standalone service is BUILT and verified locally (boot, own DB pool, JWT
+auth, all routers). What remains is infrastructure — an operator (or matrx-ship)
+executes this; it needs AWS/DNS access the codebase doesn't have.
+
+## 1. Build the image (from the aidream monorepo root)
+
+```bash
+docker build -f packages/matrx-files/Dockerfile -t matrx-files:latest .
+```
+
+## 2. Runtime environment (the COMPLETE registry — `standalone/app.py::_env_config`)
+
+| Var | Required | Notes |
+|---|---|---|
+| `SUPABASE_MATRIX_HOST/_PORT/_USER/_PASSWORD` + `SUPABASE_MATRIX_DATABASE_NAME` | yes* | Same pooler values aidream uses (Supavisor transaction pooler). *Or the package trio `MATRX_FILES_POSTGRES_HOST/_PORT/_NAME/_USER/_PASSWORD`. |
+| `AWS_S3_DEFAULT_BUCKET` (+ `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` or an instance role) | yes | Instance role preferred on EC2 — same region as the buckets is the whole point. |
+| `SUPABASE_URL` (or `SUPABASE_MATRIX_URL`) | yes | Auth: JWKS derived as `{url}/auth/v1/.well-known/jwks.json` (ES256). |
+| `SUPABASE_JWT_SECRET` | no | Only if HS256 tokens must also verify. |
+| `MATRX_FILES_BASE_DIR` | no | Scratch dir; image defaults `/data/matrx-files`. |
+
+No env at all = smoke mode (health only). PARTIAL env = loud refusal to start
+(by design — never silently serve broken routes).
+
+## 3. Run
+
+```bash
+docker run -d --name matrx-files -p 8080:8080 --env-file matrx-files.env \
+  -v /data/matrx-files:/data/matrx-files matrx-files:latest
+# probe
+curl -s localhost:8080/files-service/health   # {"status":"ok","package":"matrx-files"}
+```
+
+## 4. Platform integration checklist (operator)
+
+- [ ] EC2 instance (us-east-1, same region as `AWS_S3_DEFAULT_BUCKET`), instance role with S3 access.
+- [ ] Register the service in **matrx-ship** (deploy/version tracking) like other projects.
+- [ ] Dedicated domain (decision on record: e.g. `files.matrxserver.com`) → TLS via the standard edge (Cloudflare) → port 8080.
+- [ ] aidream keeps serving the same routes in-process — traffic cutover for byte-heavy endpoints is a LATER, separate switch (route-by-route at the client/edge).
+- [ ] PyPI publishing: tag `matrx-files/v0.1.0` triggers `.github/workflows/publish-package.yml` (matrx-utils v2.0.0 must be published first for a PyPI-only install; the Docker image above doesn't need PyPI).
+
+## Verification once live
+
+```bash
+curl -s https://files.matrxserver.com/api-or-bare/files-service/health
+# authenticated smoke: any real Supabase JWT
+curl -s -H "Authorization: Bearer $JWT" https://.../files/<known-file-id>
+```

matrx_files-0.1.0/Dockerfile

@@ -0,0 +1,31 @@
+# Matrx Files — standalone microservice image.
+#
+# Build from the MONOREPO ROOT (siblings resolve from the local workspace —
+# the local matrx-utils/orm/connect versions lead what's on PyPI):
+#   docker build -f packages/matrx-files/Dockerfile -t matrx-files .
+#
+# The image contains ONLY matrx-files + its declared dependencies. If the
+# build ever needs anything else from the repo, the package is violating
+# INDEPENDENCE.md — fix the package, not this file.
+FROM python:3.13-slim
+
+# tesseract: pytesseract OCR backend (same binary the CI gate installs).
+RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
+    tesseract-ocr \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN pip install --no-cache-dir uv
+
+WORKDIR /app
+# Declared local siblings first (leaf order), then the package itself.
+COPY packages/matrx-utils   /deps/matrx-utils
+COPY packages/matrx-connect /deps/matrx-connect
+COPY packages/matrx-orm     /deps/matrx-orm
+COPY packages/matrx-files   /app
+
+RUN uv pip install --system /deps/matrx-utils /deps/matrx-connect /deps/matrx-orm \
+    && uv pip install --system ".[standalone]"
+
+ENV MATRX_FILES_BASE_DIR=/data/matrx-files
+EXPOSE 8080
+CMD ["uvicorn", "matrx_files.standalone.app:create_app", "--factory", "--host", "0.0.0.0", "--port", "8080"]

matrx_files-0.1.0/INDEPENDENCE.md

@@ -0,0 +1,61 @@
+# INDEPENDENCE.md — the contract every Matrx package signs
+
+> **A Matrx package is extremely opinionated, yet fully independent.** It may expect an
+> exact Postgres structure, exact AWS buckets, exact conventions — but everything it
+> expects is **handed to it** through `configure_*()`, or it provisions its **own exact
+> duplicate copy**. It NEVER reaches into a host application.
+>
+> — This is Arman's standing order. Agents have repeatedly destroyed package
+> independence by importing "just one thing" from the host. That is why this file and
+> its enforcement gates exist. If you are an AI agent reading this: the rules below are
+> not style preferences. They are mechanically enforced, and a violation FAILS the build.
+
+## The three laws
+
+1. **No host imports, ever.** Nothing under the package may import `aidream`, any
+   host-repo root module, or an undeclared sibling. If the package needs a host
+   capability, it grows a `configure_*()` kwarg with a standalone fallback.
+   *Test: would this import resolve in a fresh project that contains ONLY this package
+   and its declared dependencies? No → forbidden.*
+
+2. **Opinionated inputs, validated loudly, never defaulted silently.** Required config
+   is validated **all-errors-at-once at configure/boot time** with a screaming banner
+   that names every missing value and how to supply it. A missing value must be
+   impossible to miss — crash, never fall back to a legacy path. Optional seams get a
+   real standalone fallback (the package's own duplicate copy), never a stub that
+   pretends to work.
+
+3. **The package ships its own truth.** Its database schema (models + migrations), its
+   API layer (FastAPI routers), and its standalone app entry live INSIDE the package.
+   Installing the package gives a customer the entire structure that drives it.
+   - **Host mode:** the host resolves the package's `db/db_requirements.py` manifest
+     and calls `configure_db(...)`; the host mounts the package's routers.
+   - **Standalone mode:** `standalone/app.py` boots the same routers as a real
+     microservice against the same (or a fresh) database, using the package's own
+     migrations.
+   The two modes run the SAME code. A feature that only works in host mode is a defect.
+
+## Enforcement — the layers that SCREAM
+
+Each layer is sufficient alone; each screams when it fires (per the extinction doctrine):
+
+| Layer | What | When |
+|---|---|---|
+| `scripts/check_imports.py` | AST scan — forbidden host/undeclared-sibling imports | pre-commit / CI, instant |
+| **Standalone-boot gate** `scripts/independence_gate.sh` | Fresh venv, install the package from its OWN pyproject (+ declared local siblings only), **boot the standalone app, run smoke tests** | CI + release. Proof by execution — the gate no agent can rationalize past |
+| Boot validation | `configure()`/app factory raises `IndependenceConfigError` listing ALL missing config with a red banner | every process start |
+| Host boundary ratchet | the host repo's `check_package_boundaries.py` | host CI |
+
+**Never weaken a gate to make a build pass.** If a gate fires, the code is wrong, not
+the gate. Deleting, skipping, baselining-up, or `# noqa`-ing an independence gate is
+itself a defect — file it and fix the import instead.
+
+## What "opinionated" licenses — and what it does not
+
+✅ Allowed: expecting exact table names/schemas, exact bucket layouts, exact envelope
+shapes, exact conventions — **received via configure() or shipped as the package's own
+schema/migrations**.
+
+❌ Not licensed: reading the host's env vars, assuming the host's filesystem layout,
+importing the host's settings object, "temporarily" importing a host helper, or a
+fallback that silently no-ops instead of either working or crashing.

matrx_files-0.1.0/PKG-INFO

@@ -0,0 +1,74 @@
+Metadata-Version: 2.4
+Name: matrx-files
+Version: 0.1.0
+Summary: All things files: cloud storage, media management, conversion, PDF, sharing — the platform's independent file service.
+Project-URL: Homepage, https://github.com/AI-Matrix-Engine/aidream-current
+Project-URL: Repository, https://github.com/AI-Matrix-Engine/aidream-current
+Project-URL: Issues, https://github.com/AI-Matrix-Engine/aidream-current/issues
+Author-email: Matrx <admin@aimatrx.com>
+License: MIT
+Keywords: cdn,files,matrx,media,pdf,s3
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.13
+Requires-Dist: boto3-stubs[s3]>=1.42.59
+Requires-Dist: boto3>=1.42.59
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: imageio-ffmpeg>=0.5
+Requires-Dist: matrx-connect
+Requires-Dist: matrx-orm>=3.1
+Requires-Dist: matrx-utils
+Requires-Dist: mozjpeg-lossless-optimization>=1.1.0
+Requires-Dist: numpy<2.5,>=1.26
+Requires-Dist: onnxruntime>=1.16
+Requires-Dist: opencv-python-headless>=4.9
+Requires-Dist: pandas>=3.0.1
+Requires-Dist: pillow-avif-plugin>=1.4.6
+Requires-Dist: pillow-heif>=0.18.0
+Requires-Dist: pillow>=11.2.1
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: pymupdf>=1.27.0
+Requires-Dist: pypdfium2>=4.30.0
+Requires-Dist: pytesseract>=0.3.13
+Requires-Dist: python-pptx>=1.0.2
+Requires-Dist: pyyaml>=6.0.3
+Requires-Dist: rembg>=2.0.69
+Requires-Dist: requests>=2.32.5
+Requires-Dist: supabase>=2.28.0
+Provides-Extra: pdf-cloud-ocr
+Requires-Dist: azure-ai-documentintelligence; extra == 'pdf-cloud-ocr'
+Requires-Dist: boto3; extra == 'pdf-cloud-ocr'
+Requires-Dist: google-cloud-documentai; extra == 'pdf-cloud-ocr'
+Provides-Extra: pdf-forms
+Requires-Dist: pypdfform>=1.4; extra == 'pdf-forms'
+Provides-Extra: pdf-generate
+Requires-Dist: reportlab>=4.0; extra == 'pdf-generate'
+Provides-Extra: pdf-html
+Requires-Dist: weasyprint>=60; extra == 'pdf-html'
+Provides-Extra: pdf-ml-llm
+Requires-Dist: pymupdf4llm>=0.0.30; extra == 'pdf-ml-llm'
+Provides-Extra: pdf-ocr
+Requires-Dist: ocrmypdf>=16; extra == 'pdf-ocr'
+Provides-Extra: pdf-redact-ner
+Requires-Dist: presidio-analyzer>=2.2; extra == 'pdf-redact-ner'
+Requires-Dist: presidio-anonymizer>=2.2; extra == 'pdf-redact-ner'
+Requires-Dist: spacy>=3.7; extra == 'pdf-redact-ner'
+Provides-Extra: pdf-signatures
+Requires-Dist: pyhanko>=0.20; extra == 'pdf-signatures'
+Provides-Extra: pdf-tables
+Requires-Dist: tabula-py>=2.9; extra == 'pdf-tables'
+Provides-Extra: pdf-tables-extra
+Requires-Dist: camelot-py>=0.11; extra == 'pdf-tables-extra'
+Provides-Extra: standalone
+Requires-Dist: fastapi>=0.115; extra == 'standalone'
+Requires-Dist: uvicorn>=0.34; extra == 'standalone'
+Description-Content-Type: text/markdown
+
+# Matrx Files
+
+All things files: cloud storage, media management, conversion, PDF, sharing — the platform's independent file service.
+
+Contract: [INDEPENDENCE.md](INDEPENDENCE.md).

matrx_files-0.1.0/README.md

@@ -0,0 +1,5 @@
+# Matrx Files
+
+All things files: cloud storage, media management, conversion, PDF, sharing — the platform's independent file service.
+
+Contract: [INDEPENDENCE.md](INDEPENDENCE.md).