dinary 0.2.0__tar.gz → 0.4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (224) hide show
  1. dinary-0.4.0/.deploy.example/.env +59 -0
  2. dinary-0.4.0/.deploy.example/README.md +39 -0
  3. dinary-0.4.0/.deploy.example/import_sources.json +4 -0
  4. dinary-0.4.0/.deploy.example/litestream.yml +64 -0
  5. {dinary-0.2.0 → dinary-0.4.0}/.github/workflows/ci.yml +8 -0
  6. {dinary-0.2.0 → dinary-0.4.0}/.gitignore +3 -1
  7. dinary-0.4.0/.plans/analytics.md +348 -0
  8. dinary-0.4.0/.plans/architecture.md +1036 -0
  9. dinary-0.4.0/.plans/cloud-security.md +408 -0
  10. dinary-0.4.0/.plans/frontend-evaluation.md +76 -0
  11. dinary-0.4.0/.plans/income.md +156 -0
  12. {dinary-0.2.0 → dinary-0.4.0}/.plans/phase0.md +19 -9
  13. {dinary-0.2.0 → dinary-0.4.0}/.plans/phase1.md +153 -8
  14. dinary-0.4.0/.plans/storage-migration.md +958 -0
  15. {dinary-0.2.0 → dinary-0.4.0}/.plans/task.md +21 -1
  16. {dinary-0.2.0 → dinary-0.4.0}/.pre-commit-config.yaml +3 -3
  17. dinary-0.4.0/AGENTS.md +87 -0
  18. {dinary-0.2.0 → dinary-0.4.0}/PKG-INFO +40 -18
  19. dinary-0.4.0/README.md +92 -0
  20. {dinary-0.2.0 → dinary-0.4.0}/docker-compose.yml +1 -1
  21. {dinary-0.2.0 → dinary-0.4.0}/docs/mkdocs.yml +6 -5
  22. {dinary-0.2.0 → dinary-0.4.0}/docs/src/en/cloudflare-setup.md +3 -3
  23. {dinary-0.2.0 → dinary-0.4.0}/docs/src/en/deploy-oracle.md +18 -10
  24. {dinary-0.2.0 → dinary-0.4.0}/docs/src/en/deploy-selfhost.md +11 -7
  25. dinary-0.4.0/docs/src/en/google-sheets-setup.md +97 -0
  26. {dinary-0.2.0 → dinary-0.4.0}/docs/src/en/index.md +5 -6
  27. dinary-0.4.0/docs/src/en/installation.md +31 -0
  28. dinary-0.4.0/docs/src/en/operations.md +471 -0
  29. {dinary-0.2.0 → dinary-0.4.0}/docs/src/en/pwa-install.md +1 -1
  30. {dinary-0.2.0 → dinary-0.4.0}/docs/src/ru/cloudflare-setup.md +3 -3
  31. {dinary-0.2.0 → dinary-0.4.0}/docs/src/ru/deploy-oracle.md +19 -11
  32. {dinary-0.2.0 → dinary-0.4.0}/docs/src/ru/deploy-selfhost.md +11 -7
  33. dinary-0.4.0/docs/src/ru/google-sheets-setup.md +97 -0
  34. {dinary-0.2.0 → dinary-0.4.0}/docs/src/ru/index.md +5 -6
  35. dinary-0.4.0/docs/src/ru/installation.md +31 -0
  36. dinary-0.4.0/docs/src/ru/operations.md +485 -0
  37. {dinary-0.2.0 → dinary-0.4.0}/docs/src/ru/pwa-install.md +1 -1
  38. dinary-0.4.0/docs/src/ru/taxonomy.md +149 -0
  39. dinary-0.4.0/invoke.yml +6 -0
  40. {dinary-0.2.0 → dinary-0.4.0}/package-lock.json +792 -1
  41. {dinary-0.2.0 → dinary-0.4.0}/package.json +1 -0
  42. {dinary-0.2.0 → dinary-0.4.0}/pyproject.toml +11 -4
  43. dinary-0.4.0/src/dinary/__about__.py +1 -0
  44. dinary-0.4.0/src/dinary/api/admin_catalog.py +499 -0
  45. dinary-0.4.0/src/dinary/api/catalog.py +328 -0
  46. dinary-0.4.0/src/dinary/api/expenses.py +341 -0
  47. dinary-0.4.0/src/dinary/background/rate_prefetch_task.py +105 -0
  48. dinary-0.4.0/src/dinary/background/sheet_logging_task.py +99 -0
  49. dinary-0.4.0/src/dinary/config.py +374 -0
  50. dinary-0.4.0/src/dinary/imports/README.md +104 -0
  51. dinary-0.4.0/src/dinary/imports/__init__.py +13 -0
  52. dinary-0.4.0/src/dinary/imports/bootstrap.md +221 -0
  53. dinary-0.4.0/src/dinary/imports/expense_import.py +1306 -0
  54. dinary-0.4.0/src/dinary/imports/income.md +89 -0
  55. dinary-0.4.0/src/dinary/imports/income_import.py +325 -0
  56. dinary-0.4.0/src/dinary/imports/report_2d_3d.py +613 -0
  57. dinary-0.4.0/src/dinary/imports/seed.py +1022 -0
  58. dinary-0.4.0/src/dinary/imports/verify_equivalence.py +219 -0
  59. dinary-0.4.0/src/dinary/imports/verify_income.py +107 -0
  60. {dinary-0.2.0 → dinary-0.4.0}/src/dinary/main.py +23 -7
  61. dinary-0.4.0/src/dinary/migrations/0001_initial_schema.rollback.sql +15 -0
  62. dinary-0.4.0/src/dinary/migrations/0001_initial_schema.sql +169 -0
  63. dinary-0.4.0/src/dinary/migrations/0002_exchange_rates_source_target.rollback.sql +8 -0
  64. dinary-0.4.0/src/dinary/migrations/0002_exchange_rates_source_target.sql +13 -0
  65. dinary-0.4.0/src/dinary/migrations/README.md +15 -0
  66. dinary-0.4.0/src/dinary/reports/__init__.py +12 -0
  67. dinary-0.4.0/src/dinary/reports/expenses.py +420 -0
  68. dinary-0.4.0/src/dinary/reports/income.py +285 -0
  69. dinary-0.4.0/src/dinary/reports/verify_budget.py +261 -0
  70. dinary-0.4.0/src/dinary/reports/verify_income.py +220 -0
  71. dinary-0.4.0/src/dinary/services/catalog_writer.py +1329 -0
  72. dinary-0.4.0/src/dinary/services/db_migrations.py +134 -0
  73. dinary-0.4.0/src/dinary/services/exchange_rates.py +117 -0
  74. dinary-0.4.0/src/dinary/services/ledger_repo.py +1107 -0
  75. dinary-0.4.0/src/dinary/services/nbs.py +100 -0
  76. dinary-0.4.0/src/dinary/services/rate_helpers.py +74 -0
  77. dinary-0.4.0/src/dinary/services/seed_config.py +560 -0
  78. dinary-0.4.0/src/dinary/services/sheet_logging.py +495 -0
  79. dinary-0.4.0/src/dinary/services/sheet_mapping.py +759 -0
  80. dinary-0.4.0/src/dinary/services/sheets.py +753 -0
  81. dinary-0.4.0/src/dinary/services/sql_loader.py +76 -0
  82. dinary-0.4.0/src/dinary/services/sqlite_types.py +165 -0
  83. dinary-0.4.0/src/dinary/sql/__init__.py +0 -0
  84. dinary-0.4.0/src/dinary/sql/get_category_by_name.sql +3 -0
  85. dinary-0.4.0/src/dinary/sql/get_existing_expense.sql +4 -0
  86. dinary-0.4.0/src/dinary/sql/get_month_expenses.sql +14 -0
  87. dinary-0.4.0/src/dinary/sql/insert_expense.sql +6 -0
  88. dinary-0.4.0/src/dinary/sql/list_categories.sql +10 -0
  89. dinary-0.4.0/src/dinary/sql/logging_projection.sql +33 -0
  90. dinary-0.4.0/src/dinary/sql/resolve_mapping.sql +3 -0
  91. dinary-0.4.0/src/dinary/sql/resolve_mapping_for_year.sql +5 -0
  92. dinary-0.4.0/src/dinary/sql/seed_load_categories.sql +1 -0
  93. dinary-0.4.0/src/dinary/tools/__init__.py +8 -0
  94. dinary-0.4.0/src/dinary/tools/sql.py +183 -0
  95. {dinary-0.2.0 → dinary-0.4.0}/static/css/style.css +230 -0
  96. dinary-0.4.0/static/index.html +127 -0
  97. dinary-0.4.0/static/js/api.js +329 -0
  98. dinary-0.4.0/static/js/app.js +812 -0
  99. dinary-0.4.0/static/js/catalog-add.js +287 -0
  100. dinary-0.4.0/static/js/catalog.js +397 -0
  101. {dinary-0.2.0 → dinary-0.4.0}/static/js/offline-queue.js +25 -8
  102. {dinary-0.2.0 → dinary-0.4.0}/static/sw.js +6 -1
  103. dinary-0.4.0/tasks.py +3400 -0
  104. dinary-0.4.0/tests/conftest.py +111 -0
  105. dinary-0.4.0/tests/js/catalog-cache.test.js +187 -0
  106. dinary-0.4.0/tests/js/default-group-selection.test.js +170 -0
  107. {dinary-0.2.0 → dinary-0.4.0}/tests/js/no-data-loss.test.js +208 -82
  108. dinary-0.4.0/tests/test_admin_catalog.py +539 -0
  109. dinary-0.4.0/tests/test_api.py +1016 -0
  110. dinary-0.4.0/tests/test_api_catalog.py +217 -0
  111. dinary-0.4.0/tests/test_catalog_writer.py +502 -0
  112. dinary-0.4.0/tests/test_config.py +294 -0
  113. dinary-0.4.0/tests/test_currency_rates.py +432 -0
  114. dinary-0.4.0/tests/test_expense_import.py +289 -0
  115. dinary-0.4.0/tests/test_ledger_repo.py +1188 -0
  116. dinary-0.4.0/tests/test_main.py +250 -0
  117. dinary-0.4.0/tests/test_migrations.py +323 -0
  118. dinary-0.4.0/tests/test_qr_parser.py +32 -0
  119. dinary-0.4.0/tests/test_rate_prefetch_task.py +331 -0
  120. dinary-0.4.0/tests/test_report_2d_3d.py +879 -0
  121. dinary-0.4.0/tests/test_reports_expenses.py +329 -0
  122. dinary-0.4.0/tests/test_reports_income.py +206 -0
  123. dinary-0.4.0/tests/test_reports_verify_budget.py +194 -0
  124. dinary-0.4.0/tests/test_reports_verify_income.py +181 -0
  125. dinary-0.4.0/tests/test_seed_config.py +334 -0
  126. dinary-0.4.0/tests/test_sheet_logging.py +837 -0
  127. dinary-0.4.0/tests/test_sheet_mapping.py +776 -0
  128. dinary-0.4.0/tests/test_sheets.py +683 -0
  129. {dinary-0.2.0 → dinary-0.4.0}/tests/test_sql_loader.py +13 -17
  130. dinary-0.4.0/tests/test_tasks.py +2537 -0
  131. dinary-0.4.0/tests/test_tools_sql.py +263 -0
  132. dinary-0.4.0/uv.lock +1772 -0
  133. dinary-0.2.0/.env.example +0 -5
  134. dinary-0.2.0/.plans/architecture.md +0 -781
  135. dinary-0.2.0/.plans/deploy-oracle-no-db.md +0 -168
  136. dinary-0.2.0/.plans/frontend-evaluation.md +0 -41
  137. dinary-0.2.0/README.md +0 -73
  138. dinary-0.2.0/docs/includes/install_pipx_macos.sh +0 -2
  139. dinary-0.2.0/docs/src/en/google-sheets-setup.md +0 -57
  140. dinary-0.2.0/docs/src/en/installation.md +0 -29
  141. dinary-0.2.0/docs/src/en/operations.md +0 -87
  142. dinary-0.2.0/docs/src/ru/google-sheets-setup.md +0 -57
  143. dinary-0.2.0/docs/src/ru/installation.md +0 -29
  144. dinary-0.2.0/docs/src/ru/operations.md +0 -87
  145. dinary-0.2.0/invoke.yml +0 -5
  146. dinary-0.2.0/src/dinary/__about__.py +0 -1
  147. dinary-0.2.0/src/dinary/api/categories.py +0 -36
  148. dinary-0.2.0/src/dinary/api/expenses.py +0 -108
  149. dinary-0.2.0/src/dinary/config.py +0 -34
  150. dinary-0.2.0/src/dinary/migrations/README.md +0 -15
  151. dinary-0.2.0/src/dinary/migrations/budget/0001_initial_schema.rollback.sql +0 -3
  152. dinary-0.2.0/src/dinary/migrations/budget/0001_initial_schema.sql +0 -25
  153. dinary-0.2.0/src/dinary/migrations/config/0001_initial_schema.rollback.sql +0 -8
  154. dinary-0.2.0/src/dinary/migrations/config/0001_initial_schema.sql +0 -54
  155. dinary-0.2.0/src/dinary/services/category_store.py +0 -42
  156. dinary-0.2.0/src/dinary/services/db_migrations.py +0 -107
  157. dinary-0.2.0/src/dinary/services/duckdb_repo.py +0 -407
  158. dinary-0.2.0/src/dinary/services/exchange_rate.py +0 -28
  159. dinary-0.2.0/src/dinary/services/import_sheet.py +0 -229
  160. dinary-0.2.0/src/dinary/services/seed_config.py +0 -206
  161. dinary-0.2.0/src/dinary/services/sheets.py +0 -357
  162. dinary-0.2.0/src/dinary/services/sql_loader.py +0 -70
  163. dinary-0.2.0/src/dinary/services/sync.py +0 -362
  164. dinary-0.2.0/src/dinary/sql/find_travel_event.sql +0 -3
  165. dinary-0.2.0/src/dinary/sql/get_existing_expense.sql +0 -3
  166. dinary-0.2.0/src/dinary/sql/get_month_expenses.sql +0 -14
  167. dinary-0.2.0/src/dinary/sql/insert_expense.sql +0 -5
  168. dinary-0.2.0/src/dinary/sql/list_sheet_categories.sql +0 -3
  169. dinary-0.2.0/src/dinary/sql/resolve_mapping.sql +0 -3
  170. dinary-0.2.0/src/dinary/sql/reverse_lookup_5d.sql +0 -6
  171. dinary-0.2.0/src/dinary/sql/reverse_lookup_travel.sql +0 -3
  172. dinary-0.2.0/src/dinary/sql/seed_load_categories.sql +0 -1
  173. dinary-0.2.0/src/dinary/sql/seed_load_groups.sql +0 -1
  174. dinary-0.2.0/src/dinary/sql/seed_load_members.sql +0 -1
  175. dinary-0.2.0/src/dinary/sql/seed_load_tags.sql +0 -1
  176. dinary-0.2.0/static/index.html +0 -80
  177. dinary-0.2.0/static/js/api.js +0 -54
  178. dinary-0.2.0/static/js/app.js +0 -325
  179. dinary-0.2.0/static/js/categories.js +0 -66
  180. dinary-0.2.0/tasks.py +0 -427
  181. dinary-0.2.0/tests/conftest.py +0 -11
  182. dinary-0.2.0/tests/test_api.py +0 -349
  183. dinary-0.2.0/tests/test_duckdb.py +0 -599
  184. dinary-0.2.0/tests/test_migrations.py +0 -124
  185. dinary-0.2.0/tests/test_seed_config.py +0 -157
  186. dinary-0.2.0/tests/test_services.py +0 -119
  187. dinary-0.2.0/tests/test_sheets.py +0 -754
  188. dinary-0.2.0/tests/test_sync.py +0 -476
  189. dinary-0.2.0/uv.lock +0 -1584
  190. {dinary-0.2.0 → dinary-0.4.0}/.coveragerc +0 -0
  191. {dinary-0.2.0 → dinary-0.4.0}/.github/workflows/docs.yml +0 -0
  192. {dinary-0.2.0 → dinary-0.4.0}/.github/workflows/pip_publish.yml +0 -0
  193. {dinary-0.2.0 → dinary-0.4.0}/.github/workflows/static.yml +0 -0
  194. {dinary-0.2.0 → dinary-0.4.0}/.plans/sql-vs-ibis-comparison.md +0 -0
  195. {dinary-0.2.0 → dinary-0.4.0}/Dockerfile +0 -0
  196. {dinary-0.2.0 → dinary-0.4.0}/LICENSE.txt +0 -0
  197. {dinary-0.2.0 → dinary-0.4.0}/activate.sh +0 -0
  198. {dinary-0.2.0 → dinary-0.4.0}/docs/src/en/images/about.jpg +0 -0
  199. {dinary-0.2.0 → dinary-0.4.0}/docs/src/en/reference.md +0 -0
  200. {dinary-0.2.0 → dinary-0.4.0}/pytest.ini +0 -0
  201. {dinary-0.2.0 → dinary-0.4.0}/scripts/__init__.py +0 -0
  202. {dinary-0.2.0 → dinary-0.4.0}/scripts/build-docs.sh +0 -0
  203. {dinary-0.2.0 → dinary-0.4.0}/scripts/build.sh +0 -0
  204. {dinary-0.2.0 → dinary-0.4.0}/scripts/docs-render-config.sh +0 -0
  205. {dinary-0.2.0 → dinary-0.4.0}/scripts/upload.sh +0 -0
  206. {dinary-0.2.0 → dinary-0.4.0}/scripts/verup.sh +0 -0
  207. {dinary-0.2.0 → dinary-0.4.0}/scripts/verup_action.sh +0 -0
  208. {dinary-0.2.0 → dinary-0.4.0}/src/dinary/__init__.py +0 -0
  209. {dinary-0.2.0 → dinary-0.4.0}/src/dinary/api/__init__.py +0 -0
  210. {dinary-0.2.0 → dinary-0.4.0}/src/dinary/api/qr.py +0 -0
  211. {dinary-0.2.0/src/dinary/services → dinary-0.4.0/src/dinary/background}/__init__.py +0 -0
  212. {dinary-0.2.0/src/dinary/sql → dinary-0.4.0/src/dinary/services}/__init__.py +0 -0
  213. {dinary-0.2.0 → dinary-0.4.0}/src/dinary/services/qr_parser.py +0 -0
  214. {dinary-0.2.0 → dinary-0.4.0}/static/icons/icon-180.png +0 -0
  215. {dinary-0.2.0 → dinary-0.4.0}/static/icons/icon-192.png +0 -0
  216. {dinary-0.2.0 → dinary-0.4.0}/static/icons/icon-512.png +0 -0
  217. {dinary-0.2.0 → dinary-0.4.0}/static/js/qr-scanner-lib.js +0 -0
  218. {dinary-0.2.0 → dinary-0.4.0}/static/js/qr-scanner-worker.min.js +0 -0
  219. {dinary-0.2.0 → dinary-0.4.0}/static/js/qr-scanner.js +0 -0
  220. {dinary-0.2.0 → dinary-0.4.0}/static/manifest.json +0 -0
  221. {dinary-0.2.0 → dinary-0.4.0}/tests/js/offline-queue.test.js +0 -0
  222. {dinary-0.2.0 → dinary-0.4.0}/tests/js/setup.js +0 -0
  223. {dinary-0.2.0 → dinary-0.4.0}/tests/test_dinary.py +0 -0
  224. {dinary-0.2.0 → dinary-0.4.0}/vitest.config.js +0 -0
@@ -0,0 +1,59 @@
1
+ # Copy to .deploy/.env and fill in your values (.deploy/ is gitignored)
2
+ # cp -r .deploy.example .deploy
3
+ DINARY_DEPLOY_HOST=ubuntu@<PUBLIC_IP>
4
+ # DINARY_TUNNEL=tailscale # tailscale (default) | cloudflare | none
5
+
6
+ # Optional: Litestream SFTP replica (VM2). Only consumed by
7
+ # ``inv setup-replica`` and the replica URL rendered by
8
+ # ``inv litestream-setup``; ``inv deploy`` ignores it. Must be a
9
+ # host reachable over SSH as the ``ubuntu`` user (Tailscale MagicDNS
10
+ # name recommended so rotations do not break deploys). Leave unset
11
+ # on hosts that only run the primary.
12
+ # DINARY_REPLICA_HOST=ubuntu@dinary-replica
13
+
14
+ # First-deploy ONLY: the ISO-4217 accounting currency that will be
15
+ # baked into every expenses.amount / income.amount row on disk. On
16
+ # the very first ``init_db`` against an empty dinary.db the value
17
+ # is copied into ``app_metadata.accounting_currency`` and becomes the
18
+ # DB's source of truth. After that the env var is purely advisory —
19
+ # leave it commented out and the server will read the anchored value
20
+ # back from the DB on startup. If you DO leave it set, init_db checks
21
+ # it against the stored value and refuses to start on mismatch (the
22
+ # guard that catches accidental typos silently corrupting the ledger).
23
+ # Default on a fresh DB: EUR.
24
+ # DINARY_ACCOUNTING_CURRENCY=EUR
25
+
26
+ # Optional: the PWA / API user-facing default currency. This is what
27
+ # the UI works in and the fallback for ``POST /api/expenses`` requests
28
+ # that omit ``currency``. It only influences INPUT defaulting — it is
29
+ # NOT persisted and does not affect stored amounts (those are always in
30
+ # the accounting currency above). Safe to flip any time (e.g. if the
31
+ # operator relocates). Default: RSD.
32
+ # DINARY_APP_CURRENCY=RSD
33
+
34
+ # Optional: enable sheet logging (append each expense to Google Sheets).
35
+ # Accepts a spreadsheet ID or a full browser URL. Leave empty to disable.
36
+ # DINARY_SHEET_LOGGING_SPREADSHEET=https://docs.google.com/spreadsheets/d/YOUR_SPREADSHEET_ID/edit
37
+
38
+ # Optional: how often the in-process periodic drain re-tries the
39
+ # sheet_logging_jobs queue. Default 300s. 0 disables periodic drain
40
+ # (per-expense fire-and-forget still runs).
41
+ # DINARY_SHEET_LOGGING_DRAIN_INTERVAL_SEC=300
42
+
43
+ # Optional: hard cap on _drain_one_job calls per single drain_pending
44
+ # invocation, summed across years. Default 15. Bounds Sheets API quota
45
+ # usage in the worst case (mass backlog after a long outage).
46
+ # DINARY_SHEET_LOGGING_DRAIN_MAX_ATTEMPTS_PER_ITERATION=15
47
+
48
+ # Optional: synchronous sleep between two consecutive _drain_one_job
49
+ # calls, executed inside the asyncio.to_thread worker (does NOT block
50
+ # the FastAPI event loop). Default 1.0s = instantaneous ceiling of
51
+ # ~1 attempt/sec inside one sweep; combined with max_attempts the
52
+ # sustained rate stays at 3-9 Sheets API calls/min. Set to 0 to
53
+ # disable (tests).
54
+ # DINARY_SHEET_LOGGING_DRAIN_INTER_ROW_DELAY_SEC=1.0
55
+
56
+ # Optional: TTL in days for the sheet_logging_jobs queue. Rows whose
57
+ # expense.date is older than today minus this many days are silently
58
+ # skipped (left in the table). Default 90. 0 disables the TTL filter.
59
+ # DINARY_SHEET_LOGGING_DRAIN_MAX_AGE_DAYS=90
@@ -0,0 +1,39 @@
1
+ # .deploy.example — deploy config template
2
+
3
+ Copy this directory to `.deploy/` and fill in your values:
4
+
5
+ ```bash
6
+ cp -r .deploy.example .deploy
7
+ # edit .deploy/.env
8
+ ```
9
+
10
+ `.deploy/` is gitignored; the operator's local copy is the authoritative
11
+ source of deploy configuration across every server they run.
12
+
13
+ ## Files
14
+
15
+ - **`.env`** — required runtime config (`DINARY_DEPLOY_HOST`, optional
16
+ `DINARY_SHEET_LOGGING_SPREADSHEET`, etc.). `inv deploy` reads this
17
+ locally and syncs it to the server as `/home/ubuntu/dinary/.deploy/.env`.
18
+ - **`import_sources.json`** — OPTIONAL. Only needed if you run
19
+ `inv import-*` tasks to import historical expenses from your own
20
+ Google Sheets. Non-import users can leave this file alone — it is
21
+ never read at runtime. See the `imports/` directory at the repo
22
+ root for details on the schema and workflows.
23
+ - **`litestream.yml`** — OPTIONAL. Litestream (v0.5.x) replicator
24
+ config for hot off-site backup of `data/dinary.db` to an SFTP
25
+ target (typically a second Oracle Cloud Free Tier VM). Copy to
26
+ `.deploy/litestream.yml`, fill in the SFTP `host`/`user`/`path`
27
+ fields, then run `inv litestream-setup` once. See
28
+ [`docs/src/en/operations.md`](../docs/src/en/operations.md)
29
+ for the end-to-end replica bootstrap workflow. `inv setup` does
30
+ NOT auto-run `inv litestream-setup` even when the config is
31
+ present locally — the sidecar requires an SFTP host whose
32
+ `authorized_keys` already trusts VM 1's ed25519 key, a
33
+ cross-host relationship the bootstrap script cannot arrange on
34
+ your behalf.
35
+
36
+ If you copy `import_sources.json` as-is and never run `inv import-*`,
37
+ nothing breaks: the file is only consulted by import tasks, which
38
+ non-import users never invoke. Same for `litestream.yml`: absent
39
+ means no replication, which is a valid single-VM deployment.
@@ -0,0 +1,4 @@
1
+ [
2
+ {"year": 2026, "spreadsheet_id": "REPLACE_WITH_YOUR_SPREADSHEET_ID"},
3
+ {"year": 2019, "spreadsheet_id": "REPLACE_WITH_YOUR_SPREADSHEET_ID", "income_worksheet_name": "Balance", "income_layout_key": "balance_rub"}
4
+ ]
@@ -0,0 +1,64 @@
1
+ # Litestream config template for dinary (v0.5.x schema).
2
+ #
3
+ # Copy this file to ``.deploy/litestream.yml`` and fill in the placeholders
4
+ # that match the SFTP replica target you maintain on VM 2 (or any other
5
+ # SFTP host that has your VM 1 SSH key in ``~/.ssh/authorized_keys``).
6
+ #
7
+ # ``inv litestream-setup`` uploads this file to ``/etc/litestream.yml`` on
8
+ # VM 1, installs the Litestream binary pinned in ``tasks.py``, and enables
9
+ # a systemd unit that continuously ships LTX segments to the replica while
10
+ # the app writes. Stopping the sidecar has no effect on the app path —
11
+ # Litestream is a passive replicator.
12
+ #
13
+ # See ``.plans/storage-migration.md`` (Phase 2) for the rationale behind
14
+ # SFTP + 1 h snapshot interval + 7-day retention. The governing knob for
15
+ # a "laptop back from a long vacation" restore is ``snapshot.interval``,
16
+ # not ``snapshot.retention`` — ``litestream restore`` always starts from
17
+ # the latest snapshot and replays LTX forward, so *any* snapshot at all
18
+ # is sufficient for a refresh.
19
+ #
20
+ # Schema notes (Litestream v0.5.x):
21
+ # * ``replica`` is SINGULAR (v0.4.x ``replicas:`` plural is rejected).
22
+ # * ``retention`` and ``snapshot-interval`` moved OUT of the per-replica
23
+ # block into the top-level ``snapshot:`` block as ``interval`` and
24
+ # ``retention`` respectively. Leaving them on the replica makes
25
+ # Litestream reject the config.
26
+
27
+ # Global snapshot cadence and retention — applies to every db below.
28
+ snapshot:
29
+ # Write a full consistent snapshot every hour. This bounds the LTX
30
+ # replay time on every ``litestream restore`` invocation (laptop
31
+ # refresh, VM 2 disaster recovery) to the snapshot window.
32
+ interval: 1h
33
+ # One week of history. The operator-side disaster recovery case
34
+ # ("what did the DB look like three days ago") is what retention
35
+ # serves; the laptop-side analytics case does not care.
36
+ retention: 168h
37
+
38
+ dbs:
39
+ - path: /home/ubuntu/dinary/data/dinary.db
40
+ replica:
41
+ type: sftp
42
+ # REPLACE this placeholder with the SSH ``host:port`` of your
43
+ # replica VM. Must be a reachable hostname from VM 1 — a
44
+ # Tailscale MagicDNS name, a Cloudflare tunnel alias, a bare
45
+ # FQDN, or a ``~/.ssh/config`` ``Host`` entry you already have.
46
+ # Litestream talks raw SSH/22, so the port suffix is literal
47
+ # (append ``:22`` even if you use a non-standard SSH port;
48
+ # change ``22`` to the actual port in that case).
49
+ host: <REPLICA_HOST_OR_TAILSCALE_NAME>:22
50
+ # REPLACE with the username on the replica host that owns the
51
+ # ``path:`` directory below.
52
+ user: <REPLICA_USERNAME>
53
+ # Private key path on VM 1. ``inv setup`` generated an ``ed25519``
54
+ # keypair as part of the one-time VM install; copy the ``.pub``
55
+ # side into the replica host's ``~/.ssh/authorized_keys``. The
56
+ # default path matches what ``inv setup`` leaves on disk — only
57
+ # change it if your setup differs.
58
+ key-path: /home/ubuntu/.ssh/id_ed25519
59
+ # REPLACE the whole right-hand side with an existing, writable
60
+ # absolute path on the replica host where Litestream will
61
+ # materialize snapshots + the LTX tree. Must exist and be
62
+ # writable by ``user`` (Litestream does not mkdir it). Typical
63
+ # layout: ``/home/<replica-user>/replicas/dinary``.
64
+ path: <ABSOLUTE_REPLICA_DINARY_DIR>
@@ -7,6 +7,14 @@ name: CI
7
7
  env:
8
8
  PRIMARY_PYTHON_VERSION: '3.13'
9
9
  PRIMARY_PLATFORM: 'ubuntu-latest'
10
+ # Force Python's UTF-8 mode so text I/O (e.g.
11
+ # ``yoyo.read_migrations`` opening ``*.sql`` files) defaults to
12
+ # utf-8 on every platform. Without this the Windows matrix entries
13
+ # fall back to cp1252 and crash on any non-ASCII byte in migration
14
+ # comments (e.g. ``"отпуск"`` / em-dashes), producing a fleet of
15
+ # ``UnicodeDecodeError: 'charmap' codec can't decode byte 0x81``
16
+ # failures at fixture-collection time.
17
+ PYTHONUTF8: '1'
10
18
  PYTEST_CMD: >-
11
19
  python -m pytest
12
20
  --junitxml=pytest.xml
@@ -11,6 +11,7 @@ pytest.xml
11
11
  pytest-coverage.txt
12
12
  *.py-e
13
13
  .venv/
14
+ data/
14
15
  **/_mkdocs.yml*
15
16
  **/ru/images/
16
17
  **/ru/reference.md
@@ -19,7 +20,8 @@ allure-results/
19
20
  allure-report/
20
21
  dist/
21
22
  credentials.json
22
- .env
23
+ /.env
24
+ /.deploy/
23
25
  node_modules/
24
26
  data/
25
27
  _static/
@@ -0,0 +1,348 @@
1
+ # Analytics layer (OLAP on the laptop)
2
+
3
+ > **Status.** Stub. This document captures architectural decisions that
4
+ > are already firm (monorepo placement with a separate
5
+ > `dinary-analytics` package, runtime tiering, SQL-first design) and
6
+ > leaves placeholders for decisions to be made once implementation
7
+ > starts. Implementation itself is expected to begin only after the
8
+ > storage migration (`storage-migration.md` Phases 0–4) has landed.
9
+
10
+ ## 1. Scope
11
+
12
+ The analytics layer covers everything that reads the ledger for
13
+ insight rather than to write back into it:
14
+
15
+ - Existing CLI reports (`inv report-expenses`, `inv report-income`,
16
+ `inv import-report-2d-3d`).
17
+ - Forthcoming AI-driven interactive dashboards.
18
+ - Ad-hoc exploration via notebooks.
19
+ - Periodic complex reports (monthly / yearly roll-ups, envelope
20
+ accounting summaries, budget vs actual).
21
+
22
+ It explicitly does **not** cover: OLTP writes, sheet logging, imports
23
+ from Google Sheets, migrations, API surface. Those stay in the
24
+ existing service layer and keep SQLite (via the `storage/`
25
+ abstraction in `storage-migration.md` Phase 1) as their sole engine.
26
+
27
+ ## 2. Relationship to storage-migration.md
28
+
29
+ This plan builds on top of the storage migration and does not
30
+ repeat its content. Key cross-references:
31
+
32
+ - Storage engine on the laptop: `.plans/storage-migration.md` §9.2
33
+ (zero-copy DuckDB-over-SQLite ATTACH).
34
+ - Why DuckDB is the right query engine despite row-oriented storage:
35
+ `.plans/storage-migration.md` §9.1.
36
+ - Replica refresh workflow (`inv pull-replica`):
37
+ `.plans/storage-migration.md` §10 Phase 3.
38
+ - Daily cloud snapshots on VM 2: `.plans/storage-migration.md` §10
39
+ Phase 3.5.
40
+
41
+ The invariant this plan inherits: **analytics reads a read-only
42
+ SQLite replica; it never writes back to `ledger.*`**. Any
43
+ materialized caches are DuckDB-local and disposable.
44
+
45
+ ## 3. Repository placement: monorepo (decided)
46
+
47
+ Analytics code lives in the same `dinary` repository as the
48
+ server, for these reasons (unchanged from the design discussion that
49
+ produced this plan):
50
+
51
+ - Schema changes in the server ripple into analytics instantly; one
52
+ PR edits the migration and the dependent report together.
53
+ - Shared utilities (Decimal / Currency coercers, category tree
54
+ walkers, envelope classifiers, source_type normalizers, sheet
55
+ column mappers) are imported directly rather than vendored or
56
+ published as a side package.
57
+ - AI coding assistants see server schema, business rules, and
58
+ analytics in one context — critical for the "AI-driven dashboards"
59
+ use case.
60
+ - Single `inv` task surface for both sides.
61
+
62
+ Heavyweight analytical dependencies (DuckDB, Polars, Marimo,
63
+ plotting, LLM SDKs) are kept out of the server's dependency tree by
64
+ shipping analytics as a **separate Python package** inside the
65
+ monorepo, with its own `pyproject.toml` and its own dependency
66
+ closure. VM 1 never installs that package and therefore never
67
+ resolves any of its transitive deps; the laptop installs both.
68
+
69
+ Reasons for separate-package over a same-package optional extra:
70
+
71
+ - **Architectural isolation**, not conventional. An optional
72
+ `[project.optional-dependencies]` extra relies on every contributor
73
+ (and every future AI-generated patch) remembering to not `import
74
+ dinary.analytics` from server code. A separate package makes the
75
+ mistake impossible at install time: the analytics package is simply
76
+ not on the server's `sys.path`.
77
+ - **No runtime guards needed.** The §6 invariant "server-side code
78
+ must remain import-clean when the analytics group is not installed"
79
+ collapses from a CI rule we have to police into a property the
80
+ build system enforces by construction.
81
+ - **Independent version pins.** DuckDB / Polars / Marimo can move on
82
+ their own cadence without touching the server's lock. A CVE or
83
+ breaking bump in an analytics transitive dep does not freeze
84
+ server deploys.
85
+ - **Cleaner VM 1 image.** No dead weight in the server venv and no
86
+ accidental `import duckdb` from server code surviving a review.
87
+
88
+ Proposed layout under the repo root:
89
+
90
+ ```
91
+ packages/
92
+ dinary/ # server (FastAPI, ledger_repo, imports, tasks)
93
+ pyproject.toml
94
+ src/dinary/...
95
+ dinary-analytics/ # laptop-only: DuckDB-over-SQLite, reports, notebooks
96
+ pyproject.toml
97
+ src/dinary_analytics/
98
+ __init__.py
99
+ connection.py # open_ledger() helper: duckdb + ATTACH sqlite
100
+ queries/ # named reusable SQL (.sql files or constants)
101
+ reports/ # Python functions returning DataFrames
102
+ caches.py # optional CREATE TABLE AS SELECT definitions
103
+ notebooks/ # Marimo *.py notebooks (tracked in git)
104
+ analytics/ # laptop runtime data: replica + caches (gitignored)
105
+ ```
106
+
107
+ Shared, engine-agnostic code (schema-level types, Decimal/Currency
108
+ coercers, category-tree walkers, envelope classifiers) lives in
109
+ `dinary` and is consumed by `dinary-analytics` via an ordinary
110
+ dependency declaration. The dependency only goes one way:
111
+ `dinary-analytics` depends on `dinary`; `dinary` **never** depends on
112
+ `dinary-analytics`.
113
+
114
+ Workspace tooling: both packages are managed as a
115
+ [uv workspace](https://docs.astral.sh/uv/concepts/projects/workspaces/)
116
+ (or equivalent) so `uv sync` on the laptop resolves both together
117
+ and `uv sync --package dinary` on VM 1 resolves only the server.
118
+ The exact workspace wiring is in §7.
119
+
120
+ ## 4. Runtime tiers
121
+
122
+ Three levels of "where analytics runs" trade install footprint against
123
+ interactivity and AI capability. The project targets Tier 1 first;
124
+ Tier 2 is deferred until a concrete need arises; Tier 3 is rejected.
125
+
126
+ ### 4.1 Tier 1 — Python on developer laptop (primary, decided)
127
+
128
+ - Dependencies installed by `uv sync` from the workspace root,
129
+ which resolves both `dinary` and `dinary-analytics` into the
130
+ laptop venv. Incremental footprint over the server-only set
131
+ ~200 MB (DuckDB, Polars, Marimo, Altair/matplotlib). Zero
132
+ additional install for the developer who already runs `inv dev`.
133
+ - Notebooks are [Marimo](https://marimo.io/) reactive `*.py` files —
134
+ dif-friendly, tracked in git, executed via `inv notebook`
135
+ (`marimo edit notebooks/<name>.py`).
136
+ - Analytical logic is Python-over-SQL: business rules expressed as
137
+ SQL strings executed against the DuckDB-attached SQLite replica;
138
+ Python handles orchestration, plotting, and AI-tool glue.
139
+ - Full AI integration: LLM SDK calls from notebook cells, MCP servers
140
+ for DuckDB, Cursor / Claude exercising the notebook directly.
141
+ - Runtime: everything is local to the laptop. No extra services to
142
+ deploy.
143
+
144
+ This is the default tier for all single-user / developer-mode use.
145
+
146
+ ### 4.2 Tier 2 — DuckDB-WASM in the browser (optional, deferred)
147
+
148
+ Considered and pre-approved as an additive path if/when browser-only
149
+ access matters (phone dashboards, sharing with a non-technical user,
150
+ multi-device view).
151
+
152
+ Building blocks:
153
+
154
+ - [DuckDB-WASM](https://duckdb.org/docs/api/wasm/overview) for
155
+ in-browser SQL execution (~25 MB cached).
156
+ - [Evidence.dev](https://evidence.dev/) or
157
+ [Marimo-WASM](https://docs.marimo.io/guides/wasm.html) for the
158
+ dashboard framework (static site build output).
159
+ - SQLite replica served over HTTPS from a Tailscale-reachable
160
+ endpoint (VM 2 with a static file server) using HTTP Range
161
+ requests; DuckDB-WASM supports this natively.
162
+
163
+ Implementation cost at that point is roughly one afternoon of
164
+ wiring, *provided* the SQL queries from Tier 1 were written without
165
+ pandas-specific transformations (see §5 SQL-first design).
166
+
167
+ Not built now. Mentioned so the path stays open.
168
+
169
+ ### 4.3 Tier 3 — Static HTML from nightly render (rejected)
170
+
171
+ Pre-rendered static dashboards generated by a cron job. Rejected
172
+ because it sacrifices interactivity and AI-driven exploration,
173
+ which are the main reasons to build the analytics layer in the
174
+ first place. If Tier 2 becomes necessary we use DuckDB-WASM
175
+ (interactive in the browser), not pre-baked HTML.
176
+
177
+ ## 5. Design principle: SQL-first business logic
178
+
179
+ A deliberate constraint, motivated by both performance and the
180
+ Tier 1 → Tier 2 migration path:
181
+
182
+ **Analytical business logic is written as SQL executed by DuckDB,
183
+ not as Python manipulating DataFrames.** Python owns orchestration,
184
+ caching, plotting, and AI glue; SQL owns the "what the numbers
185
+ mean" part.
186
+
187
+ Concrete example:
188
+
189
+ ```python
190
+ # Discouraged: business logic hidden inside pandas
191
+ def monthly_totals(df):
192
+ return df.groupby(pd.Grouper(key="date", freq="ME"))["amount_rsd"].sum()
193
+
194
+ # Preferred: business logic in SQL, Python is only the call site
195
+ def monthly_totals(ledger):
196
+ return ledger.sql("""
197
+ SELECT date_trunc('month', date) AS month,
198
+ SUM(amount_rsd) AS total
199
+ FROM expenses
200
+ GROUP BY month
201
+ ORDER BY month
202
+ """).to_df()
203
+ ```
204
+
205
+ Reasons:
206
+
207
+ 1. **Portability to Tier 2.** DuckDB-WASM runs the same SQL dialect.
208
+ A Tier 1 report written as SQL ports to a Tier 2 browser dashboard
209
+ as a copy-paste, not a rewrite.
210
+ 2. **Performance.** DuckDB's vectorized engine beats pandas on every
211
+ aggregation shape relevant here. Leaving the work in SQL keeps
212
+ the fast path intact.
213
+ 3. **Reviewability.** SQL is easier to read for "is this business
214
+ rule correct?" than a chain of pandas calls with implicit index
215
+ semantics.
216
+ 4. **AI-assisted coding.** LLMs write reliable DuckDB SQL; they are
217
+ less reliable at writing idiomatic pandas, especially with
218
+ date arithmetic and group-by semantics.
219
+
220
+ Not a dogma: there are places where Python is the right tool
221
+ (plotting, interactive widgets, LLM tool-use loops, I/O to
222
+ CSV/Parquet imports). The rule applies to *business semantics*
223
+ — "what does 'monthly expense total by envelope' mean" belongs
224
+ in SQL. "How do I render that as an interactive chart" belongs
225
+ in Python.
226
+
227
+ ## 6. Invariants (inherited + plan-specific)
228
+
229
+ - Analytics is strictly read-only against `ledger.*`. `ATTACH` with
230
+ `READ_ONLY` is enforced at the connection level. No code path
231
+ opens the replica writable.
232
+ - Analytics never touches the live OLTP SQLite on VM 1 directly.
233
+ The replica on VM 2 / the laptop is the only legitimate source.
234
+ - Any materialized DuckDB-native caches (§4.1 optional
235
+ `CREATE TABLE ... AS SELECT`) are regenerated from the replica,
236
+ never hand-edited. They are treated as disposable; a valid
237
+ recovery from any cache bug is `DROP TABLE` + rerun the builder.
238
+ - Analytics ships as a separate Python package (`dinary-analytics`);
239
+ the server package (`dinary`) does not depend on it. VM 1 installs
240
+ only `dinary`, so analytics deps (DuckDB, Polars, Marimo, plotting,
241
+ LLM SDKs) are physically absent from the server venv — no runtime
242
+ guard or CI lint needed to keep them out.
243
+
244
+ ## 7. Package layout and dependency wiring (to be finalized)
245
+
246
+ Exact split to be decided when implementation starts. Sketch of the
247
+ two `pyproject.toml` files:
248
+
249
+ ```toml
250
+ # packages/dinary/pyproject.toml — server, runs on VM 1
251
+ [project]
252
+ name = "dinary"
253
+ dependencies = [
254
+ "fastapi",
255
+ "pydantic",
256
+ "gspread",
257
+ # ... existing server deps; NO duckdb, polars, marimo, plotting, LLM
258
+ ]
259
+ ```
260
+
261
+ ```toml
262
+ # packages/dinary-analytics/pyproject.toml — laptop only
263
+ [project]
264
+ name = "dinary-analytics"
265
+ dependencies = [
266
+ "dinary", # shared types, Decimal/Currency helpers, schema
267
+ "duckdb>=1.1",
268
+ "polars>=1.5",
269
+ "marimo>=0.9",
270
+ "altair>=5.4",
271
+ # + LLM SDK once chosen
272
+ ]
273
+ ```
274
+
275
+ Workspace root `pyproject.toml` declares both as uv workspace
276
+ members so a single `uv sync` on the laptop resolves the combined
277
+ closure, while `uv sync --package dinary` on VM 1 pulls only the
278
+ server side.
279
+
280
+ Outstanding decisions before committing this to the repo:
281
+
282
+ - Concrete workspace tool choice (uv workspaces vs hatch
283
+ workspaces vs plain editable installs) and the resulting
284
+ `uv.lock` / CI layout.
285
+ - Exact DuckDB version pin and extension list (sqlite, json, icu).
286
+ - Plotting stack: Altair + Vega alone, or also matplotlib for
287
+ static exports.
288
+ - LLM SDK choice (Anthropic / OpenAI / litellm wrapper): drives
289
+ how `dinary_analytics.ai` is shaped.
290
+ - Migration of any existing analytics-ish code (current CLI report
291
+ renderers in `src/dinary/reports/`) — decide whether those move
292
+ into `dinary-analytics` or stay in `dinary` because they back
293
+ `inv report-*` tasks that must run on the laptop against the
294
+ replica, not on VM 1.
295
+
296
+ ## 8. Placeholder: dashboard framework
297
+
298
+ To be decided during the first concrete dashboard. Candidates and
299
+ their fit:
300
+
301
+ - **Marimo** (Tier 1 primary): reactive notebooks, excellent Python
302
+ integration, single-file, good for iterative exploration.
303
+ - **Evidence.dev** (Tier 2 candidate): Markdown + SQL, static site
304
+ output, native DuckDB, excellent published-dashboard UX.
305
+ - **Streamlit**: popular but stateful-per-session, less ideal for
306
+ AI tool-use flows, heavier than Marimo for our scale.
307
+ - **Rill Developer**: DuckDB-native, opinionated dashboards,
308
+ strong for time-series metrics.
309
+
310
+ The decision gates on the first concrete requirement, not now.
311
+
312
+ ## 9. Placeholder: AI integration
313
+
314
+ To be decided. Likely shape:
315
+
316
+ - MCP server wrapping DuckDB queries for use from Claude Desktop /
317
+ Cursor.
318
+ - Prompt scaffolding for "describe this expense trend" / "classify
319
+ this anomaly" / "suggest an envelope rebalancing" tasks.
320
+ - Guardrails preventing LLM-generated SQL from running with write
321
+ access (already guaranteed by the read-only invariant in §6,
322
+ reinforced by connection-level config).
323
+
324
+ Deferred until there is a concrete dashboard that needs it.
325
+
326
+ ## 10. Placeholder: module catalog and first reports
327
+
328
+ When implementation starts, the first wave will likely be:
329
+
330
+ 1. Port `inv report-expenses` and `inv report-income` to read
331
+ through `dinary_analytics.connection` (DuckDB + ATTACH) instead
332
+ of the current server-local SQLite read. Zero semantic change;
333
+ mechanical.
334
+ 2. Port `inv import-report-2d-3d` similarly.
335
+ 3. Add a first exploratory Marimo notebook (e.g.
336
+ `notebooks/expenses-explore.py`) to exercise the connection
337
+ helper and the SQL-first pattern.
338
+ 4. Revisit the module layout in §3 with real usage data.
339
+
340
+ ## 11. What this plan explicitly does *not* cover
341
+
342
+ - Storage choice on the server (covered by `storage-migration.md`).
343
+ - Replication mechanics (covered by `storage-migration.md` §10
344
+ Phases 2–3.5).
345
+ - PWA UI, API surface, sheet logging, imports — untouched by the
346
+ analytics work.
347
+ - A public multi-user analytics service. The scope is one
348
+ single-user ledger owned by the repository maintainer.