authsome 0.3.1__tar.gz → 0.4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (358) hide show
  1. authsome-0.4.0/.claude/commands/run-evals.md +495 -0
  2. {authsome-0.3.1 → authsome-0.4.0}/.claude-plugin/marketplace.json +1 -1
  3. authsome-0.4.0/.github/release-please-manifest.json +3 -0
  4. {authsome-0.3.1 → authsome-0.4.0}/.gitignore +7 -0
  5. {authsome-0.3.1 → authsome-0.4.0}/AGENTS.md +21 -10
  6. {authsome-0.3.1 → authsome-0.4.0}/CHANGELOG.md +94 -0
  7. authsome-0.4.0/CONTEXT.md +251 -0
  8. {authsome-0.3.1 → authsome-0.4.0}/CONTRIBUTING.md +4 -1
  9. {authsome-0.3.1 → authsome-0.4.0}/PKG-INFO +47 -20
  10. {authsome-0.3.1 → authsome-0.4.0}/README.md +44 -19
  11. authsome-0.4.0/TODOS.md +274 -0
  12. authsome-0.4.0/assets/authsome-logo-dark.svg +31 -0
  13. authsome-0.4.0/assets/authsome-logo-light.svg +31 -0
  14. authsome-0.4.0/docs/UBIQUITOUS_LANGUAGE.md +128 -0
  15. authsome-0.4.0/docs/adr/0003-principal-owned-vault.md +33 -0
  16. authsome-0.4.0/docs/adr/0003-proxy-unmatched-pass-through.md +47 -0
  17. authsome-0.4.0/docs/agents/domain.md +37 -0
  18. authsome-0.4.0/docs/agents/issue-tracker.md +22 -0
  19. authsome-0.4.0/docs/agents/triage-labels.md +15 -0
  20. {authsome-0.3.1 → authsome-0.4.0}/docs/internal/manual-testing.md +61 -48
  21. authsome-0.4.0/docs/refactor.md +503 -0
  22. {authsome-0.3.1 → authsome-0.4.0}/docs/site/README.md +1 -1
  23. {authsome-0.3.1 → authsome-0.4.0}/docs/site/changelog.mdx +5 -5
  24. {authsome-0.3.1 → authsome-0.4.0}/docs/site/compared.mdx +1 -1
  25. {authsome-0.3.1 → authsome-0.4.0}/docs/site/concepts/credential-storage.mdx +24 -23
  26. authsome-0.4.0/docs/site/concepts/profiles-vs-connections.mdx +86 -0
  27. {authsome-0.3.1 → authsome-0.4.0}/docs/site/concepts/provider-registry.mdx +4 -4
  28. {authsome-0.3.1 → authsome-0.4.0}/docs/site/concepts/the-daemon.mdx +3 -3
  29. {authsome-0.3.1 → authsome-0.4.0}/docs/site/docs.json +2 -0
  30. authsome-0.4.0/docs/site/favicon.svg +106 -0
  31. {authsome-0.3.1 → authsome-0.4.0}/docs/site/guides/custom-providers.mdx +10 -10
  32. {authsome-0.3.1 → authsome-0.4.0}/docs/site/guides/headless-device-code.mdx +4 -4
  33. {authsome-0.3.1 → authsome-0.4.0}/docs/site/guides/login-with-oauth.mdx +12 -12
  34. {authsome-0.3.1 → authsome-0.4.0}/docs/site/guides/multiple-connections.mdx +14 -14
  35. {authsome-0.3.1 → authsome-0.4.0}/docs/site/guides/profiles.mdx +1 -1
  36. {authsome-0.3.1 → authsome-0.4.0}/docs/site/guides/run-agents-with-proxy.mdx +6 -6
  37. {authsome-0.3.1 → authsome-0.4.0}/docs/site/guides/use-api-keys.mdx +15 -15
  38. authsome-0.4.0/docs/site/images/login-github-authsome.png +0 -0
  39. {authsome-0.3.1 → authsome-0.4.0}/docs/site/index.mdx +2 -2
  40. {authsome-0.3.1 → authsome-0.4.0}/docs/site/installation.mdx +40 -28
  41. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/anthropic-sdk.mdx +5 -5
  42. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/claude-code.mdx +23 -19
  43. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/codex.mdx +18 -14
  44. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/cowork.mdx +4 -4
  45. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/cursor.mdx +17 -9
  46. authsome-0.4.0/docs/site/integrations/agents/hermes.mdx +166 -0
  47. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/index.mdx +7 -4
  48. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/langchain.mdx +3 -3
  49. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/llamaindex.mdx +6 -6
  50. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/nanoclaw.mdx +9 -7
  51. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/openai-agents-sdk.mdx +5 -5
  52. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/opencode.mdx +11 -7
  53. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/agents/python.mdx +3 -3
  54. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/ahrefs.mdx +11 -11
  55. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/apollo.mdx +11 -11
  56. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/ashby.mdx +11 -11
  57. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/beehiiv.mdx +11 -11
  58. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/brevo.mdx +11 -11
  59. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/buffer.mdx +11 -11
  60. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/calendly.mdx +11 -11
  61. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/clearbit.mdx +11 -11
  62. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/dub.mdx +11 -11
  63. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/g2.mdx +11 -11
  64. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/hunter.mdx +11 -11
  65. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/index.mdx +5 -5
  66. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/instantly.mdx +11 -11
  67. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/intercom.mdx +11 -11
  68. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/keywords-everywhere.mdx +11 -11
  69. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/klaviyo.mdx +11 -11
  70. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/lemlist.mdx +11 -11
  71. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/livestorm.mdx +11 -11
  72. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/mailchimp.mdx +11 -11
  73. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/mention-me.mdx +11 -11
  74. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/openai.mdx +15 -15
  75. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/optimizely.mdx +11 -11
  76. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/postmark.mdx +11 -11
  77. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/resend.mdx +11 -11
  78. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/rewardful.mdx +11 -11
  79. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/savvycal.mdx +11 -11
  80. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/semrush.mdx +11 -11
  81. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/sendgrid.mdx +11 -11
  82. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/tolt.mdx +11 -11
  83. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/typeform.mdx +11 -11
  84. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/wistia.mdx +11 -11
  85. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/api-key/zapier.mdx +11 -11
  86. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/atlassian.mdx +10 -10
  87. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/discord.mdx +10 -10
  88. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/github.mdx +52 -27
  89. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/gitlab.mdx +11 -11
  90. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/google.mdx +11 -11
  91. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/hubspot.mdx +10 -10
  92. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/index.mdx +3 -3
  93. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/klaviyo-oauth.mdx +10 -10
  94. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/linear.mdx +10 -10
  95. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/microsoft.mdx +11 -11
  96. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/notion-dcr.mdx +8 -8
  97. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/notion.mdx +8 -8
  98. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/postiz.mdx +8 -8
  99. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/slack.mdx +10 -10
  100. {authsome-0.3.1 → authsome-0.4.0}/docs/site/integrations/oauth/x.mdx +10 -10
  101. authsome-0.4.0/docs/site/logo/dark.svg +31 -0
  102. authsome-0.4.0/docs/site/logo/light.svg +31 -0
  103. authsome-0.4.0/docs/site/quickstart.mdx +303 -0
  104. {authsome-0.3.1 → authsome-0.4.0}/docs/site/reference/audit-log.mdx +3 -3
  105. {authsome-0.3.1 → authsome-0.4.0}/docs/site/reference/bundled-providers.mdx +5 -5
  106. {authsome-0.3.1 → authsome-0.4.0}/docs/site/reference/cli.mdx +51 -51
  107. {authsome-0.3.1 → authsome-0.4.0}/docs/site/reference/daemon-api.mdx +29 -5
  108. {authsome-0.3.1 → authsome-0.4.0}/docs/site/reference/environment-variables.mdx +14 -14
  109. {authsome-0.3.1 → authsome-0.4.0}/docs/site/reference/file-layout.mdx +10 -10
  110. {authsome-0.3.1 → authsome-0.4.0}/docs/site/reference/python-library.mdx +4 -4
  111. authsome-0.4.0/docs/site/roadmap.mdx +55 -0
  112. {authsome-0.3.1 → authsome-0.4.0}/docs/site/security/disclosure.mdx +1 -1
  113. {authsome-0.3.1 → authsome-0.4.0}/docs/site/security/encryption.mdx +23 -7
  114. {authsome-0.3.1 → authsome-0.4.0}/docs/site/security/hosted-deployment.mdx +1 -1
  115. {authsome-0.3.1 → authsome-0.4.0}/docs/site/security/threat-model.mdx +1 -1
  116. authsome-0.4.0/docs/site/troubleshooting/auth-errors.mdx +91 -0
  117. {authsome-0.3.1 → authsome-0.4.0}/docs/site/troubleshooting/daemon-issues.mdx +13 -13
  118. {authsome-0.3.1 → authsome-0.4.0}/docs/site/troubleshooting/doctor.mdx +5 -5
  119. {authsome-0.3.1 → authsome-0.4.0}/docs/site/troubleshooting/oauth-callbacks.mdx +7 -7
  120. {authsome-0.3.1 → authsome-0.4.0}/docs/site/troubleshooting/proxy-networking.mdx +7 -7
  121. {authsome-0.3.1 → authsome-0.4.0}/docs/site/troubleshooting/token-refresh.mdx +12 -12
  122. authsome-0.4.0/evals/.gitignore +2 -0
  123. authsome-0.4.0/evals/evals.json +85 -0
  124. authsome-0.4.0/evals/generate_report.py +278 -0
  125. {authsome-0.3.1 → authsome-0.4.0}/pyproject.toml +6 -1
  126. authsome-0.4.0/skills/authsome/SKILL.md +112 -0
  127. authsome-0.4.0/skills/authsome/references/adding-provider.md +19 -0
  128. authsome-0.4.0/skills/authsome/references/feedback.md +85 -0
  129. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/__init__.py +0 -3
  130. authsome-0.4.0/src/authsome/auth/__init__.py +1 -0
  131. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/flows/api_key.py +2 -2
  132. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/flows/base.py +2 -2
  133. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/flows/dcr_pkce.py +2 -2
  134. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/flows/device_code.py +2 -2
  135. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/flows/pkce.py +2 -2
  136. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/models/config.py +1 -1
  137. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/models/connection.py +12 -6
  138. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/sessions.py +5 -2
  139. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/utils.py +2 -10
  140. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/cli/client.py +72 -14
  141. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/cli/client_config.py +17 -1
  142. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/cli/daemon_control.py +60 -16
  143. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/cli/main.py +128 -26
  144. authsome-0.4.0/src/authsome/identity/__init__.py +73 -0
  145. authsome-0.3.1/src/authsome/identity/keys.py → authsome-0.4.0/src/authsome/identity/local.py +62 -21
  146. authsome-0.4.0/src/authsome/identity/principal.py +61 -0
  147. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/identity/proof.py +1 -1
  148. authsome-0.4.0/src/authsome/identity/registry.py +19 -0
  149. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/proxy/runner.py +4 -2
  150. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/proxy/server.py +79 -14
  151. authsome-0.4.0/src/authsome/server/analytics.py +63 -0
  152. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/app.py +29 -10
  153. authsome-0.3.1/src/authsome/auth/service.py → authsome-0.4.0/src/authsome/server/credential_service.py +152 -95
  154. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/dependencies.py +108 -7
  155. authsome-0.4.0/src/authsome/server/hosted_auth.py +114 -0
  156. authsome-0.4.0/src/authsome/server/identity_bootstrap.py +116 -0
  157. authsome-0.4.0/src/authsome/server/ownership.py +116 -0
  158. authsome-0.4.0/src/authsome/server/proxy_catalog.py +73 -0
  159. authsome-0.4.0/src/authsome/server/registries.py +266 -0
  160. authsome-0.4.0/src/authsome/server/routes/_deps.py +180 -0
  161. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/routes/auth.py +103 -8
  162. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/routes/connections.py +40 -5
  163. authsome-0.4.0/src/authsome/server/routes/health.py +153 -0
  164. authsome-0.4.0/src/authsome/server/routes/identities.py +45 -0
  165. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/routes/providers.py +19 -1
  166. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/routes/proxy.py +18 -4
  167. authsome-0.4.0/src/authsome/server/routes/ui.py +804 -0
  168. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/schemas.py +6 -0
  169. authsome-0.4.0/src/authsome/server/ui/pages.py +421 -0
  170. authsome-0.4.0/src/authsome/server/ui_sessions.py +146 -0
  171. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/static/app.js +32 -2
  172. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/static/style.css +85 -0
  173. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/templates/_layout.html +14 -8
  174. authsome-0.4.0/src/authsome/ui/templates/app_detail_managed.html +24 -0
  175. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/templates/app_detail_oauth.html +0 -46
  176. authsome-0.4.0/src/authsome/ui/templates/app_provider.html +128 -0
  177. authsome-0.4.0/src/authsome/ui/templates/applications.html +57 -0
  178. authsome-0.4.0/src/authsome/ui/templates/connections.html +51 -0
  179. authsome-0.4.0/src/authsome/ui/templates/identity.html +16 -0
  180. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/templates/overview.html +1 -1
  181. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/utils.py +40 -1
  182. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/vault/__init__.py +62 -7
  183. authsome-0.4.0/src/authsome/vault/crypto.py +365 -0
  184. {authsome-0.3.1 → authsome-0.4.0}/tests/auth/test_flows.py +18 -0
  185. {authsome-0.3.1 → authsome-0.4.0}/tests/auth/test_models.py +6 -2
  186. {authsome-0.3.1 → authsome-0.4.0}/tests/auth/test_service.py +21 -5
  187. {authsome-0.3.1 → authsome-0.4.0}/tests/auth/test_service_provider_clients.py +63 -40
  188. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/conftest.py +9 -1
  189. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_client_signing.py +97 -4
  190. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_daemon.py +11 -1
  191. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_identity.py +1 -1
  192. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_init.py +12 -4
  193. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_ui.py +4 -4
  194. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_whoami.py +25 -1
  195. {authsome-0.3.1 → authsome-0.4.0}/tests/common/test_utils.py +4 -0
  196. {authsome-0.3.1 → authsome-0.4.0}/tests/conftest.py +1 -0
  197. {authsome-0.3.1 → authsome-0.4.0}/tests/identity/test_identity.py +27 -1
  198. {authsome-0.3.1 → authsome-0.4.0}/tests/identity/test_proof.py +1 -1
  199. authsome-0.4.0/tests/identity/test_registry.py +41 -0
  200. {authsome-0.3.1 → authsome-0.4.0}/tests/proxy/test_proxy.py +82 -6
  201. authsome-0.4.0/tests/server/test_analytics.py +43 -0
  202. {authsome-0.3.1 → authsome-0.4.0}/tests/server/test_auth_sessions.py +8 -2
  203. authsome-0.4.0/tests/server/test_hosted_auth.py +83 -0
  204. authsome-0.4.0/tests/server/test_identity_bootstrap.py +66 -0
  205. authsome-0.4.0/tests/server/test_ownership.py +63 -0
  206. authsome-0.4.0/tests/server/test_pop_auth.py +239 -0
  207. {authsome-0.3.1 → authsome-0.4.0}/tests/server/test_provider_operation_policy.py +14 -1
  208. authsome-0.4.0/tests/server/test_ui_dashboard.py +480 -0
  209. authsome-0.4.0/tests/server/test_ui_sessions.py +252 -0
  210. {authsome-0.3.1 → authsome-0.4.0}/tests/vault/test_crypto.py +117 -3
  211. authsome-0.4.0/tests/vault/test_rekey.py +167 -0
  212. {authsome-0.3.1 → authsome-0.4.0}/uv.lock +155 -117
  213. authsome-0.3.1/.github/release-please-manifest.json +0 -3
  214. authsome-0.3.1/assets/authsome-logo-dark.svg +0 -6
  215. authsome-0.3.1/assets/authsome-logo-light.svg +0 -6
  216. authsome-0.3.1/docs/UBIQUITOUS_LANGUAGE.md +0 -111
  217. authsome-0.3.1/docs/site/concepts/profiles-vs-connections.mdx +0 -102
  218. authsome-0.3.1/docs/site/favicon.svg +0 -4
  219. authsome-0.3.1/docs/site/logo/dark.svg +0 -6
  220. authsome-0.3.1/docs/site/logo/light.svg +0 -6
  221. authsome-0.3.1/docs/site/quickstart.mdx +0 -109
  222. authsome-0.3.1/docs/site/roadmap.mdx +0 -103
  223. authsome-0.3.1/skills/authsome/SKILL.md +0 -84
  224. authsome-0.3.1/skills/authsome/evals/evals.json +0 -29
  225. authsome-0.3.1/src/authsome/auth/__init__.py +0 -8
  226. authsome-0.3.1/src/authsome/identity/__init__.py +0 -44
  227. authsome-0.3.1/src/authsome/identity/registry.py +0 -87
  228. authsome-0.3.1/src/authsome/server/routes/_deps.py +0 -96
  229. authsome-0.3.1/src/authsome/server/routes/health.py +0 -111
  230. authsome-0.3.1/src/authsome/server/routes/identities.py +0 -30
  231. authsome-0.3.1/src/authsome/server/routes/ui.py +0 -485
  232. authsome-0.3.1/src/authsome/server/ui/pages.py +0 -199
  233. authsome-0.3.1/src/authsome/server/ui_sessions.py +0 -128
  234. authsome-0.3.1/src/authsome/ui/templates/connections.html +0 -60
  235. authsome-0.3.1/src/authsome/vault/crypto.py +0 -153
  236. authsome-0.3.1/tests/server/test_pop_auth.py +0 -125
  237. authsome-0.3.1/tests/server/test_ui_sessions.py +0 -115
  238. {authsome-0.3.1 → authsome-0.4.0}/.github/ISSUE_TEMPLATE/bug_report.yml +0 -0
  239. {authsome-0.3.1 → authsome-0.4.0}/.github/ISSUE_TEMPLATE/feature_request.yml +0 -0
  240. {authsome-0.3.1 → authsome-0.4.0}/.github/dependabot.yml +0 -0
  241. {authsome-0.3.1 → authsome-0.4.0}/.github/pull_request_template.md +0 -0
  242. {authsome-0.3.1 → authsome-0.4.0}/.github/release-please-config.json +0 -0
  243. {authsome-0.3.1 → authsome-0.4.0}/.github/workflows/pr-title.yml +0 -0
  244. {authsome-0.3.1 → authsome-0.4.0}/.github/workflows/publish-rc.yml +0 -0
  245. {authsome-0.3.1 → authsome-0.4.0}/.github/workflows/publish.yml +0 -0
  246. {authsome-0.3.1 → authsome-0.4.0}/.github/workflows/release-please.yml +0 -0
  247. {authsome-0.3.1 → authsome-0.4.0}/.github/workflows/test.yml +0 -0
  248. {authsome-0.3.1 → authsome-0.4.0}/.pre-commit-config.yaml +0 -0
  249. {authsome-0.3.1 → authsome-0.4.0}/CLAUDE.md +0 -0
  250. {authsome-0.3.1 → authsome-0.4.0}/LICENSE +0 -0
  251. {authsome-0.3.1 → authsome-0.4.0}/assets/authsome-how-it-works-dark.svg +0 -0
  252. {authsome-0.3.1 → authsome-0.4.0}/assets/authsome-how-it-works-light.svg +0 -0
  253. {authsome-0.3.1 → authsome-0.4.0}/docs/adr/0001-provider-client-record-server-scope.md +0 -0
  254. {authsome-0.3.1 → authsome-0.4.0}/docs/adr/0002-server-registered-identities.md +0 -0
  255. {authsome-0.3.1 → authsome-0.4.0}/docs/internal/authsome-design.md +0 -0
  256. {authsome-0.3.1 → authsome-0.4.0}/docs/internal/cli-design-review.md +0 -0
  257. {authsome-0.3.1 → authsome-0.4.0}/docs/register-provider.md +0 -0
  258. {authsome-0.3.1 → authsome-0.4.0}/docs/site/concepts/architecture.mdx +0 -0
  259. {authsome-0.3.1 → authsome-0.4.0}/docs/site/concepts/proxy-injection.mdx +0 -0
  260. {authsome-0.3.1 → authsome-0.4.0}/docs/site/reference/provider-schema.mdx +0 -0
  261. {authsome-0.3.1 → authsome-0.4.0}/docs/site/security/daemon-trust-boundary.mdx +0 -0
  262. {authsome-0.3.1 → authsome-0.4.0}/docs/site/snippets/masked-input-note.mdx +0 -0
  263. {authsome-0.3.1 → authsome-0.4.0}/docs/site/snippets/multi-connections-cta.mdx +0 -0
  264. {authsome-0.3.1 → authsome-0.4.0}/docs/site/snippets/whats-next-apikey.mdx +0 -0
  265. {authsome-0.3.1 → authsome-0.4.0}/docs/site/snippets/whats-next-oauth.mdx +0 -0
  266. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/audit/__init__.py +0 -0
  267. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/__init__.py +0 -0
  268. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/ahrefs.json +0 -0
  269. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/apollo.json +0 -0
  270. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/ashby.json +0 -0
  271. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/atlassian.json +0 -0
  272. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/beehiiv.json +0 -0
  273. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/brevo.json +0 -0
  274. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/buffer.json +0 -0
  275. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/calendly.json +0 -0
  276. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/clearbit.json +0 -0
  277. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/discord.json +0 -0
  278. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/dub.json +0 -0
  279. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/g2.json +0 -0
  280. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/github.json +0 -0
  281. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/gitlab.json +0 -0
  282. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/google.json +0 -0
  283. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/hubspot.json +0 -0
  284. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/hunter.json +0 -0
  285. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/instantly.json +0 -0
  286. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/intercom.json +0 -0
  287. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/keywords-everywhere.json +0 -0
  288. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/klaviyo-oauth.json +0 -0
  289. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/klaviyo.json +0 -0
  290. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/lemlist.json +0 -0
  291. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/linear.json +0 -0
  292. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/livestorm.json +0 -0
  293. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/mailchimp.json +0 -0
  294. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/mention-me.json +0 -0
  295. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/microsoft.json +0 -0
  296. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/notion.json +0 -0
  297. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/notion_dcr.json +0 -0
  298. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/openai.json +0 -0
  299. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/optimizely.json +0 -0
  300. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/postiz.json +0 -0
  301. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/postmark.json +0 -0
  302. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/resend.json +0 -0
  303. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/rewardful.json +0 -0
  304. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/savvycal.json +0 -0
  305. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/semrush.json +0 -0
  306. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/sendgrid.json +0 -0
  307. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/slack.json +0 -0
  308. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/tolt.json +0 -0
  309. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/typeform.json +0 -0
  310. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/wistia.json +0 -0
  311. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/x.json +0 -0
  312. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/bundled_providers/zapier.json +0 -0
  313. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/flows/__init__.py +0 -0
  314. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/input_provider.py +0 -0
  315. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/models/__init__.py +0 -0
  316. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/models/enums.py +0 -0
  317. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/auth/models/provider.py +0 -0
  318. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/cli/__init__.py +0 -0
  319. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/cli/context.py +0 -0
  320. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/cli/helpers.py +0 -0
  321. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/errors.py +0 -0
  322. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/paths.py +0 -0
  323. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/proxy/__init__.py +0 -0
  324. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/proxy/certs.py +0 -0
  325. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/proxy/router.py +0 -0
  326. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/py.typed +0 -0
  327. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/__init__.py +0 -0
  328. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/daemon.py +0 -0
  329. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/routes/__init__.py +0 -0
  330. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/ui/__init__.py +0 -0
  331. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/ui/web_theme.py +0 -0
  332. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/server/urls.py +0 -0
  333. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/store/__init__.py +0 -0
  334. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/store/interfaces.py +0 -0
  335. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/store/local.py +0 -0
  336. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/__init__.py +0 -0
  337. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/templates/_app_detail_shell.html +0 -0
  338. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/templates/app_detail_apikey.html +0 -0
  339. {authsome-0.3.1 → authsome-0.4.0}/src/authsome/ui/templates/app_detail_disconnected.html +0 -0
  340. {authsome-0.3.1 → authsome-0.4.0}/tests/__init__.py +0 -0
  341. {authsome-0.3.1 → authsome-0.4.0}/tests/auth/__init__.py +0 -0
  342. {authsome-0.3.1 → authsome-0.4.0}/tests/auth/test_url_template.py +0 -0
  343. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/__init__.py +0 -0
  344. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_doctor.py +0 -0
  345. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_get.py +0 -0
  346. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_helpers.py +0 -0
  347. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_import_env.py +0 -0
  348. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_list.py +0 -0
  349. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_login.py +0 -0
  350. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_logout.py +0 -0
  351. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_register.py +0 -0
  352. {authsome-0.3.1 → authsome-0.4.0}/tests/cli/test_revoke.py +0 -0
  353. {authsome-0.3.1 → authsome-0.4.0}/tests/common/__init__.py +0 -0
  354. {authsome-0.3.1 → authsome-0.4.0}/tests/common/test_audit.py +0 -0
  355. {authsome-0.3.1 → authsome-0.4.0}/tests/common/test_errors.py +0 -0
  356. {authsome-0.3.1 → authsome-0.4.0}/tests/common/test_logging.py +0 -0
  357. {authsome-0.3.1 → authsome-0.4.0}/tests/proxy/__init__.py +0 -0
  358. {authsome-0.3.1 → authsome-0.4.0}/tests/vault/__init__.py +0 -0
@@ -0,0 +1,495 @@
1
+ # Run Authsome Evals
2
+
3
+ Interactive eval runner for the authsome skill. You orchestrate everything
4
+ inline — agent invocation, transcript parsing, grading, and result saving.
5
+ There is no separate Python runner script.
6
+
7
+ ## Pre-session setup
8
+
9
+ Run this once before starting an eval session.
10
+
11
+ **1. Install the latest authsome CLI:**
12
+
13
+ ```bash
14
+ uv sync
15
+ ```
16
+
17
+ Verify:
18
+
19
+ ```bash
20
+ uv run authsome --version
21
+ ```
22
+
23
+ **2. Create a fresh identity and verify the existing one still works:**
24
+
25
+ ```bash
26
+ # Check the current identity is healthy
27
+ uv run authsome doctor
28
+
29
+ # Create a new identity for the eval session
30
+ uv run authsome profile create --json
31
+ ```
32
+
33
+ Save the new `profile` handle. Then switch to it:
34
+
35
+ ```bash
36
+ uv run authsome profile use <new-handle>
37
+ ```
38
+
39
+ **3. Confirm the new identity starts clean:**
40
+
41
+ ```bash
42
+ uv run authsome list
43
+ ```
44
+
45
+ Expected: no providers connected. If any show `connected`, the wrong
46
+ profile may be active — check with `cat ~/.authsome/client/config.json`.
47
+
48
+ **4. Restart the daemon using the dev version:**
49
+
50
+ The daemon may be running as a globally tool-installed binary while the CLI runs via `uv run`. This version mismatch causes PoP auth failures (spurious 401s) that confuse agents into running `authsome init` mid-eval, corrupting the eval profile. Restart to ensure both CLI and daemon use the same code:
51
+
52
+ ```bash
53
+ uv run authsome daemon restart
54
+ ```
55
+
56
+ Verify:
57
+
58
+ ```bash
59
+ uv run authsome list
60
+ ```
61
+
62
+ Expected: the same clean state as before. If the daemon fails to restart, check for port conflicts with `lsof -i :7998`.
63
+
64
+ **5. Remove hermes GitHub skills to avoid interference with authsome triggering:**
65
+
66
+ ```bash
67
+ rm -rf ~/.hermes/skills/github
68
+ ```
69
+
70
+ This prevents hermes from using its bundled GitHub skills instead of loading authsome.
71
+
72
+ **6. Verify hermes and claude are working:**
73
+
74
+ ```bash
75
+ hermes chat -Q -q "reply with the single word OK" -t ""
76
+ claude -p "reply with the single word OK" --output-format text
77
+ ```
78
+
79
+ Expected: both respond with `OK`. Hermes runs the eval agents; claude is the LLM judge.
80
+
81
+ ---
82
+
83
+ ## Arguments
84
+
85
+ - No args — run all non-optional evals (ids 1–6)
86
+ - `--id N` — run only eval with that id
87
+ - `--all` — include optional evals (currently id 7: Agentic Installation)
88
+
89
+ ---
90
+
91
+ ## Steps
92
+
93
+ ### 1. Load evals
94
+
95
+ Read `evals/evals.json`. Show the user a table of which
96
+ evals will run (id, name, agent, requires_human, optional).
97
+
98
+ Read `~/.authsome/client/config.json` and save `active_identity` as
99
+ `EVAL_HANDLE` — this is the fresh profile created during pre-session setup.
100
+
101
+ Create the run directory and save as `RUN_DIR`:
102
+
103
+ ```bash
104
+ mkdir -p "evals/results/$(date +%Y%m%d_%H%M%S)"
105
+ ```
106
+
107
+ ---
108
+
109
+ ### 2. Per-eval loop
110
+
111
+ For each eval to run, **in order**:
112
+
113
+ #### a. State-check
114
+
115
+ Run `uv run authsome list` and **show the full output to the user**.
116
+ Compare it against the eval's `environment` field and explicitly state
117
+ whether it matches. If it matches, proceed. If not, show the mismatch
118
+ and fix it inline using `uv run authsome` commands (e.g.
119
+ `uv run authsome logout github`). Re-check until state matches.
120
+
121
+ If the required state cannot be reached automatically (e.g. gh CLI login
122
+ requires interactive browser auth that isn't part of the eval), ask the
123
+ user whether to skip. If they say skip, write a null verdict and record
124
+ it in grading.json, then move to the next eval:
125
+
126
+ ```bash
127
+ # Write null verdict
128
+ cat > RUN_DIR/verdict_N.json <<'EOF'
129
+ {
130
+ "outcome": {"passed": null, "evidence": "skipped by user"},
131
+ "trajectory_efficiency": {"passed": null, "evidence": "skipped by user"}
132
+ }
133
+ EOF
134
+
135
+ # Capture authsome state
136
+ uv run authsome list > RUN_DIR/authsome_state_N.txt 2>&1
137
+
138
+ # Append to grading.json (same save script as step f, RATE_LIMITED=false)
139
+ ```
140
+
141
+ Then run the step-f save script for this eval and continue to the next one.
142
+
143
+ For `requires_human` evals, also show `human_instructions` now.
144
+
145
+ #### b. Install the skill for the agent
146
+
147
+ ```bash
148
+ # For hermes evals
149
+ rm -rf ~/.hermes/skills/authsome
150
+ cp -r skills/authsome ~/.hermes/skills/authsome
151
+
152
+ # For claude evals
153
+ rm -rf .claude/skills/authsome
154
+ mkdir -p .claude/skills
155
+ cp -r skills/authsome .claude/skills/authsome
156
+ ```
157
+
158
+ #### c. Run the agent
159
+
160
+ Before running the agent, read `max_turns` from the eval object (default
161
+ `12` if absent) and store it as `MAX_TURNS`.
162
+
163
+ **Hermes evals:**
164
+
165
+ ```bash
166
+ hermes chat -v -q "PROMPT" --yolo --max-turns MAX_TURNS \
167
+ 2>&1 | tee RUN_DIR/transcript_N.txt
168
+ ```
169
+
170
+ The combined stdout+stderr is the transcript. Save `RATE_LIMITED=false`
171
+ unless the output contains: `rate limit`, `429`, `too many requests`,
172
+ `usage limit`, or `quota exceeded`.
173
+
174
+ **Claude evals — turn 1:**
175
+
176
+ ```bash
177
+ claude --dangerously-skip-permissions --verbose --output-format stream-json \
178
+ --max-turns MAX_TURNS -p "PROMPT" > RUN_DIR/raw_N_t1.jsonl 2>&1
179
+ ```
180
+
181
+ Then parse the raw stream-json, extract the human-readable transcript,
182
+ and detect whether the agent is waiting for a human action:
183
+
184
+ ```bash
185
+ uv run python - RUN_DIR/raw_N_t1.jsonl > RUN_DIR/transcript_N.txt 2> RUN_DIR/meta_N.txt <<'PYEOF'
186
+ import sys, json, re
187
+
188
+ RATE_LIMIT_SIGNALS = ["rate limit", "429", "too many requests", "usage limit", "quota exceeded"]
189
+ path = sys.argv[1]
190
+ lines_out = []
191
+ session_id = None
192
+
193
+ for line in open(path):
194
+ line = line.strip()
195
+ if not line:
196
+ continue
197
+ try:
198
+ ev = json.loads(line)
199
+ except json.JSONDecodeError:
200
+ lines_out.append(line)
201
+ continue
202
+ t = ev.get("type", "")
203
+ if t == "assistant" and "message" in ev:
204
+ for block in ev["message"].get("content", []):
205
+ if block.get("type") == "text":
206
+ lines_out.append(f"[assistant] {block['text']}")
207
+ elif block.get("type") == "tool_use":
208
+ inp = json.dumps(block.get("input", {}))
209
+ lines_out.append(f"[tool_use] {block['name']}({inp})")
210
+ elif t == "user" and "message" in ev:
211
+ for block in ev["message"].get("content", []):
212
+ if block.get("type") == "tool_result":
213
+ content = block.get("content", "")
214
+ if isinstance(content, list):
215
+ content = " ".join(
216
+ c.get("text", "") for c in content if isinstance(c, dict)
217
+ )
218
+ lines_out.append(f"[tool_result] {str(content)[:800]}")
219
+ elif t == "result":
220
+ if ev.get("result"):
221
+ lines_out.append(f"[result] {ev['result']}")
222
+ if ev.get("error"):
223
+ lines_out.append(f"[error] {ev['error']}")
224
+ if ev.get("session_id"):
225
+ session_id = ev["session_id"]
226
+
227
+ transcript = "\n".join(lines_out)
228
+ print(transcript)
229
+
230
+ # Emit metadata to stderr for the caller to read
231
+ url_match = re.search(r'http://127\.0\.0\.1:\d+/\S+', transcript)
232
+ if url_match:
233
+ print(f"WAITING_URL={url_match.group()}", file=sys.stderr)
234
+ if session_id:
235
+ print(f"SESSION_ID={session_id}", file=sys.stderr)
236
+ rate_limited = any(sig in transcript.lower() for sig in RATE_LIMIT_SIGNALS)
237
+ print(f"RATE_LIMITED={'true' if rate_limited else 'false'}", file=sys.stderr)
238
+ PYEOF
239
+ ```
240
+
241
+ Read `RUN_DIR/meta_N.txt` and extract:
242
+
243
+ ```bash
244
+ SESSION_ID=$(grep "^SESSION_ID=" RUN_DIR/meta_N.txt | cut -d= -f2)
245
+ WAITING_URL=$(grep "^WAITING_URL=" RUN_DIR/meta_N.txt | cut -d= -f2)
246
+ RATE_LIMITED=$(grep "^RATE_LIMITED=" RUN_DIR/meta_N.txt | cut -d= -f2)
247
+ ```
248
+
249
+ #### d. Human handoff (requires_human evals only)
250
+
251
+ **Case 1 — Agent-initiated interrupt (`expected_interrupt` is set):**
252
+
253
+ If the eval has an `expected_interrupt` field, read `RUN_DIR/transcript_N.txt` and judge
254
+ whether the agent's final message matches the described interrupt — i.e. the agent paused
255
+ mid-task to ask the user a clarifying question or request input instead of proceeding
256
+ autonomously. If it matches, auto-resume without human input by sending `next_turn_instruction`
257
+ back to the session:
258
+
259
+ ```bash
260
+ claude --resume SESSION_ID \
261
+ --dangerously-skip-permissions --verbose --output-format stream-json \
262
+ --max-turns MAX_TURNS -p "NEXT_TURN_INSTRUCTION" > RUN_DIR/raw_N_t2.jsonl 2>&1
263
+ ```
264
+
265
+ Parse the continuation with the same parse script (substitute `raw_N_t2.jsonl` and
266
+ `meta_N_t2.txt`) and append to `RUN_DIR/transcript_N.txt`. Update `RATE_LIMITED` from
267
+ `meta_N_t2.txt`. Then continue to step e for grading — do not prompt the human unless
268
+ `WAITING_URL` is non-empty in the resumed turn.
269
+
270
+ **Case 2 — Browser auth flow (`WAITING_URL` is non-empty):**
271
+
272
+ If `WAITING_URL` is non-empty, the agent started an auth flow and is
273
+ suspended at its session boundary. Wait 5 seconds, then check whether
274
+ the auth flow completed automatically:
275
+
276
+ ```bash
277
+ sleep 5 && uv run authsome list
278
+ ```
279
+
280
+ If the relevant provider now shows `connected`, proceed directly to resuming
281
+ the session (skip the user prompt). If it is still `not_connected`, show the user:
282
+
283
+ > The agent is waiting. Please complete the auth flow at: `WAITING_URL`
284
+ > Tell me "done" when finished.
285
+
286
+ Wait for the user to reply "done". Then resume the claude session and
287
+ append the continuation to the transcript:
288
+
289
+ ```bash
290
+ claude --resume SESSION_ID \
291
+ --dangerously-skip-permissions --verbose --output-format stream-json \
292
+ --max-turns MAX_TURNS -p "done" > RUN_DIR/raw_N_t2.jsonl 2>&1
293
+ ```
294
+
295
+ Parse turn 2 with the same script above (substitute `raw_N_t2.jsonl` and
296
+ `meta_N_t2.txt`), then append its transcript to `RUN_DIR/transcript_N.txt`:
297
+
298
+ ```bash
299
+ uv run python - RUN_DIR/raw_N_t2.jsonl >> RUN_DIR/transcript_N.txt 2> RUN_DIR/meta_N_t2.txt <<'PYEOF'
300
+ # ... same parse script as above ...
301
+ PYEOF
302
+
303
+ # Update RATE_LIMITED if turn 2 was rate-limited
304
+ RATE_LIMITED_T2=$(grep "^RATE_LIMITED=" RUN_DIR/meta_N_t2.txt | cut -d= -f2)
305
+ [ "$RATE_LIMITED_T2" = "true" ] && RATE_LIMITED=true
306
+ ```
307
+
308
+ If `WAITING_URL` is empty and no agent-initiated interrupt was detected for a `requires_human`
309
+ eval, the agent finished in one turn (e.g. it polled for completion itself) — no resume needed.
310
+
311
+ #### e. Grade the transcript
312
+
313
+ Call claude as the LLM judge with the full eval criteria and the transcript:
314
+
315
+ ```bash
316
+ uv run python - RUN_DIR/transcript_N.txt EVAL_ID <<'PYEOF' > RUN_DIR/verdict_N.json
317
+ import sys, json, subprocess
318
+ from pathlib import Path
319
+
320
+ transcript = Path(sys.argv[1]).read_text()
321
+ eval_id = int(sys.argv[2])
322
+ evals = json.loads(Path("evals/evals.json").read_text())["evals"]
323
+ eval_ = next(e for e in evals if e["id"] == eval_id)
324
+
325
+ JUDGE_PROMPT = """\
326
+ You are an eval grader for an agent called Authsome. You receive:
327
+ - An agent transcript (stdout+stderr from a live agent run)
328
+ - Environment pre-conditions describing the starting state
329
+ - An outcome criterion (did the task succeed?)
330
+ - An optional trajectory_efficiency criterion (did the agent take the right number of meaningful steps?)
331
+
332
+ Return a JSON object with this exact structure:
333
+ {
334
+ "outcome": {"passed": true, "evidence": "one sentence quoting or describing transcript evidence"},
335
+ "trajectory_efficiency": {"passed": true, "evidence": "one sentence quoting or describing transcript evidence"}
336
+ }
337
+
338
+ Rules:
339
+ - Grade outcome and trajectory_efficiency independently.
340
+ - When counting meaningful steps for trajectory_efficiency, **ignore scaffolding steps**:
341
+ skill loading or calling skill tool, using one extra step to parse and format a
342
+ response, returning results to the user, reading --help, version checks, and similar
343
+ overhead. Only task-relevant actions count (API calls, auth flows, etc).
344
+ - The actual number of LLM calls will be higher than the expected step count — this is normal.
345
+ - If trajectory_efficiency criterion is absent, return {"passed": null, "evidence": "not evaluated"} for it.
346
+ - Be strict: burden of proof to pass is on the transcript.
347
+ - evidence must quote or specifically reference the transcript, not repeat the criterion.\
348
+ """
349
+
350
+ prompt = f"""{JUDGE_PROMPT}
351
+
352
+ Environment: {eval_["environment"]}
353
+
354
+ Outcome criterion: {eval_["outcome"]}
355
+
356
+ Trajectory efficiency criterion: {eval_.get("trajectory_efficiency", "(not provided — skip this grade)")}
357
+
358
+ Full transcript:
359
+ ---
360
+ {transcript}
361
+ ---
362
+
363
+ Return ONLY valid JSON, no markdown fences."""
364
+
365
+ result = subprocess.run(
366
+ ["claude", "-p", prompt, "--output-format", "text"],
367
+ capture_output=True, text=True, timeout=120,
368
+ )
369
+
370
+ if result.returncode != 0:
371
+ raise RuntimeError(f"claude judge failed (exit {result.returncode}): {result.stderr[:300]}")
372
+
373
+ raw = result.stdout.strip()
374
+ if "```" in raw:
375
+ for part in raw.split("```"):
376
+ part = part.strip().lstrip("json").strip()
377
+ if part.startswith("{"):
378
+ raw = part
379
+ break
380
+
381
+ verdict = json.loads(raw)
382
+ if "trajectory_efficiency" not in eval_:
383
+ verdict["trajectory_efficiency"] = {"passed": None, "evidence": "not evaluated"}
384
+
385
+ print(json.dumps(verdict, indent=2))
386
+ PYEOF
387
+ ```
388
+
389
+ Read `RUN_DIR/verdict_N.json` and print the result line:
390
+
391
+ ```
392
+ [result] outcome=✓/✗ trajectory=✓/✗/—
393
+ outcome : <evidence>
394
+ trajectory: <evidence>
395
+ ```
396
+
397
+ #### f. Append result to grading.json
398
+
399
+ ```bash
400
+ uv run python - RUN_DIR/grading.json EVAL_ID "$RATE_LIMITED" <<'PYEOF'
401
+ import sys, json
402
+ from datetime import datetime
403
+ from pathlib import Path
404
+
405
+ grading_path = Path(sys.argv[1])
406
+ eval_id = int(sys.argv[2])
407
+ rate_limited = sys.argv[3] == "true"
408
+
409
+ evals = json.loads(Path("evals/evals.json").read_text())["evals"]
410
+ eval_ = next(e for e in evals if e["id"] == eval_id)
411
+
412
+ run_dir = grading_path.parent
413
+ verdict = json.loads((run_dir / f"verdict_{eval_id}.json").read_text())
414
+ authsome_state = (run_dir / f"authsome_state_{eval_id}.txt").read_text() \
415
+ if (run_dir / f"authsome_state_{eval_id}.txt").exists() else ""
416
+
417
+ result_entry = {
418
+ "id": eval_id,
419
+ "name": eval_.get("name", ""),
420
+ "prompt": eval_["prompt"],
421
+ "agent": eval_.get("agent", "claude"),
422
+ "environment": eval_["environment"],
423
+ "authsome_state": authsome_state,
424
+ "requires_human": eval_.get("requires_human", False),
425
+ "rate_limited": rate_limited,
426
+ **verdict,
427
+ }
428
+
429
+ existing = {"results": []}
430
+ if grading_path.exists():
431
+ existing = json.loads(grading_path.read_text())
432
+
433
+ all_results = existing["results"] + [result_entry]
434
+ passed = sum(1 for r in all_results if r["outcome"]["passed"] is True)
435
+ failed = sum(1 for r in all_results if r["outcome"]["passed"] is False)
436
+ skipped = sum(1 for r in all_results if r["outcome"]["passed"] is None)
437
+
438
+ grading = {
439
+ "skill_name": "authsome",
440
+ "timestamp": datetime.now().isoformat(timespec="seconds"),
441
+ "summary": {"passed": passed, "failed": failed, "skipped": skipped, "total": len(all_results)},
442
+ "results": all_results,
443
+ }
444
+ grading_path.write_text(json.dumps(grading, indent=2))
445
+
446
+ summary = grading["summary"]
447
+ print(f"Done: {summary['passed']} passed / {summary['failed']} failed / {summary['skipped']} skipped out of {summary['total']}")
448
+ print(f"Results: {grading_path}")
449
+ PYEOF
450
+ ```
451
+
452
+ Before running the save script, write the current authsome state to
453
+ `RUN_DIR/authsome_state_N.txt` so it's captured in the grading record:
454
+
455
+ ```bash
456
+ uv run authsome list > RUN_DIR/authsome_state_N.txt 2>&1
457
+ ```
458
+
459
+ #### g. State-check and continue
460
+
461
+ After showing the verdict, immediately prepare for the next eval:
462
+
463
+ 1. Run `uv run authsome list` and compare against the next eval's `environment` field.
464
+ 2. Fix any mismatches inline (e.g. `uv run authsome revoke github`). Re-check until state matches.
465
+ 3. If you need to login (e.g. for provider X) and the environment says provider X is NOT connected, it is okay to use `uv run authsome login <provider>` and `uv run authsome list` to poll the status of the provider a few seconds later to see if the login was successful.
466
+ 4. Only pause and ask the user when:
467
+ - If polling fails during login (show the URL, wait for "done")
468
+ - A `requires_human` eval where the user must act during the run
469
+
470
+ ---
471
+
472
+ ### 3. Teardown
473
+
474
+ Delete the eval profile's key files:
475
+
476
+ ```bash
477
+ rm ~/.authsome/client/identities/EVAL_HANDLE.json
478
+ rm ~/.authsome/client/identities/EVAL_HANDLE.key
479
+ ```
480
+
481
+ Then switch back to the user's original profile:
482
+
483
+ ```bash
484
+ uv run authsome profile use <original-handle>
485
+ ```
486
+
487
+ ---
488
+
489
+ ### 4. Generate report
490
+
491
+ ```bash
492
+ uv run python evals/generate_report.py RUN_DIR/grading.json
493
+ ```
494
+
495
+ Tell the user the report path. The script opens it automatically.
@@ -18,7 +18,7 @@
18
18
  "name": "Agentr",
19
19
  "url": "https://github.com/agentrhq"
20
20
  },
21
- "homepage": "https://authsome.agentr.dev",
21
+ "homepage": "https://authsome.ai",
22
22
  "license": "MIT",
23
23
  "keywords": ["auth", "oauth2", "credentials", "agent-identity", "broker"],
24
24
  "category": "auth"
@@ -0,0 +1,3 @@
1
+ {
2
+ ".": "0.4.0"
3
+ }
@@ -244,5 +244,12 @@ __marimo__/
244
244
  # Git worktrees
245
245
  .worktrees/
246
246
  .claude/worktrees/
247
+
247
248
  # Local authsome home directory
248
249
  .authsome/
250
+
251
+ # Superpowers agent docs (local planning artifacts)
252
+ docs/superpowers/
253
+
254
+ # Authsome skill (generated at eval time)
255
+ .claude/skills/authsome/
@@ -56,6 +56,8 @@ These rules govern all changes to this codebase — apply them without exception
56
56
 
57
57
  **Deep modules over shallow ones.** Prefer a small surface area with rich internals over many thin wrappers. More files is not more modular.
58
58
 
59
+ **Composition over inheritance.** Prefer small collaborators wired together through explicit dependencies over inheritance hierarchies. Use inheritance only when there is a real subtype relationship and composition would make the design less clear.
60
+
59
61
  **Single responsibility and separation of concerns.** Auth authenticates. Vault stores credentials. CLI presents output. A flow must not write to storage; storage must not know about OAuth. If a function is hard to name, it's doing too many things.
60
62
 
61
63
  **No premature optimization.** Don't add caching, batching, or concurrency before a measured performance problem exists. Simple and slow is fixable; complex and wrong is not.
@@ -102,15 +104,24 @@ These rules govern all changes to this codebase — apply them without exception
102
104
 
103
105
  ## Architecture
104
106
 
105
- **Identity (`src/authsome/identity/`)** manages local Ed25519 key pairs and `did:key` DIDs. `ensure_local_identity(home, active_handle)` returns the identity named in `GlobalConfig.active_identity`, or creates a new one if none exists. Key material lives at `~/.authsome/identities/<handle>.key` (mode `0600`); metadata at `~/.authsome/identities/<handle>.json`. Identity = Profile: the handle is both the cryptographic identity name and the credential namespace key.
107
+ **Identity (`src/authsome/identity/local.py`)** manages local Ed25519 key pairs and `did:key` DIDs. Key material lives at `~/.authsome/identities/<handle>.key` (mode `0600`); metadata at `~/.authsome/identities/<handle>.json`. An Identity is a cryptographic agent — it is not a credential namespace. Credential namespacing is owned by a Vault (see below).
106
108
 
107
- **PoP Auth (`src/authsome/identity/proof.py`)** implements Proof-of-Possession JWT creation and validation. Every protected daemon request carries `Authorization: PoP <jwt>` signed with the local Ed25519 key. The JWT is bound to the specific HTTP method, path, and body SHA-256. The daemon validates the signature, checks the `jti` replay cache, and confirms `sub` (handle) `iss` (DID) via the Identity Registry.
109
+ **Principal & Vault domain models (`src/authsome/identity/principal.py`)** define the two concepts that own credentials. A **Principal** is a non-cryptographic logical partition (human or team) identified by an opaque `PrincipalId`. A **Vault** is a named credential store owned by exactly one Principal and identified by an opaque `VaultId`. Credentials are scoped to a vault: `vault:<vault_id>:...`. An Identity claims membership in a Principal via an `IdentityClaimRecord`; the claim must be accepted before vault access is granted.
108
110
 
109
- **Identity Registry (`src/authsome/identity/registry.py`)** is the daemon-owned authoritative handle→DID mapping, persisted at `~/.authsome/server/identity_registry.json`.
111
+ **Five server-owned registries** persist in `~/.authsome/server/` and are implemented in `src/authsome/server/registries.py`:
112
+ | Registry | File | Authoritative for |
113
+ |----------|------|-------------------|
114
+ | `IdentityRegistry` | `identity_registry.json` | Handle → DID mapping (PoP JWT validation) |
115
+ | `PrincipalRegistry` | `principal_registry.json` | PrincipalId → email |
116
+ | `VaultRegistry` | `vault_registry.json` | VaultId → VaultHandle |
117
+ | `IdentityClaimRegistry` | `identity_claim_registry.json` | Identity → Principal claim + ClaimStatus |
118
+ | `PrincipalVaultBindingRegistry` | `principal_vault_binding_registry.json` | Principal → default Vault binding |
119
+
120
+ **PoP Auth (`src/authsome/identity/proof.py`)** implements Proof-of-Possession JWT creation and validation. Every protected daemon request carries `Authorization: PoP <jwt>` signed with the local Ed25519 key. The JWT is bound to the specific HTTP method, path, and body SHA-256. The daemon validates the signature, checks the `jti` replay cache, and confirms `sub` (handle) → `iss` (DID) via the Identity Registry.
110
121
 
111
- **AuthService (`src/authsome/auth/service.py`)** is the authentication and credential lifecycle layer. It owns OAuth flows, token refresh, login/logout/revoke. Constructed with `vault` and `identity` (the handle); all store keys are namespaced as `profile:<handle>:...`.
122
+ **AuthService (`src/authsome/server/credential_service.py`)** is the authentication and credential lifecycle coordinator. It owns OAuth flows, token refresh, login/logout/revoke. Lives in `server/` because it coordinates `auth/` flows with `vault/` storage and `audit/` logging. Constructed with `(vault, identity, principal_id, vault_id)`; all credential store keys are namespaced as `vault:<vault_id>:...`. The caller (server dependency injection) resolves `vault_id` from the `PrincipalVaultBindingRegistry` before constructing `AuthService`.
112
123
 
113
- **Flows (`src/authsome/auth/flows/`)** implement the `AuthFlow.authenticate()` interface. Each flow returns a `ConnectionRecord`.
124
+ **Flows (`src/authsome/auth/flows/`)** implement the `AuthFlow.authenticate()` interface. Each flow returns a `ConnectionRecord`. The `auth/` module is a leaf — it imports nothing from `vault/`, `audit/`, or `server/`.
114
125
 
115
126
  | Flow | Class | Notes |
116
127
  |------|-------|-------|
@@ -119,19 +130,19 @@ These rules govern all changes to this codebase — apply them without exception
119
130
  | `dcr_pkce` | `DcrPkceFlow` | Dynamic Client Registration then PKCE |
120
131
  | `api_key` | `ApiKeyFlow` | Prompts via secure browser bridge |
121
132
 
122
- **Provider Registry (`src/authsome/auth/service.py`)** resolves providers in this order: local `~/.authsome/providers/<name>.json` overrides bundled JSON in `src/authsome/bundled_providers/`. Bundled providers (GitHub, Google, Okta, Linear, OpenAI) are loaded via `importlib.resources`.
133
+ **Provider Registry** resolves providers in this order: custom providers stored in the vault under the `providers` collection override bundled JSON in `src/authsome/auth/bundled_providers/`. Bundled providers (GitHub, Google, Okta, Linear, OpenAI) are loaded via `importlib.resources`.
123
134
 
124
135
  **Vault (`src/authsome/vault/`)** is the encrypted KV store. The master key lives at `~/.authsome/server/master.key` (mode `0600`) or in the OS keyring. All credential blobs are encrypted at rest; the AuthService reads and writes plaintext through the Vault without knowing encryption details.
125
136
 
126
137
  **Storage** uses a DiskStore-backed KV at `~/.authsome/server/kv_store/`. Store keys follow the pattern:
127
138
  ```
128
- profile:<handle>:<provider>:connection:<connection_name>
129
- profile:<handle>:<provider>:metadata
130
- profile:<handle>:<provider>:state
139
+ vault:<vault_id>:<provider>:connection:<connection_name>
140
+ vault:<vault_id>:<provider>:metadata
141
+ vault:<vault_id>:<provider>:state
131
142
  server:<provider>:client
132
143
  ```
133
144
 
134
- **Config** (`GlobalConfig`) is stored in the KV store under `config/global`. Key field: `active_identity` (the handle of the current identity). Encryption mode is set via `config.encryption.mode` (`local_key` or `keyring`).
145
+ **Config** (`GlobalConfig`) is stored in the KV store under `config/global`. Key fields: `active_identity` (the handle of the current identity), `vault_id` (the active vault resolved at `authsome init`). Encryption mode is set via `config.encryption.mode` (`local_key` or `keyring`).
135
146
 
136
147
  **CLI (`src/authsome/cli/main.py`)** is Click-based. All commands support `--json` for machine-readable output. `authsome init` creates the local identity, registers it with the daemon, and writes `active_identity` to config.
137
148