MetaList 0.7.6__tar.gz → 0.9.0__tar.gz

{metalist-0.7.6/metalist.egg-info → metalist-0.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: metalist
-Version: 0.7.6
+Version: 0.9.0
 Summary: A minimalist single-user note-taking app with SSR, fast in-memory tree operations, and efficient sync.
 Project-URL: Homepage, https://github.com/evolvingstuff/metalist
 Project-URL: Repository, https://github.com/evolvingstuff/metalist
@@ -20,6 +20,7 @@ Requires-Dist: nh3==0.3.6
 Requires-Dist: openai==2.54.0
 Requires-Dist: pillow==12.3.0
 Requires-Dist: pydantic==2.13.4
+Requires-Dist: pypdf==6.19.0
 Requires-Dist: python-dotenv==1.2.2
 Requires-Dist: python-multipart==0.0.32
 Requires-Dist: starlette==1.3.1
@@ -48,6 +49,17 @@ A minimalist single-user note-taking app focused on server-side rendering (SSR),
 - Multi-tab search contexts with server-persisted scroll/search state (survives browser restarts)
 - Manual namespace backups/restores to a user-selected backup folder with retention controls
 
+## Changes in 0.9.0
+
+- AI chat can browse the web in disabled, contextual, or unrestricted mode, fetch multiple pages concurrently, and cite the specific pages used as evidence.
+- Complete-scope summaries and tag suggestions can process every permitted note in bounded parallel batches after user approval, with visible per-batch progress and final synthesis.
+- AI chat retains selected-note and tree context while enforcing redaction boundaries, and reference navigation uses temporary result collections without polluting search history.
+
+## Changes in 0.8.0
+
+- Error diagnostics now record safe request IDs, exception types, and source locations through Loguru; full tracebacks are available only in authenticated encrypted logs.
+- Releases run the full cross-platform matrix once on the `main` push. The tag publishes those exact tested distributions without rerunning the matrix.
+
 ## Changes in 0.7.5
 
 - Unexpected API 500 errors now show the exception class and MetaList source line in the browser, without exposing note contents, exception messages, or local filesystem paths.
@@ -57,7 +69,7 @@ A minimalist single-user note-taking app focused on server-side rendering (SSR),
 
 - Updates use a private, checksum-verified uv release on Windows, macOS, and Linux. The updater does not execute a user-installed uv, and it revalidates both the cached official archive and extracted executable before each use.
 - The HTTPS listener retries one bodyless safe request after a transient backend reset, records final transport failures server-side, and never replays a mutation or request body.
-- Release CI builds one distribution pair, starts independent platform/Python, installed-update, browser-smoke, and browser-soak gates in parallel, and caches only hash- or lockfile-verified dependencies. Cross-platform runtime changes must pass branch CI before merge.
+- Release CI builds one distribution pair, starts independent platform/Python, installed-update, browser-smoke, and browser-soak gates in parallel, and caches only hash- or lockfile-verified dependencies.
 
 ## Changes in 0.7.3
 
@@ -356,9 +368,9 @@ This repo now packages itself under the PyPI distribution name `metalist`. Curre
 
 Recommended release path:
 1. In the existing PyPI project `metalist`, configure GitHub Trusted Publishing for `evolvingstuff/metalist` and the workflow file `.github/workflows/publish-pypi.yml`.
-2. Push the candidate commit to a branch and wait for the `Publish to PyPI` validation workflow to pass for that exact commit. GitHub-hosted Windows, macOS, and Linux runners install the built wheel and test application startup outside the source checkout on Python 3.10 through 3.14. Every matrix job also exercises the real `metalist update` path with uv, backups, interpreter preservation, and two-namespace restart against a local release index. The old-version fixture carries the candidate updater so this validates the mechanism being shipped; it does not retroactively fix older installed updaters. You do not need those operating systems locally. Archive verification also checks runtime files, including agent Markdown resources, against both the wheel and source distribution.
-3. Only after every required check passes, create and push the release tag for that same commit. Any intervening change requires a new validation run before tagging. Branch and pull-request runs validate without publishing.
-4. The tag workflow repeats validation and publishes the same artifacts tested by its full matrix. Manual publication also requires all checks to pass. Users can then run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
+2. Develop and review the candidate on a feature branch with focused checks. Merge it into `main` and push `main` once. This runs the full `Publish to PyPI` validation matrix for that exact commit. GitHub-hosted Windows, macOS, and Linux runners install the built wheel and test application startup outside the source checkout on the supported Python endpoints, 3.10 and 3.14. Every installed-package matrix job also exercises the real `metalist update` path with uv, backups, interpreter preservation, and two-namespace restart against a local release index. The old-version fixture carries the candidate updater so this validates the mechanism being shipped; it does not retroactively fix older installed updaters. You do not need those operating systems locally. Archive verification checks runtime files, including agent Markdown resources, against both the wheel and source distribution.
+3. Wait for the exact-commit main release matrix to pass, then run `.venv/bin/python scripts/release.py X.Y.Z` to create and push the tag. Any intervening change requires a new main validation run before tagging.
+4. The tag job independently verifies every main-job conclusion and reuses the wheel and source distribution from that run for GitHub Trusted Publishing; it does not repeat the matrix. Manual dispatch cannot publish. The release driver compares PyPI hashes with the tested artifact and checks a clean public install. Users can then run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
 
 If `--input` is omitted, a file picker opens (when `tkinter` and its native `_tkinter` component are available).
 Notes tagged with `@implies` are converted into ontology rules and are not imported as notes. Legacy rules that are invalid under the current ontology grammar are reported and skipped while valid rules continue importing.

{metalist-0.7.6 → metalist-0.9.0}/README.md

@@ -12,6 +12,17 @@ A minimalist single-user note-taking app focused on server-side rendering (SSR),
 - Multi-tab search contexts with server-persisted scroll/search state (survives browser restarts)
 - Manual namespace backups/restores to a user-selected backup folder with retention controls
 
+## Changes in 0.9.0
+
+- AI chat can browse the web in disabled, contextual, or unrestricted mode, fetch multiple pages concurrently, and cite the specific pages used as evidence.
+- Complete-scope summaries and tag suggestions can process every permitted note in bounded parallel batches after user approval, with visible per-batch progress and final synthesis.
+- AI chat retains selected-note and tree context while enforcing redaction boundaries, and reference navigation uses temporary result collections without polluting search history.
+
+## Changes in 0.8.0
+
+- Error diagnostics now record safe request IDs, exception types, and source locations through Loguru; full tracebacks are available only in authenticated encrypted logs.
+- Releases run the full cross-platform matrix once on the `main` push. The tag publishes those exact tested distributions without rerunning the matrix.
+
 ## Changes in 0.7.5
 
 - Unexpected API 500 errors now show the exception class and MetaList source line in the browser, without exposing note contents, exception messages, or local filesystem paths.
@@ -21,7 +32,7 @@ A minimalist single-user note-taking app focused on server-side rendering (SSR),
 
 - Updates use a private, checksum-verified uv release on Windows, macOS, and Linux. The updater does not execute a user-installed uv, and it revalidates both the cached official archive and extracted executable before each use.
 - The HTTPS listener retries one bodyless safe request after a transient backend reset, records final transport failures server-side, and never replays a mutation or request body.
-- Release CI builds one distribution pair, starts independent platform/Python, installed-update, browser-smoke, and browser-soak gates in parallel, and caches only hash- or lockfile-verified dependencies. Cross-platform runtime changes must pass branch CI before merge.
+- Release CI builds one distribution pair, starts independent platform/Python, installed-update, browser-smoke, and browser-soak gates in parallel, and caches only hash- or lockfile-verified dependencies.
 
 ## Changes in 0.7.3
 
@@ -320,9 +331,9 @@ This repo now packages itself under the PyPI distribution name `metalist`. Curre
 
 Recommended release path:
 1. In the existing PyPI project `metalist`, configure GitHub Trusted Publishing for `evolvingstuff/metalist` and the workflow file `.github/workflows/publish-pypi.yml`.
-2. Push the candidate commit to a branch and wait for the `Publish to PyPI` validation workflow to pass for that exact commit. GitHub-hosted Windows, macOS, and Linux runners install the built wheel and test application startup outside the source checkout on Python 3.10 through 3.14. Every matrix job also exercises the real `metalist update` path with uv, backups, interpreter preservation, and two-namespace restart against a local release index. The old-version fixture carries the candidate updater so this validates the mechanism being shipped; it does not retroactively fix older installed updaters. You do not need those operating systems locally. Archive verification also checks runtime files, including agent Markdown resources, against both the wheel and source distribution.
-3. Only after every required check passes, create and push the release tag for that same commit. Any intervening change requires a new validation run before tagging. Branch and pull-request runs validate without publishing.
-4. The tag workflow repeats validation and publishes the same artifacts tested by its full matrix. Manual publication also requires all checks to pass. Users can then run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
+2. Develop and review the candidate on a feature branch with focused checks. Merge it into `main` and push `main` once. This runs the full `Publish to PyPI` validation matrix for that exact commit. GitHub-hosted Windows, macOS, and Linux runners install the built wheel and test application startup outside the source checkout on the supported Python endpoints, 3.10 and 3.14. Every installed-package matrix job also exercises the real `metalist update` path with uv, backups, interpreter preservation, and two-namespace restart against a local release index. The old-version fixture carries the candidate updater so this validates the mechanism being shipped; it does not retroactively fix older installed updaters. You do not need those operating systems locally. Archive verification checks runtime files, including agent Markdown resources, against both the wheel and source distribution.
+3. Wait for the exact-commit main release matrix to pass, then run `.venv/bin/python scripts/release.py X.Y.Z` to create and push the tag. Any intervening change requires a new main validation run before tagging.
+4. The tag job independently verifies every main-job conclusion and reuses the wheel and source distribution from that run for GitHub Trusted Publishing; it does not repeat the matrix. Manual dispatch cannot publish. The release driver compares PyPI hashes with the tested artifact and checks a clean public install. Users can then run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
 
 If `--input` is omitted, a file picker opens (when `tkinter` and its native `_tkinter` component are available).
 Notes tagged with `@implies` are converted into ontology rules and are not imported as notes. Legacy rules that are invalid under the current ontology grammar are reported and skipped while valid rules continue importing.

{metalist-0.7.6 → metalist-0.9.0}/app/api/note_requests.py

@@ -8,6 +8,10 @@ from pydantic import AfterValidator, ConfigDict, Field, with_config
 Identifier = Annotated[str, Field(min_length=1, max_length=256)]
 Text = Annotated[str, Field(max_length=1024 * 1024)]
 Content = Annotated[str, Field(max_length=16 * 1024 * 1024)]
+# Clients build undo contexts as "tab:<id>|search:<executed query>|epoch:<n>", so the
+# limit must admit any Text-sized search query (for example an "Open all references"
+# query ORing every cited note ID) plus that small fixed envelope.
+UndoContext = Annotated[str, Field(min_length=1, max_length=1024 * 1024 + 1024)]
 Nonnegative = Annotated[int, Field(ge=0)]
 
 
@@ -51,7 +55,7 @@ class ViewDiffRequest(TypedDict):
     editingNoteId: Text | None
     search: Text | None
     tabId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     clientNoteUuidHashes: dict[Identifier, Text]
     visibleRootAnchorId: Text | None
     isUntaggedView: bool
@@ -67,7 +71,7 @@ class UpdateTabSortModeRequest(TypedDict):
     tabId: Identifier
     sortMode: Literal['normal', 'created', 'updated', 'alphabetical', 'content-volume']
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
 
 @with_config(ConfigDict(strict=True))
 class CreateNewTabRequest(TypedDict):
@@ -113,21 +117,21 @@ class CreateNoteTopRequest(TypedDict):
     first_visible_note_id: Text | None
     search_query: Text | None
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class CreateSiblingRequest(TypedDict):
     search_query: Text | None
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class CreateChildRequest(TypedDict):
     search_query: Text | None
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -135,7 +139,7 @@ class UpdateNoteRequest(TypedDict):
     clientId: Identifier
     content: Content
     tags: Text
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -143,34 +147,34 @@ class SaveNoteRequest(TypedDict):
     clientId: Identifier
     content: Content
     tags: Text
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class AddSelectedTextTagRequest(TypedDict):
     selectedText: Identifier
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class MakePseudoTagProposalsRequest(TypedDict):
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class AcceptTagProposalRequest(TypedDict):
     proposal: Identifier
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class RejectTagProposalRequest(TypedDict):
     proposal: Identifier
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -178,19 +182,19 @@ class SplitNoteRequest(TypedDict):
     clientId: Identifier
     segments: Annotated[list[Content], Field(min_length=1)]
     tags: Text
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class ToggleTodoDoneRequest(TypedDict):
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class UnformatNoteContentRequest(TypedDict):
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -199,7 +203,7 @@ class ResizeNoteImageRequest(TypedDict):
     sourceKind: Literal['inline', 'file']
     occurrenceIndex: Nonnegative
     action: Literal['bigger', 'smaller', 'reset']
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -212,7 +216,7 @@ class ToggleReferenceModeEndpointRequest(TypedDict):
     occurrence_index: Nonnegative
     mode: Literal['embed', 'link']
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -222,7 +226,7 @@ class MoveNoteEndpointRequest(TypedDict):
     sibling_id: Identifier
     position: Literal['BEFORE', 'AFTER']
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -230,33 +234,33 @@ class MoveNoteToTopEndpointRequest(TypedDict):
     search_query: Text | None
     tab_id: Identifier
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class IndentNoteEndpointRequest(TypedDict):
     visible_prev_id: Text | None
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class OutdentNoteEndpointRequest(TypedDict):
     search_query: Text | None
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class CollapseEndpointRequest(TypedDict):
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class ExpandEndpointRequest(TypedDict):
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -264,7 +268,7 @@ class SetCollapsedBulkEndpointRequest(TypedDict):
     note_ids: Annotated[list[Identifier], Field(min_length=1)]
     collapsed: bool
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -273,14 +277,14 @@ class SetCollapsedInContextEndpointRequest(TypedDict):
     collapsed: bool
     recursive: bool
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class SetCollapsedSubtreeEndpointRequest(TypedDict):
     collapsed: bool
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -290,7 +294,7 @@ class PrioritizeInViewEndpointRequest(TypedDict):
     search_query: Text | None
     tab_id: Identifier
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -299,20 +303,20 @@ class AlphabetizeRootNotesEndpointRequest(TypedDict):
     search_query: Text | None
     tab_id: Identifier
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class ResetUpdatedAtToCreatedAtEndpointRequest(TypedDict):
     search_query: Text | None
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class DeleteNoteRequest(TypedDict):
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
@@ -323,13 +327,13 @@ class CopyNoteEndpointRequest(TypedDict):
 class PasteSiblingEndpointRequest(TypedDict):
     search_query: Text | None
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 
 @with_config(ConfigDict(strict=True))
 class PasteChildEndpointRequest(TypedDict):
     search_query: Text | None
     clientId: Identifier
-    undoContext: Identifier
+    undoContext: UndoContext
     viewport: Viewport
 

{metalist-0.7.6 → metalist-0.9.0}/app/api/routes/ai.py

@@ -20,6 +20,7 @@ from app.models.utils import render_note_data_read_only
 from app.services.ai_chat_stream import ChatTurnStream
 from app.services.ai_chat import ai_chat_store
 from app.services.ai_chat_rendering import find_note_citation_ids
+from app.services.ai_chat_rendering import find_web_citation_ids
 from app.services.ai_chat_rendering import render_ai_chat_markdown_to_html
 from app.services.agent.context import AgentContextBuilder
 from app.services.agent.cloud_privacy import cloud_privacy_evaluator
@@ -58,6 +59,9 @@ from app.services.openai_credentials import validate_openai_api_key
 from app.services.sync import set_clipboard, get_current_sync_uuid
 from app.services.agent.tagging_run import TaggingRun, proposal_scope_ids
 from app.services.agent.retrieval_settings import resolve_tagging_batch_tokens
+from app.services.agent.web_settings import AgentWebSettings
+from app.services.agent.web_settings import resolve_agent_web_settings
+from app.services.agent.web_evidence import web_evidence_store
 from app.services.agent.tagging import TAGGING_PROMPT_KEY, TAGGING_POLICY_KEY, DEFAULT_TAGGING_PROMPT
 from app.usecases.bulk_tag_proposals import apply_bulk_proposals, prepare_proposal_changes
 from app.services.bulk_operation import bulk_operation_guard
@@ -249,7 +253,7 @@ class AiCopyMessageResponse(BaseModel):
     tags: str
 
 
-def _render_ai_response_note_content(markdown_text: str) -> str:
+def _render_ai_response_note_content(*, markdown_text: str, session_key: str) -> str:
     if not isinstance(markdown_text, str) or markdown_text == "":
         raise ValueError("AI response Markdown must be a non-empty string")
     allowed_note_ids = find_note_citation_ids(
@@ -260,6 +264,10 @@ def _render_ai_response_note_content(markdown_text: str) -> str:
         markdown_text,
         notes=note_store,
         allowed_note_ids=allowed_note_ids,
+        allowed_web_evidence=web_evidence_store.available_references_for_ids(
+            session_key=session_key,
+            evidence_ids=find_web_citation_ids(markdown_text),
+        ),
     )
     if rendered_content == "":
         raise RuntimeError("Completed AI response rendered to empty HTML")
@@ -461,6 +469,10 @@ def get_ai_session(
                 message["content"],
                 notes=note_store,
                 allowed_note_ids=allowed_note_ids,
+                allowed_web_evidence=web_evidence_store.available_references_for_ids(
+                    session_key=session_key,
+                    evidence_ids=find_web_citation_ids(message["content"]),
+                ),
             )
         if message["role"] == "assistant" and message["thinking"] != "":
             rendered_thinking = render_markdown_to_html(message["thinking"])
@@ -502,6 +514,7 @@ def clear_ai_session(
     response.headers["Cache-Control"] = "no-store"
     session_key = token_service.get_session_key(token)
     ai_chat_store.clear_session(session_key=session_key)
+    web_evidence_store.clear_session(session_key=session_key)
     agent_trace_store.clear_trace(session_key=session_key)
     return AiClearResponse(message="Chat cleared")
 
@@ -576,7 +589,10 @@ def copy_ai_message(
         raise HTTPException(status_code=409, detail="AI response is empty")
 
     tags = "@llm"
-    note_content = _render_ai_response_note_content(content)
+    note_content = _render_ai_response_note_content(
+        markdown_text=content,
+        session_key=session_key,
+    )
     clipboard_record = {
         "id": f"ai-chat:{message_id}",
         "parent_id": None,
@@ -585,6 +601,7 @@ def copy_ai_message(
         "is_collapsed": False,
         "content": note_content,
         "tags": tags,
+        "proposed_tags": "",
     }
     set_clipboard(payload.client_id, [clipboard_record])
 
@@ -626,6 +643,7 @@ def stream_ai_chat(
         preferences=preferences,
         provider=payload.provider,
     )
+    web_settings = resolve_agent_web_settings(preferences=preferences)
     privacy_boundary = resolve_cloud_privacy_boundary(
         preferences=preferences,
         provider=payload.provider,
@@ -660,10 +678,12 @@ def stream_ai_chat(
     )
     tagging_run = TaggingRun(token=token, snapshot=frozen_scope, preferences=preferences, sync_uuid=get_current_sync_uuid(),
         batch_tokens=resolve_tagging_batch_tokens(preferences, payload.provider))
-    ai_chat_store.synchronize_disclosure_boundary(
+    disclosure_changed = ai_chat_store.synchronize_disclosure_boundary(
         session_key=session_key,
         disclosure_key=cloud_privacy_evaluator.history_disclosure_key(boundary=privacy_boundary),
     )
+    if disclosure_changed:
+        web_evidence_store.clear_session(session_key=session_key)
     turn_id = ai_chat_store.start_turn(
         session_key=session_key,
         user_content=payload.message,
@@ -677,11 +697,13 @@ def stream_ai_chat(
     )
     initial_approx_input_tokens = estimate_input_tokens(initial_messages)
 
-    turn_stream = ChatTurnStream(store=ai_chat_store, notes=note_store, session_key=session_key,
+    turn_stream = ChatTurnStream(store=ai_chat_store, notes=note_store,
+                                 web_evidence_store=web_evidence_store, session_key=session_key,
                                  turn_id=turn_id, initial_input_tokens=initial_approx_input_tokens)
     events = _stream_runtime_events(payload=payload, openai_api_key=openai_api_key,
         session_key=session_key, provider_messages=provider_messages, prompts=prompts, skills=skills,
         retrieval_settings=retrieval_settings, frozen_scope=frozen_scope, tagging_run=tagging_run,
+        web_settings=web_settings,
         initial_input_tokens=initial_approx_input_tokens)
 
     return StreamingResponse(
@@ -699,7 +721,7 @@ async def _stream_runtime_events(
     *, payload: AiChatRequest, openai_api_key: str, session_key: str,
     provider_messages: list[dict[str, str]], prompts: AgentPromptSet, skills: AgentSkillSet,
     retrieval_settings: AgentRetrievalSettings, frozen_scope: ScopedSearchSnapshot,
-    tagging_run: TaggingRun, initial_input_tokens: int,
+    tagging_run: TaggingRun, initial_input_tokens: int, web_settings: AgentWebSettings,
 ) -> AsyncIterator[dict[str, object]]:
     action, start_label = 'provider_runtime', 'Connecting to OpenAI API'
     ready_label = 'OpenAI API ready · 1,050,000-token context'
@@ -712,7 +734,8 @@ async def _stream_runtime_events(
     events = runtime.stream_scoped(session_key=session_key, base_url=base_url,
             selected_model=payload.model, thinking_level=payload.thinking_level,
             canonical_messages=provider_messages, prompts=prompts, skills=skills,
-            retrieval_settings=retrieval_settings, frozen_scope=frozen_scope, tag_handler=tagging_run.stream)
+            retrieval_settings=retrieval_settings, web_settings=web_settings,
+            frozen_scope=frozen_scope, tag_handler=tagging_run.stream)
     try:
         async for event in events:
             yield event
@@ -735,7 +758,20 @@ async def acknowledge_menu_result(payload: MenuResult, token: Annotated[str, Dep
 class BulkAnswerRequest(BaseModel):
     model_config = ConfigDict(extra="forbid")
     question_id: str = Field(..., min_length=1)
-    value: Literal["existing", "new", "proceed", "cancel", "focus_existing", "focus_new", "focus_both"]
+    value: Literal[
+        "existing",
+        "new",
+        "proceed",
+        "cancel",
+        "focus_existing",
+        "focus_new",
+        "focus_both",
+        "prefix_focus_existing",
+        "prefix_focus_new",
+        "prefix_focus_both",
+        "summarize_all",
+        "use_prefix",
+    ]
 
 
 @router.post("/proposals/answer")

{metalist-0.7.6 → metalist-0.9.0}/app/api/routes/auth.py

@@ -52,6 +52,7 @@ from app.security.sensitive_logging import traceback_frame_summary
 from app.services.encryption import EncryptionService
 from app.services.tokens import token_service
 from app.services.ai_chat import ai_chat_store
+from app.services.agent.web_evidence import web_evidence_store
 from app.services.agent.trace import agent_trace_store
 from app.services.note_store import store as note_store
 from app.services.sync import clear_all_locks
@@ -383,6 +384,7 @@ def _reset_runtime_state_after_restore() -> bool:
     invalidate_runtime_work()
     clear_sensitive_caches()
     ai_chat_store.reset()
+    web_evidence_store.reset()
     agent_trace_store.reset()
     openai_credential_store.reset()
     view_cache.clear()
@@ -563,6 +565,7 @@ def login(
     invalidate_runtime_work()
     clear_sensitive_caches()
     ai_chat_store.reset()
+    web_evidence_store.reset()
     agent_trace_store.reset()
     openai_credential_store.reset()
     view_cache.clear()
@@ -587,6 +590,7 @@ def logout(
 ):
     session_key = token_service.get_session_key(token)
     ai_chat_store.clear_session(session_key=session_key)
+    web_evidence_store.clear_session(session_key=session_key)
     agent_trace_store.clear_session(session_key=session_key)
     openai_credential_store.clear_session(session_key=session_key)
     token_service.revoke_token(token)
@@ -674,6 +678,7 @@ def create_passwordless_session(
     invalidate_runtime_work()
     clear_sensitive_caches()
     ai_chat_store.reset()
+    web_evidence_store.reset()
     agent_trace_store.reset()
     openai_credential_store.reset()
     view_cache.clear()
@@ -1089,6 +1094,7 @@ def create_password(
     invalidate_runtime_work()
     clear_sensitive_caches()
     ai_chat_store.reset()
+    web_evidence_store.reset()
     agent_trace_store.reset()
     openai_credential_store.reset()
     clear_all_locks()
@@ -1118,6 +1124,7 @@ def change_password(
     invalidate_runtime_work()
     clear_sensitive_caches()
     ai_chat_store.reset()
+    web_evidence_store.reset()
     agent_trace_store.reset()
     openai_credential_store.reset()
     clear_all_locks()
@@ -1139,6 +1146,7 @@ def remove_password(
     invalidate_runtime_work()
     clear_sensitive_caches()
     ai_chat_store.reset()
+    web_evidence_store.reset()
     agent_trace_store.reset()
     openai_credential_store.reset()
     clear_all_locks()

{metalist-0.7.6 → metalist-0.9.0}/app/exception_boundaries.py

@@ -5,6 +5,16 @@ syntax suppressions and is consumed by runtime capture and the startup gate.
 """
 
 CAPTURE_BOUNDARIES = {
+    # Web retrieval translates only expected failures from untrusted remote targets.
+    'app/services/agent/web_fetch.py:fetch_web_page:target_capture': ('PublicHttpTargetRejected',),
+    'app/services/agent/web_fetch.py:_download_one_url:transport_capture': (
+        'TimeoutException', 'NetworkError', 'HTTPError',
+        'TimeoutException', 'NetworkError', 'ProtocolError',
+    ),
+    'app/services/agent/web_fetch.py:_extract_pdf_content:pdf_capture': ('PyPdfError',),
+    'app/services/agent/web_fetch.py:_decode_web_text:encoding_capture': ('LookupError',),
+    'app/services/agent/web_fetch.py:handle_starttag:join_capture': ('ValueError',),
+    'app/services/agent/runtime.py:_open_web_action:evidence_capacity_capture': ('WebEvidenceCapacityError',),
     # In-app updates translate only external transport, installation eligibility and job lookup errors.
     'app/services/app_updates.py:check_for_update:eligibility': ('AppUpdateRejected',),
     'app/services/app_updates.py:job_path:validation': ('ValueError',),
@@ -106,11 +116,12 @@ CAPTURE_BOUNDARIES = {
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
     'app/services/link_titles.py:_fetch_one_url:capture': ('TimeoutException', 'NetworkError', 'HTTPError', 'TimeoutException', 'NetworkError', 'ProtocolError'),
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
-    'app/services/link_titles.py:_resolve_public_http_target:capture': ('gaierror',),
+    'app/services/public_http.py:resolve_public_http_target:resolve_capture': ('gaierror',),
+    'app/services/public_http.py:normalize_public_http_url:parse_capture': ('ValueError',),
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
-    'app/services/link_titles.py:connect_tcp:connect_capture': ('ConnectError', 'ConnectTimeout'),
+    'app/services/public_http.py:connect_tcp:connect_capture': ('ConnectError', 'ConnectTimeout'),
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
-    'app/services/link_titles.py:fetch_link_title:target_capture': ('_LinkTitleTargetRejected',),
+    'app/services/link_titles.py:fetch_link_title:target_capture': ('PublicHttpTargetRejected',),
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
     'app/services/namespace_deletion_worker.py:_is_process_running:kill_capture': ('ProcessLookupError', 'PermissionError'),
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
@@ -138,7 +149,7 @@ CAPTURE_BOUNDARIES = {
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
     'app/services/remote_image_proxy.py:_download_one_url:request_capture': ('TimeoutException', 'NetworkError', 'HTTPError', 'TimeoutException', 'NetworkError', 'ProtocolError'),
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
-    'app/services/remote_image_proxy.py:_download_one_url:target_rejection_capture': ('_LinkTitleTargetRejected',),
+    'app/services/remote_image_proxy.py:_download_one_url:target_rejection_capture': ('PublicHttpTargetRejected',),
     # Handle expected external I/O, provider, image or LaTeX parsing failure.
     'app/services/remote_image_proxy.py:_validated_image_mime_type:validation_capture': ('DecompressionBombError', 'OSError', 'SyntaxError', 'UnidentifiedImageError', 'ValueError'),
     # Record operation failure and re-raise; never resume the failed operation.