unique-user-memory 2026.38.0.dev0__tar.gz → 2026.38.0.dev2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- unique_user_memory-2026.38.0.dev0/README.md → unique_user_memory-2026.38.0.dev2/PKG-INFO +46 -1
- unique_user_memory-2026.38.0.dev0/PKG-INFO → unique_user_memory-2026.38.0.dev2/README.md +32 -15
- {unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/pyproject.toml +3 -3
- {unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/unique_user_memory/tests/test_user_memory.py +259 -3
- {unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/unique_user_memory/user_memory.py +113 -0
- {unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/unique_user_memory/user_memory_prompts.py +131 -8
- {unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/unique_user_memory/__init__.py +0 -0
- {unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/unique_user_memory/config.py +0 -0
- {unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/unique_user_memory/user_memory_message_log.py +0 -0
- {unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/unique_user_memory/user_memory_postprocessor.py +0 -0
|
@@ -1,3 +1,17 @@
|
|
|
1
|
+
Metadata-Version: 2.3
|
|
2
|
+
Name: unique-user-memory
|
|
3
|
+
Version: 2026.38.0.dev2
|
|
4
|
+
Summary:
|
|
5
|
+
Author: Fabian Schläpfer
|
|
6
|
+
Author-email: Fabian Schläpfer <fabian@unique.ch>
|
|
7
|
+
License: Proprietary
|
|
8
|
+
Requires-Dist: jinja2>=3.1.6
|
|
9
|
+
Requires-Dist: pydantic>=2.8.2
|
|
10
|
+
Requires-Dist: unique-sdk>=2026.38.0.dev4,<2026.38.0rc0
|
|
11
|
+
Requires-Dist: unique-toolkit>=2026.38.0.dev14,<2026.38.0rc0
|
|
12
|
+
Requires-Python: >=3.12, <4
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
|
|
1
15
|
# Unique User Memory
|
|
2
16
|
|
|
3
17
|
Persistent per-user memory for Unique AI agents.
|
|
@@ -25,7 +39,38 @@ The memory file is intentionally small and structured. It is rewritten as a full
|
|
|
25
39
|
5. `UserMemoryPostprocessor` runs after the assistant response.
|
|
26
40
|
6. The package asks the configured language model to either return `NOOP` or a complete rewritten profile.
|
|
27
41
|
7. If a rewrite runs, an **Updating your memory** Step is shown while consolidating (no settings entry yet).
|
|
28
|
-
8.
|
|
42
|
+
8. Every rewrite passes through a mandatory CID/PII scrub call (see Content Policy below) before it is assembled for upload. When the scrub cannot vouch for the rewrite, the existing memory is kept unchanged.
|
|
43
|
+
9. If the profile changed and `memory.md` uploads successfully (ingestion skipped, content hidden from chat), that Step is completed with a **Review your context memory** detail entry (same settings badge). On NOOP or failed upload the Step completes without the entry.
|
|
44
|
+
|
|
45
|
+
## Content Policy — the current user's CID/PII only (UN-24886)
|
|
46
|
+
|
|
47
|
+
The profile belongs to exactly one person: the signed-in user. It may contain
|
|
48
|
+
personal data (PII) of that user only. It must never contain
|
|
49
|
+
client-identifying data (CID) or PII of any other person or private entity —
|
|
50
|
+
not other users, not clients, not prospects, not counterparties. Facts about
|
|
51
|
+
the user's own work that involve a client are stored only in a fully
|
|
52
|
+
de-identified form ("runs quarterly portfolio reviews", never "reviews the
|
|
53
|
+
portfolio of J. Muster"). Credentials, payment data, health records, and
|
|
54
|
+
government IDs are never stored for anyone, including the user.
|
|
55
|
+
|
|
56
|
+
The policy text lives once, in `_CID_POLICY` in `user_memory_prompts.py`, and
|
|
57
|
+
is embedded in every write-path stage so the rules cannot drift:
|
|
58
|
+
|
|
59
|
+
| Stage | Role of the policy |
|
|
60
|
+
| --- | --- |
|
|
61
|
+
| Gate (`memory_gate_system_prompt`) | Turns whose only new facts are third-party CID/PII lean `NOOP` and never reach the rewrite. |
|
|
62
|
+
| Consolidation (`consolidation_system_prompt`) | The rewrite itself must not extract forbidden content. |
|
|
63
|
+
| Condensation (`condensation_system_prompt`) | Oversized (including legacy) profiles must drop violating bullets on every shrink, progressively cleaning old data. |
|
|
64
|
+
| Scrub (`scrub_system_prompt` / `scrub_user_memory`) | Mandatory final gate on every consolidation rewrite. Answers `CLEAN`, returns a cleaned body, or vetoes the write. Fails closed: on any error the existing memory is kept and the unvalidated candidate is discarded. |
|
|
65
|
+
|
|
66
|
+
Storage-level isolation (per-user home folder with exclusive owner ACL, see
|
|
67
|
+
Storage Model) already prevents cross-user *access*; this policy governs
|
|
68
|
+
cross-user and cross-client *content*.
|
|
69
|
+
|
|
70
|
+
Note for follow-up client-memory work: this package is user memory only. A
|
|
71
|
+
future client memory is a separate artefact with its own scoping and must not
|
|
72
|
+
be built by loosening this policy — client-related durable knowledge is
|
|
73
|
+
intentionally rejected here rather than stored under the user's profile.
|
|
29
74
|
|
|
30
75
|
## Storage Model
|
|
31
76
|
|
|
@@ -1,17 +1,3 @@
|
|
|
1
|
-
Metadata-Version: 2.3
|
|
2
|
-
Name: unique-user-memory
|
|
3
|
-
Version: 2026.38.0.dev0
|
|
4
|
-
Summary:
|
|
5
|
-
Author: Fabian Schläpfer
|
|
6
|
-
Author-email: Fabian Schläpfer <fabian@unique.ch>
|
|
7
|
-
License: Proprietary
|
|
8
|
-
Requires-Dist: jinja2>=3.1.6
|
|
9
|
-
Requires-Dist: pydantic>=2.8.2
|
|
10
|
-
Requires-Dist: unique-sdk>=2026.38.0.dev0,<2026.38.0rc0
|
|
11
|
-
Requires-Dist: unique-toolkit>=2026.38.0.dev0,<2026.38.0rc0
|
|
12
|
-
Requires-Python: >=3.12, <4
|
|
13
|
-
Description-Content-Type: text/markdown
|
|
14
|
-
|
|
15
1
|
# Unique User Memory
|
|
16
2
|
|
|
17
3
|
Persistent per-user memory for Unique AI agents.
|
|
@@ -39,7 +25,38 @@ The memory file is intentionally small and structured. It is rewritten as a full
|
|
|
39
25
|
5. `UserMemoryPostprocessor` runs after the assistant response.
|
|
40
26
|
6. The package asks the configured language model to either return `NOOP` or a complete rewritten profile.
|
|
41
27
|
7. If a rewrite runs, an **Updating your memory** Step is shown while consolidating (no settings entry yet).
|
|
42
|
-
8.
|
|
28
|
+
8. Every rewrite passes through a mandatory CID/PII scrub call (see Content Policy below) before it is assembled for upload. When the scrub cannot vouch for the rewrite, the existing memory is kept unchanged.
|
|
29
|
+
9. If the profile changed and `memory.md` uploads successfully (ingestion skipped, content hidden from chat), that Step is completed with a **Review your context memory** detail entry (same settings badge). On NOOP or failed upload the Step completes without the entry.
|
|
30
|
+
|
|
31
|
+
## Content Policy — the current user's CID/PII only (UN-24886)
|
|
32
|
+
|
|
33
|
+
The profile belongs to exactly one person: the signed-in user. It may contain
|
|
34
|
+
personal data (PII) of that user only. It must never contain
|
|
35
|
+
client-identifying data (CID) or PII of any other person or private entity —
|
|
36
|
+
not other users, not clients, not prospects, not counterparties. Facts about
|
|
37
|
+
the user's own work that involve a client are stored only in a fully
|
|
38
|
+
de-identified form ("runs quarterly portfolio reviews", never "reviews the
|
|
39
|
+
portfolio of J. Muster"). Credentials, payment data, health records, and
|
|
40
|
+
government IDs are never stored for anyone, including the user.
|
|
41
|
+
|
|
42
|
+
The policy text lives once, in `_CID_POLICY` in `user_memory_prompts.py`, and
|
|
43
|
+
is embedded in every write-path stage so the rules cannot drift:
|
|
44
|
+
|
|
45
|
+
| Stage | Role of the policy |
|
|
46
|
+
| --- | --- |
|
|
47
|
+
| Gate (`memory_gate_system_prompt`) | Turns whose only new facts are third-party CID/PII lean `NOOP` and never reach the rewrite. |
|
|
48
|
+
| Consolidation (`consolidation_system_prompt`) | The rewrite itself must not extract forbidden content. |
|
|
49
|
+
| Condensation (`condensation_system_prompt`) | Oversized (including legacy) profiles must drop violating bullets on every shrink, progressively cleaning old data. |
|
|
50
|
+
| Scrub (`scrub_system_prompt` / `scrub_user_memory`) | Mandatory final gate on every consolidation rewrite. Answers `CLEAN`, returns a cleaned body, or vetoes the write. Fails closed: on any error the existing memory is kept and the unvalidated candidate is discarded. |
|
|
51
|
+
|
|
52
|
+
Storage-level isolation (per-user home folder with exclusive owner ACL, see
|
|
53
|
+
Storage Model) already prevents cross-user *access*; this policy governs
|
|
54
|
+
cross-user and cross-client *content*.
|
|
55
|
+
|
|
56
|
+
Note for follow-up client-memory work: this package is user memory only. A
|
|
57
|
+
future client memory is a separate artefact with its own scoping and must not
|
|
58
|
+
be built by loosening this policy — client-related durable knowledge is
|
|
59
|
+
intentionally rejected here rather than stored under the user's profile.
|
|
43
60
|
|
|
44
61
|
## Storage Model
|
|
45
62
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[project]
|
|
2
2
|
name = "unique_user_memory"
|
|
3
|
-
version = "2026.38.0.
|
|
3
|
+
version = "2026.38.0.dev2"
|
|
4
4
|
description = ""
|
|
5
5
|
authors = [
|
|
6
6
|
{ name = "Fabian Schläpfer", email = "fabian@unique.ch" },
|
|
@@ -11,8 +11,8 @@ requires-python = ">=3.12,<4"
|
|
|
11
11
|
dependencies = [
|
|
12
12
|
"jinja2>=3.1.6",
|
|
13
13
|
"pydantic>=2.8.2",
|
|
14
|
-
"unique-sdk>=2026.38.0.
|
|
15
|
-
"unique-toolkit>=2026.38.0.
|
|
14
|
+
"unique-sdk>=2026.38.0.dev4,<2026.38.0rc0",
|
|
15
|
+
"unique-toolkit>=2026.38.0.dev14,<2026.38.0rc0",
|
|
16
16
|
]
|
|
17
17
|
|
|
18
18
|
[dependency-groups]
|
|
@@ -36,6 +36,7 @@ from unique_user_memory.user_memory import (
|
|
|
36
36
|
ensure_user_memory_folder,
|
|
37
37
|
fit_user_memory,
|
|
38
38
|
profile_body,
|
|
39
|
+
scrub_user_memory,
|
|
39
40
|
should_consolidate_memory,
|
|
40
41
|
upload_user_memory,
|
|
41
42
|
)
|
|
@@ -46,6 +47,7 @@ from unique_user_memory.user_memory_prompts import (
|
|
|
46
47
|
consolidation_system_prompt,
|
|
47
48
|
empty_profile,
|
|
48
49
|
memory_gate_system_prompt,
|
|
50
|
+
scrub_system_prompt,
|
|
49
51
|
)
|
|
50
52
|
|
|
51
53
|
_TEST_LANGUAGE_MODEL = LanguageModelInfo.from_name(DEFAULT_LANGUAGE_MODEL)
|
|
@@ -375,6 +377,7 @@ async def test_user_memory_llm_paths_forward_event_attribution(
|
|
|
375
377
|
side_effect=[
|
|
376
378
|
_completion_response("UPDATE"),
|
|
377
379
|
_completion_response(rewritten),
|
|
380
|
+
_completion_response("CLEAN"),
|
|
378
381
|
_completion_response(condensed),
|
|
379
382
|
]
|
|
380
383
|
)
|
|
@@ -415,7 +418,7 @@ async def test_user_memory_llm_paths_forward_event_attribution(
|
|
|
415
418
|
assert gate_result is True
|
|
416
419
|
assert rewritten in rewrite_result
|
|
417
420
|
assert condense_result == condensed
|
|
418
|
-
assert complete_async.await_count ==
|
|
421
|
+
assert complete_async.await_count == 4
|
|
419
422
|
assert all(
|
|
420
423
|
call.kwargs["chat_id"] == "chat_1"
|
|
421
424
|
and call.kwargs["assistant_id"] == "assistant_1"
|
|
@@ -564,7 +567,8 @@ async def test_consolidate_user_memory_runs_full_rewrite_when_gate_update(
|
|
|
564
567
|
assert "user_id: user_1" in result
|
|
565
568
|
assert "schema_version: 1" in result
|
|
566
569
|
assert "turn_count: 1" in result
|
|
567
|
-
|
|
570
|
+
# One rewrite call plus the mandatory CID/PII scrub call (UN-24886).
|
|
571
|
+
assert llm_service.complete_async.await_count == 2
|
|
568
572
|
|
|
569
573
|
|
|
570
574
|
@pytest.mark.asyncio
|
|
@@ -676,7 +680,8 @@ async def test_consolidate_user_memory_skips_gate_when_disabled(
|
|
|
676
680
|
assert "user_id: user_1" in result
|
|
677
681
|
assert "turn_count: 1" in result
|
|
678
682
|
gate.assert_not_awaited()
|
|
679
|
-
|
|
683
|
+
# One rewrite call plus the mandatory CID/PII scrub call (UN-24886).
|
|
684
|
+
assert llm_service.complete_async.await_count == 2
|
|
680
685
|
|
|
681
686
|
|
|
682
687
|
@pytest.mark.asyncio
|
|
@@ -799,6 +804,257 @@ async def test_should_consolidate_memory_falls_back_to_true_on_error(
|
|
|
799
804
|
assert result is True
|
|
800
805
|
|
|
801
806
|
|
|
807
|
+
@pytest.mark.ai
|
|
808
|
+
def test_cid_policy_is_embedded_in_every_write_path_prompt() -> None:
|
|
809
|
+
"""Purpose: Verify the shared CID/PII policy reaches all write-path prompts.
|
|
810
|
+
Why this matters: UN-24886 forbids other people's CID/PII in User Memory;
|
|
811
|
+
a prompt missing the policy silently reopens the leak.
|
|
812
|
+
Setup summary: Render all four system prompts and assert the policy marker.
|
|
813
|
+
"""
|
|
814
|
+
policy_marker = "CID / PII policy - the current user only - STRICT"
|
|
815
|
+
prompts = (
|
|
816
|
+
consolidation_system_prompt(2000),
|
|
817
|
+
condensation_system_prompt(
|
|
818
|
+
max_tokens=2000,
|
|
819
|
+
current_tokens=2200,
|
|
820
|
+
target_tokens=1800,
|
|
821
|
+
),
|
|
822
|
+
memory_gate_system_prompt(),
|
|
823
|
+
scrub_system_prompt(),
|
|
824
|
+
)
|
|
825
|
+
|
|
826
|
+
assert all(policy_marker in prompt for prompt in prompts)
|
|
827
|
+
assert all("Bank-client relationship signals" in prompt for prompt in prompts)
|
|
828
|
+
# The policy must not swallow the user's own facts: employer changes and
|
|
829
|
+
# other affiliations are identity data, not CID (regression guard for the
|
|
830
|
+
# gate answering NOOP on "I work at X now, not at Y anymore").
|
|
831
|
+
assert all(
|
|
832
|
+
"Facts about the user themselves are ALWAYS allowed" in prompt
|
|
833
|
+
for prompt in prompts
|
|
834
|
+
)
|
|
835
|
+
# Same for the user's own personal life facts (regression guard for the
|
|
836
|
+
# gate answering NOOP on the user stating their own family status).
|
|
837
|
+
assert all(
|
|
838
|
+
"The user's own life facts stated about themselves are also allowed" in prompt
|
|
839
|
+
for prompt in prompts
|
|
840
|
+
)
|
|
841
|
+
|
|
842
|
+
|
|
843
|
+
@pytest.mark.ai
|
|
844
|
+
@pytest.mark.asyncio
|
|
845
|
+
async def test_scrub_user_memory_returns_body_unchanged_on_clean(
|
|
846
|
+
monkeypatch: pytest.MonkeyPatch,
|
|
847
|
+
) -> None:
|
|
848
|
+
"""Purpose: Verify a compliant profile passes the scrub untouched.
|
|
849
|
+
Why this matters: The scrub must not mutate profiles that already comply.
|
|
850
|
+
Setup summary: Mock the LLM to answer CLEAN and compare input to output.
|
|
851
|
+
"""
|
|
852
|
+
body = _complete_profile_body("- Prefers concise answers")
|
|
853
|
+
response = MagicMock()
|
|
854
|
+
response.choices[0].message.content = "CLEAN"
|
|
855
|
+
llm_service = MagicMock()
|
|
856
|
+
llm_service.complete_async = AsyncMock(return_value=response)
|
|
857
|
+
monkeypatch.setattr(
|
|
858
|
+
"unique_user_memory.user_memory.LanguageModelService",
|
|
859
|
+
MagicMock(return_value=llm_service),
|
|
860
|
+
)
|
|
861
|
+
|
|
862
|
+
result = await scrub_user_memory(
|
|
863
|
+
body=body,
|
|
864
|
+
config=UserMemoryConfig(),
|
|
865
|
+
language_model=_TEST_LANGUAGE_MODEL,
|
|
866
|
+
event=MagicMock(),
|
|
867
|
+
logger=MagicMock(),
|
|
868
|
+
)
|
|
869
|
+
|
|
870
|
+
assert result == body
|
|
871
|
+
|
|
872
|
+
|
|
873
|
+
@pytest.mark.ai
|
|
874
|
+
@pytest.mark.asyncio
|
|
875
|
+
async def test_scrub_user_memory_returns_cleaned_profile(
|
|
876
|
+
monkeypatch: pytest.MonkeyPatch,
|
|
877
|
+
) -> None:
|
|
878
|
+
"""Purpose: Verify the scrub replaces a violating body with the cleaned one.
|
|
879
|
+
Why this matters: Third-party CID caught only at this last gate must be
|
|
880
|
+
stripped before the profile is persisted.
|
|
881
|
+
Setup summary: Mock the LLM to return a de-identified rewrite and assert it wins.
|
|
882
|
+
"""
|
|
883
|
+
body_with_cid = _complete_profile_body(
|
|
884
|
+
"- Reviews the portfolio of J. Muster (client, IBAN CH93 0076 2011 6238 5295 7)"
|
|
885
|
+
)
|
|
886
|
+
cleaned = _complete_profile_body("- Runs quarterly portfolio reviews")
|
|
887
|
+
response = MagicMock()
|
|
888
|
+
response.choices[0].message.content = cleaned
|
|
889
|
+
llm_service = MagicMock()
|
|
890
|
+
llm_service.complete_async = AsyncMock(return_value=response)
|
|
891
|
+
monkeypatch.setattr(
|
|
892
|
+
"unique_user_memory.user_memory.LanguageModelService",
|
|
893
|
+
MagicMock(return_value=llm_service),
|
|
894
|
+
)
|
|
895
|
+
|
|
896
|
+
result = await scrub_user_memory(
|
|
897
|
+
body=body_with_cid,
|
|
898
|
+
config=UserMemoryConfig(),
|
|
899
|
+
language_model=_TEST_LANGUAGE_MODEL,
|
|
900
|
+
event=MagicMock(),
|
|
901
|
+
logger=MagicMock(),
|
|
902
|
+
)
|
|
903
|
+
|
|
904
|
+
assert result == cleaned
|
|
905
|
+
assert "J. Muster" not in result
|
|
906
|
+
|
|
907
|
+
|
|
908
|
+
@pytest.mark.ai
|
|
909
|
+
@pytest.mark.asyncio
|
|
910
|
+
async def test_scrub_user_memory_fails_closed_on_llm_error(
|
|
911
|
+
monkeypatch: pytest.MonkeyPatch,
|
|
912
|
+
) -> None:
|
|
913
|
+
"""Purpose: Verify a failing scrub call vetoes the write instead of passing it.
|
|
914
|
+
Why this matters: An unvalidated candidate must never reach storage; the
|
|
915
|
+
gate falls back to UPDATE on error, so the scrub must fall back to reject.
|
|
916
|
+
Setup summary: Mock the LLM to raise and assert the scrub returns None.
|
|
917
|
+
"""
|
|
918
|
+
llm_service = MagicMock()
|
|
919
|
+
llm_service.complete_async = AsyncMock(side_effect=RuntimeError("boom"))
|
|
920
|
+
monkeypatch.setattr(
|
|
921
|
+
"unique_user_memory.user_memory.LanguageModelService",
|
|
922
|
+
MagicMock(return_value=llm_service),
|
|
923
|
+
)
|
|
924
|
+
|
|
925
|
+
result = await scrub_user_memory(
|
|
926
|
+
body=_complete_profile_body(),
|
|
927
|
+
config=UserMemoryConfig(),
|
|
928
|
+
language_model=_TEST_LANGUAGE_MODEL,
|
|
929
|
+
event=MagicMock(),
|
|
930
|
+
logger=MagicMock(),
|
|
931
|
+
)
|
|
932
|
+
|
|
933
|
+
assert result is None
|
|
934
|
+
|
|
935
|
+
|
|
936
|
+
@pytest.mark.ai
|
|
937
|
+
@pytest.mark.asyncio
|
|
938
|
+
async def test_scrub_user_memory_fails_closed_on_malformed_output(
|
|
939
|
+
monkeypatch: pytest.MonkeyPatch,
|
|
940
|
+
) -> None:
|
|
941
|
+
"""Purpose: Verify non-profile scrub output rejects the candidate.
|
|
942
|
+
Why this matters: A scrub reply that is neither CLEAN nor a well-formed
|
|
943
|
+
profile gives no guarantee about the candidate, so it must not be stored.
|
|
944
|
+
Setup summary: Mock the LLM to return prose and assert the scrub returns None.
|
|
945
|
+
"""
|
|
946
|
+
response = MagicMock()
|
|
947
|
+
response.choices[0].message.content = "I removed some sensitive data for you."
|
|
948
|
+
llm_service = MagicMock()
|
|
949
|
+
llm_service.complete_async = AsyncMock(return_value=response)
|
|
950
|
+
monkeypatch.setattr(
|
|
951
|
+
"unique_user_memory.user_memory.LanguageModelService",
|
|
952
|
+
MagicMock(return_value=llm_service),
|
|
953
|
+
)
|
|
954
|
+
|
|
955
|
+
result = await scrub_user_memory(
|
|
956
|
+
body=_complete_profile_body(),
|
|
957
|
+
config=UserMemoryConfig(),
|
|
958
|
+
language_model=_TEST_LANGUAGE_MODEL,
|
|
959
|
+
event=MagicMock(),
|
|
960
|
+
logger=MagicMock(),
|
|
961
|
+
)
|
|
962
|
+
|
|
963
|
+
assert result is None
|
|
964
|
+
|
|
965
|
+
|
|
966
|
+
@pytest.mark.ai
|
|
967
|
+
@pytest.mark.asyncio
|
|
968
|
+
async def test_consolidate_user_memory_persists_scrubbed_body(
|
|
969
|
+
monkeypatch: pytest.MonkeyPatch,
|
|
970
|
+
) -> None:
|
|
971
|
+
"""Purpose: Verify the consolidation write path stores the scrubbed rewrite.
|
|
972
|
+
Why this matters: The scrub is the enforcement layer for UN-24886; the
|
|
973
|
+
profile assembled for upload must be the cleaned body, not the raw rewrite.
|
|
974
|
+
Setup summary: Rewrite emits third-party CID, scrub emits a de-identified
|
|
975
|
+
version; assert the result contains only the latter.
|
|
976
|
+
"""
|
|
977
|
+
rewritten_with_cid = _complete_profile_body(
|
|
978
|
+
"- Reviews the portfolio of J. Muster (client)"
|
|
979
|
+
)
|
|
980
|
+
cleaned = _complete_profile_body("- Runs quarterly portfolio reviews")
|
|
981
|
+
responses = iter([rewritten_with_cid, cleaned])
|
|
982
|
+
|
|
983
|
+
def _next_response(*args: object, **kwargs: object) -> MagicMock:
|
|
984
|
+
response = MagicMock()
|
|
985
|
+
response.choices[0].message.content = next(responses)
|
|
986
|
+
return response
|
|
987
|
+
|
|
988
|
+
llm_service = MagicMock()
|
|
989
|
+
llm_service.complete_async = AsyncMock(side_effect=_next_response)
|
|
990
|
+
monkeypatch.setattr(
|
|
991
|
+
"unique_user_memory.user_memory.LanguageModelService",
|
|
992
|
+
MagicMock(return_value=llm_service),
|
|
993
|
+
)
|
|
994
|
+
monkeypatch.setattr(
|
|
995
|
+
"unique_user_memory.user_memory.should_consolidate_memory",
|
|
996
|
+
AsyncMock(return_value=True),
|
|
997
|
+
)
|
|
998
|
+
|
|
999
|
+
result = await consolidate_user_memory(
|
|
1000
|
+
current_memory=empty_profile("user_1"),
|
|
1001
|
+
user_id="user_1",
|
|
1002
|
+
user_message="I just reviewed the portfolio of J. Muster",
|
|
1003
|
+
assistant_message="noted",
|
|
1004
|
+
config=UserMemoryConfig(),
|
|
1005
|
+
language_model=_TEST_LANGUAGE_MODEL,
|
|
1006
|
+
event=MagicMock(),
|
|
1007
|
+
logger=MagicMock(),
|
|
1008
|
+
)
|
|
1009
|
+
|
|
1010
|
+
assert "Runs quarterly portfolio reviews" in result
|
|
1011
|
+
assert "J. Muster" not in result
|
|
1012
|
+
assert llm_service.complete_async.await_count == 2
|
|
1013
|
+
|
|
1014
|
+
|
|
1015
|
+
@pytest.mark.ai
|
|
1016
|
+
@pytest.mark.asyncio
|
|
1017
|
+
async def test_consolidate_user_memory_keeps_existing_when_scrub_fails(
|
|
1018
|
+
monkeypatch: pytest.MonkeyPatch,
|
|
1019
|
+
) -> None:
|
|
1020
|
+
"""Purpose: Verify a failed scrub keeps the existing memory (fail closed).
|
|
1021
|
+
Why this matters: When the enforcement layer cannot vouch for a rewrite,
|
|
1022
|
+
the unvalidated candidate must be discarded, not uploaded.
|
|
1023
|
+
Setup summary: Let the rewrite succeed, force the scrub to return None,
|
|
1024
|
+
and assert the pre-turn memory survives unchanged.
|
|
1025
|
+
"""
|
|
1026
|
+
current = empty_profile("user_1")
|
|
1027
|
+
response = MagicMock()
|
|
1028
|
+
response.choices[0].message.content = _complete_profile_body()
|
|
1029
|
+
llm_service = MagicMock()
|
|
1030
|
+
llm_service.complete_async = AsyncMock(return_value=response)
|
|
1031
|
+
monkeypatch.setattr(
|
|
1032
|
+
"unique_user_memory.user_memory.LanguageModelService",
|
|
1033
|
+
MagicMock(return_value=llm_service),
|
|
1034
|
+
)
|
|
1035
|
+
monkeypatch.setattr(
|
|
1036
|
+
"unique_user_memory.user_memory.should_consolidate_memory",
|
|
1037
|
+
AsyncMock(return_value=True),
|
|
1038
|
+
)
|
|
1039
|
+
monkeypatch.setattr(
|
|
1040
|
+
"unique_user_memory.user_memory.scrub_user_memory",
|
|
1041
|
+
AsyncMock(return_value=None),
|
|
1042
|
+
)
|
|
1043
|
+
|
|
1044
|
+
result = await consolidate_user_memory(
|
|
1045
|
+
current_memory=current,
|
|
1046
|
+
user_id="user_1",
|
|
1047
|
+
user_message="remember I like concise answers",
|
|
1048
|
+
assistant_message="noted",
|
|
1049
|
+
config=UserMemoryConfig(),
|
|
1050
|
+
language_model=_TEST_LANGUAGE_MODEL,
|
|
1051
|
+
event=MagicMock(),
|
|
1052
|
+
logger=MagicMock(),
|
|
1053
|
+
)
|
|
1054
|
+
|
|
1055
|
+
assert result == current
|
|
1056
|
+
|
|
1057
|
+
|
|
802
1058
|
@pytest.mark.asyncio
|
|
803
1059
|
async def test_consolidate_user_memory_invokes_update_callbacks_on_rewrite(
|
|
804
1060
|
monkeypatch: pytest.MonkeyPatch,
|
|
@@ -38,6 +38,8 @@ from unique_user_memory.user_memory_prompts import (
|
|
|
38
38
|
empty_profile,
|
|
39
39
|
memory_gate_system_prompt,
|
|
40
40
|
memory_gate_user_prompt,
|
|
41
|
+
scrub_system_prompt,
|
|
42
|
+
scrub_user_prompt,
|
|
41
43
|
)
|
|
42
44
|
|
|
43
45
|
MEMORY_FILENAME = "memory.md"
|
|
@@ -846,6 +848,101 @@ async def consolidate_user_memory(
|
|
|
846
848
|
await on_update_end()
|
|
847
849
|
|
|
848
850
|
|
|
851
|
+
async def scrub_user_memory(
|
|
852
|
+
*,
|
|
853
|
+
body: str,
|
|
854
|
+
config: UserMemoryConfig,
|
|
855
|
+
language_model: LanguageModelInfo,
|
|
856
|
+
event: ChatEvent,
|
|
857
|
+
logger: Logger,
|
|
858
|
+
invocation_stats: list[LanguageModelInvocationStats] | None = None,
|
|
859
|
+
) -> str | None:
|
|
860
|
+
"""Enforce the CID/PII policy on a candidate profile body (UN-24886).
|
|
861
|
+
|
|
862
|
+
Runs after every consolidation rewrite and before the profile is
|
|
863
|
+
assembled for upload. Returns the body unchanged when the model
|
|
864
|
+
answers ``CLEAN``, the cleaned body when bullets were removed or
|
|
865
|
+
de-identified, and ``None`` when the scrub cannot vouch for the
|
|
866
|
+
candidate (LLM error, malformed output). The caller MUST treat
|
|
867
|
+
``None`` as fail-closed and keep the existing memory.
|
|
868
|
+
"""
|
|
869
|
+
try:
|
|
870
|
+
llm_service = LanguageModelService(event)
|
|
871
|
+
except Exception as exc:
|
|
872
|
+
logger.warning(
|
|
873
|
+
"[user-memory] cannot construct LanguageModelService for scrub: [%s] %s",
|
|
874
|
+
type(exc).__name__,
|
|
875
|
+
exc,
|
|
876
|
+
)
|
|
877
|
+
return None
|
|
878
|
+
|
|
879
|
+
messages = LanguageModelMessages(
|
|
880
|
+
[
|
|
881
|
+
LanguageModelSystemMessage(content=scrub_system_prompt()),
|
|
882
|
+
LanguageModelUserMessage(
|
|
883
|
+
content=scrub_user_prompt(_sanitize_for_xml_context(body))
|
|
884
|
+
),
|
|
885
|
+
]
|
|
886
|
+
)
|
|
887
|
+
|
|
888
|
+
try:
|
|
889
|
+
response = await llm_service.complete_async(
|
|
890
|
+
messages=messages,
|
|
891
|
+
model_name=language_model.name,
|
|
892
|
+
other_options={
|
|
893
|
+
"max_tokens": config.max_tokens + _LLM_OUTPUT_HEADROOM_TOKENS
|
|
894
|
+
},
|
|
895
|
+
)
|
|
896
|
+
except Exception as exc:
|
|
897
|
+
logger.warning(
|
|
898
|
+
"[user-memory] scrub LLM call failed (model=%s): [%s] %s",
|
|
899
|
+
language_model.name,
|
|
900
|
+
type(exc).__name__,
|
|
901
|
+
exc,
|
|
902
|
+
)
|
|
903
|
+
return None
|
|
904
|
+
|
|
905
|
+
if invocation_stats is not None and response.usage is not None:
|
|
906
|
+
invocation_stats.append(
|
|
907
|
+
LanguageModelInvocationStats.from_usage(
|
|
908
|
+
language_model.name,
|
|
909
|
+
response.usage,
|
|
910
|
+
source="user_memory_scrub",
|
|
911
|
+
)
|
|
912
|
+
)
|
|
913
|
+
|
|
914
|
+
try:
|
|
915
|
+
raw = response.choices[0].message.content or ""
|
|
916
|
+
except Exception as exc:
|
|
917
|
+
logger.warning(
|
|
918
|
+
"[user-memory] could not extract content from scrub response: [%s] %s",
|
|
919
|
+
type(exc).__name__,
|
|
920
|
+
exc,
|
|
921
|
+
)
|
|
922
|
+
return None
|
|
923
|
+
|
|
924
|
+
if not isinstance(raw, str):
|
|
925
|
+
logger.warning(
|
|
926
|
+
"[user-memory] scrub returned non-string content (%s)",
|
|
927
|
+
type(raw).__name__,
|
|
928
|
+
)
|
|
929
|
+
return None
|
|
930
|
+
|
|
931
|
+
if raw.strip().upper() == "CLEAN":
|
|
932
|
+
return body
|
|
933
|
+
|
|
934
|
+
cleaned = _without_legacy_profile_title(profile_body(_strip_code_fences(raw)))
|
|
935
|
+
if not _is_well_formed_profile(cleaned):
|
|
936
|
+
logger.warning(
|
|
937
|
+
"[user-memory] scrub output did not look like a profile (%d chars)",
|
|
938
|
+
len(cleaned),
|
|
939
|
+
)
|
|
940
|
+
return None
|
|
941
|
+
|
|
942
|
+
logger.info("[user-memory] scrub removed or de-identified profile content")
|
|
943
|
+
return cleaned
|
|
944
|
+
|
|
945
|
+
|
|
849
946
|
async def _rewrite_user_memory(
|
|
850
947
|
*,
|
|
851
948
|
safe_current: str,
|
|
@@ -945,6 +1042,22 @@ async def _rewrite_user_memory(
|
|
|
945
1042
|
)
|
|
946
1043
|
return safe_current
|
|
947
1044
|
|
|
1045
|
+
scrubbed_body = await scrub_user_memory(
|
|
1046
|
+
body=candidate_body,
|
|
1047
|
+
config=config,
|
|
1048
|
+
language_model=language_model,
|
|
1049
|
+
event=event,
|
|
1050
|
+
logger=logger,
|
|
1051
|
+
invocation_stats=invocation_stats,
|
|
1052
|
+
)
|
|
1053
|
+
if scrubbed_body is None:
|
|
1054
|
+
logger.warning(
|
|
1055
|
+
"[user-memory] scrub could not validate the rewritten profile - "
|
|
1056
|
+
"keeping existing memory"
|
|
1057
|
+
)
|
|
1058
|
+
return safe_current
|
|
1059
|
+
candidate_body = scrubbed_body
|
|
1060
|
+
|
|
948
1061
|
if safe_current and candidate_body == profile_body(safe_current):
|
|
949
1062
|
logger.debug("[user-memory] memory body unchanged - skipping update")
|
|
950
1063
|
return safe_current
|
|
@@ -37,6 +37,63 @@ def empty_profile(user_id: str) -> str:
|
|
|
37
37
|
)
|
|
38
38
|
|
|
39
39
|
|
|
40
|
+
# Shared across the consolidation, condensation, gate, and scrub prompts so
|
|
41
|
+
# the CID/PII rules cannot drift between the write-path stages (UN-24886).
|
|
42
|
+
_CID_POLICY = """\
|
|
43
|
+
# CID / PII policy - the current user only - STRICT
|
|
44
|
+
|
|
45
|
+
The profile belongs to exactly one person: the signed-in user. It may
|
|
46
|
+
contain personal data (PII) of that user only. It must NEVER contain
|
|
47
|
+
client-identifying data (CID) or PII of any other person or private
|
|
48
|
+
entity - not other users, not clients, not prospects, not counterparties.
|
|
49
|
+
|
|
50
|
+
NEVER store, about any person or private entity other than the user:
|
|
51
|
+
|
|
52
|
+
- Direct identifiers: full or partial names with identifying context,
|
|
53
|
+
addresses, email addresses, phone numbers, dates of birth,
|
|
54
|
+
passport / national ID / tax ID numbers, employee IDs, client IDs,
|
|
55
|
+
customer numbers, CRM or ticket IDs.
|
|
56
|
+
- Financial identifiers: account numbers, IBANs, card numbers,
|
|
57
|
+
portfolio / custody / mandate numbers, transaction or payment
|
|
58
|
+
references, loan, mortgage, or insurance policy numbers.
|
|
59
|
+
- Bank-client relationship signals: any statement that a person or
|
|
60
|
+
private entity is a client, customer, prospect, account holder,
|
|
61
|
+
beneficial owner, borrower, investor, or KYC/onboarding subject.
|
|
62
|
+
- Financial or personal context tied to an identifiable person or
|
|
63
|
+
private entity other than the user: holdings, balances, transactions,
|
|
64
|
+
source of wealth, income, creditworthiness, risk rating, KYC/AML
|
|
65
|
+
status, or that person's or entity's health, family, legal disputes,
|
|
66
|
+
or tax situation.
|
|
67
|
+
- Indirect identification: combinations of attributes that could single
|
|
68
|
+
out a non-public person or private entity even without a name.
|
|
69
|
+
|
|
70
|
+
These rules apply even when the identifying data is abbreviated,
|
|
71
|
+
misspelled, obfuscated, or split across words.
|
|
72
|
+
|
|
73
|
+
Facts about the user themselves are ALWAYS allowed, even when they name
|
|
74
|
+
an organization: the user's own employer (current and past), team, role,
|
|
75
|
+
job title, location, and the organizations, products, and tools they say
|
|
76
|
+
they work at or with. "Works at COMPANY X", "moved from COMPANY X to
|
|
77
|
+
COMPANY Y" are the user's own identity facts, not CID. The prohibitions
|
|
78
|
+
above target OTHER people, and entities in a client, prospect, or
|
|
79
|
+
counterparty relationship - never the user's own employment or
|
|
80
|
+
affiliations.
|
|
81
|
+
|
|
82
|
+
The user's own life facts stated about themselves are also allowed -
|
|
83
|
+
family status, languages, hobbies, general location - as long as they
|
|
84
|
+
do not name or otherwise identify another specific individual.
|
|
85
|
+
|
|
86
|
+
When a durable fact about the user's own work involves a client or
|
|
87
|
+
counterparty, keep only a fully de-identified version: "runs quarterly
|
|
88
|
+
portfolio reviews" is fine; "reviews the portfolio of J. Muster" is not.
|
|
89
|
+
|
|
90
|
+
Also NEVER store, for anyone including the user: credentials, API keys,
|
|
91
|
+
passwords, OTPs, payment card data, health-record details, or government
|
|
92
|
+
ID numbers. If any of these appear in a conversation, ignore the value
|
|
93
|
+
entirely.\
|
|
94
|
+
"""
|
|
95
|
+
|
|
96
|
+
|
|
40
97
|
_CONSOLIDATION_SYSTEM_PROMPT_TEMPLATE = """\
|
|
41
98
|
You are a memory-consolidation engine for the Unique AI platform.
|
|
42
99
|
|
|
@@ -160,15 +217,13 @@ ADD/UPDATE for facts that are:
|
|
|
160
217
|
|
|
161
218
|
NEVER extract:
|
|
162
219
|
|
|
163
|
-
-
|
|
164
|
-
health-record details, government IDs. If the user pastes any of
|
|
165
|
-
these, ignore the value entirely.
|
|
166
|
-
- Information about other named individuals beyond the immediate
|
|
167
|
-
professional context.
|
|
220
|
+
- Anything forbidden by the CID / PII policy below.
|
|
168
221
|
- Transient turn-level state: one-off factual answers, code snippets,
|
|
169
222
|
error messages, file contents, search results.
|
|
170
223
|
- Anything stated as third-party information or retrieved context.
|
|
171
224
|
|
|
225
|
+
{{ cid_policy }}
|
|
226
|
+
|
|
172
227
|
# Word budget - STRICT
|
|
173
228
|
|
|
174
229
|
The complete body MUST be <= {{ max_words }} words (corresponding to {{ max_tokens }} tokens).
|
|
@@ -202,6 +257,7 @@ def consolidation_system_prompt(max_tokens: int) -> str:
|
|
|
202
257
|
max_tokens=max_tokens,
|
|
203
258
|
section_list=section_list,
|
|
204
259
|
now_datetime=now.strftime("%Y-%m-%d %H:%M UTC"),
|
|
260
|
+
cid_policy=_CID_POLICY,
|
|
205
261
|
)
|
|
206
262
|
|
|
207
263
|
|
|
@@ -243,6 +299,12 @@ fact about the user. This is lossy compression, not deletion of meaning.
|
|
|
243
299
|
# Hard rules
|
|
244
300
|
|
|
245
301
|
- NEVER invent, embellish, or add facts that are not already present.
|
|
302
|
+
- DELETE any bullet that violates the CID / PII policy below - CID or
|
|
303
|
+
PII of any person or private entity other than the user. This deletion
|
|
304
|
+
is mandatory even when the bullet is otherwise high-signal and does
|
|
305
|
+
not count against the shrink priorities above. When such a bullet also
|
|
306
|
+
carries a durable fact about the user's own work, keep only a fully
|
|
307
|
+
de-identified rewrite.
|
|
246
308
|
- Keep exactly these section headings, in this order, even if a section
|
|
247
309
|
becomes empty (use the literal string `_(empty)_`):
|
|
248
310
|
|
|
@@ -252,6 +314,8 @@ fact about the user. This is lossy compression, not deletion of meaning.
|
|
|
252
314
|
keep two conflicting bullets.
|
|
253
315
|
- Use `-` markdown bullets, no nesting beyond two levels, no emojis.
|
|
254
316
|
|
|
317
|
+
{{ cid_policy }}
|
|
318
|
+
|
|
255
319
|
# Output
|
|
256
320
|
|
|
257
321
|
Return the complete rewritten profile body, starting with
|
|
@@ -276,6 +340,7 @@ def condensation_system_prompt(
|
|
|
276
340
|
target_words=int(target_tokens * 0.75),
|
|
277
341
|
reduction_pct=max(reduction_pct, 1),
|
|
278
342
|
max_tokens=max_tokens,
|
|
343
|
+
cid_policy=_CID_POLICY,
|
|
279
344
|
)
|
|
280
345
|
|
|
281
346
|
|
|
@@ -358,17 +423,18 @@ only the single word `UPDATE` or `NOOP`.
|
|
|
358
423
|
|
|
359
424
|
# What NEVER justifies UPDATE (lean NOOP)
|
|
360
425
|
|
|
361
|
-
-
|
|
362
|
-
- Facts about other people beyond the immediate professional context.
|
|
426
|
+
- Anything forbidden by the CID / PII policy below.
|
|
363
427
|
- Transient turn state: one-off answers, code snippets, error messages,
|
|
364
428
|
file contents, search results.
|
|
365
429
|
- Anything stated as third-party or retrieved context.
|
|
366
430
|
- Facts already captured in `<existing_memory>`.
|
|
431
|
+
|
|
432
|
+
{{ cid_policy }}
|
|
367
433
|
"""
|
|
368
434
|
|
|
369
435
|
|
|
370
436
|
def memory_gate_system_prompt() -> str:
|
|
371
|
-
return Template(_GATE_SYSTEM_PROMPT_TEMPLATE).render()
|
|
437
|
+
return Template(_GATE_SYSTEM_PROMPT_TEMPLATE).render(cid_policy=_CID_POLICY)
|
|
372
438
|
|
|
373
439
|
|
|
374
440
|
_GATE_USER_PROMPT_TEMPLATE = """\
|
|
@@ -400,3 +466,60 @@ def memory_gate_user_prompt(
|
|
|
400
466
|
user_message=(user_message or "").strip(),
|
|
401
467
|
assistant_message=(assistant_message or "").strip(),
|
|
402
468
|
)
|
|
469
|
+
|
|
470
|
+
|
|
471
|
+
_SCRUB_SYSTEM_PROMPT_TEMPLATE = """\
|
|
472
|
+
You are the final content gate for the user-memory system on the Unique
|
|
473
|
+
AI platform. You receive a candidate Markdown memory profile just before
|
|
474
|
+
it is persisted. Your ONLY job is to enforce the CID / PII policy below;
|
|
475
|
+
you never improve, extend, or restyle the profile.
|
|
476
|
+
|
|
477
|
+
{{ cid_policy }}
|
|
478
|
+
|
|
479
|
+
# Your task
|
|
480
|
+
|
|
481
|
+
Inspect every bullet of the candidate profile:
|
|
482
|
+
|
|
483
|
+
- Remove each bullet that violates the policy. When a violating bullet
|
|
484
|
+
also carries a durable fact about the user's own work, replace it with
|
|
485
|
+
a fully de-identified rewrite instead of deleting it.
|
|
486
|
+
- Leave every compliant bullet unchanged, word for word. Never add
|
|
487
|
+
facts, never merge or reorder bullets, never rename or reorder
|
|
488
|
+
sections.
|
|
489
|
+
- A section left without bullets keeps its heading with the literal
|
|
490
|
+
placeholder `_(empty)_`.
|
|
491
|
+
|
|
492
|
+
# Output
|
|
493
|
+
|
|
494
|
+
- When every bullet complies, reply with EXACTLY the single uppercase
|
|
495
|
+
word `CLEAN` and nothing else.
|
|
496
|
+
- Otherwise return the complete cleaned profile body, starting with
|
|
497
|
+
`## Identity` and containing exactly these section headings in this
|
|
498
|
+
order:
|
|
499
|
+
|
|
500
|
+
{{ section_list }}
|
|
501
|
+
|
|
502
|
+
Do NOT add a document-level `# User Memory` title. Do NOT emit a diff or
|
|
503
|
+
commentary. Do NOT wrap the output in ``` fences.
|
|
504
|
+
"""
|
|
505
|
+
|
|
506
|
+
|
|
507
|
+
def scrub_system_prompt() -> str:
|
|
508
|
+
section_list = "\n".join(f"- ## {heading}" for heading in SECTION_HEADINGS)
|
|
509
|
+
return Template(_SCRUB_SYSTEM_PROMPT_TEMPLATE).render(
|
|
510
|
+
cid_policy=_CID_POLICY,
|
|
511
|
+
section_list=section_list,
|
|
512
|
+
)
|
|
513
|
+
|
|
514
|
+
|
|
515
|
+
_SCRUB_USER_PROMPT_TEMPLATE = """\
|
|
516
|
+
<candidate_profile>
|
|
517
|
+
{{ profile }}
|
|
518
|
+
</candidate_profile>
|
|
519
|
+
|
|
520
|
+
Reply with CLEAN or the complete cleaned profile body now.
|
|
521
|
+
"""
|
|
522
|
+
|
|
523
|
+
|
|
524
|
+
def scrub_user_prompt(profile: str) -> str:
|
|
525
|
+
return Template(_SCRUB_USER_PROMPT_TEMPLATE).render(profile=profile)
|
|
File without changes
|
{unique_user_memory-2026.38.0.dev0 → unique_user_memory-2026.38.0.dev2}/unique_user_memory/config.py
RENAMED
|
File without changes
|
|
File without changes
|