aicordon-haystack 0.1.0__tar.gz

aicordon_haystack-0.1.0/.gitignore

@@ -0,0 +1,41 @@
+# Secrets never live next to code. This line is a fuse, not a description of what is here.
+.env
+.env.*
+
+__pycache__/
+*.py[cod]
+.venv/
+dist/
+build/
+*.egg-info/
+
+# The injection bank and any datasets never travel: the bank is both a resource of the project and
+# its only honest held-out test. Insurance against an accidental `git add`, not a precaution in the
+# abstract — memory is not the thing to rely on here.
+datasets/
+data/pools/
+*.jsonl
+
+# Three exceptions, each of them ours end to end and rebuildable from the script beside it: the ten
+# demonstration exchanges and the two ground-truth manifests. Without them a clone can neither read
+# the example sets nor run `show_material.py` / `show_request.py` over them.
+!integrations/examples/dialogues.jsonl
+!integrations/examples/dialog_manifest.jsonl
+!integrations/examples/manifest.jsonl
+
+# The sources the base is built from stay out: the slot dictionaries and the rule file are at once
+# the recipe for the base and the instructions for walking around it. The product does not claim its
+# rules are readable; what ships is the built base, one file (`src/aicordon/picket/data/*.bin`).
+src/aicordon/picket/data/slots/
+src/aicordon/picket/data/*.json
+
+# Research notes live in the research repository and do not travel. This line is not caution in the
+# abstract: an append to RESULTS.md from this directory has already dropped fifty lines of
+# measurement prose in here once. Better the mistake is caught by git than by eye.
+RESULTS.md
+HANDOFF.md
+PLAN*.md
+
+# Working notes and release plans: ours, not part of what ships. Same rule as the research
+# notes above -- the repository holds the product, not the scaffolding around a release.
+TODO/

aicordon_haystack-0.1.0/CATALOG-CARD.md

@@ -0,0 +1,133 @@
+<!--
+STAGING COPY of the integration card for the Haystack catalogue.
+
+When submitting: copy this file to `integrations/aicordon-haystack.md` in a fork of
+`deepset-ai/haystack-integrations` and open a PR. Schema confirmed 2026-08-19 from that repo's
+README:
+  - required frontmatter: layout, name, description, authors (>=1 with a name), type
+  - at least one of `pypi` or `repo` must be present for a merge
+  - allowed `type`: Document Store | Model Provider | Data Ingestion | Monitoring Tool |
+    Evaluation Framework | Custom Component | Tool Integration | Other
+  - one card per PACKAGE, not per component: `aicordon-haystack` ships two components under one card.
+
+SUBMIT ONLY AFTER `aicordon-haystack` IS ON PyPI. Order:
+  1. merge the integration branch into `main` and push - the `repo` link and the README links in
+     this card resolve off `main`;
+  2. publish `aicordon` >= 1.1.0;
+  3. publish `aicordon-haystack`;
+  4. open this PR.
+
+`logo` is optional and deliberately unset: it lives in the catalogue repository, so it is a separate
+PR whenever we want one.
+-->
+---
+layout: integration
+name: AI Cordon Picket
+description: Detect prompt injection in what an LLM is given - documents at ingest and the turn it answers - with a local rule base, no GPU or network.
+authors:
+    - name: Mike Gribov
+      socials:
+        github: mihail-gribov
+pypi: https://pypi.org/project/aicordon-haystack
+repo: https://github.com/AICordon/aicordon
+report_issue: https://github.com/AICordon/aicordon/issues
+type: Custom Component
+version: Haystack 3.0
+toc: true
+---
+
+# AI Cordon Picket for Haystack
+
+Check what an LLM is given for prompt injection - in both places it can arrive.
+
+| component | reads | with |
+|---|---|---|
+| `PromptInjectionFilter` | **material**: documents at ingest, before they are chunked and embedded | Picket's `ipi` rules |
+| `PromptInjectionGuard` | **the request**: the turn the model is about to answer | Picket's `dpi` rules |
+
+The two rule sets are disjoint, and neither is a stricter version of the other - this is not a
+sensitivity knob. Pick by role: material is what the model works on, the request is what it answers.
+Your code knows which is which; it puts them in different places when it assembles the call.
+
+The check is a rule, not a model: no GPU, no network, no key, a few hundred kilobytes of base, and a
+fraction of a millisecond per turn on one core.
+
+## Installation
+
+```bash
+pip install aicordon-haystack
+```
+
+## Material at ingest
+
+```python
+from haystack import Pipeline
+from haystack.components.preprocessors import DocumentSplitter
+from haystack.components.writers import DocumentWriter
+from haystack_integrations.components.preprocessors.aicordon import PromptInjectionFilter
+
+pipe = Pipeline()
+pipe.add_component("ipi_filter", PromptInjectionFilter(mode="redact"))
+pipe.add_component("splitter", DocumentSplitter(split_by="word", split_length=200))
+pipe.add_component("writer", DocumentWriter(document_store=store))
+
+pipe.connect("converter.documents", "ipi_filter.documents")
+pipe.connect("ipi_filter.documents", "splitter.documents")
+pipe.connect("splitter.documents", "writer.documents")
+pipe.connect("ipi_filter.rejected", "quarantine.documents")   # optional; nothing disappears quietly
+```
+
+The component sits **before the splitter**: a cut here takes the injection out of the chunks, the
+embeddings and the store at once, with no offsets to reconcile across chunk boundaries. Modes:
+`annotate`, `blank` (keeps the length), `mask`, `redact` (default), `drop`, `fail`. The cut takes
+the whole line holding the span, not the matched characters alone.
+
+## The turn the model answers
+
+```python
+from haystack_integrations.components.validators.aicordon import PromptInjectionGuard
+
+pipe.add_component("guard", PromptInjectionGuard())          # mode="drop" is the default
+pipe.connect("prompt.messages", "guard.messages")
+pipe.connect("guard.messages", "llm.messages")               # the model is called on this path
+pipe.connect("guard.blocked", "refusal.messages")            # and not on this one
+```
+
+**Two sockets, one value.** On a flagged exchange `run` returns `blocked` and no `messages` key, so
+the generator is not called at all. Connect `blocked` to whatever answers the user instead. The
+decision is for the exchange, not for one message. This side never edits a turn: a typed jailbreak
+is not spliced into anything - it *is* the turn.
+
+## Measured
+
+Not the detector's recall - that ships with the detector - but what the pipeline delivers with the
+component and without.
+
+**Material** - [Quadrat-IPI v1.0.1](https://huggingface.co/datasets/mihailgribov/quadrat-ipi),
+1000 injected + 1000 clean documents, `mode="redact"`:
+
+| | whole corpus | injections that ask the model to reveal something |
+|---|---|---|
+| payload reaches the store intact, without the filter | 100% | 100% |
+| payload reaches the store intact, with it | **85.4%** | **42.8%** |
+| payload gone without a trace | 13.1% | **52.3%** |
+| clean documents dropped or trimmed | 0 of 1000 | 0 of 1000 |
+
+**The request** - held-out forum jailbreaks from
+[TrustAIRLab in-the-wild](https://huggingface.co/datasets/TrustAIRLab/in-the-wild-jailbreak-prompts)
+(537) against 20 000 real [WildChat](https://huggingface.co/datasets/allenai/WildChat-1M) turns,
+`mode="drop"`:
+
+| | |
+|---|---|
+| attacks reaching the model, without the guard | 100% (537 of 537) |
+| attacks reaching the model, with it | **65.2%** |
+| turns not answered, out of 20 000 real ones | 0.070% (14) |
+| verdicts differing from the bare detector | **0** |
+
+Adding the component costs **0.39 ms for a turn of median length**; loading the base costs 15 ms,
+once per process. No findings does not mean no injection.
+
+## License
+
+Apache-2.0, the same as the detector it wraps.

aicordon_haystack-0.1.0/CHANGELOG.md

@@ -0,0 +1,32 @@
+# Changelog
+
+## 0.1.0 — 2026-08-21
+
+First release. Two components, one per place a pipeline holds text a model is about to read.
+
+* **`PromptInjectionFilter`** — `haystack_integrations.components.preprocessors.aicordon`. Reads
+  documents at ingest with the `ipi` rules, before chunking and embedding. Six modes (`annotate`,
+  `blank`, `mask`, `redact`, `drop`, `fail`); the cut takes the whole line holding the span, or the
+  sentence when the line is long. Two output sockets, so a dropped document goes to quarantine
+  instead of disappearing.
+* **`PromptInjectionGuard`** — `haystack_integrations.components.validators.aicordon`. Reads the
+  message list on its way into the generator with the `dpi` rules. The decision is for the whole
+  exchange: when it fires, `run` returns `blocked` and **no** `messages` key, which is how a
+  pipeline branches — the generator is not called at all. This side never edits a turn.
+
+Both are thin; the policy lives in `aicordon.guard`, which ships with the detector. Hence
+`aicordon>=1.1.0`: against 1.0.0 the install resolves and the import fails.
+
+Measured on the pipeline, not on the detector:
+
+| | |
+|---|---|
+| injected payload reaching the store, `mode="redact"` | 85.4% with the filter, 100% without |
+| clean documents damaged | 0 of 1000 |
+| held-out forum jailbreaks reaching the generator, `mode="drop"` | 65.2% with the guard, 100% without |
+| real WildChat turns left unanswered | 0.070% (14 of 20 000), an upper bound |
+| **verdicts differing from the bare detector** | **0** |
+| cost per turn of median length | 0.39 ms; 15 ms once, to load the base |
+
+The last row licenses the others: a wrapper that loses or adds text makes every number above it
+meaningless.

aicordon_haystack-0.1.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Mikhail Gribov
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

aicordon_haystack-0.1.0/NOTES.md

@@ -0,0 +1,100 @@
+# Haystack: the contract, established from the sources
+
+Checked 2026-08-13. The clone in `upstream/` at `ba92ec9` is **3.1.0-rc0**, i.e. the development
+branch; the last RELEASED version is **v3.0.0** (2026-07-20). The wrapper is written against 3.0.x,
+and the clone is what tells us whether the contract is drifting by 3.1.
+
+Repository: `deepset-ai/haystack`, Apache-2.0, 26 201 stars. The catalogue of third-party
+integrations is a separate repository, `deepset-ai/haystack-integrations` (a card via PR); what
+deepset maintains itself lives in `haystack-core-integrations`.
+
+## What a component is obliged to do
+
+Source of truth: the docstring of `haystack/core/component/component.py`, which says it outright.
+
+* The class is marked `@component`.
+* **`run()`** is mandatory; output types are declared with `@component.output_types(...)`, and a
+  `dict` with those same keys is returned.
+* **`__init__` must be cheap** — it is called when the pipeline is assembled and validated. Heavy
+  initialisation goes into the optional `warm_up()`, which the pipeline calls before a run.
+* **`__init__` parameters must be primitives** (strings, numbers, lists and dicts of them). Objects
+  and functions are forbidden: parameters must be JSON-serialisable or the pipeline cannot be saved
+  or loaded. If an object is needed, take its import path as a string and resolve it inside.
+* **Do not modify the input in place**: work on a copy and return that (`dataclasses.replace` for a
+  single field, `deepcopy` for anything richer). Otherwise the edit leaks into other branches of the
+  pipeline.
+
+## What follows for us
+
+1. The detector is heavy state (a 376 KB base, ~17 ms to load): raise it in `warm_up()`, not in
+   `__init__`. Otherwise the cost is paid on every pipeline assembly, validation included.
+2. Pass only strings into `__init__`: the mode (`redact` / `drop` / `annotate`), the metadata
+   prefix. No "hand the detector object in here" — that breaks saving the pipeline.
+3. Copy documents before editing their text: in an indexing pipeline the same list may go into a
+   second branch.
+4. Give the component two outputs: the clean stream and the rejected one. Connections are named, so
+   the rejected pile goes to its own store in one `connect` line instead of vanishing quietly.
+
+## Where we sit
+
+Two places, one per role the text plays in the prompt.
+
+**Material** — the indexing pipeline: `converter → (our component) → cleaner → splitter → embedder →
+writer`. BEFORE the splitter: what is cut then reaches neither the embeddings nor the store, and no
+chunk boundaries need stitching.
+
+**The request** — the chat pipeline: `prompt builder → (our component) → chat generator`, with a
+second wire `blocked → whoever answers instead of the model`. RIGHT NEXT to the generator, on the
+message list about to enter it: anything between the check and the call is one more place the text
+could change.
+
+## Traps the mock-up caught (2026-08-13)
+
+1. **Settings are lost silently when a pipeline is saved.** Without a `to_dict` of its own, Haystack
+   restores parameters through `getattr(obj, "<parameter name>")`, and where the attribute is
+   missing it **substitutes the default from the signature** and says nothing. A component built
+   with `mode="annotate"` came back out of YAML as `redact`. The cure: keep the parameters on the
+   object under those same names AND declare `to_dict`/`from_dict` via
+   `default_to_dict`/`default_from_dict`. Covered by a YAML round trip in the tests.
+2. **The order of chunks out of a store is not guaranteed**, and `split_overlap` repeats the tail of
+   the previous chunk. A measurement that glued chunks back together reported 92.5% where the answer
+   is 100% because of it. The acceptance measurement now has no splitter in it at all: it stands
+   AFTER us and cannot affect the result.
+3. **Logging is not a mode.** It is wanted under `drop` as much as under `annotate`, so it goes
+   through `haystack.logging` at `warning` level rather than being a value of the parameter.
+
+## What carries over to LlamaIndex or LangChain
+
+The policy (`aicordon.guard`) carries over as it is — modes, cut boundaries, metadata. A wrapper is
+the host's document type translated into a string and back, plus the host's contract: for LlamaIndex
+`TransformComponent.__call__(nodes)`, for LangChain `BaseDocumentTransformer.transform_documents`.
+Check their serialisation separately — Haystack is not the only framework that loses a mode without
+a word.
+
+## The request side: contract and traps (2026-08-18)
+
+The second component is `PromptInjectionGuard`, in
+`haystack_integrations.components.validators.aicordon`. Category `validators` rather than
+`preprocessors`: it prepares nothing, it decides whether to call the model.
+
+1. **A branch in Haystack is an ABSENT key in the returned dict, not an empty list.** The receiver
+   gets `_NoOutputProduced` (`core/pipeline/component_checks.py`) and does not run at all. Returning
+   `{"messages": [], "blocked": [...]}` means calling the generator with an empty list. So `run`
+   returns exactly one of the two keys. Covered by a test with a control: on a clean turn the next
+   component IS in the pipeline output, on a flagged one it is not.
+2. **`ChatMessage.text` is the FIRST text part, not the whole message.** In a message with an image
+   the parts come as a list, and an attack in the second text part is invisible to `text` — with no
+   error of any kind. We read `texts` and join them. Covered by a test.
+3. **The text of a tool result is NOT in `texts`.** For role `tool` the content sits in
+   `tool_call_result.result` and `texts` is empty. A wrapper reading only `texts` would check the
+   role against an empty string and write "read, clean", which is worse than a crash. We read `texts`
+   plus the call results; the calls themselves (`tool_calls`) we do not, they are the model's output
+   rather than what it was given. Covered by a test.
+4. **Edit a message only through a copy**: `ChatMessage` carries `@_warn_on_inplace_mutation`, and
+   the same list may go into a second branch of the pipeline. The copy is
+   `dataclasses.replace(msg, _meta=...)` (the underscored dataclass fields are their real names in
+   `__init__`).
+5. **The same serialisation trap as in the filter**, now with a dict parameter: `roles` has to sit
+   on the object under its own name and be listed in `to_dict`. Covered by a YAML round trip.
+6. **The decision is for the exchange, not for a message.** Dropping the flagged turn and calling
+   the model with the rest is not allowed: the model then answers the message before it.