liblore 0.1.0__tar.gz

liblore-0.1.0/LICENSES/GPL-2.0-or-later.txt

@@ -0,0 +1,266 @@
+Valid-License-Identifier: GPL-2.0-only
+Valid-License-Identifier: GPL-2.0-or-later
+SPDX-URL: https://spdx.org/licenses/GPL-2.0-only.html
+Usage-Guide:
+  To use this license in source code, put one of the following SPDX
+  tag/value pairs into a comment according to the placement
+  guidelines in the licensing rules documentation.
+  For 'GNU General Public License (GPL) version 2 only' use:
+    SPDX-License-Identifier: GPL-2.0-only
+  For 'GNU General Public License (GPL) version 2 or any later version' use:
+    SPDX-License-Identifier: GPL-2.0-or-later
+License-Text:
+
+		GNU GENERAL PUBLIC LICENSE
+		   Version 2, June 1991
+
+Copyright (C) 1989, 1991 Free Software Foundation, Inc.
+51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA
+
+Everyone is permitted to copy and distribute verbatim copies
+of this license document, but changing it is not allowed.
+
+   			  Preamble
+
+The licenses for most software are designed to take away your freedom to share
+and change it. By contrast, the GNU General Public License is intended to
+guarantee your freedom to share and change free software--to make sure the
+software is free for all its users. This General Public License applies to most
+of the Free Software Foundation's software and to any other program whose
+authors commit to using it. (Some other Free Software Foundation software is
+covered by the GNU Lesser General Public License instead.) You can apply it to
+your programs, too.
+
+When we speak of free software, we are referring to freedom, not price. Our
+General Public Licenses are designed to make sure that you have the freedom to
+distribute copies of free software (and charge for this service if you wish),
+that you receive source code or can get it if you want it, that you can change
+the software or use pieces of it in new free programs; and that you know you
+can do these things.
+
+To protect your rights, we need to make restrictions that forbid anyone to deny
+you these rights or to ask you to surrender the rights. These restrictions
+translate to certain responsibilities for you if you distribute copies of the
+software, or if you modify it.
+
+For example, if you distribute copies of such a program, whether gratis or for
+a fee, you must give the recipients all the rights that you have. You must make
+sure that they, too, receive or can get the source code. And you must show them
+these terms so they know their rights.
+We protect your rights with two steps: (1) copyright the software, and (2)
+offer you this license which gives you legal permission to copy, distribute
+and/or modify the software.
+
+Also, for each author's protection and ours, we want to make certain that
+everyone understands that there is no warranty for this free software. If the
+software is modified by someone else and passed on, we want its recipients to
+know that what they have is not the original, so that any problems introduced
+by others will not reflect on the original authors' reputations.
+
+Finally, any free program is threatened constantly by software patents. We wish
+to avoid the danger that redistributors of a free program will individually
+obtain patent licenses, in effect making the program proprietary. To prevent
+this, we have made it clear that any patent must be licensed for everyone's
+free use or not licensed at all.
+
+The precise terms and conditions for copying, distribution and modification
+follow.
+
+	TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+0. This License applies to any program or other work which contains a notice
+placed by the copyright holder saying it may be distributed under the terms of
+this General Public License. The "Program", below, refers to any such program
+or work, and a "work based on the Program" means either the Program or any
+derivative work under copyright law: that is to say, a work containing the
+Program or a portion of it, either verbatim or with modifications and/or
+translated into another language. (Hereinafter, translation is included without
+limitation in the term "modification".) Each licensee is addressed as "you".
+Activities other than copying, distribution and modification are not covered by
+this License; they are outside its scope. The act of running the Program is not
+restricted, and the output from the Program is covered only if its contents
+constitute a work based on the Program (independent of having been made by
+running the Program). Whether that is true depends on what the Program does.
+
+1. You may copy and distribute verbatim copies of the Program's source code as
+you receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice and
+disclaimer of warranty; keep intact all the notices that refer to this License
+and to the absence of any warranty; and give any other recipients of the
+Program a copy of this License along with the Program.
+You may charge a fee for the physical act of transferring a copy, and you may
+at your option offer warranty protection in exchange for a fee.
+
+2. You may modify your copy or copies of the Program or any portion of it, thus
+forming a work based on the Program, and copy and distribute such modifications
+or work under the terms of Section 1 above, provided that you also meet all of
+these conditions:
+
+      a) You must cause the modified files to carry prominent notices stating
+      that you changed the files and the date of any change.
+      
+      b) You must cause any work that you distribute or publish, that in whole
+      or in part contains or is derived from the Program or any part thereof,
+      to be licensed as a whole at no charge to all third parties under the
+      terms of this License.
+      
+      c) If the modified program normally reads commands interactively when
+      run, you must cause it, when started running for such interactive use in
+      the most ordinary way, to print or display an announcement including an
+      appropriate copyright notice and a notice that there is no warranty (or
+      else, saying that you provide a warranty) and that users may redistribute
+      the program under these conditions, and telling the user how to view a
+      copy of this License. (Exception: if the Program itself is interactive
+      but does not normally print such an announcement, your work based on the
+      Program is not required to print an announcement.)
+      
+These requirements apply to the modified work as a whole. If identifiable
+sections of that work are not derived from the Program, and can be reasonably
+considered independent and separate works in themselves, then this License, and
+its terms, do not apply to those sections when you distribute them as separate
+works. But when you distribute the same sections as part of a whole which is a
+work based on the Program, the distribution of the whole must be on the terms
+of this License, whose permissions for other licensees extend to the entire
+whole, and thus to each and every part regardless of who wrote it.
+Thus, it is not the intent of this section to claim rights or contest your
+rights to work written entirely by you; rather, the intent is to exercise the
+right to control the distribution of derivative or collective works based on
+the Program.
+
+In addition, mere aggregation of another work not based on the Program with the
+Program (or with a work based on the Program) on a volume of a storage or
+distribution medium does not bring the other work under the scope of this
+License.
+
+3. You may copy and distribute the Program (or a work based on it, under
+Section 2) in object code or executable form under the terms of Sections 1 and
+2 above provided that you also do one of the following:
+
+      a) Accompany it with the complete corresponding machine-readable source
+      code, which must be distributed under the terms of Sections 1 and 2 above
+      on a medium customarily used for software interchange; or,
+
+      b) Accompany it with a written offer, valid for at least three years, to
+      give any third party, for a charge no more than your cost of physically
+      performing source distribution, a complete machine-readable copy of the
+      corresponding source code, to be distributed under the terms of Sections
+      1 and 2 above on a medium customarily used for software interchange; or,
+      
+      c) Accompany it with the information you received as to the offer to
+      distribute corresponding source code. (This alternative is allowed only
+      for noncommercial distribution and only if you received the program in
+      object code or executable form with such an offer, in accord with
+      Subsection b above.)
+      
+The source code for a work means the preferred form of the work for making
+modifications to it. For an executable work, complete source code means all the
+source code for all modules it contains, plus any associated interface
+definition files, plus the scripts used to control compilation and installation
+of the executable. However, as a special exception, the source code distributed
+need not include anything that is normally distributed (in either source or
+binary form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component itself
+accompanies the executable.
+
+If distribution of executable or object code is made by offering access to copy
+from a designated place, then offering equivalent access to copy the source
+code from the same place counts as distribution of the source code, even though
+third parties are not compelled to copy the source along with the object code.
+
+4. You may not copy, modify, sublicense, or distribute the Program except as
+expressly provided under this License. Any attempt otherwise to copy, modify,
+sublicense or distribute the Program is void, and will automatically terminate
+your rights under this License. However, parties who have received copies, or
+rights, from you under this License will not have their licenses terminated so
+long as such parties remain in full compliance.
+
+5. You are not required to accept this License, since you have not signed it.
+However, nothing else grants you permission to modify or distribute the Program
+or its derivative works. These actions are prohibited by law if you do not
+accept this License. Therefore, by modifying or distributing the Program (or
+any work based on the Program), you indicate your acceptance of this License to
+do so, and all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+6. Each time you redistribute the Program (or any work based on the Program),
+the recipient automatically receives a license from the original licensor to
+copy, distribute or modify the Program subject to these terms and conditions.
+You may not impose any further restrictions on the recipients' exercise of the
+rights granted herein. You are not responsible for enforcing compliance by
+third parties to this License.
+
+7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues), conditions
+are imposed on you (whether by court order, agreement or otherwise) that
+contradict the conditions of this License, they do not excuse you from the
+conditions of this License. If you cannot distribute so as to satisfy
+simultaneously your obligations under this License and any other pertinent
+obligations, then as a consequence you may not distribute the Program at all.
+
+For example, if a patent license would not permit royalty-free redistribution
+of the Program by all those who receive copies directly or indirectly through
+you, then the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under any
+particular circumstance, the balance of the section is intended to apply and
+the section as a whole is intended to apply in other circumstances.
+It is not the purpose of this section to induce you to infringe any patents or
+other property right claims or to contest validity of any such claims; this
+section has the sole purpose of protecting the integrity of the free software
+distribution system, which is implemented by public license practices. Many
+people have made generous contributions to the wide range of software
+distributed through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing to
+distribute software through any other system and a licensee cannot impose that
+choice.
+
+This section is intended to make thoroughly clear what is believed to be a
+consequence of the rest of this License.
+
+8. If the distribution and/or use of the Program is restricted in certain
+countries either by patents or by copyrighted interfaces, the original
+copyright holder who places the Program under this License may add an explicit
+geographical distribution limitation excluding those countries, so that
+distribution is permitted only in or among countries not thus excluded. In such
+case, this License incorporates the limitation as if written in the body of
+this License.
+
+9. The Free Software Foundation may publish revised and/or new versions of the
+General Public License from time to time. Such new versions will be similar in
+spirit to the present version, but may differ in detail to address new problems
+or concerns.
+Each version is given a distinguishing version number. If the Program specifies
+a version number of this License which applies to it and "any later version",
+you have the option of following the terms and conditions either of that
+version or of any later version published by the Free Software Foundation. If
+the Program does not specify a version number of this License, you may choose
+any version ever published by the Free Software Foundation.
+10. If you wish to incorporate parts of the Program into other free programs
+whose distribution conditions are different, write to the author to ask for
+permission. For software which is copyrighted by the Free Software Foundation,
+write to the Free Software Foundation; we sometimes make exceptions for this.
+Our decision will be guided by the two goals of preserving the free status of
+all derivatives of our free software and of promoting the sharing and reuse of
+software generally.
+
+NO WARRANTY
+
+11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR
+THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE
+STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE
+PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED,
+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND
+PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU
+ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL
+ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE
+PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR
+INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA
+BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
+FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER
+OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+   	       END OF TERMS AND CONDITIONS

liblore-0.1.0/PKG-INFO

@@ -0,0 +1,378 @@
+Metadata-Version: 2.4
+Name: liblore
+Version: 0.1.0
+Summary: Shared library for public-inbox / lore.kernel.org access
+Author-email: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
+License-Expression: GPL-2.0-or-later
+Project-URL: Homepage, https://git.kernel.org/pub/scm/utils/liblore/liblore.git
+Project-URL: Repository, https://git.kernel.org/pub/scm/utils/liblore/liblore.git
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Communications :: Email
+Classifier: Topic :: Communications :: Email :: Mailing List Servers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSES/GPL-2.0-or-later.txt
+Requires-Dist: requests>=2.31
+Dynamic: license-file
+
+# liblore
+
+A Python library for working with [public-inbox](https://public-inbox.org/)
+servers, particularly [lore.kernel.org](https://lore.kernel.org/). It fetches
+email threads, parses mbox files, and provides utilities for working with
+email messages from mailing list archives.
+
+## Requirements
+
+- Python 3.11 or newer
+- `requests` >= 2.31
+
+## Installation
+
+Install from PyPI:
+
+```shell
+pip install liblore
+```
+
+Or install from source:
+
+```shell
+pip install .
+```
+
+## Quick Start
+
+The main entry point is the `LoreNode` class. It connects to a public-inbox
+endpoint and lets you fetch threads, search for messages, and work with raw
+mbox data. Use it as a context manager so the underlying HTTP session is
+cleaned up automatically:
+
+```python
+from liblore import LoreNode
+
+with LoreNode("https://lore.kernel.org/all") as node:
+    msgs = node.get_thread_by_msgid(
+        "20250101-example@kernel.org",
+        sort=True,
+    )
+    for msg in msgs:
+        print(msg["Subject"])
+```
+
+If you omit the URL, it defaults to `https://lore.kernel.org/all`.
+
+## API Reference
+
+### LoreNode
+
+```python
+from liblore import LoreNode
+
+node = LoreNode(url="https://lore.kernel.org/all")
+```
+
+#### Fetching Threads
+
+**`node.get_thread_by_msgid(msgid, *, strict=True, sort=False, since=None)`**
+
+Fetch a thread by its message ID. This is the highest-level method and the
+one you will reach for most often.
+
+- `strict` (default `True`) -- filter results to only messages that belong
+  to the thread rooted at `msgid`. When a query returns messages from
+  unrelated threads (common with broad date ranges), strict mode discards
+  them.
+- `sort` -- sort the returned messages by their `Received` header timestamp.
+- `since` -- a date string appended as a `d:` filter. This uses
+  public-inbox's approxidate syntax, so you can write things like
+  `"20240115"`, `"2.weeks.ago"`, or `"last.month"`.
+
+Returns a `list[EmailMessage]`. Raises `LookupError` if no messages match.
+
+```python
+with LoreNode() as node:
+    # Fetch a thread, sorted by date, only looking at recent messages
+    msgs = node.get_thread_by_msgid(
+        "20250101-example@kernel.org",
+        strict=True,
+        sort=True,
+        since="20250101",
+    )
+```
+
+**`node.get_thread_by_query(query)`**
+
+Run a search query and return a deduplicated `list[EmailMessage]`. The query
+uses public-inbox's
+[Xapian search syntax](https://public-inbox.org/HOWTO#search), which supports
+prefixes like `msgid:`, `s:` (subject), `f:` (from), `d:` (date range), and
+more.
+
+```python
+with LoreNode() as node:
+    # Find all messages from a sender in the last month
+    msgs = node.get_thread_by_query("f:alice@example.com d:last.month..")
+```
+
+#### Batch Fetching
+
+When you need to fetch multiple threads, the batch methods handle the loop for
+you and add a 100 ms cooldown between requests so you're being a good citizen
+to the server.
+
+**`node.batch_get_thread_by_msgid(msgids, *, strict=True, sort=False, since=None)`**
+
+Fetch threads for a list of message IDs. Calls `get_thread_by_msgid()` for
+each one with a brief pause between requests. Returns a
+`list[list[EmailMessage]]` in the same order as the input.
+
+```python
+with LoreNode() as node:
+    threads = node.batch_get_thread_by_msgid(
+        ["msg1@example.com", "msg2@example.com", "msg3@example.com"],
+        sort=True,
+        since="2.weeks.ago",
+    )
+    for thread in threads:
+        print(f"Thread with {len(thread)} messages")
+```
+
+**`node.batch_get_thread_by_query(queries)`**
+
+Run multiple search queries. Same pattern -- calls `get_thread_by_query()` per
+query with a 100 ms cooldown. Returns a `list[list[EmailMessage]]`.
+
+```python
+with LoreNode() as node:
+    results = node.batch_get_thread_by_query([
+        "s:fix f:alice@example.com",
+        "s:feature f:bob@example.com",
+    ])
+```
+
+#### Raw Mbox Access
+
+These methods return raw mbox bytes rather than parsed messages. They are
+useful when you need the unprocessed data, or when you want to feed the
+output into your own parser.
+
+**`node.get_mbox_by_msgid(msgid)`** -- fetch a thread's mbox by message ID.
+
+**`node.get_mbox_by_query(query)`** -- run a search query and return the
+matching mbox.
+
+```python
+with LoreNode() as node:
+    raw = node.get_mbox_by_msgid("20250101-example@kernel.org")
+    with open("thread.mbox", "wb") as f:
+        f.write(raw)
+```
+
+#### Single Messages
+
+**`node.get_message_by_msgid(msgid)`** -- fetch a single raw message (bytes)
+by its message ID. Useful when you need exactly one message rather than an
+entire thread.
+
+#### Session Configuration
+
+**`node.set_user_agent(app_name, version, plus=None)`** -- set a custom
+`User-Agent` header. Being a good citizen of public infrastructure means
+identifying your tool:
+
+```python
+node.set_user_agent("my-tool", "1.0")
+# User-Agent: my-tool/1.0
+```
+
+**`node.set_requests_session(session)`** -- inject your own
+`requests.Session`. Handy when you need custom timeouts, proxies, or
+authentication. Note that the session's `User-Agent` is not overwritten
+when you provide your own.
+
+**`node.validate()`** -- check that the configured URL actually points to a
+public-inbox server. Raises `RemoteError` if it does not.
+
+**`node.close()`** -- close the HTTP session. Called automatically when
+using `LoreNode` as a context manager.
+
+### How the API Layers Fit Together
+
+The methods build on each other in layers, from raw bytes up to filtered,
+sorted thread views:
+
+```
+get_mbox_by_msgid / get_mbox_by_query      ->  raw mbox bytes
+        |
+get_thread_by_query                        ->  split + dedupe -> list[EmailMessage]
+        |
+get_thread_by_msgid                        ->  strict + sort  -> list[EmailMessage]
+        |
+batch_get_thread_by_msgid / batch_get_...  ->  rate-limited loop -> list[list[EmailMessage]]
+```
+
+You can tap into whichever layer suits your needs. Need raw bytes for
+archiving? Use the `get_mbox_*` methods. Need parsed messages with
+deduplication? Use `get_thread_by_query`. Want the full convenience of
+strict filtering and date sorting? Use `get_thread_by_msgid`.
+
+### Utility Functions
+
+The `liblore.utils` module provides lower-level helpers for parsing and
+inspecting email messages.
+
+#### Header Handling
+
+```python
+from liblore.utils import clean_header, get_clean_msgid
+
+# Decode RFC 2047 encoded headers
+decoded = clean_header("=?utf-8?q?Re=3A_Some_Subject?=")
+
+# Extract a clean message ID (without angle brackets) from a message
+msgid = get_clean_msgid(msg)               # reads Message-Id by default
+msgid = get_clean_msgid(msg, "In-Reply-To")  # or any other header
+```
+
+#### Parsing Messages
+
+```python
+from liblore.utils import parse_message
+
+# Parse raw email bytes into an EmailMessage
+msg = parse_message(raw_bytes)
+```
+
+#### Extracting Message Content
+
+```python
+from liblore.utils import (
+    msg_get_subject,
+    msg_get_author,
+    msg_get_payload,
+    msg_get_recipients,
+)
+
+# Get the decoded subject line
+subject = msg_get_subject(msg)
+
+# Strip [PATCH v3 2/5] and Re: prefixes to get the bare subject
+bare = msg_get_subject(msg, strip_prefixes=True)
+
+# Get the author as a (name, email) tuple
+name, addr = msg_get_author(msg)
+
+# Get the plain-text body, stripping the signature
+body = msg_get_payload(msg)
+
+# Get the body without quoted lines or signature
+body = msg_get_payload(msg, strip_quoted=True, strip_signature=True)
+
+# Get all recipient email addresses (To + Cc + From)
+recipients = msg_get_recipients(msg)
+```
+
+#### Sorting and Threading
+
+```python
+from liblore.utils import sort_msgs_by_received, get_strict_thread
+
+# Sort messages by their Received timestamp (falls back to Date)
+sorted_msgs = sort_msgs_by_received(msgs)
+
+# Filter a list of messages to only those in a specific thread
+thread = get_strict_thread(msgs, "20250101-example@kernel.org")
+
+# Break the thread at msgid, ignoring its parent references
+thread = get_strict_thread(msgs, msgid, noparent=True)
+```
+
+#### Mbox Splitting
+
+```python
+from liblore.utils import split_mbox, split_and_dedupe
+
+# Split mboxrd bytes into a list of EmailMessage objects
+msgs = split_mbox(mbox_bytes)
+
+# Split and deduplicate by Message-ID (first occurrence wins)
+msgs = split_and_dedupe(mbox_bytes)
+```
+
+#### URL Helpers
+
+```python
+from liblore.utils import get_msgid_from_url
+
+# Extract a message ID from a lore URL
+msgid = get_msgid_from_url("https://lore.kernel.org/all/20250101-example@kernel.org/")
+# -> "20250101-example@kernel.org"
+
+# Also works with bare message IDs
+msgid = get_msgid_from_url("<20250101-example@kernel.org>")
+# -> "20250101-example@kernel.org"
+```
+
+### Exceptions
+
+All exceptions inherit from `LibloreError`, so you can catch them broadly or
+handle specific cases:
+
+```python
+from liblore import LibloreError, RemoteError, PublicInboxError
+
+try:
+    msgs = node.get_thread_by_msgid("nonexistent@example.com")
+except RemoteError:
+    # HTTP request failed (server error, network issue, etc.)
+    ...
+except PublicInboxError:
+    # Something went wrong with the public-inbox operation
+    ...
+except LibloreError:
+    # Catch-all for any liblore error
+    ...
+```
+
+## Development
+
+Install with development dependencies:
+
+```shell
+pip install -e ".[dev]"
+```
+
+Run the test suite:
+
+```shell
+pytest
+```
+
+Type checking:
+
+```shell
+mypy src/liblore/ --strict
+```
+
+Linting:
+
+```shell
+ruff check src/liblore/
+```
+
+## Bug Reports
+
+Send bug reports and patches to [tools@kernel.org](mailto:tools@kernel.org).
+
+## Licence
+
+GPL-2.0-or-later. See [LICENSES/GPL-2.0-or-later.txt](LICENSES/GPL-2.0-or-later.txt)
+for the full text.
+
+Copyright The Linux Foundation.