reader 3.15__tar.gz → 3.17__tar.gz

{reader-3.15 → reader-3.17}/CHANGES.rst

@@ -6,6 +6,90 @@ Changelog
   :no-index:
 
 
+Version 3.17
+------------
+
+Released 2025-03-08
+
+.. attention::
+
+    This is the last release to support Python 3.10;
+    see :issue:`354` for details.
+
+* Support PyPy 3.11. (:issue:`354`)
+
+* Start working on a web app :ref:`re-design <app roadmap>`
+  based on `htmx`_ and `Bootstrap`_,
+  beginning with a new version of the main entries page,
+  and dark mode support;
+  some :ref:`screenshots <app screenshots>`.
+  (:issue:`318`)
+
+* :attr:`Entry.feed_resolved_title`:
+  return both the source and feed titles only if they are different.
+
+* The :mod:`~reader._plugins.cli_status` plugin
+  now creates only one entry per command,
+  with the newest run first.
+
+.. _htmx: https://htmx.org/
+.. _Bootstrap: https://getbootstrap.com/
+
+
+Version 3.16
+------------
+
+Released 2024-12-08
+
+* Parse and store :attr:`Entry.source` for Atom and RSS feeds. (:issue:`267`)
+
+  * Add :attr:`Entry.feed_resolved_title` and :attr:`Feed.resolved_title` properties.
+  * The ``feed`` search column now indexes :attr:`Entry.feed_resolved_title`,
+    instead of feed :attr:`~Feed.user_title` or :attr:`~Feed.title`.
+  * Allow filtering entries by the entry source.
+
+* Add :func:`reader.utils.archive_entries` for
+  copying entries to an "archived" feed.
+  (:issue:`290`)
+
+  * Add :meth:`~Reader.copy_entry`.
+  * Allow archiving entries from the web app.
+
+* Fix bug causing :class:`Reader` operations
+  from a thread other than the one that created the instance
+  to happen with foreign key constraint enforcement disabled
+  (e.g. deleting a feed from another thread would not delete its entries).
+
+  This bug exists since using :class:`Reader` instances from other threads
+  became allowed in `2.15 <Version 2.15_>`_.
+
+  Serving the web application with ``python -m reader serve``
+  is known to be affected.
+  Serving it with uWSGI without threads (the default)
+  should not be affected.
+
+  .. attention::
+
+    **Your database may be in an inconsistent state because of this bug.**
+
+    It is recommended you run `PRAGMA foreign_key_check`_ on your database.
+
+    If you are upgrading from a version prior to 3.16
+    (i.e. were not using a pre-release version of *reader*),
+    the migration will do so for you.
+    If there are inconsistencies, you will get this error::
+
+      StorageError: integrity error: after migrating to version 43:
+        integrity error: FOREIGN KEY constraint failed
+
+* Fix :meth:`~Reader.enable_search` / :meth:`~Reader.update_search`
+  not working when the search database is missing but change tracking is enabled
+  (e.g. when restoring the main database from backup).
+  (:issue:`362`)
+
+.. _PRAGMA foreign_key_check: https://www.sqlite.org/pragma.html#pragma_foreign_key_check
+
+
 Version 3.15
 ------------
 

{reader-3.15 → reader-3.17}/MANIFEST.in

@@ -1,5 +1,6 @@
 recursive-include src/reader/_app/templates *
 recursive-include src/reader/_app/static *
+recursive-include src/reader/_app/v2 *
 recursive-include src/reader/_plugins/templates *
 recursive-include src/reader *.pyi
 recursive-include src/reader py.typed

{reader-3.15/src/reader.egg-info → reader-3.17}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.2
 Name: reader
-Version: 3.15
+Version: 3.17
 Summary: A Python feed reader library.
 Author: lemon24
 License: BSD-3-Clause
@@ -50,6 +50,8 @@ Provides-Extra: app
 Requires-Dist: flask>=0.10; extra == "app"
 Requires-Dist: humanize!=4.7.*,>=4; extra == "app"
 Requires-Dist: PyYAML; extra == "app"
+Requires-Dist: WTForms; extra == "app"
+Requires-Dist: jinja2-fragments; extra == "app"
 Provides-Extra: unstable-plugins
 Requires-Dist: requests; extra == "unstable-plugins"
 Requires-Dist: mutagen; extra == "unstable-plugins"

{reader-3.15 → reader-3.17}/docs/api.rst

@@ -40,6 +40,9 @@ Data objects
 .. autoclass:: Enclosure
     :members:
 
+.. autoclass:: EntrySource
+    :members:
+
 .. autoclass:: EntrySearchResult
     :members:
 
@@ -187,3 +190,10 @@ Constants
 .. autodata:: reader.core.DEFAULT_RESERVED_NAME_SCHEME
 
 .. autodata:: reader.plugins.DEFAULT_PLUGINS
+
+
+
+Utilities
+---------
+
+.. autofunction:: reader.utils.archive_entries

{reader-3.15 → reader-3.17}/docs/app.rst

@@ -67,9 +67,34 @@ If running on a personal computer, you can use cron to run ``serve`` at boot::
 .. _Werkzeug's development server: http://werkzeug.pocoo.org/docs/0.14/serving/#werkzeug.serving.run_simple
 
 
+.. _app screenshots:
+
 Screenshots
 -----------
 
+Following are screenshots of the web app re-design
+mentioned in the :ref:`roadmap <app roadmap>`.
+For the legacy web app, see `Screenshots (legacy)`_.
+
+Main page
+~~~~~~~~~
+
+.. figure:: screenshots/entries-v2-dark.png
+    :width: 240px
+
+    main page (dark mode)
+
+.. figure:: screenshots/entries-v2-filters-light.png
+    :width: 240px
+
+    main page – more filters (light mode)
+
+
+Screenshots (legacy)
+--------------------
+
+Following are screenshots of the original (legacy) web app.
+
 Main page
 ~~~~~~~~~
 

{reader-3.15 → reader-3.17}/docs/dev.rst

@@ -34,6 +34,9 @@ but I will prioritize supporting :doc:`contributors <contributing>`
 
 * OPML support, :issue:`165`
 * :ref:`deleting entries <deleting entries>`
+
+  * archiving important entries of deleted feeds, :issue:`290`
+
 * :ref:`feed interaction statistics <counts api>`
 
 * security
@@ -69,7 +72,6 @@ but I will prioritize supporting :doc:`contributors <contributing>`
 
 * add more fields to data objects
 
-  * entry source, :issue:`276`
   * extra data, as an escape hatch, :issue:`277`
 
 * :ref:`multiple storage implementations <multiple storage implementations>`
@@ -120,8 +122,14 @@ Long term, I'd like to:
 * switch to `htmx`_ instead of using a home-grown solution
 * spin it off into a separate package/project
 
-.. _htmx: https://htmx.org/
+**2025 update**:
+I've started working on a re-design based on `htmx`_ and `Bootstrap`_;
+some :ref:`screenshots <app screenshots>`.
+The new app will be available in parallel with the old one
+until it reaches feature parity.
 
+.. _htmx: https://htmx.org/
+.. _Bootstrap: https://getbootstrap.com/
 
 
 .. _compat:
@@ -360,6 +368,8 @@ Speeding up ``get_entry_counts(feed=...)``:
 * having an index on entries(feed) yielded a 4x improvement: :issue:`251`
 * even better, we should cache commonly-used counts: :issue:`306#issuecomment-1694655504`
 
+Official guidance: https://www.sqlite.org/queryplanner-ng.html#howtofix
+
 
 Parser
 ~~~~~~
@@ -512,6 +522,16 @@ From the initial issue:
 * use cases: :issue:`149#issuecomment-700066794`
 * initial requirements: :issue:`149#issuecomment-700532183`
 
+Splitting a feed into two feeds is discussed in :issue:`221`.
+The main takeaway at the time (2021) was that
+not all entry data can be retrieved through the storage API,
+so it's not fully possible to round-trip an entry.
+Based on a quick 2024 survey,
+the only affected attributes
+are :attr:`~.EntryUpdateIntent.feed_order` (can't get; not critical)
+and :attr:`~Entry.original_feed_url` (can't set; will be required for
+archiving important entries of deleted feeds, :issue:`290`).
+
 
 Resource tags / metadata
 ~~~~~~~~~~~~~~~~~~~~~~~~
@@ -651,6 +671,8 @@ Why I want to postpone batch update/set methods:
 tl:dr: Performance is likely a non-issue with SQLite,
 convenience can be added on top as a plugin.
 
+(2025) Why it may be worth adding batch interfaces anyway (even if underneath the storage implementation doesn't actually batch) – it allows for future optimization: https://blog.glyph.im/2022/12/potato-programming.html
+
 See the 2.12 reader._app.ResourceTags class for an idea of how to
 represent a bunch of tags in a reserved-name-scheme-agnostic way
 (useful e.g. for when get_entries() should return tags x, y, z of each entry).
@@ -708,16 +730,16 @@ Wrapping underlying storage exceptions
 
 Which exception to wrap, and which not: :issue:`21#issuecomment-365442439`.
 
-In version 3.10 (November 2023), all internal APIs were changed
-to use timezone-aware datetimes, with the timezone set to UTC,
-in preparation for support for any timezone.
-
 
 Timezone handling
 ~~~~~~~~~~~~~~~~~
 
 Aware vs. naive, and what's needed to go fully aware: :issue:`233#issuecomment-881618002`.
 
+In version 3.10 (November 2023), all internal APIs were changed
+to use timezone-aware datetimes, with the timezone set to UTC,
+in preparation for support for any timezone.
+
 
 OPML support
 ~~~~~~~~~~~~

{reader-3.15 → reader-3.17}/pyproject.toml

@@ -77,6 +77,9 @@ app = [
     "humanize>=4,!=4.7.*",
     # for config
     "PyYAML",
+    # for v2
+    "WTForms",
+    "jinja2-fragments",
 ]
 
 # UNSTABLE PLUGINS

{reader-3.15 → reader-3.17}/src/reader/__init__.py

@@ -37,7 +37,7 @@ Print the titles of the unread entries::
 
 """
 
-__version__ = '3.15'
+__version__ = '3.17'
 
 # isort: off
 
@@ -52,6 +52,7 @@ from .types import (
     Entry as Entry,
     Content as Content,
     Enclosure as Enclosure,
+    EntrySource as EntrySource,
     EntrySearchResult as EntrySearchResult,
     HighlightedString as HighlightedString,
     FeedCounts as FeedCounts,

{reader-3.15 → reader-3.17}/src/reader/_app/__init__.py

@@ -38,6 +38,7 @@ from reader import ReaderError
 from reader._plugins import Loader
 from reader.types import _get_entry_content
 from reader.types import TristateFilterInput
+from reader.utils import archive_entries
 
 from .api_thing import APIError
 from .api_thing import APIThing
@@ -104,6 +105,7 @@ def get_reader():
 def stream_template(template_name_or_list, **kwargs):
     template = current_app.jinja_env.get_template(template_name_or_list)
     stream = template.stream(**kwargs)
+    # TODO: increase to at least 1-2k, like this we have 50% overhead
     stream.enable_buffering(50)
     return Response(stream_with_context(stream))
 
@@ -157,12 +159,8 @@ class EntryProxy:
     def title(self):
         highlight = self._search_result.metadata.get('.title')
         if highlight:
-            return str(highlight)
-        return None
-
-    @property
-    def feed(self):
-        return FeedProxy(self._search_result, self._entry)
+            return highlighted(highlight)
+        return self._entry.title
 
     @property
     def summary(self):
@@ -184,21 +182,12 @@ class EntryProxy:
     def get_content(self, prefer_summary=False):
         return _get_entry_content(self, prefer_summary)
 
-
-@dataclass(frozen=True)
-class FeedProxy:
-    _search_result: EntrySearchResult
-    _entry: Entry
-
-    def __getattr__(self, name):
-        return getattr(self._entry.feed, name)
-
     @property
-    def title(self):
-        highlight = self._search_result.metadata.get('.feed.title')
+    def feed_resolved_title(self):
+        highlight = self._search_result.metadata.get('.feed_resolved_title')
         if highlight:
-            return str(highlight)
-        return self._entry.feed.title
+            return highlighted(highlight)
+        return self._entry.feed_resolved_url
 
 
 @dataclass
@@ -643,6 +632,14 @@ def mark_all_as_unread(data):
         get_reader().mark_entry_as_unread((feed_url, entry_id))
 
 
+@form_api(really=True)
+@readererror_to_apierror()
+def archive_all(data):
+    feed_url = data['feed-url']
+    entry_ids = json.loads(data['entry-id'])
+    archive_entries(get_reader(), [(feed_url, eid) for eid in entry_ids])
+
+
 @form_api
 @readererror_to_apierror()
 def mark_as_important(data):
@@ -840,6 +837,10 @@ def create_app(config):
 
     app.register_blueprint(blueprint)
 
+    from . import v2
+
+    app.register_blueprint(v2.blueprint, url_prefix='/v2')
+
     # NOTE: this is part of the app extension API
     app.reader_additional_enclosure_links = []
     app.reader_additional_links = []

{reader-3.15 → reader-3.17}/src/reader/_app/templates/entries.html

@@ -172,6 +172,9 @@
             {{- macros.confirm_button('.form_api', 'mark-all-as-unread', 'all unread', leave_disabled=true, next=next, context={'feed-url': feed.url, 'entry-id': entries_data | tojson}) -}}
         {% endif %}
 
+        {{- macros.confirm_button('.form_api', 'archive-all', 'archive all', leave_disabled=true, next=url_for('.entries', feed='reader:archived'), context={'feed-url': feed.url, 'entry-id': entries_data | tojson}) -}}
+        {# TODO: should redirect to archived # entry #}
+
         {{ macros.confirm_button('.form_api', 'delete-feed', 'delete feed', leave_disabled=true, next=url_for('.entries'), context={'feed-url': feed.url}) }}
 
         {{ macros.text_input_button('.form_api', 'update-feed-title', 'update feed title', 'feed-title', 'feed title', leave_disabled=true, next=next, context={'feed-url': feed.url}, value=feed.user_title) }}
@@ -265,7 +268,7 @@
 <ul class="controls">
 
 <li>
-    <a href="{{ url_for('.entries', feed=entry.feed.url) }}">{{ macros.feed_title(entry.feed) }}</a>
+    <a href="{{ url_for('.entries', feed=entry.feed.url) }}">{{ entry.feed_resolved_title or feed.url }}</a>
 <li>
     {%- set published = entry.published or entry.updated_not_none -%}
     <span title="{{ published }}">{{ published | humanize_naturaltime }}</span>

{reader-3.15 → reader-3.17}/src/reader/_app/templates/entry.html

@@ -19,7 +19,7 @@
 
 <li>
     {% if entry.author %} by {{ entry.author }}{% endif %}
-    in <a href="{{ url_for('.entries', feed=entry.feed.url) }}">{{ macros.feed_title(entry.feed) }}</a>
+    in <a href="{{ url_for('.entries', feed=entry.feed.url) }}">{{ entry.feed_resolved_title or feed.url }}</a>
 <li>
     {%- set published = entry.published or entry.updated_not_none -%}
     <span title="{{ published }}">{{ published | humanize_naturaltime }}</span>

{reader-3.15 → reader-3.17}/src/reader/_app/templates/layout.html

@@ -21,6 +21,7 @@ window.onload = function () {
 <a href="{{ url_for('reader.feeds') }}">feeds</a>
 <a href="{{ url_for('reader.tags') }}">tags</a>
 <a href="{{ url_for('reader.metadata') }}">metadata</a>
+<a href="{{ url_for('v2.entries') }}">v2</a>
 
 {{ macros.text_input_button_get(
     'reader.preview', 'add feed', 'url', 'url',

{reader-3.15 → reader-3.17}/src/reader/_app/templates/macros.html

@@ -6,17 +6,11 @@
 {%- endmacro %}
 
 {% macro feed_title(feed) %}
-{%- if feed.user_title and feed.user_title.strip() -%}
-    {{ feed.user_title }}
-{%- elif feed.title and feed.title.strip() -%}
-    {{ feed.title }}
-{%- else -%}
-    {{ feed.url }}
-{%- endif -%}
+{{- feed.resolved_title or feed.url -}}
 {% endmacro %}
 
 {% macro feed_title_secondary(feed) %}
-{%- if feed.user_title and feed.user_title.strip() and feed.title and feed.title.strip() -%}
+{%- if feed.title and feed.resolved_title != feed.title -%}
     {{ feed.title }}
 {%- endif -%}
 {% endmacro %}

reader-3.17/src/reader/_app/v2/__init__.py

@@ -0,0 +1,123 @@
+import itertools
+from functools import partial
+
+from flask import abort
+from flask import Blueprint
+from flask import current_app
+from flask import redirect
+from flask import request
+from flask import url_for
+from jinja2_fragments.flask import render_block
+
+from reader import InvalidSearchQueryError
+
+from .. import EntryProxy
+from .. import get_reader
+from .. import stream_template
+from .forms import EntryFilter
+from .forms import SearchEntryFilter
+
+
+blueprint = Blueprint(
+    'v2', __name__, template_folder='templates', static_folder='static'
+)
+
+
+@blueprint.route('/')
+def entries():
+    reader = get_reader()
+
+    # TODO: search improvements
+    # TODO: paqgination
+    # TODO: read time
+
+    if request.args.get('q', '').strip():
+        form = SearchEntryFilter(request.args)
+    else:
+        form = EntryFilter(request.args)
+
+    form_args = form.args
+    if q := form_args.pop('Q', ''):
+        form_args['q'] = q
+        return redirect(url_for('.entries', **form_args))
+    if form_args != request.args.to_dict():
+        return redirect(url_for('.entries', **form_args))
+
+    feed = None
+    if form.feed.data:
+        feed = reader.get_feed(form.feed.data, None)
+        if not feed:
+            abort(404)
+
+    kwargs = dict(form.data)
+    if query := kwargs.pop('search', None):
+
+        def get_entries(**kwargs):
+            for sr in reader.search_entries(query, **kwargs):
+                yield EntryProxy(sr, reader.get_entry(sr))
+
+    else:
+        get_entries = reader.get_entries
+
+    entries = []
+    if form.validate():
+        try:
+            entries = eager_iterator(get_entries(**kwargs, limit=64))
+        except StopIteration:
+            pass
+        except InvalidSearchQueryError as e:
+            form.search.errors.append(f"invalid query: {e}")
+
+    return stream_template(
+        'v2/entries.html',
+        form=form,
+        entries=entries,
+        feed=feed,
+    )
+
+
+def eager_iterator(it):
+    it = iter(it)
+    try:
+        return itertools.chain([next(it)], it)
+    except StopIteration:
+        return it
+
+
+@blueprint.route('/mark-as', methods=['POST'])
+def mark_as():
+    reader = get_reader()
+
+    entry = request.form['feed-url'], request.form['entry-id']
+
+    if 'read' in request.form:
+        match request.form['read']:
+            case 'true':
+                reader.set_entry_read(entry, True)
+            case 'false':
+                reader.set_entry_read(entry, False)
+            case _:
+                abort(422)
+
+    if 'important' in request.form:
+        match request.form['important']:
+            case 'true':
+                reader.set_entry_important(entry, True)
+            case 'false':
+                reader.set_entry_important(entry, False)
+            case 'none':
+                reader.set_entry_important(entry, None)
+            case _:
+                abort(422)
+
+    if request.headers.get('hx-request') == 'true':
+        return render_block(
+            'v2/entries.html',
+            'entry_form',
+            entry=reader.get_entry(entry),
+            next=request.form['next'],
+            # equivalent to {% import "v2/macros.html" as macros %}
+            macros=current_app.jinja_env.get_template('v2/macros.html').module,
+        )
+
+    return redirect(request.form['next'], code=303)