conflens 0.1.14__py3-none-any.whl

conflens-0.1.14.dist-info/METADATA

@@ -0,0 +1,349 @@
+Metadata-Version: 2.5
+Name: conflens
+Version: 0.1.14
+Summary: Browse, theme-classify, and topic-model conference papers with a NiceGUI UI.
+Project-URL: homepage, https://github.com/picaultj/conflens
+Project-URL: repository, https://github.com/picaultj/conflens
+Author-email: Jérôme Picault <jerome.picault@rte-france.com>
+Maintainer-email: Jérôme Picault <jerome.picault@rte-france.com>
+License: MPL-2.0
+License-File: LICENSE
+Keywords: LLM,conferences,report,topic analysis,topics
+Classifier: Development Status :: 4 - Beta
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.13
+Requires-Dist: anthropic>=0.40
+Requires-Dist: litellm>=1.40
+Requires-Dist: nicegui>=2.0
+Requires-Dist: openai>=1.40
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: python-pptx>=0.6.21
+Provides-Extra: bertopic
+Requires-Dist: bertopic>=0.16; extra == 'bertopic'
+Description-Content-Type: text/markdown
+
+# ConfLens: a Conference Paper Analyzer
+
+[![Lint](https://img.shields.io/github/actions/workflow/status/picaultj/conflens/lint.yml?branch=main&logo=github&label=lint)](https://github.com/picaultj/conflens/actions/workflows/lint.yml)
+[![Tests](https://img.shields.io/github/actions/workflow/status/picaultj/conflens/test.yml?branch=main&logo=github&label=tests)](https://github.com/picaultj/conflens/actions/workflows/test.yml)
+[![PyPI version](https://img.shields.io/pypi/v/conflens?logo=pypi&logoColor=white&label=pypi)](https://pypi.org/project/conflens/)
+[![Downloads](https://pepy.tech/badge/conflens)](https://pepy.tech/project/conflens)
+[![Python versions](https://img.shields.io/pypi/pyversions/conflens?logo=python&logoColor=white)](https://pypi.org/project/conflens/)
+[![Stars](https://img.shields.io/github/stars/picaultj/conflens?logo=github&style=flat)](https://github.com/picaultj/conflens/stargazers)
+[![Last commit](https://img.shields.io/github/last-commit/picaultj/conflens?logo=github)](https://github.com/picaultj/conflens/commits/main)
+[![Issues](https://img.shields.io/github/issues/picaultj/conflens?logo=github)](https://github.com/picaultj/conflens/issues)
+[![License: MPL 2.0](https://img.shields.io/badge/license-MPL%202.0-brightgreen.svg)](LICENSE)
+[![uv](https://img.shields.io/badge/managed%20by-uv-DE5FE9.svg?logo=uv&logoColor=white)](https://docs.astral.sh/uv/)
+[![Built with NiceGUI](https://img.shields.io/badge/UI-NiceGUI-2b6cb0.svg)](https://nicegui.io)
+
+A desktop-style web app (built with [NiceGUI](https://nicegui.io)) that:
+
+1. **Browses** papers from a chosen **source** and retrieves their abstracts:
+   - the [ACL Anthology](https://aclanthology.org) (default; e.g. `acl-2026`),
+   - **EMNLP** and **NAACL** proceedings (also on the ACL Anthology; e.g.
+     `emnlp-2024`, `naacl-2024`),
+   - **IJCAI** accepted-paper pages (e.g. <https://2026.ijcai.org/accepted-papers/>),
+   - **OpenReview** venues — **ICLR / NeurIPS** and more — via the public JSON API,
+     by venue id (e.g. `ICLR.cc/2024/Conference`, `NeurIPS.cc/2024/Conference`), and
+   - **PSCC** (Power Systems Computation Conference) proceedings, by year
+     (e.g. `2024`) — titles + PDFs (abstracts aren't published, so classification
+     is title-based), and
+   - **ISGT Europe** (IEEE PES) via the open **DBLP** index, by venue + year
+     (e.g. `isgteurope 2024`) — titles + DOI links, title-based (the generic
+     DBLP adapter works for any DBLP-indexed conference).
+
+   The scraper is pluggable — adding another conference is one adapter in
+   `conference_analyzer/sources.py`.
+2. **Classifies** each paper with an LLM against a customizable
+   **theme** (default *Agentic AI*), keeping only those whose core contribution
+   matches.
+3. **Discovers topics** within the selected papers (LLM-based by default, with an
+   optional BERTopic backend) and shows, for each topic, how many papers it
+   contains and a direct **PDF link** to the full text of every paper.
+
+
+![overview](docs/overview.png)
+
+## Contents
+
+- [Quick start](#quick-start) · [Run with Docker](#run-with-docker) · [LLM providers](#llm-providers)
+- [Architecture](#architecture) · [How it works](#how-it-works)
+- [Configuration](#configuration-in-the-ui) · [Features](#features)
+- [Caching](#caching) · [Cost](#cost) · [BERTopic](#optional-bertopic)
+- [Development](#development)
+
+## Quick start
+
+Install from PyPI (Python 3.13+) and run:
+
+```bash
+pip install conflens
+conflens                                   # → http://localhost:6868
+```
+
+Or from a clone, with [uv](https://docs.astral.sh/uv/):
+
+```bash
+uv sync                                   # Claude, OpenAI, LiteLLM work out of the box
+cp .env.example .env                       # then fill in your provider key(s)
+uv run conflens                           # or: uv run python run.py
+```
+
+(`conflens` is the console command once the package is installed;
+`conference-analyzer` remains as an alias.)
+
+Then open <http://localhost:6868>.
+
+**Configuration.** `conflens` needs an LLM provider key, supplied any of three
+ways:
+
+- **Environment variable** — `export ANTHROPIC_API_KEY=…` (or `OPENAI_API_KEY`,
+  plus `OPENAI_BASE_URL` for an OpenAI-compatible endpoint; `LITELLM_API_KEY`).
+- **A `.env` file** in the directory you launch `conflens` from — same keys, one
+  `NAME=value` per line. It's loaded automatically from the current working
+  directory (from a clone, copy [`.env.example`](.env.example) as a starting
+  point). Real environment variables take precedence over `.env`.
+- **The in-app “API key” field** at runtime.
+
+OpenReview venues (ICLR / NeurIPS) additionally need `OPENREVIEW_TOKEN`, or
+`OPENREVIEW_USERNAME` + `OPENREVIEW_PASSWORD` — set the same way, or typed into
+the fields that appear in the UI when you pick the OpenReview source.
+
+`uv` provisions the right Python automatically (pinned to 3.13 via
+`.python-version`); you don't need to install it yourself.
+
+### Run with Docker
+
+```bash
+cp .env.example .env          # fill in your provider key(s)
+docker compose up --build     # → http://localhost:6868
+```
+
+The image is built with `uv` on Python 3.13 and includes all LLM
+providers. A named volume persists the cache (scrapes + classifications) across
+restarts. To clear the cache in a container:
+
+```bash
+docker compose run --rm app --clear-cache
+```
+
+Plain Docker (no compose) works too:
+
+```bash
+docker build -t conference-analyzer .
+docker run --rm -p 6868:6868 --env-file .env \
+  -v conf-cache:/home/app/.cache/conflens conference-analyzer
+```
+
+To also build the optional BERTopic backend into the image:
+`docker build --build-arg EXTRAS="--extra bertopic" -t conference-analyzer .`
+
+### LLM providers
+
+The classifier and topic engine work with three providers, chosen in the UI:
+
+| Provider | API key (env var or the in-app field) | Notes |
+|----------|----------------------------------------|-------|
+| **Anthropic** (default) | `ANTHROPIC_API_KEY` | Claude models, native structured output |
+| **OpenAI** | `OPENAI_API_KEY` | OpenAI or any OpenAI-compatible base URL |
+| **LiteLLM** | `LITELLM_API_KEY` / `OPENAI_API_KEY` | point **LLM endpoint** at your own LiteLLM URL |
+
+In the app, pick the
+**LLM provider**, set the **Model**, and (for LiteLLM / OpenAI-compatible
+servers) the **LLM endpoint**. An **API key** field overrides the env var when
+set — handy for a self-hosted endpoint.
+
+> **Note on the default event:** ACL 2026 proceedings may not be published yet.
+> The app handles this gracefully and tells you so — try a past event such as
+> `acl-2024` to see a full run.
+
+## Architecture
+
+A NiceGUI front end drives a linear pipeline — **browse → classify →
+topic-model → summarize** — over a pluggable *source* (which conference) and a
+pluggable *LLM provider* (which model), with every expensive step cached on disk.
+
+```mermaid
+flowchart LR
+    UI["NiceGUI UI<br/>app.py"] --> PIPE["pipeline.py"]
+    PIPE --> SRC["sources.py<br/>ACL · EMNLP · NAACL · IJCAI · OpenReview · PSCC · ISGT Europe"]
+    PIPE --> CLS["classifier.py"]
+    PIPE --> TOP["topics.py<br/>model + summarize"]
+    CLS --> LLM["llm.py<br/>Anthropic · OpenAI · LiteLLM"]
+    TOP --> LLM
+    PIPE --> RES["AnalysisResult"] --> EXP["PPTX · BibTeX · JSON · CSV"]
+    SRC -. cache .-> CACHE[("~/.cache/conflens")]
+    CLS -. cache .-> CACHE
+    TOP -. cache .-> CACHE
+```
+
+See **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)** for component, sequence,
+data-model and caching diagrams, plus extension points.
+
+## How it works
+
+| Stage | Module | Notes |
+|-------|--------|-------|
+| Sources | `conflens/sources.py` | Pluggable adapters (ACL Anthology, EMNLP, NAACL, IJCAI, OpenReview, PSCC, DBLP/ISGT Europe) behind one interface; registry + factory. |
+| Scrape listing | `conflens/scraper.py` | ACL Anthology adapter: parses the event page; abstracts + authors fetched per paper and cached. |
+| Near-duplicates | `conflens/dedup.py` | Flags near-identical titles (union-find + `difflib`); dependency-free. |
+| Classify | `conflens/classifier.py` | Batched, structured-output calls; relevance + confidence + a one-line reason per paper (cached). |
+| Topic model | `conflens/topics.py` | `llm` backend derives a taxonomy and assigns papers (primary + optional secondary topic); `bertopic` backend optional. |
+| Summarize | `conflens/topics.py` | Per-topic description + common findings across the topic's papers (cached). |
+| LLM providers | `conflens/llm.py` | One `structured()` interface over Anthropic / OpenAI / LiteLLM. |
+| Orchestrate | `conflens/pipeline.py` | Runs the stages with progress reporting and cooperative cancel. |
+| UI / exports | `conflens/app.py`, `pptx_export.py`, `bibtex.py` | NiceGUI; ECharts chart; interactive results view; PPTX / BibTeX / JSON / CSV export. |
+
+## Configuration (in the UI)
+
+- **Source** — `ACL Anthology`, `EMNLP (ACL Anthology)`, `NAACL (ACL Anthology)`,
+  `IJCAI`, `OpenReview (ICLR / NeurIPS)`, `PSCC`, or `ISGT Europe (via DBLP)`.
+  Switching prefills the base URL and target below and relabels them. (ACL
+  Anthology, EMNLP and NAACL share one adapter — any of them accepts any
+  Anthology event slug, e.g. `acl-2024`, `emnlp-2023`, `naacl-2024`.)
+- **Base URL** — the site/API root (e.g. `https://aclanthology.org`,
+  `https://2026.ijcai.org`, `https://api2.openreview.net`); change it to point at
+  a mirror, another year, or the v1 API host (`https://api.openreview.net`).
+- **Event / target** — for ACL, a slug (`acl-2024`, `emnlp-2023`, …) or full
+  event URL; for IJCAI, the accepted-papers path or full URL; for OpenReview, the
+  **venue id** (`ICLR.cc/2024/Conference`, `NeurIPS.cc/2024/Conference`) or a
+  venue group URL; for PSCC, a **year** (`2024`, `2022`, …) or a full listing URL;
+  for ISGT Europe, a DBLP **venue + year** (`isgteurope 2024`) or proceedings key.
+
+  > **Title-based sources** — PSCC and DBLP-backed venues (ISGT Europe) don't
+  > expose abstracts, so classification runs on titles alone (still useful, just
+  > coarser). The DBLP adapter is generic: any DBLP-indexed conference works by
+  > passing `<venue> <year>`. IEEE-Xplore-only venues that DBLP doesn't index
+  > (e.g. **PowerTech**, **PES General Meeting**) can't be added without an IEEE
+  > Xplore API key.
+
+  > **OpenReview auth** — OpenReview gates anonymous note queries behind an
+  > anti-bot challenge (HTTP 403), so ICLR / NeurIPS venues need credentials:
+  > set `OPENREVIEW_TOKEN`, or `OPENREVIEW_USERNAME` + `OPENREVIEW_PASSWORD`
+  > (see `.env.example`); the adapter logs in and authenticates automatically.
+- **Theme** — any phrase; defaults to *Agentic AI*.
+- **Theme definition** — optional free text clarifying what the theme includes
+  or excludes (e.g. *"tool-using LLM agents; exclude pure RL"*). It is threaded
+  into classification, topic discovery, and summaries to sharpen relevance, and
+  is part of the classification cache key.
+- **LLM provider** — Anthropic (default), OpenAI, or LiteLLM.
+- **Model** — free-text; defaults per provider (e.g. `claude-opus-4-8`,
+  `gpt-4o-mini`). Suggestions appear as placeholder text.
+- **LLM endpoint / API key** — a custom base URL (LiteLLM or any
+  OpenAI-compatible server) and an optional key override.
+- **Topic engine** — `LLM` (no extra deps) or `BERTopic` (requires the
+  optional `bertopic` install).
+- **Max papers**, **target topics**, **minimum confidence** — tuning knobs.
+
+## Features
+
+- Per-topic bar chart of paper counts.
+- For each topic, an LLM-generated **description** and **5–10 common findings**
+  (synthesised across the topic's papers, not paper-specific), shown above the
+  paper list.
+- **Multi-topic assignment** — a paper can belong to a primary topic and, when it
+  genuinely spans two areas, a secondary one; secondary memberships show an
+  *"Also in: …"* line, and the CSV export lists every assigned topic.
+- **Near-duplicate detection** — papers with near-identical titles (e.g. a
+  preprint and its camera-ready) are clustered with a dependency-free fuzzy match
+  and flagged with a *near-dup* badge naming the representative; the count of
+  groups appears in the summary and a `duplicate_of` column is added to the CSV.
+- A **keyword search** that filters the papers by their title or abstract —
+  comma-separated keywords (each may contain spaces), matched with AND. Matched
+  keywords are **highlighted** in the title and abstract, topics with no match
+  are hidden, and a **"Search all topics"** toggle flattens every match into one
+  ranked list (a multi-topic paper appears once, tagged with its topics).
+- A **live confidence slider** in the results view: because the raw model
+  judgement is cached, re-thresholding re-filters the papers, counts, and chart
+  instantly — no re-run.
+- **Sort** papers by confidence, title, or year, and **filter by author** —
+  all applied live on top of the search and confidence filters.
+- **Save / load a run** — the JSON export is a complete snapshot; use *"Load
+  saved run"* to reopen it later and browse, search, and re-threshold without
+  re-analysing.
+- Expandable abstracts and a per-paper relevance rationale.
+- One-click **PDF** links to every paper's full text.
+- A **Cancel** button and elapsed-time readout for long runs; LLM calls
+  automatically retry transient errors (rate limits / 5xx / timeouts).
+- Export results as a **PPTX** slide deck, **BibTeX**, **JSON**, or **CSV**.
+  The deck (built with `python-pptx`) has a title slide, a papers-per-topic
+  chart, and per topic an overview slide (description + common findings) plus
+  slides listing its papers with clickable PDF links.
+- On-disk caching of scraped data so re-runs are fast (see below).
+
+## Caching
+
+Everything expensive is cached on disk under `~/.cache/conflens`:
+
+- the **event listing** is cached per event/page URL,
+- each paper's **abstract + authors** is cached per paper id, and
+- **classification results** are cached per *(provider + model, theme)* and keyed
+  by each paper's title+abstract hash. The raw model judgement is stored, so the
+  **minimum-confidence** threshold is applied at read time — adjusting it never
+  triggers a re-call.
+- **topic summaries** (description + common findings) are cached per
+  *(provider + model, theme)* and keyed by the topic's exact paper membership.
+
+So re-running with a **different theme** reuses the cached scrape (only
+classification re-runs), and **re-running the same theme + model is essentially
+free**. Changing the model or a paper's abstract re-classifies only what's
+affected. Tick **“Refresh from source”** in the UI to bypass every cache and
+rebuild from scratch.
+
+To wipe the cache from the command line:
+
+```bash
+uv run conflens --clear-cache                     # default ~/.cache/conflens
+uv run conflens --clear-cache --cache-dir /path/to/cache
+```
+
+Other flags: `--host`, `--port` (run `conflens --help`).
+
+## Cost
+
+Classification batches ~20 papers per request, so a 150-paper run is a handful
+of API calls. Pick a small model (e.g. `claude-haiku-4-5` or `gpt-4o-mini`) to
+minimise cost.
+
+## Optional: BERTopic
+
+```bash
+uv sync --extra bertopic
+```
+
+Then pick **BERTopic** as the topic engine in the UI. It clusters
+sentence-embeddings instead of asking the LLM to organise topics.
+
+## Development
+
+```bash
+uv sync              # installs the dev group (pytest + ruff) too
+uv run pytest        # run the test suite
+uv run ruff check .  # lint
+uv build             # build the wheel/sdist
+```
+
+CI is GitHub Actions, one workflow per concern, each running on **every pushed
+commit** and every PR:
+
+- **`lint.yml`** — ruff on every commit.
+- **`test.yml`** — pytest + package build.
+- **`release.yml`** — version bump + tag on PR merge (see below).
+
+The tests are network- and API-free (parsers run on HTML fixtures; the LLM
+stages use a fake client), so they're fast and deterministic.
+
+### Releasing
+
+Releases are automated (`.github/workflows/release.yml`): when a PR is **merged
+into `main`**, the workflow bumps the version in `pyproject.toml`, commits it
+back to `main`, creates a matching `vX.Y.Z` **tag**, then **builds and publishes
+the package to [PyPI](https://pypi.org/project/conflens/)** (`pip install
+conflens`). The bump is a **patch** by default — add a **`minor`** or
+**`major`** label to the PR to bump those instead.
+
+Publishing needs a repo secret **`PYPI_API_TOKEN`** (a PyPI API token; the
+workflow uploads with `twine` as `__token__`). Add it under *Settings → Secrets
+and variables → Actions*. Without it the version bump/tag still succeed and only
+the publish step fails.
+

conflens-0.1.14.dist-info/RECORD

@@ -0,0 +1,20 @@
+conflens/__init__.py,sha256=qPoGyihNoQ1vCTVu9JTvEonAaXCGtnXmLIlb6nuGXbQ,236
+conflens/app.py,sha256=5na3cXS3TGKGuNxmM19_jqvg33S91LdGExhTpNdkFcE,37425
+conflens/bibtex.py,sha256=Ew0ZjNH7xcOpm5AgPmoNLUUflrn1KBSgiq9LHORVBmI,1731
+conflens/cache.py,sha256=fi86l8GkCR-qqH0WMdWEk7-ohAW4woNOYZ_rVmKDgMM,1080
+conflens/classifier.py,sha256=0sdZNR1KvqOp13i_9mEg-8cuWVUjutqs-zSMy0PuMDw,6275
+conflens/cli.py,sha256=MTUaWw0YksvHTLqFcrySbHjWXRTRPeDdhaMAQKK1iPY,2174
+conflens/dedup.py,sha256=mxZ6flnI9ksDqbgStRO2IstjTyfpFpGK7gzfuT7NqV4,3228
+conflens/llm.py,sha256=lWCpxXmi79geMeR8FBca2xPuTqed6wRHn8qh7_bxriE,12380
+conflens/models.py,sha256=qPGiClMs2dFw0fC6WfpF_sCihd3tz5_AlimF_Lp26do,3995
+conflens/pipeline.py,sha256=3QJgwWc7KH6_3p1HOEvEjNk4q4cMijvuv_EXBwLDZgs,8149
+conflens/pptx_export.py,sha256=De_Vu0QbzZgMQKhCoSG6nuxdBa0rnPv-KLM3C7Tt4M0,9110
+conflens/scraper.py,sha256=IZ_0r-csHdth-kHuPqK9_HGy0DgECksFnmv8TLENwfg,10531
+conflens/sources.py,sha256=61_HYrHHVKVrzdaQWfoabMSKpX1Ik1A5Il3erYzLZ4c,34266
+conflens/topics.py,sha256=TXs6nbfFL0FBIlIHV7ciX6ynlUf76x9lJhmoNf6QUjc,13583
+conflens/view.py,sha256=F_9mswjZQK6c3XAV1FZEbExEOtUYkJw_YRaJ6b-nP2w,6586
+conflens-0.1.14.dist-info/METADATA,sha256=3fRiLiXFK12-8VY2XQ7qDbNc1uhMHD4tWAIlnn-SALo,17518
+conflens-0.1.14.dist-info/WHEEL,sha256=zOwg4jB6zX2kU910N-cMawjivD6tO8NEWvE12je1bVk,87
+conflens-0.1.14.dist-info/entry_points.txt,sha256=JDyWB2Ds1ydDvByyGwrvPsZE3IRynGmjsSNgnqMl1DM,87
+conflens-0.1.14.dist-info/licenses/LICENSE,sha256=Pz2eACSxkhsGfW9_iN60pgy-enjnbGTj8df8O3ebnQQ,16726
+conflens-0.1.14.dist-info/RECORD,,

conflens-0.1.14.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.32.0
+Root-Is-Purelib: true
+Tag: py3-none-any

conflens-0.1.14.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+conference-analyzer = conflens.cli:main
+conflens = conflens.cli:main