nber-cli 0.2.0__tar.gz → 0.3.1__tar.gz

nber_cli-0.3.1/CHANGELOG.md

@@ -0,0 +1,52 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [0.3.1] - 2026-06-03
+
+### Added
+- `db init` and `db migrate` top-level subcommands replace `feed init` and `feed migrate`. The database is now general-purpose and stores feed cache, behavior logs, and paper metadata cache.
+- `info_cache` table caches paper metadata fetched via `info` and the MCP `get_paper_info` tool. Subsequent lookups return immediately from the cache.
+- `query_log`, `download_log`, and `info_log` tables record search keywords, download outcomes, and paper info lookups for later auditing.
+- `schema_version` field written to `~/.nber-cli/config.json` so future schema migrations can roll forward safely.
+
+### Changed
+- Default database renamed from `feed.db` to `nber.db`. Existing `~/.nber-cli/feed.db` installations continue to work without manual steps.
+- Database schema upgraded from `user_version = 1` to `user_version = 2`. Existing v1 databases are upgraded automatically on next CLI invocation; original `feed_items` rows are preserved.
+- Database code consolidated into `nber_cli.db`. The original `init_feed_database`, `migrate_feed_database`, and `get_feed_database_path` helpers are kept as thin compatibility wrappers.
+
+## [0.3.0] - 2026-06-03
+
+### Added
+- `feed init` subcommand: initialize a local SQLite feed cache and write its path to user config.
+- `feed fetch` subcommand: fetch NBER's new working papers RSS feed, cache seen items, and show newly discovered papers by default.
+- `feed fetch --max-items`: limit displayed feed output. When used without `--display-all`, display-all behavior is enabled automatically.
+- `feed migrate` subcommand: move the feed cache database and SQLite sidecar files to a new path, then update user config.
+- `feed clean` subcommand: clean cached feed database records by age, date range, or all records after interactive confirmation.
+- Python API exports for feed cache helpers and feed result models.
+
+### Changed
+- Added user config support at `~/.nber-cli/config.json` for the feed cache database path.
+- Expanded English and Chinese documentation for feed commands, configuration, Python API, and release notes.
+
+## [0.2.0] - 2026-05-31
+
+### Added
+- `download` subcommand: single paper ID or batch mode (`--batch`), explicit file path (`--file`), target directory (`--save-base`).
+- `info` subcommand: paper metadata with `--all` flag for full details and `--format json` option.
+- `search` subcommand: full-text search with date filters (`--start-date`, `--end-date`), pagination (`--page`, `--per-page`), `--format json` option.
+- `mcp-server` subcommand: MCP server for AI agent integration with stdio and streamable_http transports.
+
+### Changed
+- Reworked CLI into subcommand syntax (`nber-cli <subcommand>`).
+- Simplified downloader to direct async HTTP PDF fetches (removed database-backed state tracking).
+- Removed legacy web UI module and script entrypoint.
+
+## [0.1.4] - 2025-08-09
+
+### Added
+- Added `--version` / `-v` flag to display current version.
+- Added comprehensive help message with examples.
+- Added `__main__.py` file to support `python -m nber_cli` usage.
+- Added argument grouping for better CLI organization.
+- Added automatic help display when no arguments are provided.

{nber_cli-0.2.0 → nber_cli-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nber-cli
-Version: 0.2.0
+Version: 0.3.1
 Summary: A command line interface for reaching the National Bureau of Economic Research (NBER) paper without brower.
 Project-URL: Homepage, https://github.com/sepinetam/nber-cli
 Project-URL: Repository, https://github.com/sepinetam/nber-cli

nber_cli-0.3.1/docs/en/changelog.md

@@ -0,0 +1,56 @@
+# Changelog
+
+All notable changes to this project will be documented here.
+
+## 0.3.1 - 2026-06-03
+
+### Added
+
+- Added `nber-cli db init` and `nber-cli db migrate` for initializing and relocating the local database. These replace `feed init` and `feed migrate`.
+- Added `info_cache` table so repeated `nber-cli info` and MCP `get_paper_info` calls return immediately from cache.
+- Added `query_log`, `download_log`, and `info_log` tables for recording search keywords, download outcomes, and info lookups.
+- Added `schema_version` field in `~/.nber-cli/config.json` for forward-compatible schema upgrades.
+
+### Changed
+
+- Renamed default database file from `feed.db` to `nber.db`. Existing `~/.nber-cli/feed.db` installations keep working without manual steps.
+- Upgraded database schema from version 1 to version 2 with automatic upgrade on next invocation.
+- Consolidated database code into `nber_cli.db`. Old `init_feed_database` and `migrate_feed_database` helpers remain as thin compatibility wrappers.
+
+## 0.3.0 - 2026-06-03
+
+### Added
+
+- Added `nber-cli feed init` for creating a local SQLite feed cache.
+- Added `nber-cli feed fetch` for fetching NBER's new working papers RSS feed and showing newly cached items.
+- Added `nber-cli feed fetch --max-items` for limiting displayed feed output.
+- Added `nber-cli feed migrate` for moving the feed cache database and updating user config.
+- Added `nber-cli feed clean` for cleaning cached feed database records with confirmation.
+- Added Python API documentation for feed cache helpers and feed data models.
+
+### Changed
+
+- Added user config documentation for `~/.nber-cli/config.json` and `feed.db-path`.
+- Expanded English and Chinese feed cache documentation across CLI, getting started, configuration, and Python API pages.
+
+## 0.2.0 - 2026-05-27
+
+### Changed
+
+- Reworked the CLI into `nber-cli download ...` subcommand syntax.
+- Added `--file/-f` and `--save-base/-s` path handling behavior.
+- Added `--batch/-b` multi-ID download mode.
+- Removed database-backed download state tracking.
+- Simplified the downloader to direct async HTTP PDF fetches.
+- Updated documentation for the v0.2 command model.
+- Removed the legacy web UI module and script entrypoint.
+
+## 0.1.4 - 2025-08-09
+
+### Added
+
+- Added `--version` / `-v` flag to display current version.
+- Added comprehensive help message with examples.
+- Added `__main__.py` support for `python -m nber_cli`.
+- Added argument grouping for better CLI organization.
+- Added automatic help display when no arguments are provided.

{nber_cli-0.2.0 → nber_cli-0.3.1}/docs/en/cli.md

@@ -22,6 +22,8 @@ Running `nber-cli` without a subcommand prints the top-level help and exits succ
 | `download` | Download one or more paper PDFs. |
 | `info` | Show metadata and abstract for one paper. |
 | `search` | Search NBER working papers. |
+| `db` | Manage the local SQLite database. |
+| `feed` | Manage the NBER new working papers RSS feed cache. |
 | `mcp-server` | Start the MCP server for agents. |
 
 ## download
@@ -139,6 +141,122 @@ nber-cli search "inflation" -f json
 
 When only `--start-date` is provided, NBER-CLI automatically uses the current date as the end date.
 
+## feed
+
+`feed` works with NBER's new working papers RSS feed and the local SQLite database. The database tracks which RSS items have already been seen, so `feed fetch` can show only newly discovered papers by default.
+
+### feed fetch
+
+Fetch the RSS feed, store all fetched items in the cache, and display only new items by default:
+
+```bash
+nber-cli feed fetch
+```
+
+Display all fetched RSS items, including items already present in the cache:
+
+```bash
+nber-cli feed fetch --display-all true
+nber-cli feed fetch --display-all
+```
+
+Limit displayed output:
+
+```bash
+nber-cli feed fetch --max-items 5
+```
+
+When `--max-items` is provided and `--display-all` is omitted, `--display-all` defaults to `true`. This makes `nber-cli feed fetch --max-items 5` show the first five fetched RSS items instead of showing nothing when there are no new items.
+
+Return JSON:
+
+```bash
+nber-cli feed fetch --format json
+nber-cli feed fetch -f json
+```
+
+### feed clean
+
+Clean cached feed database records. This deletes records from the local cache, not from NBER. Deleted cache records may be fetched again as new items if they still appear in the RSS feed.
+
+Clean records not seen for 30 days:
+
+```bash
+nber-cli feed clean
+nber-cli feed clean --days 30
+```
+
+Clean all cached records:
+
+```bash
+nber-cli feed clean --all
+```
+
+Clean records by last-seen date:
+
+```bash
+nber-cli feed clean --end-date 2026-05-31
+nber-cli feed clean --start-date 2026-05-01 --end-date 2026-05-31
+```
+
+`--end-date` without `--start-date` cleans from the earliest cached record through the end date. `--start-date` and `--end-date` are inclusive. Passing only `--start-date` is invalid.
+
+Before deleting anything, `feed clean` prints how many cached records match and asks for confirmation:
+
+```text
+This operation is irreversible.
+Deleted cache records may be fetched again as new items if they still appear in the RSS feed.
+Continue? [y/N]:
+```
+
+Only `y` or `Y` continues. Any other response aborts without deleting records.
+
+### feed Options
+
+| Subcommand | Option | Description |
+| --- | --- | --- |
+| `fetch` | `--display-all [true|false]` | Display all fetched RSS items instead of only new items. |
+| `fetch` | `--format`, `-f` | Output format: `list` or `json`. Defaults to `list`. |
+| `fetch` | `--max-items` | Maximum number of feed items to display. |
+| `clean` | `--days` | Clean cached records not seen for this many days. Defaults to `30`. |
+| `clean` | `--all` | Clean all cached feed records. |
+| `clean` | `--start-date` | Clean cached records last seen on or after this date, formatted `YYYY-MM-DD`. |
+| `clean` | `--end-date` | Clean cached records last seen on or before this date, formatted `YYYY-MM-DD`. |
+
+## db
+
+`db` manages the local SQLite database used by `info`, `search`, `download`, and `feed` for cache and behavior logs.
+
+### db init
+
+Initialize the database and write its path to the user config:
+
+```bash
+nber-cli db init
+nber-cli db init --db-path ~/.nber-cli/nber.db
+```
+
+If `--db-path` is omitted, the default database path is `~/.nber-cli/nber.db`.
+
+If an existing `~/.nber-cli/feed.db` from 0.3.0 is present and no `nber.db` exists yet, NBER-CLI uses that legacy file in place. Schema is automatically upgraded from version 1 to version 2 on first use.
+
+### db migrate
+
+Move the database to a new path and update the user config:
+
+```bash
+nber-cli db migrate ~/data/nber.db
+```
+
+Migration moves the SQLite database file and any SQLite sidecar files such as `-wal`, `-shm`, and `-journal`. The target path must not already exist.
+
+### db Options
+
+| Subcommand | Option | Description |
+| --- | --- | --- |
+| `init` | `--db-path` | SQLite database path. Defaults to `~/.nber-cli/nber.db`. |
+| `migrate` | `new_db_path` | New SQLite database path. |
+
 ## mcp-server
 
 Start the default stdio MCP server:
@@ -172,4 +290,4 @@ For client configuration and tool details, see [MCP Server](mcp.md).
 
 ## Output Formats
 
-`info` and `search` default to `list`, a readable text format. Use `--format json` when piping output into scripts or agent workflows.
+`info`, `search`, and `feed fetch` default to `list`, a readable text format. Use `--format json` when piping output into scripts or agent workflows.

{nber_cli-0.2.0 → nber_cli-0.3.1}/docs/en/configuration.md

@@ -1,6 +1,6 @@
 # Configuration
 
-NBER-CLI currently uses built-in runtime defaults rather than a user configuration file. The defaults are intentionally conservative and work for both CLI and MCP usage.
+Most NBER-CLI runtime behavior uses built-in defaults. The local database also uses a small user config file to remember the SQLite database path selected by `nber-cli db init` or `nber-cli db migrate`.
 
 ## Runtime Defaults
 
@@ -15,6 +15,63 @@ NBER-CLI currently uses built-in runtime defaults rather than a user configurati
 
 These values live in `NBERCLIConfig` and `NBER_CLI_CONFIG`.
 
+## User Config File
+
+The user config file is:
+
+```text
+~/.nber-cli/config.json
+```
+
+Current schema:
+
+```json
+{
+  "schema_version": 2,
+  "feed": {
+    "db-path": "/Users/name/.nber-cli/nber.db"
+  }
+}
+```
+
+`feed.db-path` points to the SQLite database used by `info`, `search`, `download`, and `feed`. The historical `feed` key name is preserved for backward compatibility; the database itself is general-purpose.
+
+`schema_version` records the current database schema version. NBER-CLI updates it after `db init` or schema upgrades.
+
+## Local Database
+
+Default database path:
+
+```text
+~/.nber-cli/nber.db
+```
+
+Initialize the default database:
+
+```bash
+nber-cli db init
+```
+
+Initialize at a custom path:
+
+```bash
+nber-cli db init --db-path ~/data/nber.db
+```
+
+Move an existing database and update the config:
+
+```bash
+nber-cli db migrate ~/data/nber.db
+```
+
+If you upgraded from 0.3.0 and still have `~/.nber-cli/feed.db`, NBER-CLI will keep using that legacy file when no `nber.db` is present. The schema is upgraded automatically on first invocation.
+
+The database holds:
+
+- `feed_items` and `feed_fetches`: RSS cache used by `feed fetch` and `feed clean`.
+- `info_cache`: paper metadata cache used by `info` and the MCP `get_paper_info` tool.
+- `query_log`, `download_log`, `info_log`: behavior logs for search keywords, download outcomes, and info lookups.
+
 ## Output Paths
 
 Single download default:

{nber_cli-0.2.0 → nber_cli-0.3.1}/docs/en/getting-started.md

@@ -73,6 +73,36 @@ Use `--all` to include related fields and published-version information when NBE
 nber-cli info w25000 --all
 ```
 
+## Follow New Papers with the Feed Cache
+
+Initialize the local database:
+
+```bash
+nber-cli db init
+```
+
+Fetch the NBER new working papers RSS feed:
+
+```bash
+nber-cli feed fetch
+```
+
+The first fetch stores the current RSS items in the cache and displays them as new. Later fetches show only items that were not already cached.
+
+Limit output while showing the latest fetched items:
+
+```bash
+nber-cli feed fetch --max-items 5
+```
+
+Clean old cache records:
+
+```bash
+nber-cli feed clean --days 30
+```
+
+`feed clean` asks for confirmation before deleting cached records.
+
 ## Download a PDF
 
 Download into the current directory:

{nber_cli-0.2.0 → nber_cli-0.3.1}/docs/en/index.md

@@ -8,6 +8,7 @@ NBER-CLI focuses on the common research loop around NBER working papers:
 
 - Search for papers by keyword, author, title, abstract, or paper number.
 - Inspect a paper's title, authors, date, abstract, URL, and related metadata.
+- Follow new working papers through NBER's RSS feed and a local cache.
 - Download a PDF by paper ID.
 - Run batch downloads into a target directory.
 - Serve paper search, lookup, and download operations as MCP tools.
@@ -19,6 +20,7 @@ Run without installing:
 ```bash
 uvx nber-cli search "Labor Economic"
 uvx nber-cli info w25000
+uvx nber-cli feed fetch --max-items 5
 uvx nber-cli download w34567
 ```
 
@@ -28,6 +30,7 @@ Install as a reusable command:
 uv tool install nber-cli
 nber-cli search "Labor Economic"
 nber-cli info w25000
+nber-cli feed fetch --max-items 5
 nber-cli download w34567
 ```
 
@@ -66,4 +69,4 @@ Add it to an MCP client:
 
 ## Project Status
 
-The current public command model is `nber-cli` v0.2.0. The CLI is intentionally small and script-friendly: text output is optimized for human reading, and `--format json` is available where structured output matters.
+The current public command model is `nber-cli` v0.3.0. The CLI is intentionally small and script-friendly: text output is optimized for human reading, and `--format json` is available where structured output matters.

{nber_cli-0.2.0 → nber_cli-0.3.1}/docs/en/python-api.md

@@ -2,6 +2,8 @@
 
 NBER-CLI exposes the same core functionality as importable Python functions. The API is asynchronous because lookup, search, and downloads perform network I/O.
 
+Feed cache helpers are synchronous because they perform local SQLite work and a synchronous RSS fetch.
+
 ## Install
 
 ```bash
@@ -90,6 +92,75 @@ async def main() -> None:
 asyncio.run(main())
 ```
 
+## Work with the Local Database
+
+Initialize the default database:
+
+```python
+from nber_cli import init_database
+
+db_path = init_database()
+print(db_path)
+```
+
+Fetch the NBER RSS feed and display new cached items:
+
+```python
+from nber_cli import feed_results, fetch_feed
+
+result = fetch_feed()
+payload = feed_results(result)
+print(payload["new_count"])
+for item in payload["results"]:
+    print(item["id"], item["title"])
+```
+
+Display all fetched RSS items and limit the returned items:
+
+```python
+from nber_cli import fetch_feed
+
+result = fetch_feed(display_all=True, max_items=5)
+```
+
+Move the database:
+
+```python
+from pathlib import Path
+
+from nber_cli import migrate_database
+
+old_path, new_path = migrate_database(Path("~/data/nber.db"))
+```
+
+Clean cached feed database records:
+
+```python
+from nber_cli import clean_feed_cache
+
+preview = clean_feed_cache(days=30, dry_run=True)
+print(preview.matched_count)
+
+result = clean_feed_cache(days=30)
+print(result.deleted_count)
+```
+
+`clean_feed_cache` deletes local cache records only. If deleted records still appear in the RSS feed, a later `fetch_feed` call may return them as new items again.
+
+Read or write the paper metadata cache directly:
+
+```python
+from nber_cli import read_info_cache, write_info_cache
+from nber_cli import get_nber
+import asyncio
+
+paper = read_info_cache(None, "w25000")
+if paper is None:
+    paper = asyncio.run(get_nber(25000))
+    write_info_cache(None, paper)
+print(paper.title)
+```
+
 ## Data Models
 
 ### NBER
@@ -118,6 +189,43 @@ asyncio.run(main())
 | `start_date` | `str` or `None` | Applied start date. |
 | `end_date` | `str` or `None` | Applied end date. |
 
+### NBERFeedItem
+
+| Field | Type | Description |
+| --- | --- | --- |
+| `paper_id` | `str` | Paper ID, for example `w35254`. |
+| `title` | `str` | Paper title parsed from the RSS item. |
+| `authors` | `list[str]` | Author names parsed from the RSS title. |
+| `abstract` | `str` | RSS item description. |
+| `url` | `str` | Canonical NBER paper URL without the RSS fragment. |
+| `source_url` | `str` | Original RSS item URL. |
+| `guid` | `str` | RSS item GUID. |
+
+### NBERFeedFetchResult
+
+| Field | Type | Description |
+| --- | --- | --- |
+| `source_url` | `str` | RSS feed URL. |
+| `database_path` | `Path` | SQLite cache database path. |
+| `total_fetched` | `int` | Number of RSS items fetched. |
+| `new_count` | `int` | Number of fetched items that were not already in the cache. |
+| `display_all` | `bool` | Whether returned items include all fetched items. |
+| `items` | `list[NBERFeedItem]` | Items selected for display or structured output. |
+| `max_items` | `int` or `None` | Display limit when provided. |
+
+### NBERFeedCleanResult
+
+| Field | Type | Description |
+| --- | --- | --- |
+| `database_path` | `Path` | SQLite cache database path. |
+| `matched_count` | `int` | Number of cache records matching the clean criteria. |
+| `deleted_count` | `int` | Number of cache records deleted. |
+| `mode` | `str` | Clean mode: `days`, `all`, or `date-range`. |
+| `days` | `int` or `None` | Day threshold for `days` mode. |
+| `start_date` | `str` or `None` | Inclusive start date for `date-range` mode. |
+| `end_date` | `str` or `None` | Inclusive end date for `date-range` mode. |
+| `dry_run` | `bool` | Whether the operation only counted matching records. |
+
 ### DownloadFailure
 
 | Field | Type | Description |
@@ -137,12 +245,13 @@ asyncio.run(main())
 Use formatter helpers when you want stable dictionaries for JSON output or MCP-style responses:
 
 ```python
-from nber_cli import info, related, search_results
+from nber_cli import feed_results, info, related, search_results
 ```
 
 - `info(paper)` returns core metadata.
 - `related(paper)` returns related optional fields.
 - `search_results(results)` returns a structured search payload.
+- `feed_results(result)` returns a structured feed fetch payload.
 
 For human-readable text output, use the text formatters:
 

nber_cli-0.3.1/docs/zh/changelog.md

@@ -0,0 +1,56 @@
+# 更新日志
+
+这里记录项目的重要变更。
+
+## 0.3.1 - 2026-06-03
+
+### Added
+
+- 添加 `nber-cli db init` 和 `nber-cli db migrate`，用于初始化和迁移本地数据库，替代原先的 `feed init` 和 `feed migrate`。
+- 添加 `info_cache` 表，重复执行 `nber-cli info` 或 MCP `get_paper_info` 时直接从缓存返回。
+- 添加 `query_log`、`download_log`、`info_log` 表，记录搜索关键词、下载结果和 info 查询。
+- 在 `~/.nber-cli/config.json` 中写入 `schema_version` 字段，便于后续的 schema 升级。
+
+### Changed
+
+- 默认数据库文件从 `feed.db` 改名为 `nber.db`。已经安装了 `~/.nber-cli/feed.db` 的用户无需手动迁移。
+- 数据库 schema 从版本 1 升级到版本 2，下次启动时自动完成升级。
+- 公共数据库代码迁移到 `nber_cli.db`,原先的 `init_feed_database` 和 `migrate_feed_database` 作为薄薄的兼容层保留。
+
+## 0.3.0 - 2026-06-03
+
+### Added
+
+- 添加 `nber-cli feed init`，用于创建本地 SQLite feed 缓存。
+- 添加 `nber-cli feed fetch`，用于获取 NBER 最新工作论文 RSS feed 并显示新缓存的条目。
+- 添加 `nber-cli feed fetch --max-items`，用于限制 feed 输出数量。
+- 添加 `nber-cli feed migrate`，用于移动 feed 缓存数据库并更新用户配置。
+- 添加 `nber-cli feed clean`，用于在确认后清理 feed 缓存数据库记录。
+- 补充 feed 缓存辅助函数和 feed 数据模型的 Python API 文档。
+
+### Changed
+
+- 添加 `~/.nber-cli/config.json` 和 `feed.db-path` 的用户配置文档。
+- 扩展中英文 CLI、快速开始、配置和 Python API 页面中的 feed 缓存文档。
+
+## 0.2.0 - 2026-05-27
+
+### Changed
+
+- 将 CLI 重构为 `nber-cli download ...` 子命令语法。
+- 添加 `--file/-f` 和 `--save-base/-s` 路径处理行为。
+- 添加 `--batch/-b` 多编号下载模式。
+- 移除基于数据库的下载状态跟踪。
+- 将下载器简化为直接异步 HTTP PDF 获取。
+- 更新 v0.2 命令模型文档。
+- 移除旧 web UI 模块和脚本入口。
+
+## 0.1.4 - 2025-08-09
+
+### Added
+
+- 添加 `--version` / `-v` 参数用于显示当前版本。
+- 添加更完整的帮助信息和示例。
+- 添加 `__main__.py` 以支持 `python -m nber_cli`。
+- 添加参数分组以改善 CLI 结构。
+- 不带参数运行时自动显示帮助信息。