pyproc 0.3.2__tar.gz → 0.3.3__tar.gz

{pyproc-0.3.2 → pyproc-0.3.3}/.gitignore

@@ -54,4 +54,4 @@ coverage.xml
 .pytest_cache/
 *.csv
 *.idx
-statistic.txt
+statistic.txt.claude/

{pyproc-0.3.2 → pyproc-0.3.3}/CHANGELOG.md

@@ -4,6 +4,59 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 0.3.2
+
+### Added
+- [mcp] Tambah tool `get_master_klpd` untuk referensi K/L/PD dari LKPP ISB Satu Data
+- [mcp] Tambah tool `get_master_lpse` untuk referensi LPSE dari LKPP ISB Satu Data
+- [mcp] Tambah tool `get_tender_umum_publik` untuk unduh data tender dari ISB Satu Data (sumber alternatif)
+- [mcp] Tambah tool `set_ssl_verify` untuk mengaktifkan/menonaktifkan verifikasi sertifikat TLS/SSL
+- [mcp] Tambah tool `get_ssl_verify` untuk mengecek status verifikasi sertifikat TLS/SSL
+- [mcp] Tambah notifikasi progress untuk `create_procurement_search_index`
+- [cli] Subcommand `satudata` grouping `masterlpse` dan `tenderumum` untuk akses ISB Satu Data
+- [env] Tambah variabel lingkungan `PYPROC_SSL_VERIFY` untuk konfigurasi SSL saat startup
+
+## 0.3.1
+
+### Changed
+- [mcp] Hapus batas `max_packages` pada `create_procurement_search_index`, tambah pagination dan progress logging
+
+## 0.3
+
+### Added
+- [mcp] Server MCP dengan dukungan penuh untuk MCP protocol (stdio transport)
+- [mcp] Tool pencarian paket tender, non-tender, pencatatan, swakelola, dan darurat
+- [mcp] Tool detil paket individu dan bulk (sampai 20 paket per panggilan)
+- [mcp] Tool pencarian host LPSE berdasarkan nama instansi dan validasi host
+- [mcp] Tool kategori pengadaan (tanpa panggilan jaringan)
+- [mcp] Tool opsi pencarian (penjelasan strategi direct keyword vs local FTS index)
+- [mcp] Local SQLite full-text search index (create, search, list, delete)
+- [cli] Subcommand `masterklpd` untuk query referensi K/L/PD dari ISB API
+- [cli] Subcommand `masterlpse` untuk browser LPSE interaktif dan unduh data tender
+- [cli] Subcommand `tenderumum` untuk unduh data tender dari ISB API
+- [library] Method `Lpse.get_master_klpd()`, `Lpse.get_master_lpse()`, `Lpse.get_tender_umum_publik()`
+- [library] Dukungan context manager untuk class `Lpse`
+
+## 0.2
+
+### Added
+- [cli] Subcommand `daftarhost` untuk unduh daftar host LPSE dalam format JSON dari GitHub Gist
+- [cli] Argument `--instansi-id` untuk filter pencarian berdasarkan kode K/L/PD
+- [cli] Argument `--tipe-swakelola-id` untuk filter tipe swakelola
+- [library] Module `cache.py` untuk SQLite cache store dengan context manager
+- [library] Parameter `verify` pada constructor `Lpse` untuk kontrol verifikasi SSL
+- [test] Unit test framework dengan 102 test case (cache, CLI, API layer)
+
+### Changed
+- [cli] Restruktur CLI dengan subcommand dispatch system, backward compatible
+- [cli] Ganti flag `--non-tender` menjadi `--jenis-paket` dengan dukungan 5 tipe paket
+- [build] Migrasi ke `pyproject.toml`, upgrade GitHub Actions
+
+### Fixed
+- [downloader] Filter tahun anggaran pada infinite scrolling
+- [downloader] Detil referer header
+- [library] Method `get_paket` request method
+
 ## 0.1.7
 
 ### Fix

{pyproc-0.3.2 → pyproc-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pyproc
-Version: 0.3.2
+Version: 0.3.3
 Summary: MCP tools for real-time Indonesian public procurement data from SPSE/Inaproc
 Project-URL: Homepage, https://github.com/wakataw/pyproc
 Project-URL: Issues, https://github.com/wakataw/pyproc/issues
@@ -46,7 +46,7 @@ Description-Content-Type: text/markdown
 </p>
 
 <p align="center">
-  <a href="https://pypi.org/project/pyproc/"><img src="https://img.shields.io/badge/version-v0.3.2-blue" alt="PyPI version"></a>
+  <a href="https://pypi.org/project/pyproc/"><img src="https://img.shields.io/badge/version-v0.3.3-blue" alt="PyPI version"></a>
   <a href="https://pypi.org/project/pyproc/"><img src="https://img.shields.io/badge/python-≥3.9-yellow" alt="Python ≥3.9"></a>
   <a href="https://github.com/wakataw/pyproc/actions/workflows/test.yml?query=branch%3Amaster"><img src="https://github.com/wakataw/pyproc/actions/workflows/test.yml/badge.svg?branch=master" alt="Test Status"></a>
   <a href="https://github.com/wakataw/pyproc/blob/master/LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="License: MIT"></a>
@@ -160,18 +160,20 @@ Download procurement data in bulk from the command line.
 
 ```bash
 # Download tender data from Kemenkeu LPSE, export as JSON
-pyproc kemenkeu --keyword "mobil dinas" --tahun-anggaran 2025 --output-format json
+pyproc spse kemenkeu --keyword "mobil dinas" --tahun-anggaran 2025 --output-format json
 
 # Download non-tender data from multiple LPSE hosts
-pyproc jakarta,sumbarprov --jenis-paket non_tender --tahun-anggaran 2025
+pyproc spse jakarta,sumbarprov --jenis-paket non_tender --tahun-anggaran 2025
 
 # Download pencatatan non-tender data. This is distinct from non_tender.
-pyproc nasional --jenis-paket pencatatan_non_tender --tahun-anggaran 2026
+pyproc spse nasional --jenis-paket pencatatan_non_tender --tahun-anggaran 2026
 
 # Download with custom output filename
-pyproc "kemenkeu;output_kemenkeu" --output-format csv --separator ";"
+pyproc spse "kemenkeu;output_kemenkeu" --output-format csv --separator ";"
 ```
 
+The `spse` subcommand is the default, so `pyproc kemenkeu` is shorthand for `pyproc spse kemenkeu`.
+
 See [CLI Usage](#cli-usage) for the full argument reference.
 
 ---
@@ -201,6 +203,11 @@ The MCP server exposes the following tools:
 | `get_swakelola_details_bulk` | Get details for multiple swakelola packages |
 | `get_pengadaan_darurat_details_bulk` | Get details for multiple pengadaan darurat packages |
 | `get_procurement_categories` | List supported procurement categories (no network call) |
+| `get_master_klpd` | Get LKPP Satu Data master K/L/PD references (alternative data source) |
+| `get_master_lpse` | Get LKPP Satu Data master LPSE references (alternative data source) |
+| `get_tender_umum_publik` | Get tender data from LKPP ISB Satu Data (alternative data source / fallback) |
+| `set_ssl_verify` | Enable/disable TLS/SSL certificate verification for all SPSE requests |
+| `get_ssl_verify` | Return current SSL verification setting |
 | `validate_lpse_host` | Check if an LPSE host is accessible |
 | `create_procurement_search_index` | Download a bounded package set into a local SQLite full-text index |
 | `search_procurement_index` | Search a local SQLite full-text index |
@@ -253,6 +260,24 @@ The LLM should start with direct keyword search. If results are weak or the user
 
 ---
 
+## ISB Satu Data API (Alternative Data Source)
+
+In addition to direct realtime SPSE/Inaproc queries, PyProc also integrates with the **LKPP ISB (Indonesia Satu Data) API**. This is a curated, government-maintained dataset that provides:
+
+- **Master LPSE** — reference list of all LPSE units with their codes and names
+- **Master K/L/PD** — reference list of all government institutions (Kementerian, Lembaga, Perangkat Daerah)
+- **Tender Umum Publik** — public tender data indexed by LPSE code and budget year
+
+The ISB Satu Data API serves as an **alternative data source** when:
+
+- Direct SPSE/Inaproc queries fail or return errors (e.g., server timeouts, blocked hosts)
+- You need reference data (LPSE codes, institution codes) before making SPSE queries
+- The user explicitly wants the curated ISB dataset rather than realtime SPSE data
+
+Use the `satudata` CLI subcommand or the `get_master_lpse`, `get_master_klpd`, and `get_tender_umum_publik` MCP tools to access ISB data.
+
+---
+
 ## Example LLM Workflows
 
 Here are examples of what you can ask an LLM when PyProc MCP tools are connected:
@@ -317,6 +342,7 @@ The MCP server is configured via environment variables:
 | `PYPROC_TIMEOUT` | `30` | HTTP request timeout in seconds |
 | `PYPROC_RATE_LIMIT_DELAY` | `1.0` | Minimum seconds between requests |
 | `PYPROC_LOG_LEVEL` | `INFO` | Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) |
+| `PYPROC_SSL_VERIFY` | `0` | Enable SSL/TLS certificate verification (`1`, `true`, `yes`, `on` to enable) |
 
 ---
 
@@ -468,7 +494,9 @@ JenisPengadaan.JASA_LAINNYA               # Other Services
 
 ## CLI Usage
 
-### Basic Download
+### SPSE (Direct Realtime Data)
+
+Download procurement data directly from SPSE/Inaproc servers. The `spse` subcommand is the default.
 
 ```bash
 pyproc kemenkeu
@@ -480,7 +508,7 @@ This downloads tender data from the Kemenkeu LPSE and exports it as `kemenkeu.cs
 
 | Argument | Example | Default | Description |
 |---|---|---|---|
-| `lpse_host` | `pyproc kemenkeu` | Required | LPSE host or text file with host list |
+| `lpse_host` | `pyproc spse kemenkeu` | Required | LPSE host or text file with host list |
 | `-k, --keyword` | `--keyword "mobil dinas"` | `""` | Search keyword filter |
 | `-t, --tahun-anggaran` | `--tahun-anggaran 2025` | Current year | Budget year (single, comma-separated, or range) |
 | `--kategori` | `--kategori PEKERJAAN_KONSTRUKSI` | None | Procurement category |
@@ -501,10 +529,10 @@ This downloads tender data from the Kemenkeu LPSE and exports it as `kemenkeu.cs
 
 ```bash
 # Download from multiple LPSE hosts
-pyproc jakarta,kemenkeu,sumbarprov
+pyproc spse jakarta,kemenkeu,sumbarprov
 
 # With custom output filenames
-pyproc "jakarta;file_jakarta,kemenkeu;file_kemenkeu"
+pyproc spse "jakarta;file_jakarta,kemenkeu;file_kemenkeu"
 ```
 
 ### Download Host List
@@ -517,6 +545,71 @@ pyproc daftarlpse
 pyproc daftarhost
 ```
 
+### Master KLPD
+
+Query K/L/PD reference data from the LKPP ISB Satu Data API:
+
+```bash
+# Show all K/L/PD references
+pyproc masterklpd
+
+# Search by institution name or code
+pyproc masterklpd --query kemenkeu
+
+# Filter by jenis (K, L, PD)
+pyproc masterklpd --jenis K
+
+# Get specific K/L/PD by code
+pyproc masterklpd --kd-klpd K68
+
+# Limit results
+pyproc masterklpd --limit 5
+```
+
+### Satu Data (Alternative Source)
+
+Access tender data from the LKPP ISB Satu Data API as an alternative to direct realtime SPSE queries.
+
+#### Master LPSE (Interactive Browser)
+
+```bash
+# Launch interactive LPSE browser
+pyproc satudata masterlpse
+
+# With custom timeout
+pyproc satudata masterlpse --timeout 60
+
+# Specify output format and file
+pyproc satudata masterlpse --output csv --output-file hasil.csv
+```
+
+#### Tender Umum Publik (Direct Download)
+
+Download tender data for a specific LPSE code and budget year:
+
+```bash
+# Download tender data for a specific LPSE and year
+pyproc satudata tenderumum --kode-lpse 119 --tahun-anggaran 2026
+
+# With custom options
+pyproc satudata tenderumum --kode-lpse 119 --tahun-anggaran 2026 --timeout 60 --output json
+```
+
+**Required arguments for `tenderumum`**:
+
+| Argument | Example | Description |
+|---|---|---|
+| `--tahun-anggaran` | `2026` | Budget year |
+| `--kode-lpse` | `119` | LPSE code from master LPSE |
+
+**Optional arguments**:
+
+| Argument | Default | Description |
+|---|---|---|
+| `--timeout` | `30` | Request timeout in seconds |
+| `--output` | `json` | Output format: `json` or `csv` |
+| `--output-file` | Auto-generated | Output file path |
+
 ---
 
 ## Responsible Use and Disclaimer
@@ -582,11 +675,3 @@ pyproc/
 ## License
 
 MIT License. See [LICENSE](LICENSE) for details.
-
----
-
-## Donatur ☕️
-
-Orang-orang yang berjasa menyediakan kopi sehingga pengembangan paket tetap berjalan:
-
-- Angga Rinaldi Rizal (50 cangkir ☕️)

{pyproc-0.3.2 → pyproc-0.3.3}/README.md

@@ -11,7 +11,7 @@
 </p>
 
 <p align="center">
-  <a href="https://pypi.org/project/pyproc/"><img src="https://img.shields.io/badge/version-v0.3.2-blue" alt="PyPI version"></a>
+  <a href="https://pypi.org/project/pyproc/"><img src="https://img.shields.io/badge/version-v0.3.3-blue" alt="PyPI version"></a>
   <a href="https://pypi.org/project/pyproc/"><img src="https://img.shields.io/badge/python-≥3.9-yellow" alt="Python ≥3.9"></a>
   <a href="https://github.com/wakataw/pyproc/actions/workflows/test.yml?query=branch%3Amaster"><img src="https://github.com/wakataw/pyproc/actions/workflows/test.yml/badge.svg?branch=master" alt="Test Status"></a>
   <a href="https://github.com/wakataw/pyproc/blob/master/LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="License: MIT"></a>
@@ -125,18 +125,20 @@ Download procurement data in bulk from the command line.
 
 ```bash
 # Download tender data from Kemenkeu LPSE, export as JSON
-pyproc kemenkeu --keyword "mobil dinas" --tahun-anggaran 2025 --output-format json
+pyproc spse kemenkeu --keyword "mobil dinas" --tahun-anggaran 2025 --output-format json
 
 # Download non-tender data from multiple LPSE hosts
-pyproc jakarta,sumbarprov --jenis-paket non_tender --tahun-anggaran 2025
+pyproc spse jakarta,sumbarprov --jenis-paket non_tender --tahun-anggaran 2025
 
 # Download pencatatan non-tender data. This is distinct from non_tender.
-pyproc nasional --jenis-paket pencatatan_non_tender --tahun-anggaran 2026
+pyproc spse nasional --jenis-paket pencatatan_non_tender --tahun-anggaran 2026
 
 # Download with custom output filename
-pyproc "kemenkeu;output_kemenkeu" --output-format csv --separator ";"
+pyproc spse "kemenkeu;output_kemenkeu" --output-format csv --separator ";"
 ```
 
+The `spse` subcommand is the default, so `pyproc kemenkeu` is shorthand for `pyproc spse kemenkeu`.
+
 See [CLI Usage](#cli-usage) for the full argument reference.
 
 ---
@@ -166,6 +168,11 @@ The MCP server exposes the following tools:
 | `get_swakelola_details_bulk` | Get details for multiple swakelola packages |
 | `get_pengadaan_darurat_details_bulk` | Get details for multiple pengadaan darurat packages |
 | `get_procurement_categories` | List supported procurement categories (no network call) |
+| `get_master_klpd` | Get LKPP Satu Data master K/L/PD references (alternative data source) |
+| `get_master_lpse` | Get LKPP Satu Data master LPSE references (alternative data source) |
+| `get_tender_umum_publik` | Get tender data from LKPP ISB Satu Data (alternative data source / fallback) |
+| `set_ssl_verify` | Enable/disable TLS/SSL certificate verification for all SPSE requests |
+| `get_ssl_verify` | Return current SSL verification setting |
 | `validate_lpse_host` | Check if an LPSE host is accessible |
 | `create_procurement_search_index` | Download a bounded package set into a local SQLite full-text index |
 | `search_procurement_index` | Search a local SQLite full-text index |
@@ -218,6 +225,24 @@ The LLM should start with direct keyword search. If results are weak or the user
 
 ---
 
+## ISB Satu Data API (Alternative Data Source)
+
+In addition to direct realtime SPSE/Inaproc queries, PyProc also integrates with the **LKPP ISB (Indonesia Satu Data) API**. This is a curated, government-maintained dataset that provides:
+
+- **Master LPSE** — reference list of all LPSE units with their codes and names
+- **Master K/L/PD** — reference list of all government institutions (Kementerian, Lembaga, Perangkat Daerah)
+- **Tender Umum Publik** — public tender data indexed by LPSE code and budget year
+
+The ISB Satu Data API serves as an **alternative data source** when:
+
+- Direct SPSE/Inaproc queries fail or return errors (e.g., server timeouts, blocked hosts)
+- You need reference data (LPSE codes, institution codes) before making SPSE queries
+- The user explicitly wants the curated ISB dataset rather than realtime SPSE data
+
+Use the `satudata` CLI subcommand or the `get_master_lpse`, `get_master_klpd`, and `get_tender_umum_publik` MCP tools to access ISB data.
+
+---
+
 ## Example LLM Workflows
 
 Here are examples of what you can ask an LLM when PyProc MCP tools are connected:
@@ -282,6 +307,7 @@ The MCP server is configured via environment variables:
 | `PYPROC_TIMEOUT` | `30` | HTTP request timeout in seconds |
 | `PYPROC_RATE_LIMIT_DELAY` | `1.0` | Minimum seconds between requests |
 | `PYPROC_LOG_LEVEL` | `INFO` | Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) |
+| `PYPROC_SSL_VERIFY` | `0` | Enable SSL/TLS certificate verification (`1`, `true`, `yes`, `on` to enable) |
 
 ---
 
@@ -433,7 +459,9 @@ JenisPengadaan.JASA_LAINNYA               # Other Services
 
 ## CLI Usage
 
-### Basic Download
+### SPSE (Direct Realtime Data)
+
+Download procurement data directly from SPSE/Inaproc servers. The `spse` subcommand is the default.
 
 ```bash
 pyproc kemenkeu
@@ -445,7 +473,7 @@ This downloads tender data from the Kemenkeu LPSE and exports it as `kemenkeu.cs
 
 | Argument | Example | Default | Description |
 |---|---|---|---|
-| `lpse_host` | `pyproc kemenkeu` | Required | LPSE host or text file with host list |
+| `lpse_host` | `pyproc spse kemenkeu` | Required | LPSE host or text file with host list |
 | `-k, --keyword` | `--keyword "mobil dinas"` | `""` | Search keyword filter |
 | `-t, --tahun-anggaran` | `--tahun-anggaran 2025` | Current year | Budget year (single, comma-separated, or range) |
 | `--kategori` | `--kategori PEKERJAAN_KONSTRUKSI` | None | Procurement category |
@@ -466,10 +494,10 @@ This downloads tender data from the Kemenkeu LPSE and exports it as `kemenkeu.cs
 
 ```bash
 # Download from multiple LPSE hosts
-pyproc jakarta,kemenkeu,sumbarprov
+pyproc spse jakarta,kemenkeu,sumbarprov
 
 # With custom output filenames
-pyproc "jakarta;file_jakarta,kemenkeu;file_kemenkeu"
+pyproc spse "jakarta;file_jakarta,kemenkeu;file_kemenkeu"
 ```
 
 ### Download Host List
@@ -482,6 +510,71 @@ pyproc daftarlpse
 pyproc daftarhost
 ```
 
+### Master KLPD
+
+Query K/L/PD reference data from the LKPP ISB Satu Data API:
+
+```bash
+# Show all K/L/PD references
+pyproc masterklpd
+
+# Search by institution name or code
+pyproc masterklpd --query kemenkeu
+
+# Filter by jenis (K, L, PD)
+pyproc masterklpd --jenis K
+
+# Get specific K/L/PD by code
+pyproc masterklpd --kd-klpd K68
+
+# Limit results
+pyproc masterklpd --limit 5
+```
+
+### Satu Data (Alternative Source)
+
+Access tender data from the LKPP ISB Satu Data API as an alternative to direct realtime SPSE queries.
+
+#### Master LPSE (Interactive Browser)
+
+```bash
+# Launch interactive LPSE browser
+pyproc satudata masterlpse
+
+# With custom timeout
+pyproc satudata masterlpse --timeout 60
+
+# Specify output format and file
+pyproc satudata masterlpse --output csv --output-file hasil.csv
+```
+
+#### Tender Umum Publik (Direct Download)
+
+Download tender data for a specific LPSE code and budget year:
+
+```bash
+# Download tender data for a specific LPSE and year
+pyproc satudata tenderumum --kode-lpse 119 --tahun-anggaran 2026
+
+# With custom options
+pyproc satudata tenderumum --kode-lpse 119 --tahun-anggaran 2026 --timeout 60 --output json
+```
+
+**Required arguments for `tenderumum`**:
+
+| Argument | Example | Description |
+|---|---|---|
+| `--tahun-anggaran` | `2026` | Budget year |
+| `--kode-lpse` | `119` | LPSE code from master LPSE |
+
+**Optional arguments**:
+
+| Argument | Default | Description |
+|---|---|---|
+| `--timeout` | `30` | Request timeout in seconds |
+| `--output` | `json` | Output format: `json` or `csv` |
+| `--output-file` | Auto-generated | Output file path |
+
 ---
 
 ## Responsible Use and Disclaimer
@@ -547,11 +640,3 @@ pyproc/
 ## License
 
 MIT License. See [LICENSE](LICENSE) for details.
-
----
-
-## Donatur ☕️
-
-Orang-orang yang berjasa menyediakan kopi sehingga pengembangan paket tetap berjalan:
-
-- Angga Rinaldi Rizal (50 cangkir ☕️)

{pyproc-0.3.2 → pyproc-0.3.3}/pyproc/__init__.py

@@ -1,6 +1,6 @@
 from .lpse import Lpse, JenisPengadaan, TipeSwakelola, By, KontrakStatus
 
-__version__ = '0.3.2'
+__version__ = '0.3.3'
 __author__ = 'Agung Pratama'
 __all__ = [
     'Lpse',