inat-channel 0.8.2 → 0.9.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0c91fbb67ba53ef0148d0c9074551a3c8591cb05b6d3a578858018459213f8c4
-  data.tar.gz: b7cefeea01cf40c26574cee11f8ac62657a57435732ed095b3badeebf5c79709
+  metadata.gz: 4eb7d2097a0e7dfa5bb82a35bb44a2760796b60a2b7bc9e2005b40de25cb2eb4
+  data.tar.gz: 1e853788786d2e9a4c2dd260a3a1fd133be1e5ace65bf79f8866dbdb45b2bb59
 SHA512:
-  metadata.gz: dc896ae4f4a74f64364d1607aabdf2c6acc3d2a3e86e0d5d82e498f5da491201fba99d4c5f1ab769694365914874d401faa85dae97d951695895adfa074731b8
-  data.tar.gz: 67f3314b776a1352a0451570e9d6589abbe9467cfa44b1a124cf4f4d00ed2c7fda8db6060b2e7d2d0021dfde43858abc6484d87569b33b232300932e6f88dc8d
+  metadata.gz: 07ba71faf1531c5048799439424d335ee066371e55d7c09ae412a33f11b10b55c81e3cdcc71c2f1c705bf0f1c3c25a2210c55048096a3767a27ad302f2ec9f3e
+  data.tar.gz: d30cec087845059eb8a8ae0b8fb1a4ce804942900101262b5bb5d3c4173046bdfe5a286e5824066f8c0db078323ef2f8d25c421c49b42de05c230ef27a989773

data/README.md

@@ -1,150 +1,383 @@
-# iNat Telegram Poster
+| **EN** | [ru](README-ru.md) |
+|----------|----------|
+
+# inat-channel — automatic posting of observations
 
 [![GitHub License](https://img.shields.io/github/license/inat-get/inat-channel)](LICENSE)
-[![Gem Version](https://badge.fury.io/rb/inat-channel.svg?icon=si%3Arubygems&d=5)](https://badge.fury.io/rb/inat-channel)
+[![Gem Version](https://badge.fury.io/rb/inat-channel.svg?icon=si%3Arubygems&d=7)](https://badge.fury.io/rb/inat-channel)
 [![Ruby](https://github.com/inat-get/inat-channel/actions/workflows/ruby.yml/badge.svg)](https://github.com/inat-get/inat-channel/actions/workflows/ruby.yml) 
 ![Coverage](coverage-badge.svg)
 
-**Автоматический бот**, который ежедневно публикует в Telegram-каналы случайные, популярные наблюдения из iNaturalist, согласно гибким настройкам API-запросов.
+A script that sends a random iNaturalist observation from a selected sample to Telegram.
+
+## What does it do?
+
++ Obtains a sample via the [iNaturalist API](https://api.inaturalist.org/v2/docs/#/Observations/get_observations) based on an arbitrary query (supported by the API).
+
++ Sends a random observation from the sample to a specified Telegram channel, excluding those already sent. If there are no new observations in the fresh sample, it takes from a saved pool.
+
++ Saves unsent observations obtained via the query into the pool.
+
++ Taxon uniqueness and pool depth are configurable.
+
++ Sends a message to the administrator in case of failures.
+
+## Installation and running
+
+### Using Bundler
+
+Create a `Gemfile` with the single line:
+```
+gem 'inat-channel', '~> 0.9.0'
+```
+
+Run:
+```
+$ bundle install
+```
+
+Then use:
+```
+$ bundle exec inat-channel [options]
+```
+
+### Manually
+
+Install:
+```
+$ gem install inat-channel
+```
+
+Run:
+```
+$ inat-channel [options]
+```
 
-## Версия
+There are few command line parameters, and all are optional:
 
-+ **0.8.0** — *предварительный* релиз.
+```
+$ inat-channel --help
+Usage: inat-channel [options]
+    -c, --config FILE                Config file (default: inat-channel.yml)
+    -l, --log-level LEVEL            Log level (default: warn)
+        --debug                      Set log level to debug
+        --version                    Show version info and exit
+    -h, --help                       Show help and exit
+```
 
-## Основные возможности
+## Configuration
 
-- Поддержка гибких запросов iNaturalist API: проекты, таксоны, места, пользователи и др.
-- Очередь публикаций с приоритетом свежих наблюдений, затем пулом и архивом отправленных (без дубликатов).
-- Поддержка до 10 фотографий и геолокационных ссылок в одном посте.
-- Вывод эмодзи и хештегов, соответствующих иерархии таксонов.
-- Автоматическая генерация ссылок на проекты и места из настройки.
-- Механизм блокировки для безопасного параллельного запуска.
-- Автоматические повторы запросов, уведомления администратору, логирование.
+Main settings are described in a YAML configuration file. Optionally, you can specify an ERB message template and extract the `places` group of settings (see below) into a separate YAML file.
 
-## Быстрый старт
+Most settings have default values and may be omitted, but some are mandatory.
 
-```bash
-# 1. Установка
-bundle install
+### Environment variables
 
-# 2. Создайте конфиг (например, config.yaml)
-cat > config.yaml << EOF
+Additionally, **two mandatory parameters** must be set **in environment variables**: `TELEGRAM_BOT_TOKEN` is responsible for the Telegram bot token,
+which is issued during creation via [@BotFather](https://t.me/BotFather) — you need to add this bot (your bot) to your channel as an admin
+and give it rights to post messages; in the `ADMIN_TELEGRAM_ID` variable, specify your personal ID — notifications will be sent to it.
+You can find this ID using the bot [@Getmyid_bot](https://t.me/Getmyid_bot). These parameters cannot be set in the config file for security reasons.
+
+### Configuration file
+
+Example:
+```
 base_query:
-  project_id: 12345
+  project_id: 175821
+  locale: ru
   popular: true
+  photo_license: 'cc-by,cc-by-nc,cc-by-nd,cc-by-sa,cc-by-nc-nd,cc-by-nc-sa,cc0'
   quality_grade: research
+  
+days_back:
+  fresh: 30
+  pool: 180
+  sent: 181
+  used: 360
+  
+unique_taxon: priority
+
+
+lock_file:
+  path: data/data.lock
+  ttl: 300
+  
+data_files:
+  root: data
+  pool: data/pool.json
+  sent: data/sent.json
+  used: data/used.json
+  
+api:
+  retries: 5
+  interval: 1.0
+  randomness: 0.5
+  backoff: 2
+  page_delay: 1.0
+  per_page: 200
+  
+tg_bot:
+  retries: 5
+  interval: 1.0
+  randomness: 0.5
+  backoff: 2
+  chat_id: '@<my_test_channel>'
+  template: message.erb
+  desc_limit: 512
+  link_zoom: 12
+  
+log_level: info
+notify_level: warn
+
+
+places: places.yml
+```
+
+This is a fully working (if you fix `tg_bot.chat_id`) but somewhat verbose config. Let's break down the parameters.
+
+#### API query parameters
+
+```
+base_query:
+  project_id: 175821
   locale: ru
-days_back: 30
-chat_id: -1001234567890
-retries: 5
-EOF
+  popular: true
+  photo_license: 'cc-by,cc-by-nc,cc-by-nd,cc-by-sa,cc-by-nc-nd,cc-by-nc-sa,cc0'
+  quality_grade: research
+```
+
+The composition of these parameters can be arbitrary but at least one must be specified. Some notes:
+
++ As `project_id` you can specify not only a numeric ID but also the project `slug`, which is easier to obtain — part of its URL.
 
-# 3. Установите переменные окружения
-export TELEGRAM_BOT_TOKEN="ваш_токен_бота"
-export ADMIN_TELEGRAM_ID="ID_администратора_в_Telegram"
++ Locale affects taxon names; without it, English names are returned.
 
-# 4. Запуск
-bin/inat-channel -c config.yaml
++ *It is strongly recommended to specify photo licenses*, otherwise the script may collect observations whose photos you do not have rights to redistribute.
+
++ Observations on iNaturalist vary in quality. The parameters `quality_grade: research` and `popular: true` do not guarantee anything but tend to improve the sample on average.
+
+#### Recency parameters
 
-# 5. Настройте cron для ежедневного запуска
-echo "0 9 * * * cd /путь/к/проекту && bin/inat-channel -c config.yaml >> log/cron.log 2>&1" | crontab -
 ```
+days_back:
+  fresh: 30
+  pool: 180
+  sent: 181
+  used: 360
 
-## Конфигурация (пример)
+unique_taxon: priority
+```
 
-```yaml
-base_query:          # Параметры запроса к iNaturalist API (Hash)
-  project_id: 12345
-  popular: true
-  quality_grade: research
-  locale: ru
-days_back: 30        # Кол-во дней назад для фильтрации (integer > 0)
-chat_id: -1001234567890  # ID Telegram канала или группы
-retries: 5           # Кол-во повторных попыток запросов (API и Telegram)
+Group of parameters controlling recency and data lifetime.
+
++ `days_back.fresh` is *mandatory* and defines how many days of data are requested each run.
+  You might want to reduce it according to the script run frequency, but keep in mind that to achieve `research` status,
+  and especially to be counted as popular (means at least one user favorited the observation), some extra time is needed.
+  Though 30 days may be too much, at least a week is recommended.
+
++ `days_back.pool` controls the age of observations in the pool. The pool is used if no fresh observations are available or all are already sent.
+  Observations older (in days) than this are removed from the pool. Default is `days_back.fresh * 3`. This parameter also controls initial pool fill on first run.
+
++ `days_back.sent` controls how long the list of sent observations is kept. Age is counted from the send date. It is recommended to set it equal 
+   to `days_back.pool` or one day more, to avoid duplicates (they shouldn't get into the pool but just in case...). Default is `days_back.pool + 1`.
+
++ `days_back.used` controls how long information about *used taxa* is kept. Default is `365` days (1 year).
+
++ `unique_taxon` manages taxon uniqueness. Possible values:
 
-# Опционально — пути хранения данных (лучше использовать разные для параллельных запусков)
-pool_file: "data/pool.json"
-sent_file: "data/sent.json"
-lock_file: "data/bot.lock"
+  + `strict` — observations with the same taxon are not sent (and removed from pool);
+
+  + `priority` — observations with taxa never sent before get priority;
+
+  + `ignore` — uniqueness is ignored; taxon data is not saved.
+
+Default is `ignore`.
+
+#### File parameters
+
+Data is stored across runs in several JSON files. Also, to avoid concurrent runs corrupting data, parallel execution is forbidden, controlled by a lock file.
 
-# Автоссылки по place_ids
-places:
-  group:
-    - place_ids: [1, 2]
-      link: "https://inaturalist.org/projects/12345"
-      text: "Moscow Region Project"
+```
+lock_file:
+  path: data/data.lock
+  ttl: 300
+  
+data_files:
+  root: data
+  pool: data/pool.json
+  sent: data/sent.json
+  used: data/used.json
 ```
 
-## Запуск нескольких экземпляров
+The `lock_file` group defines the lock file name and its TTL in seconds — after that, the lock file is considered expired, allowing a new run. This prevents permanent blocking in case of crashes (though most errors should delete the lock file anyway).
 
-Можно использовать несколько конфигураций для различных проектов или регионов одновременно, указав для каждого отдельные `pool_file`, `sent_file` и `lock_file`.
+The `data_files` group sets data file locations. `data_files.root` can specify a root folder for all data files without individual paths. In the example, `root` is unused since absolute paths are given.
 
-```bash
-config/
-├── moscow.yaml       # данные: data/moscow_pool.json + moscow.lock
-└── spb.yaml          # данные: data/spb_pool.json + spb.lock
+#### Service parameters
 
-bin/inat-channel -c config/moscow.yaml &
-bin/inat-channel -c config/spb.yaml &
 ```
+api:
+  retries: 5
+  interval: 1.0
+  randomness: 0.5
+  backoff: 2
+  page_delay: 1.0
+  per_page: 200
+  
+tg_bot:
+  retries: 5
+  interval: 1.0
+  randomness: 0.5
+  backoff: 2
+  chat_id: '@<my_test_channel>'
+  template: message.erb
+  desc_limit: 512
+  link_zoom: 12
+```  
+
+These two groups control interaction with **[iNaturalist API v2](https://api.inaturalist.org/v2/docs/)** and **[Telegram Bot API](https://core.telegram.org/bots/api)**.
+
+The first four keys in each group are identical and configure [`faraday-retry`](https://github.com/lostisland/faraday-retry), handling retries on errors. Defaults shown above can be changed for fine tuning but usually aren't needed.
+
+iNaturalist API has rate limits; `api.page_delay` adds a delay between normal requests (not retries). Parameter `api.per_page` sets the number of items per request; 200 is the maximum. Adjust these only if iNaturalist changes API conditions.
+
+Additional parameters in `tg_bot`:
 
-Важно: файлы пула и отправленных должны быть уникальными для каждой конфигурации, чтобы избежать гонок и сбоев.
++ `chat_id` — channel ID where messages will be sent.
 
-## Структура директорий и данных
++ `template` — [ERB](https://docs.ruby-lang.org/en/3.4/ERB.html) message template, explained below.
+
++ `desc_limit` — limits the `description` field length to fit Telegram limits. Reduce it if your template adds more text.
+
++ `link_zoom` — zoom level applied in map service links.
+
+#### Logging and notifications
+
+Two parameters:
 
 ```
-├── config.yaml        # Конфигурация (пример)
-├── data/
-│   ├── pool.json      # Кэш новых UUID объектов для публикации
-│   ├── sent.json      # UUID уже опубликованных объектов + id сообщений Telegram
-│   └── bot.lock       # Лок-файл блокировки работы бота
-├── log/               # Логи запуска бота
-└── bin/inat-channel   # Основной исполняемый файл бота
+log_level: info
+notify_level: warn
 ```
 
-## Защита от параллельных запусков
++ `log_level` replicates command line `--log-level` (command line has priority).
 
-- Lock-файл с TTL 30 минут.
-- Автоудаление устаревших блокировок.
-- Завершение по сигналам INT и TERM (graceful shutdown).
-- Ошибка, если бот уже запущен с тем же конфигом.
++ `notify_level` similarly controls admin notification levels. Currently not implemented; notification cases are hardcoded.
 
-Пример:
+#### Places
 
-```bash
-$ bin/inat-channel -c config.yaml    # PID 12345 захватил lock
-$ bin/inat-channel -c config.yaml    # Ошибка: процесс с PID 12345 уже запущен
 ```
+places: places.yml
+```
+
+In the example, places are extracted to a separate file. This is handy because place settings can be large and reusable across configs.
 
-## Пример поста в Telegram
+Example `places.yml` looks like:
 
 ```
-🪶 <b>Обыкновенный снегирь</b> <i>(Pyrrhula pyrrhula)</i>
+regions:
+  - place_ids: [ 139490 ]
+    link: https://www.inaturalist.org/places/139490
+    text: Sverdlovsk Oblast
+    tag: SV
+  - place_ids: [ 139365 ]
+    link: https://www.inaturalist.org/places/139365
+    text: Republic of Bashkortostan
+    tag: BK
+  - place_ids: [ 139506 ]
+    link: https://www.inaturalist.org/places/139506
+    text: Chelyabinsk Oblast
+    tag: CL
+  - place_ids: [ 139361 ]
+    link: https://www.inaturalist.org/places/139361
+    text: Perm Krai
+    tag: PE
+  - place_ids: [ 139358 ]
+    link: https://www.inaturalist.org/places/139358
+    text: Orenburg Oblast
+    tag: OB
+  - place_ids: [ 12867 ]
+    link: https://www.inaturalist.org/places/12867
+    text: Khanty-Mansi Autonomous Okrug
+    tag: KM
+  - place_ids: [ 11809 ]
+    link: https://www.inaturalist.org/places/11809
+    text: Komi Republic
+    tag: KO
+  - place_ids: [ 13219 ]
+    link: https://www.inaturalist.org/places/13219
+    text: Yamalo-Nenets Autonomous Okrug
+    tag: YN
+```
+
+A few words on why this is needed. iNaturalist defines many territories, each observation belongs to several. These places vary in quality and purpose of tagging, so it's uninteresting to show them all. But some are worth highlighting. For that, we specify a selected list.
+
+The list is split into groups. The example has one group — `regions`, but there can be several: e.g., one for administrative/municipal units, another for protected areas, parks, etc. Options vary.
 
-📷 #123456 — 👤 <a href="...">Ivan Ivanov</a> @ 📅 2025-11-15
+The script iterates over this list and, if it finds an intersection between the observation's `place_ids` and the list's `place_ids`, it adds a corresponding item (with `link`, `text`, and `tag`) internally for the template. From each group, only one place max is added; groups are independent. So the max places listed equals the number of groups configured.
 
-🗺️ <a href="...">Moscow Region Project</a>
+If you do not use a separate file, the whole structure should be nested under the `places` parameter with proper indentation.
 
-↳ 🗺️ 55.7558°N, 37.6173°E
+## Message template
+
+As mentioned, you can specify an ERB message template in the config. If not provided, a builtin default template is used:
 
-#Animalia • #Aves • #Pyrrhula_pyrrhula
 ```
+<%= taxon.icon %> <a href="<%= taxon.url %>"><%= taxon.title %></a>
+
 
-## Опции командной строки
+<%= observation.icon %> <%= observation.url %>
+<%= datetime.icon %> <%= datetime %>
+<%= user.icon %> <a href="<%= user.url %>"><%= user.title %></a>
+<% if observation.description -%>
+<blockquote><%= observation.description.text %></blockquote>
+<% end -%>
 
-```bash
-bin/inat-channel --help
 
-# Доступные параметры:
-# -c, --config FILE     Путь к конфигу (по умолчанию inat-channel.yaml)
-# -l, --log-level LEVEL Уровень логирования (debug/info/warn/error)
-# --debug               Установить уровень логирования в debug
+<%= location.icon %> <%= location.dms %> -  <a href="<%= location.google %>">G</a> <a href="<%= location.osm %>">OSM</a>
+<% if places && places.size > 0 -%>
+<%   places.each do |place| -%>
+<%= place.icon %> <a href="<%= place.link %>"><%= place.text %></a> <%= '• #' + place.tag if place.tag %>
+<%   end -%>
+<% else -%>
+<%= icons[:place] %> <%= observation.place_guess %>
+<% end -%>
+
+
+<%= taxon.to_tags&.join(' • ') %>
 ```
 
-## Благодарности
+[<img align="right" width="40%" src="example.png">](example.png)
+
+This template uses many emojis, which may look gaudy but makes it language-independent. You will most likely want to use your own template for your channel.
+[An example](https://github.com/inat-get/channel-ural/blob/main/message.erb) is available in the [`channel-ural`](https://github.com/inat-get/channel-ural) project.
+
+The template variables (standard or immutable data classes) are:
+
++ `observation`: `Observation`  
++ `datetime`: `DateTime` = `observation.datetime`  
++ `date`: `Date` = `observation.date` (= `datetime.to_date`)  
++ `location`: `Location` = `observation.location`  
++ `places`: `Array<Place>` = `observation.places`  
++ `taxon`: `Taxon` = `observation.taxon`  
++ `user`: `User` = `observation.user`  
++ `icons`: `Hash` — emoji set for icons  
++ `taxa_icons`: `Hash` — emoji set for taxon icons, selected bottom-up by hierarchy  
++ `data`: `Hash` — template data — you can specify front-matter in YAML format for your custom template, passed here; you can also override `icons` and `taxa_icons`.  
++ `config`: `Hash` — configuration data
+
+Data class definitions are in [`data_types.rb`](https://github.com/inat-get/inat-channel/blob/next/lib/inat-channel/data_types.rb).
+
+## Usage example
+
+The script fully supports GitHub Actions. See how it works in the [`inat-get/channel-ural`](https://github.com/inat-get/channel-ural) project.
+It contains four configs and four workflows that run them, to post different observation categories at different times.
+
+## Version
+
++ **0.9.0** can be considered a beta or pre-release version. After some improvements, version 1.0 will be released.
 
-- [iNaturalist API v2](https://www.inaturalist.org/pages/api+reference)
-- [Faraday HTTP Client](https://github.com/lostisland/faraday)
+## License
 
-**Лицензия**: [GPLv3](LICENSE)
++ **[GNU GPL v3+](LICENSE)**,
 

data/bin/inat-channel

@@ -1,29 +1,28 @@
 #!/usr/bin/ruby
 require_relative '../lib/inat-channel'
-include INatChannel
+include IC
 
-INCh::LOGGER.info "=== iNatChannel Run ==="
+logger.info "=== iNatChannel Run ==="
 
-news = INCh::API::load_news
-INCh::LOGGER.info "Found #{news.size} fresh UUIDs (days_back=#{CONFIG[:days_back]})"
+news = load_news
+logger.info "Found #{ news.size } fresh UUIDs (days_back=#{ CONFIG.dig :days_back, :fresh })"
 
-uuid = INCh::Data::select_uuid news
+uuid = select_uuid news
 
 if uuid
-  obs = INCh::API::load_observation(uuid)
+  obs = load_observation(uuid)
   if obs
-    msg_id = INCh::Telegram::send_observation(obs)
-    INCh::Data::sent[uuid] = { msg_id: msg_id, sent_at: Time.now.to_s }
-    INCh::LOGGER.info "Posted #{obs[:id]} (#{INCh::Message::list_photos(obs).size} photos)"  # очень плохо, не должно быть тут вызова list_photos
+    msg_id = send_observation(obs)
+    IC::confirm_sending! msg_id
   else
-    INCh::LOGGER.warn "Failed to load #{uuid} — returning to pool"
-    INCh::Data::pool << uuid 
-    INCh::Telegram::notify_admin "Failed to load #{uuid} — returning to pool"
+    IC::revert_sending!
+    logger.warn "Failed to load #{uuid} — returning to pool"
+    notify_admin "Failed to load #{uuid} — returning to pool"
   end
 
-  INCh::Data::save
+  save_data
 else
-  INCh::Telegram::notify_admin 'No observation to send.'
-  INCh::LOGGER.warn 'No observation to send.'
+  notify_admin 'No observation to send.'
+  logger.warn 'No observation to send.'
 end