@danhachuel/thunderbolt 0.3.73 → 0.3.75
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/MANUAL-INSTALACAO.md +13 -5
- package/README.md +9 -1
- package/app/main.py +39 -17
- package/hermes_ui/automation_worker.py +9 -0
- package/hermes_ui/blueprints.py +4 -3
- package/hermes_ui/cuts.py +2 -1
- package/hermes_ui/domain.py +13 -1
- package/hermes_ui/notifications.py +2 -0
- package/hermes_ui/pipeline_worker.py +48 -7
- package/hermes_ui/provider_routing.py +3 -5
- package/hermes_ui/storage.py +11 -0
- package/integrations/session_info_health.py +238 -0
- package/integrations/upload_routing.py +11 -2
- package/integrations/youtube_direct_credentials.py +22 -11
- package/integrations/youtube_direct_upload.py +14 -1
- package/package.json +1 -1
package/MANUAL-INSTALACAO.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Este manual descreve a instalação local da UI Thunderbolt, baseada no MoneyPrinterTurbo, utilizando o pacote npm `@danhachuel/thunderbolt`. O fluxo recomendado instala automaticamente o ambiente Python, as dependências da aplicação, as dependências do MoneyPrinterTurbo, o Streamlit e o suporte FFmpeg através de `imageio-ffmpeg`.
|
|
4
4
|
|
|
5
|
-
> **Versão deste manual:** 0.3.
|
|
5
|
+
> **Versão deste manual:** 0.3.75
|
|
6
6
|
> **Pacote npm:** `@danhachuel/thunderbolt`
|
|
7
7
|
> **Porta padrão da UI:** `localhost:3030`
|
|
8
8
|
> **Repositório:** [github.com/DanHachuel/thunderbolt](https://github.com/DanHachuel/thunderbolt)
|
|
@@ -132,13 +132,13 @@ Execute:
|
|
|
132
132
|
Windows PowerShell ou MobaXterm:
|
|
133
133
|
|
|
134
134
|
```powershell
|
|
135
|
-
npx.cmd --yes --prefer-online @danhachuel/thunderbolt@0.3.
|
|
135
|
+
npx.cmd --yes --prefer-online @danhachuel/thunderbolt@0.3.75 install
|
|
136
136
|
```
|
|
137
137
|
|
|
138
138
|
Linux/macOS:
|
|
139
139
|
|
|
140
140
|
```bash
|
|
141
|
-
npx --yes --prefer-online @danhachuel/thunderbolt@0.3.
|
|
141
|
+
npx --yes --prefer-online @danhachuel/thunderbolt@0.3.75 install
|
|
142
142
|
```
|
|
143
143
|
|
|
144
144
|
A instalação normal é **segura para actualizações**: preserva `storage`, Blueprints, Brandings, configurações e artefactos do utilizador. Remove apenas `.venv`, o clone técnico do MoneyPrinterTurbo e dependências que serão recriadas. Uma pasta antiga sem dados do utilizador, como `C:\Users\<utilizador>\AppData\Local\hermes` da tentativa incompleta, pode ser removida; uma pasta antiga que contenha Blueprints, Brandings ou storage é preservada e apenas avisada no terminal. Feche processos Python, Node, Streamlit e MobaXterm que estejam a usar as pastas antes de executar.
|
|
@@ -146,7 +146,7 @@ A instalação normal é **segura para actualizações**: preserva `storage`, Bl
|
|
|
146
146
|
Se quiser apagar absolutamente tudo de forma intencional, use o comando destrutivo separado:
|
|
147
147
|
|
|
148
148
|
```powershell
|
|
149
|
-
npx.cmd --yes --prefer-online @danhachuel/thunderbolt@0.3.
|
|
149
|
+
npx.cmd --yes --prefer-online @danhachuel/thunderbolt@0.3.75 install --purge-data
|
|
150
150
|
```
|
|
151
151
|
|
|
152
152
|
O parâmetro `--purge-data` apaga Blueprints, Brandings, configurações, storage e artefactos locais. Não o use numa actualização normal.
|
|
@@ -574,7 +574,7 @@ Ao abrir a página, o Thunderbolt não prepara dados públicos, não descarrega
|
|
|
574
574
|
|
|
575
575
|
Os parâmetros da UI são número de clusters entre 2 e 10, suporte mínimo entre 0,01 e 0,50, país, engagement, intervalo de datas e tags, todos dentro da área principal da aba. O núcleo normaliza os dados, calcula engagement, aplica filtros, faz transformação logarítmica e standardização, executa K-Means e calcula itemsets/regras com FP-Growth. Não são apresentados resultados até ao primeiro clique em **Analisar Nichos**; o mesmo botão aplica alterações posteriores aos filtros. Os resultados são DataFrames de clusters, itemsets frequentes, regras de associação e dados analisados; o gráfico de dispersão é criado nativamente com Plotly.
|
|
576
576
|
|
|
577
|
-
As dependências adicionais — `scikit-learn`, `mlxtend`, `plotly`, `seaborn`, `matplotlib` e `kagglehub` — são instaladas pelo procedimento normal de `npx`. Em instalações existentes, execute novamente `npx.cmd --yes --prefer-online @danhachuel/thunderbolt@0.3.
|
|
577
|
+
As dependências adicionais — `scikit-learn`, `mlxtend`, `plotly`, `seaborn`, `matplotlib` e `kagglehub` — são instaladas pelo procedimento normal de `npx`. Em instalações existentes, execute novamente `npx.cmd --yes --prefer-online @danhachuel/thunderbolt@0.3.75 install`; o instalador detecta e reutiliza o que já estiver válido.
|
|
578
578
|
|
|
579
579
|
### Niche Finder Apify
|
|
580
580
|
|
|
@@ -803,3 +803,11 @@ A publicação YouTube segue primeiro a lógica adaptada do `PublishingSchedulin
|
|
|
803
803
|
|
|
804
804
|
### Growth — novas áreas reservadas
|
|
805
805
|
O grupo **Growth** inclui agora as abas **Analista Facebook Pages** e **Analista Bilibili**. Ambas são páginas vazias nesta versão e não executam operações nem pedem credenciais.
|
|
806
|
+
|
|
807
|
+
## 15. Documentação técnica e execução retomável
|
|
808
|
+
Consulte [`docs/api-internal.md`](docs/api-internal.md) no repositório GitHub para os contratos internos, diagramas de sequência, escritas atómicas de JSON, health check de `sessionInfo` e orquestrador em cascata.
|
|
809
|
+
|
|
810
|
+
O worker local guarda a etapa actual e os artefactos de cada tarefa. Ao reiniciar, retoma a primeira etapa pendente; não deve regenerar ficheiros válidos já persistidos. O `sessionInfo` do Upload directo é monitorizado com uma janela preventiva de 24–48 horas, por defeito 36 horas. A renovação é manual em **Contas Google**.
|
|
811
|
+
|
|
812
|
+
### Selector de modelos LLM
|
|
813
|
+
No card **OpenAI / NVIDIA NIM**, o campo **Modelo** é uma lista suspensa. Use **Consultar modelos** para actualizar as opções do endpoint; se o identificador não estiver disponível, escolha **Escrever modelo manualmente**.
|
package/README.md
CHANGED
|
@@ -107,7 +107,7 @@ No topo da área principal da aplicação existe o menu nativo de idioma no padr
|
|
|
107
107
|
|
|
108
108
|
A UI suporta os temas **Dark** e **Light** através do menu nativo de três pontos do Streamlit, no local original do toolbar. Não existe um selector Theme adicional dentro da página. A configuração distribuída em `.streamlit/config.toml` disponibiliza as variantes nomeadas **Dark** e **Light**, e o menu nativo continua responsável por alternar entre os modos, seguindo o padrão do [MoneyPrinterTurbo](https://github.com/harry0703/MoneyPrinterTurbo). O CSS próprio do Thunderbolt usa cores semânticas, `currentColor` e `color-mix` para acompanhar o tema activo, sem alterar a posição nem a funcionalidade do toolbar, do botão Deploy e do menu principal.
|
|
109
109
|
|
|
110
|
-
## Navegação da UI 0.3.
|
|
110
|
+
## Navegação da UI 0.3.75
|
|
111
111
|
|
|
112
112
|
A barra lateral mantém os níveis principais, nesta ordem: **Início**, **Niche Finder**, **Pipeline**, **Pipeline TikTok**, **Automação**, **Edição**, **AI Influencers** e **Configurações**. **Pipeline Vídeos** é expansível e contém **Criação de Vídeos**, **Backlog Vídeos**, **Roteiros**, **Thumbnails** e **Upload**. **Pipeline Música** é expansível e contém **Criação de Músicas** e **Upload Música**. **Automação** também é expansível e contém **Automação Youtube**. **Edição** é expansível e contém **Limpador de Metadados**, **Cortes**, **Editor Python** e **Download Mídia**, nessa ordem. **AI Influencers** é expansível e contém **Personagens**, **Geração de Conteúdo IA**, **Motion Control**, **UGC Products**, **Redes Sociais**, **Tutorial Meta** e **Tutorial Supabase**, nessa ordem. A criação de Motion Control e UGC Products ocorre nas páginas standalone e guarda os artefactos localmente; não existem tabs duplicadas dentro de Geração de Conteúdo IA.
|
|
113
113
|
**Niche Finder** é expansível e contém **Niche Finder Kaggle** e **Niche Finder Apify**. **Configurações** é expansível e contém **Canais Youtube**, **Blueprints Youtube**, **MCP**, **Contas Google**, **Notificações**, **Logs** e **Configuração API**, nessa ordem. O Início reúne o dashboard e as filas do Pipeline, sem botões de acções rápidas.
|
|
@@ -418,3 +418,11 @@ Não coloque chaves, cookies, tokens YouTube ou segredos TikTok no Git. Use a co
|
|
|
418
418
|
|
|
419
419
|
## Growth
|
|
420
420
|
As páginas **Analista Facebook Pages** e **Analista Bilibili** foram adicionadas ao grupo **Growth** como áreas reservadas e permanecem vazias nesta versão, prontas para futuras implementações específicas.
|
|
421
|
+
|
|
422
|
+
## Documentação técnica interna
|
|
423
|
+
A documentação de API interna, os contratos de persistência, o health check de `sessionInfo` e os diagramas de sequência estão em [`docs/api-internal.md`](docs/api-internal.md). Os ficheiros Mermaid editáveis ficam em [`docs/diagrams/`](docs/diagrams/).
|
|
424
|
+
|
|
425
|
+
A pipeline do worker usa agora um orquestrador local em cascata, com artefactos retomáveis e transições persistidas. As escritas de estado JSON e de Blueprints usam substituição atómica para evitar documentos truncados após interrupções.
|
|
426
|
+
|
|
427
|
+
## Selector de modelos LLM
|
|
428
|
+
O campo **Modelo** em **Configuração API > API Keys > LLM — providers e modelos** é apresentado como lista suspensa permanente. A lista usa os modelos descobertos pelo endpoint e preserva o modelo guardado; a opção manual continua disponível apenas como fallback explícito.
|
package/app/main.py
CHANGED
|
@@ -26,7 +26,7 @@ from hermes_ui.domain import STAGES, create_batch, create_channel, create_tasks_
|
|
|
26
26
|
from hermes_ui.drafts import list_drafts, save_draft
|
|
27
27
|
from hermes_ui.automation_worker import load_worker_status
|
|
28
28
|
from hermes_ui.pipeline_worker import load_pipeline_worker_status, recover_stale_tasks, STALE_TASK_SECONDS, WORKER_HEARTBEAT_TIMEOUT_SECONDS
|
|
29
|
-
from hermes_ui.storage import BLUEPRINTS, DEFAULT_LLM_PROVIDER, MEDIA_DOWNLOADS, STORAGE, TIKTOK_PROMPT_MASTERS, ensure_storage, get_display_name, list_blueprint_files, list_prompt_master_files, load_blueprint_file, load_prompt_master_file, now, read_json, set_display_name, write_json
|
|
29
|
+
from hermes_ui.storage import BLUEPRINTS, DEFAULT_LLM_PROVIDER, MEDIA_DOWNLOADS, STORAGE, TIKTOK_PROMPT_MASTERS, atomic_write, ensure_storage, get_display_name, list_blueprint_files, list_prompt_master_files, load_blueprint_file, load_prompt_master_file, now, read_json, set_display_name, write_json
|
|
30
30
|
from app.modules.niche_finder.apify import ApifyError, DEFAULT_ACTOR_ID, abort_actor_run, build_actor_input, get_dataset_items, normalize_video_items, start_actor_run, wait_for_actor_run
|
|
31
31
|
from app.modules.niche_finder.core import NicheAnalysisError, run_niche_analysis
|
|
32
32
|
from app.modules.niche_finder.data_loader import DatasetError, download_kaggle_dataset
|
|
@@ -76,6 +76,7 @@ from integrations.music_uploads import JewelMusicAdapter, PushtunesAdapter, YTMu
|
|
|
76
76
|
from integrations.upload_routing import OFFICIAL_DAILY_LIMIT, official_upload_count, upload_with_default_route
|
|
77
77
|
from integrations.youtube_direct_upload import YouTubeDirectUploader
|
|
78
78
|
from integrations.youtube_direct_credentials import delete_credentials_document, direct_account_status, document_status, ensure_credentials_document, load_credentials_document, merge_credentials_document, parse_credentials_document, save_credentials_document, update_credentials_document_session_info
|
|
79
|
+
from integrations.session_info_health import check_account_session_info_health
|
|
79
80
|
from integrations.youtube_batch import account_key as youtube_batch_account_key, account_status as youtube_batch_account_status, authorize_account as authorize_youtube_batch_account, delete_account_token as delete_youtube_batch_token, list_my_channels as list_youtube_batch_channels, loopback_redirect_uri
|
|
80
81
|
from integrations.local_runtime import MoneyPrinterRuntime
|
|
81
82
|
from integrations.moneyprinter_config import sync_moneyprinter_config
|
|
@@ -1353,7 +1354,7 @@ def render_blueprints():
|
|
|
1353
1354
|
if destination.exists() and not st.checkbox("Confirmar substituição", key="confirm_blueprint_replace"):
|
|
1354
1355
|
st.warning("O ficheiro já existe. Confirme a substituição.")
|
|
1355
1356
|
else:
|
|
1356
|
-
destination
|
|
1357
|
+
atomic_write(destination, data)
|
|
1357
1358
|
record_notification("blueprint_completed", f"Blueprint guardado: {destination.stem}", "O Blueprint importado foi guardado no storage local.", metadata={"name": destination.stem}, dedupe_key=f"blueprint:{destination}:{destination.stat().st_mtime_ns}")
|
|
1358
1359
|
st.success(f"Blueprint guardado em {destination}")
|
|
1359
1360
|
st.rerun()
|
|
@@ -1393,7 +1394,7 @@ def render_blueprints():
|
|
|
1393
1394
|
raise ValueError("O JSON raiz deve ser um objecto.")
|
|
1394
1395
|
target = BLUEPRINTS / "brandings" / (Path(branding_upload.name).stem.replace(" ", "-") + ".json")
|
|
1395
1396
|
target.parent.mkdir(parents=True, exist_ok=True)
|
|
1396
|
-
target
|
|
1397
|
+
atomic_write(target, data)
|
|
1397
1398
|
record_notification("branding_completed", f"Branding guardado: {target.stem}", "O Branding importado foi guardado no storage local.", metadata={"name": target.stem}, dedupe_key=f"branding:{target}:{target.stat().st_mtime_ns}")
|
|
1398
1399
|
st.success(f"Branding guardado em {target}")
|
|
1399
1400
|
st.rerun()
|
|
@@ -4966,6 +4967,7 @@ def render_google_accounts():
|
|
|
4966
4967
|
account_label_snapshot = str(batch_account.get("label") or "Canais YouTube")
|
|
4967
4968
|
ensure_credentials_document(STORAGE, batch_account, settings, channel_state)
|
|
4968
4969
|
direct_status = direct_account_status(STORAGE, batch_account, settings)
|
|
4970
|
+
session_health = check_account_session_info_health(STORAGE, batch_account, settings)
|
|
4969
4971
|
missing_document_parts = list(direct_status.get("missing_cookies", []))
|
|
4970
4972
|
if not direct_status.get("has_session_info"):
|
|
4971
4973
|
missing_document_parts.append("sessionInfo")
|
|
@@ -4985,6 +4987,15 @@ def render_google_accounts():
|
|
|
4985
4987
|
st.subheader(f"{account_label_snapshot} — {account_email_snapshot}")
|
|
4986
4988
|
with account_header_cols[1]:
|
|
4987
4989
|
_api_status_badge("Configured" if account_ready else "Missing configuration", "ready" if account_ready else "missing")
|
|
4990
|
+
health_message = f"SessionInfo: {session_health.message}"
|
|
4991
|
+
if session_health.status == "healthy":
|
|
4992
|
+
st.caption(health_message)
|
|
4993
|
+
elif session_health.status == "expiring":
|
|
4994
|
+
st.warning(health_message, icon="⚠️")
|
|
4995
|
+
elif session_health.status == "expired":
|
|
4996
|
+
st.error(health_message, icon="⚠️")
|
|
4997
|
+
elif session_health.status == "unknown":
|
|
4998
|
+
st.warning(health_message, icon="⚠️")
|
|
4988
4999
|
with st.expander("Detalhes da conta Google", expanded=False):
|
|
4989
5000
|
with st.form(f"batch_account_form_{account_id}"):
|
|
4990
5001
|
account_cols = st.columns(2)
|
|
@@ -5428,20 +5439,31 @@ def _render_llm_card(settings: dict[str, Any], cards: list[dict[str, Any]], inde
|
|
|
5428
5439
|
api_key = ""
|
|
5429
5440
|
with model_col:
|
|
5430
5441
|
model_catalog = st.session_state.get(f"llm_model_catalog_{card_id}", [])
|
|
5431
|
-
|
|
5432
|
-
|
|
5433
|
-
|
|
5434
|
-
|
|
5435
|
-
|
|
5436
|
-
|
|
5437
|
-
|
|
5438
|
-
|
|
5439
|
-
|
|
5440
|
-
|
|
5441
|
-
|
|
5442
|
-
|
|
5443
|
-
|
|
5444
|
-
|
|
5442
|
+
if not isinstance(model_catalog, list):
|
|
5443
|
+
model_catalog = []
|
|
5444
|
+
current_model = str(card.get("model") or "").strip()
|
|
5445
|
+
manual_model = "__manual_model__"
|
|
5446
|
+
discovered_models = [str(item).strip() for item in model_catalog if str(item).strip()]
|
|
5447
|
+
options = list(dict.fromkeys(discovered_models))
|
|
5448
|
+
# Mantém modelos guardados mesmo quando a descoberta ainda não foi
|
|
5449
|
+
# executada ou quando o provider deixou de os devolver temporariamente.
|
|
5450
|
+
if current_model and current_model not in options:
|
|
5451
|
+
options.insert(0, current_model)
|
|
5452
|
+
options.append(manual_model)
|
|
5453
|
+
selected = st.selectbox(
|
|
5454
|
+
"Modelo",
|
|
5455
|
+
options,
|
|
5456
|
+
index=options.index(current_model) if current_model in options else len(options) - 1,
|
|
5457
|
+
format_func=lambda value: "Escrever modelo manualmente" if value == manual_model else value,
|
|
5458
|
+
help="Seleccione um modelo descoberto pelo provider. Use a opção manual apenas quando o endpoint exigir um identificador que não esteja na lista.",
|
|
5459
|
+
key=f"llm_card_{card_id}_model_select",
|
|
5460
|
+
)
|
|
5461
|
+
model = st.text_input(
|
|
5462
|
+
"Modelo manual",
|
|
5463
|
+
value=current_model if selected == manual_model and current_model not in discovered_models else "",
|
|
5464
|
+
help="ID do modelo usado pelo endpoint Chat Completions.",
|
|
5465
|
+
key=f"llm_card_{card_id}_model_manual",
|
|
5466
|
+
) if selected == manual_model else selected
|
|
5445
5467
|
base_url = str(card.get("base_url") or definition.default_base_url)
|
|
5446
5468
|
endpoint_col, action_col = st.columns([1.65, 1.05])
|
|
5447
5469
|
with endpoint_col:
|
|
@@ -12,6 +12,7 @@ from .creative_generation import generate_title_and_keywords, generate_topic_for
|
|
|
12
12
|
from .domain import create_batch, create_tasks_for_batch
|
|
13
13
|
from .material_sources import selected_material_source
|
|
14
14
|
from .notifications import record_notification
|
|
15
|
+
from integrations.session_info_health import check_all_accounts_session_info_health, emit_session_info_health_alerts
|
|
15
16
|
|
|
16
17
|
WORKER_STATE_FILE = "automation_worker.json"
|
|
17
18
|
LOCK_FILENAME = "automation_worker.lock"
|
|
@@ -233,6 +234,14 @@ def run_once(when: datetime | None = None) -> dict[str, Any]:
|
|
|
233
234
|
status["last_tick_local"] = _local_iso(current)
|
|
234
235
|
status["last_heartbeat_local"] = _local_iso(current)
|
|
235
236
|
status["last_error"] = ""
|
|
237
|
+
try:
|
|
238
|
+
session_health = check_all_accounts_session_info_health(storage.STORAGE, storage.read_json("settings.json", {}))
|
|
239
|
+
session_alerts = emit_session_info_health_alerts(session_health, now=current.astimezone())
|
|
240
|
+
status["session_info_health"] = [item.as_dict() for item in session_health]
|
|
241
|
+
status["session_info_alerts"] = len(session_alerts)
|
|
242
|
+
except Exception as exc:
|
|
243
|
+
# Session monitoring must never stop creation of a scheduled batch.
|
|
244
|
+
status["session_info_health_error"] = type(exc).__name__
|
|
236
245
|
created: list[dict[str, Any]] = []
|
|
237
246
|
skipped: list[str] = []
|
|
238
247
|
channels = storage.read_json("channels.json", [])
|
package/hermes_ui/blueprints.py
CHANGED
|
@@ -6,7 +6,7 @@ from datetime import datetime, timezone
|
|
|
6
6
|
from pathlib import Path
|
|
7
7
|
from typing import Any
|
|
8
8
|
|
|
9
|
-
from .storage import BLUEPRINTS, now
|
|
9
|
+
from .storage import BLUEPRINTS, atomic_write, now
|
|
10
10
|
|
|
11
11
|
|
|
12
12
|
def _slug(value: str) -> str:
|
|
@@ -140,15 +140,16 @@ def create_branding_for_blueprint(blueprint: dict[str, Any]) -> dict[str, Any]:
|
|
|
140
140
|
|
|
141
141
|
|
|
142
142
|
def save_generated_blueprint(blueprint: dict[str, Any], branding: dict[str, Any] | None = None) -> tuple[Path, Path | None]:
|
|
143
|
+
"""Persist generated Blueprint/Branding JSON using atomic replacement."""
|
|
143
144
|
BLUEPRINTS.mkdir(parents=True, exist_ok=True)
|
|
144
145
|
target = BLUEPRINTS / "canais" / f"{_slug(blueprint.get('name', 'blueprint'))}-{blueprint['id']}.json"
|
|
145
146
|
target.parent.mkdir(parents=True, exist_ok=True)
|
|
146
|
-
target
|
|
147
|
+
atomic_write(target, blueprint)
|
|
147
148
|
branding_path = None
|
|
148
149
|
if branding:
|
|
149
150
|
branding_path = BLUEPRINTS / "brandings" / f"{_slug(branding.get('name', 'branding'))}-{branding['id']}.json"
|
|
150
151
|
branding_path.parent.mkdir(parents=True, exist_ok=True)
|
|
151
|
-
branding_path
|
|
152
|
+
atomic_write(branding_path, branding)
|
|
152
153
|
return target, branding_path
|
|
153
154
|
|
|
154
155
|
|
package/hermes_ui/cuts.py
CHANGED
|
@@ -304,8 +304,9 @@ def _run_ffmpeg(command: list[str], output: Path, *, source: Path, segment: dict
|
|
|
304
304
|
|
|
305
305
|
|
|
306
306
|
def _write_manifest(record: dict[str, Any], run_dir: Path) -> Path:
|
|
307
|
+
"""Write a complete clip manifest atomically."""
|
|
307
308
|
path = run_dir / "manifest.json"
|
|
308
|
-
|
|
309
|
+
storage.atomic_write(path, record)
|
|
309
310
|
return path
|
|
310
311
|
|
|
311
312
|
|
package/hermes_ui/domain.py
CHANGED
|
@@ -8,6 +8,9 @@ from .notifications import record_notification
|
|
|
8
8
|
from .storage import append_json, now, read_json, write_json
|
|
9
9
|
|
|
10
10
|
STAGES = ["niche", "blueprint", "brand", "topic", "script", "title", "keywords", "video", "thumbnail_prompt", "thumbnail", "upload"]
|
|
11
|
+
# Ordem executada pelo worker. Cada etapa só é executada quando o seu artefacto
|
|
12
|
+
# ainda não existe; os artefactos persistidos tornam a cascata retomável.
|
|
13
|
+
CASCADE_STAGE_ORDER = ("topic", "script", "title", "keywords", "video", "thumbnail_prompt", "thumbnail", "upload")
|
|
11
14
|
LEGACY_STAGES = {"edit"}
|
|
12
15
|
VALID_STATES = {"to_do", "doing", "blocked", "done", "failed", "cancelled"}
|
|
13
16
|
|
|
@@ -148,6 +151,7 @@ def create_tasks_for_batch(batch: dict[str, Any]) -> list[dict[str, Any]]:
|
|
|
148
151
|
thumbnail_status = str(payload.get("thumbnail_status") or ("generated" if thumbnail_path else "not_generated"))
|
|
149
152
|
if thumbnail_path:
|
|
150
153
|
artifacts.setdefault("thumbnail", thumbnail_path)
|
|
154
|
+
initial_stage = "topic" if not topic and str(payload.get("topic_source") or options.get("topic_source") or "manual") in {"auto", "llm_pending"} else "script"
|
|
151
155
|
task = {
|
|
152
156
|
"id": make_id("video"),
|
|
153
157
|
"batch_id": batch["id"],
|
|
@@ -180,7 +184,15 @@ def create_tasks_for_batch(batch: dict[str, Any]) -> list[dict[str, Any]]:
|
|
|
180
184
|
"title_candidates": payload.get("title_candidates", []),
|
|
181
185
|
"keywords": payload.get("keywords", options.get("keywords", [])),
|
|
182
186
|
"ai_generation": payload.get("ai_generation", {}),
|
|
183
|
-
"stage":
|
|
187
|
+
"stage": initial_stage,
|
|
188
|
+
"orchestration": {
|
|
189
|
+
"name": "local-cascade",
|
|
190
|
+
"stage_order": list(CASCADE_STAGE_ORDER),
|
|
191
|
+
"current_stage": initial_stage,
|
|
192
|
+
"completed_stages": [],
|
|
193
|
+
"resumable": True,
|
|
194
|
+
"last_transition_at": now(),
|
|
195
|
+
},
|
|
184
196
|
"state": "to_do",
|
|
185
197
|
"progress": 0,
|
|
186
198
|
"video_ready": bool(artifacts.get("video")),
|
|
@@ -34,6 +34,8 @@ EVENT_CATALOG: tuple[dict[str, str], ...] = (
|
|
|
34
34
|
{"code": "automation_completed", "category": "Automação", "label": "Automação concluída", "description": "Quando o worker concluir o lote agendado de um canal."},
|
|
35
35
|
{"code": "automation_failed", "category": "Automação", "label": "Automação falhou", "description": "Quando uma execução automática terminar com erro."},
|
|
36
36
|
{"code": "activity_failed", "category": "Sistema", "label": "Actividade falhou", "description": "Quando uma tarefa ou operação persistida terminar em erro."},
|
|
37
|
+
{"code": "session_info_expiring", "category": "Autenticação", "label": "SessionInfo a expirar", "description": "Quando o token sessionInfo de uma conta Google/YouTube se aproximar da expiração estimada."},
|
|
38
|
+
{"code": "session_info_expired", "category": "Autenticação", "label": "SessionInfo expirado", "description": "Quando o token sessionInfo de uma conta Google/YouTube ultrapassar a expiração estimada."},
|
|
37
39
|
{"code": "upload_youtube_success", "category": "Upload", "label": "Upload YouTube concluído", "description": "Quando um vídeo for publicado num canal YouTube."},
|
|
38
40
|
{"code": "upload_tiktok_success", "category": "Upload", "label": "Upload TikTok concluído", "description": "Quando um vídeo for publicado numa conta TikTok."},
|
|
39
41
|
{"code": "upload_instagram_success", "category": "Upload", "label": "Upload Instagram concluído", "description": "Quando um vídeo for publicado num perfil Instagram."},
|
|
@@ -12,11 +12,12 @@ from pathlib import Path
|
|
|
12
12
|
from typing import Any
|
|
13
13
|
|
|
14
14
|
from integrations.moneyprinter_config import sync_moneyprinter_config
|
|
15
|
+
from integrations.session_info_health import check_all_accounts_session_info_health, emit_session_info_health_alerts
|
|
15
16
|
from integrations.upload_routing import upload_with_default_route
|
|
16
17
|
from hermes_ui.creative_generation import CreativeGenerationError, generate_creative_package, generate_title_and_keywords, generate_thumbnail_prompt, generate_topic_for_channel
|
|
17
18
|
from hermes_ui.script_documents import save_script_document
|
|
18
19
|
from hermes_ui.script_generation import generate_script_document
|
|
19
|
-
from hermes_ui.storage import STORAGE, ensure_storage, read_json, write_json
|
|
20
|
+
from hermes_ui.storage import STORAGE, atomic_write, ensure_storage, read_json, write_json
|
|
20
21
|
from hermes_ui.llm_providers import active_llm_card, provider_definition
|
|
21
22
|
from hermes_ui.media_generation import MediaGenerationError, _append_generation_constraints, generate_image_from_pool, generate_video_from_pool
|
|
22
23
|
from hermes_ui.media_providers import FULL_IA_VIDEO_PROVIDER_CODES, media_cards_for_pool, media_provider_definition
|
|
@@ -28,6 +29,7 @@ PIPELINE_LOG_FILENAME = "pipeline_worker.json"
|
|
|
28
29
|
VIDEO_TIMEOUT_SECONDS = 20 * 60
|
|
29
30
|
STALE_TASK_SECONDS = VIDEO_TIMEOUT_SECONDS + 5 * 60
|
|
30
31
|
WORKER_HEARTBEAT_TIMEOUT_SECONDS = 15
|
|
32
|
+
CASCADE_STAGE_ORDER = ("topic", "script", "title", "keywords", "video", "thumbnail_prompt", "thumbnail", "upload")
|
|
31
33
|
|
|
32
34
|
|
|
33
35
|
class PipelineError(RuntimeError):
|
|
@@ -207,14 +209,42 @@ def _task_by_id(task_id: str) -> dict[str, Any] | None:
|
|
|
207
209
|
return next((task for task in read_json("tasks.json", []) if isinstance(task, dict) and task.get("id") == task_id), None)
|
|
208
210
|
|
|
209
211
|
|
|
212
|
+
def _cascade_metadata(current: dict[str, Any], updates: dict[str, Any]) -> dict[str, Any]:
|
|
213
|
+
"""Build resumable orchestration metadata for one stage transition."""
|
|
214
|
+
previous = str(current.get("stage") or "")
|
|
215
|
+
next_stage = str(updates.get("stage") or previous)
|
|
216
|
+
raw = current.get("orchestration") if isinstance(current.get("orchestration"), dict) else {}
|
|
217
|
+
completed = [str(item) for item in raw.get("completed_stages", []) if str(item) in CASCADE_STAGE_ORDER]
|
|
218
|
+
if previous in CASCADE_STAGE_ORDER and next_stage != previous and previous not in completed:
|
|
219
|
+
completed.append(previous)
|
|
220
|
+
if str(updates.get("state") or "") == "done" and next_stage in CASCADE_STAGE_ORDER and next_stage not in completed:
|
|
221
|
+
completed.append(next_stage)
|
|
222
|
+
transition_at = raw.get("last_transition_at") or _now()
|
|
223
|
+
if next_stage != previous:
|
|
224
|
+
transition_at = _now()
|
|
225
|
+
try:
|
|
226
|
+
transition_count = int(raw.get("transition_count") or 0)
|
|
227
|
+
except (TypeError, ValueError):
|
|
228
|
+
transition_count = 0
|
|
229
|
+
return {
|
|
230
|
+
"name": "local-cascade",
|
|
231
|
+
"stage_order": list(CASCADE_STAGE_ORDER),
|
|
232
|
+
"current_stage": next_stage,
|
|
233
|
+
"completed_stages": completed,
|
|
234
|
+
"resumable": True,
|
|
235
|
+
"last_transition_at": transition_at,
|
|
236
|
+
"transition_count": transition_count + (1 if next_stage != previous else 0),
|
|
237
|
+
}
|
|
238
|
+
|
|
210
239
|
def _update(task_id: str, **updates: Any) -> dict[str, Any]:
|
|
211
240
|
from hermes_ui.domain import update_task
|
|
212
|
-
|
|
213
241
|
current = _task_by_id(task_id)
|
|
214
242
|
if not current:
|
|
215
243
|
raise PipelineError(f"Tarefa {task_id} deixou de existir durante a execução.")
|
|
216
244
|
if str(current.get("state") or "") in {"blocked", "cancelled"}:
|
|
217
245
|
raise PipelineStopped("A tarefa foi parada pelo utilizador.")
|
|
246
|
+
updates = dict(updates)
|
|
247
|
+
updates["orchestration"] = _cascade_metadata(current, updates)
|
|
218
248
|
updated = update_task(task_id, updates)
|
|
219
249
|
if not updated:
|
|
220
250
|
raise PipelineError(f"Tarefa {task_id} deixou de existir durante a execução.")
|
|
@@ -223,10 +253,9 @@ def _update(task_id: str, **updates: Any) -> dict[str, Any]:
|
|
|
223
253
|
status="running",
|
|
224
254
|
stage=str(updated.get("stage") or "pipeline"),
|
|
225
255
|
progress=int(updated.get("progress") or 0),
|
|
256
|
+
orchestration=updated.get("orchestration") or {},
|
|
226
257
|
)
|
|
227
258
|
return updated
|
|
228
|
-
|
|
229
|
-
|
|
230
259
|
def _channel_for_task(task: dict[str, Any]) -> dict[str, Any]:
|
|
231
260
|
channel_id = str(task.get("channel_id") or "")
|
|
232
261
|
return next((channel for channel in read_json("channels.json", []) if str(channel.get("id")) == channel_id), {})
|
|
@@ -254,12 +283,11 @@ def _keywords(topic: str, title: str, niche: str = "") -> list[str]:
|
|
|
254
283
|
|
|
255
284
|
|
|
256
285
|
def _save_json_artifact(task_id: str, name: str, payload: dict[str, Any]) -> str:
|
|
286
|
+
"""Persist a resumable pipeline artifact atomically."""
|
|
257
287
|
directory = STORAGE / "artifacts"
|
|
258
288
|
directory.mkdir(parents=True, exist_ok=True)
|
|
259
289
|
path = directory / f"{task_id}-{name}.json"
|
|
260
|
-
|
|
261
|
-
temporary.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
|
262
|
-
temporary.replace(path)
|
|
290
|
+
atomic_write(path, payload)
|
|
263
291
|
return str(path)
|
|
264
292
|
|
|
265
293
|
|
|
@@ -947,6 +975,7 @@ def _read_json_artifact(value: Any) -> dict[str, Any] | None:
|
|
|
947
975
|
|
|
948
976
|
|
|
949
977
|
def _run_task(task: dict[str, Any]) -> dict[str, Any]:
|
|
978
|
+
"""Run one resumable local-cascade task, reusing valid persisted artefacts."""
|
|
950
979
|
task_id = str(task.get("id") or "")
|
|
951
980
|
channel = _channel_for_task(task)
|
|
952
981
|
settings = _settings()
|
|
@@ -1213,11 +1242,23 @@ def _run_task(task: dict[str, Any]) -> dict[str, Any]:
|
|
|
1213
1242
|
|
|
1214
1243
|
|
|
1215
1244
|
def run_once() -> dict[str, Any]:
|
|
1245
|
+
"""Execute one worker tick and resume the first pending cascade task."""
|
|
1216
1246
|
ensure_storage()
|
|
1217
1247
|
lock = _acquire_lock()
|
|
1218
1248
|
if lock is None:
|
|
1219
1249
|
return {"ok": True, "busy": True}
|
|
1220
1250
|
try:
|
|
1251
|
+
try:
|
|
1252
|
+
session_health = check_all_accounts_session_info_health(STORAGE, _settings())
|
|
1253
|
+
session_alerts = emit_session_info_health_alerts(session_health)
|
|
1254
|
+
_worker_heartbeat(
|
|
1255
|
+
session_info_health=[item.as_dict() for item in session_health],
|
|
1256
|
+
session_info_alerts=len(session_alerts),
|
|
1257
|
+
session_info_health_error="",
|
|
1258
|
+
)
|
|
1259
|
+
except Exception as exc:
|
|
1260
|
+
# Monitoring is advisory and must not prevent a resumable task from running.
|
|
1261
|
+
_worker_heartbeat(session_info_health_error=type(exc).__name__)
|
|
1221
1262
|
recovered = _recover_stale_tasks()
|
|
1222
1263
|
tasks = read_json("tasks.json", [])
|
|
1223
1264
|
candidate = next((task for task in tasks if isinstance(task, dict) and task.get("state") in {"to_do", "doing"}), None)
|
|
@@ -25,7 +25,7 @@ from .llm_providers import (
|
|
|
25
25
|
normalize_llm_card,
|
|
26
26
|
provider_definition,
|
|
27
27
|
)
|
|
28
|
-
from .storage import STORAGE, ensure_storage
|
|
28
|
+
from .storage import STORAGE, atomic_write, ensure_storage
|
|
29
29
|
|
|
30
30
|
|
|
31
31
|
POOL_LLM = "llm_text"
|
|
@@ -159,10 +159,8 @@ def _load_state(path: Path, default: Any) -> Any:
|
|
|
159
159
|
|
|
160
160
|
|
|
161
161
|
def _save_state(path: Path, value: Any) -> None:
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
temporary.write_text(json.dumps(value, ensure_ascii=False, indent=2), encoding="utf-8")
|
|
165
|
-
os.replace(temporary, path)
|
|
162
|
+
"""Persist provider routing state without exposing a partial JSON file."""
|
|
163
|
+
atomic_write(path, value)
|
|
166
164
|
|
|
167
165
|
|
|
168
166
|
def nvidia_rpm_enabled(settings: Mapping[str, Any]) -> bool:
|
package/hermes_ui/storage.py
CHANGED
|
@@ -105,6 +105,8 @@ DEFAULTS: dict[str, Any] = {
|
|
|
105
105
|
"telegram_chat_id": "",
|
|
106
106
|
"telegram_proxy_url": "",
|
|
107
107
|
"telegram_timeout_seconds": 15,
|
|
108
|
+
"session_info_ttl_hours": 36,
|
|
109
|
+
"session_info_alert_hours": 6,
|
|
108
110
|
"notification_preferences": {
|
|
109
111
|
"video_completed": True,
|
|
110
112
|
"music_completed": True,
|
|
@@ -122,6 +124,8 @@ DEFAULTS: dict[str, Any] = {
|
|
|
122
124
|
"automation_completed": True,
|
|
123
125
|
"automation_failed": True,
|
|
124
126
|
"activity_failed": True,
|
|
127
|
+
"session_info_expiring": True,
|
|
128
|
+
"session_info_expired": True,
|
|
125
129
|
"upload_youtube_success": True,
|
|
126
130
|
"upload_tiktok_success": True,
|
|
127
131
|
"upload_instagram_success": True,
|
|
@@ -401,6 +405,12 @@ def ensure_storage() -> None:
|
|
|
401
405
|
|
|
402
406
|
|
|
403
407
|
def atomic_write(path: Path, data: Any) -> None:
|
|
408
|
+
"""Write JSON through a same-directory temporary file and atomic replace.
|
|
409
|
+
|
|
410
|
+
The complete payload is flushed and fsynced before replacement, so a
|
|
411
|
+
process interruption cannot leave a partially written JSON document at the
|
|
412
|
+
destination.
|
|
413
|
+
"""
|
|
404
414
|
path.parent.mkdir(parents=True, exist_ok=True)
|
|
405
415
|
fd, temp_name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
|
|
406
416
|
try:
|
|
@@ -437,6 +447,7 @@ def read_json(name: str, default: Any | None = None) -> Any:
|
|
|
437
447
|
|
|
438
448
|
|
|
439
449
|
def write_json(name: str, data: Any) -> None:
|
|
450
|
+
"""Persist a state JSON file using :func:`atomic_write`."""
|
|
440
451
|
ensure_storage()
|
|
441
452
|
atomic_write(STATE / name, data)
|
|
442
453
|
|
|
@@ -0,0 +1,238 @@
|
|
|
1
|
+
"""Health check read-only para sessões do Upload directo YouTube.
|
|
2
|
+
|
|
3
|
+
O token ``sessionInfo`` não é interpretado nem devolvido. O módulo controla a
|
|
4
|
+
idade de captura persistida no documento de credenciais, aplica uma janela
|
|
5
|
+
conservadora de expiração e permite que a UI/worker alerte antes de iniciar um
|
|
6
|
+
upload. A renovação continua manual: o utilizador deve substituir o token na
|
|
7
|
+
conta Google/YouTube.
|
|
8
|
+
"""
|
|
9
|
+
from __future__ import annotations
|
|
10
|
+
|
|
11
|
+
from dataclasses import dataclass
|
|
12
|
+
from datetime import datetime, timedelta, timezone
|
|
13
|
+
from pathlib import Path
|
|
14
|
+
from typing import Any
|
|
15
|
+
|
|
16
|
+
from integrations.youtube_direct_credentials import credentials_document_path, load_credentials_document
|
|
17
|
+
|
|
18
|
+
DEFAULT_SESSION_INFO_TTL_HOURS = 36
|
|
19
|
+
DEFAULT_SESSION_INFO_ALERT_HOURS = 6
|
|
20
|
+
MIN_SESSION_INFO_TTL_HOURS = 24
|
|
21
|
+
MAX_SESSION_INFO_TTL_HOURS = 48
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
def utc_now() -> datetime:
|
|
25
|
+
"""Return the current timezone-aware UTC timestamp."""
|
|
26
|
+
return datetime.now(timezone.utc)
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
def parse_timestamp(value: Any) -> datetime | None:
|
|
30
|
+
"""Parse ISO-8601 timestamps used in local credential documents."""
|
|
31
|
+
text = str(value or "").strip()
|
|
32
|
+
if not text:
|
|
33
|
+
return None
|
|
34
|
+
try:
|
|
35
|
+
parsed = datetime.fromisoformat(text.replace("Z", "+00:00"))
|
|
36
|
+
except ValueError:
|
|
37
|
+
return None
|
|
38
|
+
return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc)
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
def _clamp_hours(value: Any, default: int) -> int:
|
|
42
|
+
try:
|
|
43
|
+
hours = int(value)
|
|
44
|
+
except (TypeError, ValueError):
|
|
45
|
+
hours = default
|
|
46
|
+
return max(MIN_SESSION_INFO_TTL_HOURS, min(MAX_SESSION_INFO_TTL_HOURS, hours))
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
@dataclass(frozen=True)
|
|
50
|
+
class SessionInfoHealth:
|
|
51
|
+
"""Safe, serialisable status of one account's sessionInfo token."""
|
|
52
|
+
|
|
53
|
+
status: str
|
|
54
|
+
message: str
|
|
55
|
+
age_hours: float | None
|
|
56
|
+
remaining_hours: float | None
|
|
57
|
+
expires_at: str | None
|
|
58
|
+
captured_at: str | None
|
|
59
|
+
account_id: str
|
|
60
|
+
account_label: str
|
|
61
|
+
credential_file: str
|
|
62
|
+
has_session_info: bool
|
|
63
|
+
needs_manual_renewal: bool
|
|
64
|
+
|
|
65
|
+
@property
|
|
66
|
+
def ok(self) -> bool:
|
|
67
|
+
"""Whether the token is usable without a preventive warning."""
|
|
68
|
+
return self.status == "healthy"
|
|
69
|
+
|
|
70
|
+
@property
|
|
71
|
+
def warning(self) -> bool:
|
|
72
|
+
"""Whether the account is usable but close to the configured TTL."""
|
|
73
|
+
return self.status == "expiring"
|
|
74
|
+
|
|
75
|
+
def as_dict(self) -> dict[str, Any]:
|
|
76
|
+
"""Return public health fields without secrets or token material."""
|
|
77
|
+
return {
|
|
78
|
+
"status": self.status,
|
|
79
|
+
"message": self.message,
|
|
80
|
+
"age_hours": round(self.age_hours, 2) if self.age_hours is not None else None,
|
|
81
|
+
"remaining_hours": round(self.remaining_hours, 2) if self.remaining_hours is not None else None,
|
|
82
|
+
"expires_at": self.expires_at,
|
|
83
|
+
"captured_at": self.captured_at,
|
|
84
|
+
"account_id": self.account_id,
|
|
85
|
+
"account_label": self.account_label,
|
|
86
|
+
"credential_file": self.credential_file,
|
|
87
|
+
"has_session_info": self.has_session_info,
|
|
88
|
+
"needs_manual_renewal": self.needs_manual_renewal,
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
def health_check_session_info(
|
|
93
|
+
account: dict[str, Any],
|
|
94
|
+
document: dict[str, Any] | None,
|
|
95
|
+
*,
|
|
96
|
+
now: datetime | None = None,
|
|
97
|
+
ttl_hours: Any = DEFAULT_SESSION_INFO_TTL_HOURS,
|
|
98
|
+
alert_hours: Any = DEFAULT_SESSION_INFO_ALERT_HOURS,
|
|
99
|
+
credential_file: str | Path = "",
|
|
100
|
+
) -> SessionInfoHealth:
|
|
101
|
+
"""Check a sessionInfo document without making a network request.
|
|
102
|
+
|
|
103
|
+
A new or legacy document without ``sessionInfoCapturedAt`` is reported as
|
|
104
|
+
``unknown`` rather than silently trusted. The next manual save records the
|
|
105
|
+
capture time; this avoids claiming that an old token is fresh.
|
|
106
|
+
"""
|
|
107
|
+
current = now or utc_now()
|
|
108
|
+
if current.tzinfo is None:
|
|
109
|
+
current = current.replace(tzinfo=timezone.utc)
|
|
110
|
+
ttl = _clamp_hours(ttl_hours, DEFAULT_SESSION_INFO_TTL_HOURS)
|
|
111
|
+
alert = max(1, min(ttl - 1, _clamp_hours(alert_hours, DEFAULT_SESSION_INFO_ALERT_HOURS)))
|
|
112
|
+
document = document if isinstance(document, dict) else {}
|
|
113
|
+
session_info = str(document.get("sessionInfo") or document.get("session_info") or "").strip()
|
|
114
|
+
captured_value = document.get("sessionInfoCapturedAt") or document.get("session_info_captured_at")
|
|
115
|
+
captured = parse_timestamp(captured_value)
|
|
116
|
+
account_id = str(account.get("id") or document.get("account_id") or "").strip()
|
|
117
|
+
account_label = str(account.get("label") or account.get("name") or account.get("email") or account_id or "Conta Google").strip()
|
|
118
|
+
file_text = str(credential_file or "")
|
|
119
|
+
if not session_info:
|
|
120
|
+
return SessionInfoHealth("missing", "Falta sessionInfo nesta conta Google/YouTube.", None, None, None, None, account_id, account_label, file_text, False, True)
|
|
121
|
+
if captured is None:
|
|
122
|
+
return SessionInfoHealth("unknown", "sessionInfo existe, mas a data de captura não é conhecida; guarde novamente o token para activar o alerta de expiração.", None, None, None, None, account_id, account_label, file_text, True, True)
|
|
123
|
+
age = max(0.0, (current - captured.astimezone(timezone.utc)).total_seconds() / 3600)
|
|
124
|
+
remaining = float(ttl) - age
|
|
125
|
+
expires = captured.astimezone(timezone.utc) + timedelta(hours=ttl)
|
|
126
|
+
if remaining <= 0:
|
|
127
|
+
return SessionInfoHealth("expired", f"sessionInfo expirou há {abs(remaining):.1f} horas; renove-o manualmente antes de criar ou enviar conteúdos.", age, remaining, expires.isoformat(), captured.isoformat(), account_id, account_label, file_text, True, True)
|
|
128
|
+
if remaining <= alert:
|
|
129
|
+
return SessionInfoHealth("expiring", f"sessionInfo expira em aproximadamente {remaining:.1f} horas; renove-o manualmente preventivamente.", age, remaining, expires.isoformat(), captured.isoformat(), account_id, account_label, file_text, True, True)
|
|
130
|
+
return SessionInfoHealth("healthy", f"sessionInfo válido por aproximadamente mais {remaining:.1f} horas.", age, remaining, expires.isoformat(), captured.isoformat(), account_id, account_label, file_text, True, False)
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
def session_info_health_from_settings(
|
|
134
|
+
account: dict[str, Any],
|
|
135
|
+
document: dict[str, Any] | None,
|
|
136
|
+
settings: dict[str, Any] | None = None,
|
|
137
|
+
*,
|
|
138
|
+
now: datetime | None = None,
|
|
139
|
+
credential_file: str | Path = "",
|
|
140
|
+
) -> SessionInfoHealth:
|
|
141
|
+
"""Apply local settings while keeping the 24–48 hour TTL bounded."""
|
|
142
|
+
settings = settings or {}
|
|
143
|
+
return health_check_session_info(
|
|
144
|
+
account,
|
|
145
|
+
document,
|
|
146
|
+
now=now,
|
|
147
|
+
ttl_hours=settings.get("session_info_ttl_hours", DEFAULT_SESSION_INFO_TTL_HOURS),
|
|
148
|
+
alert_hours=settings.get("session_info_alert_hours", DEFAULT_SESSION_INFO_ALERT_HOURS),
|
|
149
|
+
credential_file=credential_file,
|
|
150
|
+
)
|
|
151
|
+
|
|
152
|
+
|
|
153
|
+
def check_account_session_info_health(
|
|
154
|
+
storage_root: Path,
|
|
155
|
+
account: dict[str, Any],
|
|
156
|
+
settings: dict[str, Any] | None = None,
|
|
157
|
+
*,
|
|
158
|
+
now: datetime | None = None,
|
|
159
|
+
) -> SessionInfoHealth:
|
|
160
|
+
"""Load one account document and return its safe SessionInfo health state."""
|
|
161
|
+
settings = settings or {}
|
|
162
|
+
document = load_credentials_document(storage_root, account, settings, create=False)
|
|
163
|
+
return session_info_health_from_settings(
|
|
164
|
+
account,
|
|
165
|
+
document,
|
|
166
|
+
settings,
|
|
167
|
+
now=now,
|
|
168
|
+
credential_file=credentials_document_path(storage_root, account),
|
|
169
|
+
)
|
|
170
|
+
|
|
171
|
+
|
|
172
|
+
def check_all_accounts_session_info_health(
|
|
173
|
+
storage_root: Path,
|
|
174
|
+
settings: dict[str, Any] | None = None,
|
|
175
|
+
*,
|
|
176
|
+
now: datetime | None = None,
|
|
177
|
+
) -> list[SessionInfoHealth]:
|
|
178
|
+
"""Return health states for all configured Google/YouTube accounts."""
|
|
179
|
+
settings = settings or {}
|
|
180
|
+
accounts = settings.get("youtube_batch_accounts", [])
|
|
181
|
+
if not isinstance(accounts, list):
|
|
182
|
+
return []
|
|
183
|
+
result: list[SessionInfoHealth] = []
|
|
184
|
+
for account in accounts:
|
|
185
|
+
if not isinstance(account, dict) or not str(account.get("id") or "").strip():
|
|
186
|
+
continue
|
|
187
|
+
result.append(check_account_session_info_health(storage_root, account, settings, now=now))
|
|
188
|
+
return result
|
|
189
|
+
|
|
190
|
+
|
|
191
|
+
def emit_session_info_health_alerts(
|
|
192
|
+
health_items: list[SessionInfoHealth],
|
|
193
|
+
*,
|
|
194
|
+
now: datetime | None = None,
|
|
195
|
+
) -> list[dict[str, Any]]:
|
|
196
|
+
"""Persist one deduplicated local notification per account and state."""
|
|
197
|
+
from hermes_ui.notifications import record_notification
|
|
198
|
+
|
|
199
|
+
current = now or utc_now()
|
|
200
|
+
day = current.astimezone(timezone.utc).date().isoformat()
|
|
201
|
+
created: list[dict[str, Any]] = []
|
|
202
|
+
for health in health_items:
|
|
203
|
+
if health.status not in {"unknown", "expiring", "expired"}:
|
|
204
|
+
continue
|
|
205
|
+
event_type = "session_info_expired" if health.status == "expired" else "session_info_expiring"
|
|
206
|
+
state_label = "expirado" if health.status == "expired" else "a expirar"
|
|
207
|
+
title = f"SessionInfo {state_label}: {health.account_label}"
|
|
208
|
+
entry = record_notification(
|
|
209
|
+
event_type,
|
|
210
|
+
title,
|
|
211
|
+
health.message,
|
|
212
|
+
metadata={
|
|
213
|
+
"account_id": health.account_id,
|
|
214
|
+
"account_label": health.account_label,
|
|
215
|
+
"status": health.status,
|
|
216
|
+
"captured_at": health.captured_at or "",
|
|
217
|
+
"expires_at": health.expires_at or "",
|
|
218
|
+
},
|
|
219
|
+
dedupe_key=f"session-info:{health.account_id}:{health.status}:{day}",
|
|
220
|
+
)
|
|
221
|
+
if entry:
|
|
222
|
+
created.append(entry)
|
|
223
|
+
return created
|
|
224
|
+
|
|
225
|
+
|
|
226
|
+
__all__ = [
|
|
227
|
+
"DEFAULT_SESSION_INFO_ALERT_HOURS",
|
|
228
|
+
"DEFAULT_SESSION_INFO_TTL_HOURS",
|
|
229
|
+
"MAX_SESSION_INFO_TTL_HOURS",
|
|
230
|
+
"MIN_SESSION_INFO_TTL_HOURS",
|
|
231
|
+
"SessionInfoHealth",
|
|
232
|
+
"check_account_session_info_health",
|
|
233
|
+
"check_all_accounts_session_info_health",
|
|
234
|
+
"emit_session_info_health_alerts",
|
|
235
|
+
"health_check_session_info",
|
|
236
|
+
"parse_timestamp",
|
|
237
|
+
"session_info_health_from_settings",
|
|
238
|
+
]
|
|
@@ -14,6 +14,7 @@ from typing import Any, Callable
|
|
|
14
14
|
|
|
15
15
|
from integrations.platforms import IntegrationResult, YouTubeAdapter
|
|
16
16
|
from integrations.postiz import PostizAdapter
|
|
17
|
+
from integrations.session_info_health import check_account_session_info_health, emit_session_info_health_alerts
|
|
17
18
|
from integrations.youtube_direct_credentials import document_status
|
|
18
19
|
from integrations.youtube_direct_upload import YouTubeDirectUploader
|
|
19
20
|
from hermes_ui.storage import read_json, write_json
|
|
@@ -126,8 +127,16 @@ def upload_with_default_route(
|
|
|
126
127
|
if account:
|
|
127
128
|
try:
|
|
128
129
|
status = document_status(storage_root, account, channel, settings, [channel])
|
|
129
|
-
|
|
130
|
-
|
|
130
|
+
health = check_account_session_info_health(storage_root, account, settings)
|
|
131
|
+
emit_session_info_health_alerts([health])
|
|
132
|
+
direct_ready = bool(status.get("ready")) and health.status != "expired"
|
|
133
|
+
if health.status == "expired":
|
|
134
|
+
direct_result = IntegrationResult(
|
|
135
|
+
False,
|
|
136
|
+
f"Upload directo indisponível: {health.message}",
|
|
137
|
+
{"missing": ["sessionInfo"], "session_info_health": health.as_dict()},
|
|
138
|
+
)
|
|
139
|
+
elif not direct_ready:
|
|
131
140
|
missing = list(status.get("missing_cookies", []))
|
|
132
141
|
if not status.get("has_session_info"):
|
|
133
142
|
missing.append("sessionInfo")
|
|
@@ -5,9 +5,12 @@ import os
|
|
|
5
5
|
import re
|
|
6
6
|
import secrets
|
|
7
7
|
import shutil
|
|
8
|
+
from datetime import datetime, timezone
|
|
8
9
|
from pathlib import Path
|
|
9
10
|
from typing import Any
|
|
10
11
|
|
|
12
|
+
from hermes_ui.storage import atomic_write
|
|
13
|
+
|
|
11
14
|
COOKIE_KEYS = ("SID", "SSID", "HSID", "APISID", "SAPISID")
|
|
12
15
|
DIRECT_DOCUMENT_NAME = "credentials.json"
|
|
13
16
|
DEFAULT_CHUNK_SIZE = 262144
|
|
@@ -32,6 +35,10 @@ def cookie_file_path(storage_root: Path, account: dict[str, Any]) -> Path:
|
|
|
32
35
|
return account_directory(storage_root, account) / "cookies.json"
|
|
33
36
|
|
|
34
37
|
|
|
38
|
+
def _session_info_now() -> str:
|
|
39
|
+
"""Return the UTC timestamp recorded when a sessionInfo token is saved."""
|
|
40
|
+
return datetime.now(timezone.utc).isoformat()
|
|
41
|
+
|
|
35
42
|
def _placeholder_to_empty(value: Any) -> str:
|
|
36
43
|
text = str(value or "").strip()
|
|
37
44
|
if text in {"...", "…", "<preencher>", "<valor>"}:
|
|
@@ -177,6 +184,7 @@ def _normalise_document(raw: Any, account: dict[str, Any]) -> dict[str, Any]:
|
|
|
177
184
|
"account_id": str(raw.get("account_id") or account.get("id") or "").strip(),
|
|
178
185
|
"email": str(raw.get("email") or account.get("email") or "").strip(),
|
|
179
186
|
"sessionInfo": _placeholder_to_empty(raw.get("sessionInfo") or raw.get("session_info") or raw.get("direct_session_info") or _account_session_info(account)),
|
|
187
|
+
"sessionInfoCapturedAt": str(raw.get("sessionInfoCapturedAt") or raw.get("session_info_captured_at") or account.get("sessionInfoCapturedAt") or account.get("session_info_captured_at") or "").strip(),
|
|
180
188
|
"cookies": {key: cookies.get(key, "") for key in COOKIE_KEYS},
|
|
181
189
|
"INNERTUBE_API_KEY": _placeholder_to_empty(raw.get("INNERTUBE_API_KEY") or raw.get("innertube_api_key") or raw.get("direct_innertube_api_key") or ""),
|
|
182
190
|
"chunk_size": _safe_chunk_size(raw.get("chunk_size", raw.get("direct_chunk_size", DEFAULT_CHUNK_SIZE))),
|
|
@@ -204,6 +212,7 @@ def _legacy_document(storage_root: Path, account: dict[str, Any], settings: dict
|
|
|
204
212
|
"account_id": account.get("id"),
|
|
205
213
|
"email": account.get("email"),
|
|
206
214
|
"sessionInfo": _account_session_info(account) or settings.get("direct_session_info"),
|
|
215
|
+
"sessionInfoCapturedAt": account.get("sessionInfoCapturedAt") or settings.get("direct_session_info_captured_at") or "",
|
|
207
216
|
"cookies": legacy_cookies,
|
|
208
217
|
"INNERTUBE_API_KEY": settings.get("direct_innertube_api_key"),
|
|
209
218
|
"chunk_size": settings.get("direct_chunk_size", DEFAULT_CHUNK_SIZE),
|
|
@@ -219,18 +228,13 @@ def save_credentials_document(storage_root: Path, account: dict[str, Any], docum
|
|
|
219
228
|
"account_id": normalised["account_id"],
|
|
220
229
|
"email": normalised["email"],
|
|
221
230
|
"sessionInfo": normalised["sessionInfo"],
|
|
231
|
+
"sessionInfoCapturedAt": normalised.get("sessionInfoCapturedAt", ""),
|
|
222
232
|
"cookies": normalised["cookies"],
|
|
223
233
|
# INNERTUBE_API_KEY não pertence ao documento de cookies/credenciais.
|
|
224
234
|
"chunk_size": normalised["chunk_size"],
|
|
225
235
|
"delegated_session_ids": normalised["delegated_session_ids"],
|
|
226
236
|
}
|
|
227
|
-
|
|
228
|
-
temporary.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
|
229
|
-
try:
|
|
230
|
-
os.chmod(temporary, 0o600)
|
|
231
|
-
except OSError:
|
|
232
|
-
pass
|
|
233
|
-
temporary.replace(destination)
|
|
237
|
+
atomic_write(destination, payload)
|
|
234
238
|
try:
|
|
235
239
|
os.chmod(destination, 0o600)
|
|
236
240
|
except OSError:
|
|
@@ -239,13 +243,15 @@ def save_credentials_document(storage_root: Path, account: dict[str, Any], docum
|
|
|
239
243
|
|
|
240
244
|
|
|
241
245
|
def update_credentials_document_session_info(storage_root: Path, account: dict[str, Any], session_info: str) -> Path | None:
|
|
242
|
-
"""
|
|
246
|
+
"""Save sessionInfo and its UTC capture time without exposing the token."""
|
|
243
247
|
path = credentials_document_path(storage_root, account)
|
|
244
|
-
if
|
|
245
|
-
|
|
246
|
-
|
|
248
|
+
if path.exists():
|
|
249
|
+
raw = _read_json_document(path) or {}
|
|
250
|
+
else:
|
|
251
|
+
raw = load_credentials_document(storage_root, {**account, "sessionInfo": session_info}, create=True)
|
|
247
252
|
document = _normalise_document(raw, {**account, "sessionInfo": session_info})
|
|
248
253
|
document["sessionInfo"] = str(session_info or "").strip()
|
|
254
|
+
document["sessionInfoCapturedAt"] = _session_info_now() if document["sessionInfo"] else ""
|
|
249
255
|
return save_credentials_document(storage_root, account, document)
|
|
250
256
|
|
|
251
257
|
|
|
@@ -326,8 +332,13 @@ def merge_credentials_document(
|
|
|
326
332
|
if incoming["cookies"].get(key):
|
|
327
333
|
merged["cookies"][key] = incoming["cookies"][key]
|
|
328
334
|
incoming_session = _placeholder_to_empty(session_info_override) or incoming.get("sessionInfo", "")
|
|
335
|
+
previous_session = str(merged.get("sessionInfo") or "").strip()
|
|
329
336
|
if incoming_session:
|
|
330
337
|
merged["sessionInfo"] = incoming_session
|
|
338
|
+
if incoming_session != previous_session or not merged.get("sessionInfoCapturedAt"):
|
|
339
|
+
merged["sessionInfoCapturedAt"] = _session_info_now()
|
|
340
|
+
elif not merged.get("sessionInfo"):
|
|
341
|
+
merged["sessionInfoCapturedAt"] = ""
|
|
331
342
|
# INNERTUBE_API_KEY recebida num JSON é deliberadamente ignorada: a fonte oficial
|
|
332
343
|
# é a configuração separada da secção Contas Google/YouTube.
|
|
333
344
|
if "chunk_size" in raw:
|
|
@@ -10,6 +10,7 @@ from pathlib import Path
|
|
|
10
10
|
from typing import Any
|
|
11
11
|
|
|
12
12
|
from integrations.youtube_direct_credentials import COOKIE_KEYS, account_innertube_api_key, delegated_session_id, load_credentials_document
|
|
13
|
+
from integrations.session_info_health import session_info_health_from_settings
|
|
13
14
|
from urllib.parse import quote
|
|
14
15
|
|
|
15
16
|
import requests
|
|
@@ -201,11 +202,23 @@ class YouTubeDirectUploader:
|
|
|
201
202
|
validation_error = validate_direct_upload(path, self.channel, self.settings, self.account, self.storage_root)
|
|
202
203
|
if validation_error:
|
|
203
204
|
return DirectUploadResult(False, validation_error, {"mechanism": "youtube-frontend-direct"})
|
|
205
|
+
session_health = session_info_health_from_settings(
|
|
206
|
+
self.account or {},
|
|
207
|
+
self.document,
|
|
208
|
+
self.settings,
|
|
209
|
+
credential_file=(str(self.storage_root) if self.storage_root else ""),
|
|
210
|
+
)
|
|
211
|
+
if session_health.status == "expired":
|
|
212
|
+
return DirectUploadResult(
|
|
213
|
+
False,
|
|
214
|
+
f"Upload directo bloqueado: {session_health.message}",
|
|
215
|
+
{"mechanism": "youtube-frontend-direct", "session_info_health": session_health.as_dict()},
|
|
216
|
+
)
|
|
204
217
|
try:
|
|
205
218
|
self.describe_file(path)
|
|
206
219
|
self.create_video(title, description, visibility)
|
|
207
220
|
configured_chunk_size = int(chunk_size or self.document.get("chunk_size") or CHUNK_GRANULARITY)
|
|
208
221
|
self.upload_chunks(path, configured_chunk_size)
|
|
209
|
-
return DirectUploadResult(True, f"Upload directo concluído: {self.video_id}", {"mechanism": "youtube-frontend-direct", "video_id": self.video_id, "page_id": self.channel.get("delegated_session_id", ""), "google_account_id": (self.account or {}).get("id", "")})
|
|
222
|
+
return DirectUploadResult(True, f"Upload directo concluído: {self.video_id}", {"mechanism": "youtube-frontend-direct", "video_id": self.video_id, "page_id": self.channel.get("delegated_session_id", ""), "google_account_id": (self.account or {}).get("id", ""), "session_info_health": session_health.as_dict()})
|
|
210
223
|
except (requests.RequestException, OSError, ValueError, RuntimeError) as exc:
|
|
211
224
|
return DirectUploadResult(False, f"Upload directo falhou: {exc}", {"mechanism": "youtube-frontend-direct", "video_id": self.video_id})
|
package/package.json
CHANGED