salla-gitpuller 1.1.2__tar.gz → 1.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
File without changes
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: salla_gitpuller
3
- Version: 1.1.2
3
+ Version: 1.2.0
4
4
  Summary: A lightweight utility to git pull a repository using SSH deploy keys stored in environment variables
5
5
  Author-email: Mohammed Junaid <safijunaid.ss@gmail.com>, Muhammad Zahid <zahidmuhammad127@gmail.com>
6
6
  License: MIT
@@ -58,8 +58,9 @@ state to persist across pipeline runs (see [State management](#state-management)
58
58
  from gitpuller import GitPullExecutor
59
59
 
60
60
  executor = GitPullExecutor(
61
- slack_webhook_url="https://hooks.slack.com/services/XXX", # or set CDM_SLACK_WEBHOOK_URL
62
- use_mage_ai=True, # persist alert state via Mage
61
+ use_mage_ai=True, # persist alert state via Mage
62
+ # Slack: set CDM_PROD_SLACK_BOT_TOKEN (threaded daily alerts).
63
+ # Optional: CDM_PROD_DB_SLACK_CHANNEL (defaults to C05MLHR55JT).
63
64
  )
64
65
 
65
66
  result = executor.execute_with_alerting(
@@ -74,7 +75,8 @@ print(result["discarded_changes"]) # what local drift (if any) was wiped
74
75
  ```
75
76
 
76
77
  On failure, `execute_with_alerting` sends a Slack alert (subject to suppression)
77
- and then **re-raises**, so the Mage pipeline still fails loudly.
78
+ as a **reply in today's daily thread**, then **re-raises**, so the Mage pipeline
79
+ still fails loudly.
78
80
 
79
81
  ---
80
82
 
@@ -116,14 +118,56 @@ and exit code**, which becomes the Slack alert body and the pipeline error.
116
118
 
117
119
  ## API
118
120
 
119
- ### `GitPullExecutor(slack_webhook_url=None, use_mage_ai=False, state_manager=None)`
121
+ ### `GitPullExecutor(slack_webhook_url=None, slack_bot_token=None, slack_channel=None, use_mage_ai=False, state_manager=None)`
120
122
 
121
123
  | Param | Description |
122
124
  |-------|-------------|
123
- | `slack_webhook_url` | Slack incoming-webhook URL. Falls back to the `CDM_SLACK_WEBHOOK_URL` env var. Required (one of the two must be set). |
125
+ | `slack_bot_token` | Slack bot token (`xoxb-...`). Falls back to `CDM_PROD_SLACK_BOT_TOKEN`. Preferred — enables daily-thread replies. |
126
+ | `slack_channel` | Channel ID/name. Falls back to `CDM_PROD_DB_SLACK_CHANNEL`, then `C05MLHR55JT`. |
127
+ | `slack_webhook_url` | Incoming-webhook URL. Falls back to `CDM_SLACK_WEBHOOK_URL`. Used only when no bot token is set (no threading). |
124
128
  | `use_mage_ai` | If `True`, persist alert-suppression state via Mage global variables (falls back to in-memory if Mage isn't installed). |
125
129
  | `state_manager` | Inject a custom `StateManager`; overrides `use_mage_ai`. |
126
130
 
131
+ If neither a bot token nor a webhook is configured, gitpuller still runs and
132
+ prints a warning instead of failing construction.
133
+
134
+ ### Slack daily thread
135
+
136
+ When `CDM_PROD_SLACK_BOT_TOKEN` is set, failures are posted with `chat.postMessage`:
137
+
138
+ 1. Open (or reuse) one parent message **per workspace per calendar day**:
139
+ `🚨 Git Pull Failures {workspace_name} — YYYY-MM-DD`.
140
+ 2. Post each later failure for that workspace as a **thread reply**.
141
+
142
+ Daily `thread_ts` values are stored in **one unified Redis hash** shared by every
143
+ Mage workspace (the key is **not** prefixed with `MAGE_WORKSPACE_NAME`):
144
+
145
+ ```text
146
+ gitpuller:slack_thread:{channel}:{YYYY-MM-DD}
147
+ cloud_data → JSON { workspace, thread_ts, errors: [{workspace, repo, error, at, reply_ts}, ...] }
148
+ partner → JSON { workspace, thread_ts, errors: [...] }
149
+ TTL: until midnight (CDM_SLACK_THREAD_TZ, default UTC)
150
+ ```
151
+
152
+ Each workspace keeps the Slack parent `thread_ts` plus up to 20 git errors
153
+ from that day (error text capped at 1500 chars). Legacy plain `thread_ts`
154
+ strings are still read.
155
+
156
+ Redis comes from Mage `io_config.yaml` (`REDIS_HOST` / `REDIS_PORT` /
157
+ `REDIS_PASSWORD`), or those same env vars. If Redis is unavailable, gitpuller
158
+ falls back to Slack `conversations.history`.
159
+
160
+ | Env var | Role |
161
+ |---------|------|
162
+ | `CDM_PROD_SLACK_BOT_TOKEN` | Required for threading. |
163
+ | `CDM_PROD_DB_SLACK_CHANNEL` | Channel; default `C05MLHR55JT`. |
164
+ | `CDM_SLACK_THREAD_TZ` | Timezone for the daily parent date; default `UTC`. |
165
+ | `CDM_PAUSE_SLACK_MESSAGES` | Set to `1` to skip Slack. |
166
+ | `CDM_SLACK_WEBHOOK_URL` | Legacy fallback when no bot token is set. |
167
+
168
+ The bot must be in the channel (`chat:write`). History reuse also needs
169
+ `channels:history` (or `groups:history` for a private channel).
170
+
127
171
  ### `execute_with_alerting(...)` → `dict`
128
172
 
129
173
  Runs the sync and, on failure, alerts Slack (with suppression) then re-raises.
@@ -139,6 +183,7 @@ Runs the sync and, on failure, alerts Slack (with suppression) then re-raises.
139
183
  | `suppression_hours` | `1` | Don't re-alert on the *same* error within this many hours. |
140
184
  | `key_filename` | `None` | Override the on-disk key filename. |
141
185
  | `ssh_dir` | `"/home/src/.ssh"` | Directory to write the key into. |
186
+ | `webhook_url` | `None` | Legacy webhook override for this call. Ignored when a bot token is configured. |
142
187
 
143
188
  ### `execute_git_pull(...)` → `dict`
144
189
 
@@ -222,7 +267,21 @@ Keep the version in sync in **both** `pyproject.toml` and `gitpuller/__init__.py
222
267
 
223
268
  ## Changelog
224
269
 
225
- ### 1.1.0 (current)
270
+ ### 1.2.0 (current)
271
+
272
+ - **Threaded Slack alerts.** Failures post as replies under one daily parent
273
+ **per workspace** (`🚨 Git Pull Failures {workspace} — YYYY-MM-DD`) via
274
+ `CDM_PROD_SLACK_BOT_TOKEN` and `CDM_PROD_DB_SLACK_CHANNEL`
275
+ (default `C05MLHR55JT`). Daily `thread_ts` lives in one unified Redis hash
276
+ (`gitpuller:slack_thread:{channel}:{YYYY-MM-DD}`, field = workspace, TTL
277
+ until midnight) shared across all Mage workspaces. Each workspace field is
278
+ JSON: `thread_ts` plus that day's git errors.
279
+ - Incoming webhooks (`CDM_SLACK_WEBHOOK_URL` / `slack_webhook_url`) remain as a
280
+ non-threaded fallback when no bot token is set.
281
+ - Missing Slack credentials no longer raise on `GitPullExecutor` construction;
282
+ alerts are skipped with a warning.
283
+
284
+ ### 1.1.0
226
285
 
227
286
  Reliability and clarity overhaul.
228
287
 
@@ -46,8 +46,9 @@ state to persist across pipeline runs (see [State management](#state-management)
46
46
  from gitpuller import GitPullExecutor
47
47
 
48
48
  executor = GitPullExecutor(
49
- slack_webhook_url="https://hooks.slack.com/services/XXX", # or set CDM_SLACK_WEBHOOK_URL
50
- use_mage_ai=True, # persist alert state via Mage
49
+ use_mage_ai=True, # persist alert state via Mage
50
+ # Slack: set CDM_PROD_SLACK_BOT_TOKEN (threaded daily alerts).
51
+ # Optional: CDM_PROD_DB_SLACK_CHANNEL (defaults to C05MLHR55JT).
51
52
  )
52
53
 
53
54
  result = executor.execute_with_alerting(
@@ -62,7 +63,8 @@ print(result["discarded_changes"]) # what local drift (if any) was wiped
62
63
  ```
63
64
 
64
65
  On failure, `execute_with_alerting` sends a Slack alert (subject to suppression)
65
- and then **re-raises**, so the Mage pipeline still fails loudly.
66
+ as a **reply in today's daily thread**, then **re-raises**, so the Mage pipeline
67
+ still fails loudly.
66
68
 
67
69
  ---
68
70
 
@@ -104,14 +106,56 @@ and exit code**, which becomes the Slack alert body and the pipeline error.
104
106
 
105
107
  ## API
106
108
 
107
- ### `GitPullExecutor(slack_webhook_url=None, use_mage_ai=False, state_manager=None)`
109
+ ### `GitPullExecutor(slack_webhook_url=None, slack_bot_token=None, slack_channel=None, use_mage_ai=False, state_manager=None)`
108
110
 
109
111
  | Param | Description |
110
112
  |-------|-------------|
111
- | `slack_webhook_url` | Slack incoming-webhook URL. Falls back to the `CDM_SLACK_WEBHOOK_URL` env var. Required (one of the two must be set). |
113
+ | `slack_bot_token` | Slack bot token (`xoxb-...`). Falls back to `CDM_PROD_SLACK_BOT_TOKEN`. Preferred — enables daily-thread replies. |
114
+ | `slack_channel` | Channel ID/name. Falls back to `CDM_PROD_DB_SLACK_CHANNEL`, then `C05MLHR55JT`. |
115
+ | `slack_webhook_url` | Incoming-webhook URL. Falls back to `CDM_SLACK_WEBHOOK_URL`. Used only when no bot token is set (no threading). |
112
116
  | `use_mage_ai` | If `True`, persist alert-suppression state via Mage global variables (falls back to in-memory if Mage isn't installed). |
113
117
  | `state_manager` | Inject a custom `StateManager`; overrides `use_mage_ai`. |
114
118
 
119
+ If neither a bot token nor a webhook is configured, gitpuller still runs and
120
+ prints a warning instead of failing construction.
121
+
122
+ ### Slack daily thread
123
+
124
+ When `CDM_PROD_SLACK_BOT_TOKEN` is set, failures are posted with `chat.postMessage`:
125
+
126
+ 1. Open (or reuse) one parent message **per workspace per calendar day**:
127
+ `🚨 Git Pull Failures {workspace_name} — YYYY-MM-DD`.
128
+ 2. Post each later failure for that workspace as a **thread reply**.
129
+
130
+ Daily `thread_ts` values are stored in **one unified Redis hash** shared by every
131
+ Mage workspace (the key is **not** prefixed with `MAGE_WORKSPACE_NAME`):
132
+
133
+ ```text
134
+ gitpuller:slack_thread:{channel}:{YYYY-MM-DD}
135
+ cloud_data → JSON { workspace, thread_ts, errors: [{workspace, repo, error, at, reply_ts}, ...] }
136
+ partner → JSON { workspace, thread_ts, errors: [...] }
137
+ TTL: until midnight (CDM_SLACK_THREAD_TZ, default UTC)
138
+ ```
139
+
140
+ Each workspace keeps the Slack parent `thread_ts` plus up to 20 git errors
141
+ from that day (error text capped at 1500 chars). Legacy plain `thread_ts`
142
+ strings are still read.
143
+
144
+ Redis comes from Mage `io_config.yaml` (`REDIS_HOST` / `REDIS_PORT` /
145
+ `REDIS_PASSWORD`), or those same env vars. If Redis is unavailable, gitpuller
146
+ falls back to Slack `conversations.history`.
147
+
148
+ | Env var | Role |
149
+ |---------|------|
150
+ | `CDM_PROD_SLACK_BOT_TOKEN` | Required for threading. |
151
+ | `CDM_PROD_DB_SLACK_CHANNEL` | Channel; default `C05MLHR55JT`. |
152
+ | `CDM_SLACK_THREAD_TZ` | Timezone for the daily parent date; default `UTC`. |
153
+ | `CDM_PAUSE_SLACK_MESSAGES` | Set to `1` to skip Slack. |
154
+ | `CDM_SLACK_WEBHOOK_URL` | Legacy fallback when no bot token is set. |
155
+
156
+ The bot must be in the channel (`chat:write`). History reuse also needs
157
+ `channels:history` (or `groups:history` for a private channel).
158
+
115
159
  ### `execute_with_alerting(...)` → `dict`
116
160
 
117
161
  Runs the sync and, on failure, alerts Slack (with suppression) then re-raises.
@@ -127,6 +171,7 @@ Runs the sync and, on failure, alerts Slack (with suppression) then re-raises.
127
171
  | `suppression_hours` | `1` | Don't re-alert on the *same* error within this many hours. |
128
172
  | `key_filename` | `None` | Override the on-disk key filename. |
129
173
  | `ssh_dir` | `"/home/src/.ssh"` | Directory to write the key into. |
174
+ | `webhook_url` | `None` | Legacy webhook override for this call. Ignored when a bot token is configured. |
130
175
 
131
176
  ### `execute_git_pull(...)` → `dict`
132
177
 
@@ -210,7 +255,21 @@ Keep the version in sync in **both** `pyproject.toml` and `gitpuller/__init__.py
210
255
 
211
256
  ## Changelog
212
257
 
213
- ### 1.1.0 (current)
258
+ ### 1.2.0 (current)
259
+
260
+ - **Threaded Slack alerts.** Failures post as replies under one daily parent
261
+ **per workspace** (`🚨 Git Pull Failures {workspace} — YYYY-MM-DD`) via
262
+ `CDM_PROD_SLACK_BOT_TOKEN` and `CDM_PROD_DB_SLACK_CHANNEL`
263
+ (default `C05MLHR55JT`). Daily `thread_ts` lives in one unified Redis hash
264
+ (`gitpuller:slack_thread:{channel}:{YYYY-MM-DD}`, field = workspace, TTL
265
+ until midnight) shared across all Mage workspaces. Each workspace field is
266
+ JSON: `thread_ts` plus that day's git errors.
267
+ - Incoming webhooks (`CDM_SLACK_WEBHOOK_URL` / `slack_webhook_url`) remain as a
268
+ non-threaded fallback when no bot token is set.
269
+ - Missing Slack credentials no longer raise on `GitPullExecutor` construction;
270
+ alerts are skipped with a warning.
271
+
272
+ ### 1.1.0
214
273
 
215
274
  Reliability and clarity overhaul.
216
275
 
@@ -1,8 +1,3 @@
1
- """gitpuller — auto-pull a git repo over SSH inside Mage pipelines.
2
-
3
- Public API is re-exported here so callers can ``from gitpuller import ...``.
4
- """
5
-
6
1
  from .gitpull import GitPullExecutor
7
2
  from .alert_manager import AlertManager
8
3
  from .state_manager import StateManager, InMemoryStateManager, MageAIStateManager
@@ -10,7 +5,7 @@ from .slack_notifier import SlackNotifier
10
5
  from .utils import transform_custom, get_repo_path, get_env_base_path
11
6
 
12
7
 
13
- __version__ = "1.1.0"
8
+ __version__ = "1.2.0"
14
9
  __all__ = [
15
10
  "GitPullExecutor",
16
11
  "AlertManager",
@@ -1,21 +1,12 @@
1
- """Alert de-duplication: decides whether a failure should page Slack."""
2
-
3
1
  from datetime import datetime, timedelta
4
2
  from typing import Any, Dict, Optional, Tuple
5
3
  from .state_manager import StateManager, InMemoryStateManager, MageAIStateManager
6
4
 
7
-
8
- class AlertManager:
9
- """Wraps a ``StateManager`` to suppress repeated identical alerts."""
10
-
5
+ class AlertManager:
11
6
  def __init__(self, state_manager: Optional[StateManager] = None, use_mage_ai: bool = False):
12
- # Pick the backing store for "last alert" bookkeeping:
13
7
  if state_manager is not None:
14
- # Caller supplied one explicitly.
15
8
  self.state_manager = state_manager
16
9
  elif use_mage_ai:
17
- # Persist across runs via Mage global variables when available,
18
- # otherwise degrade gracefully to in-memory (per-process) state.
19
10
  try:
20
11
  self.state_manager = MageAIStateManager()
21
12
  except ImportError:
@@ -23,43 +14,37 @@ class AlertManager:
23
14
  self.state_manager = InMemoryStateManager()
24
15
  else:
25
16
  self.state_manager = InMemoryStateManager()
26
-
17
+
27
18
  def should_send_alert(
28
- self,
29
- pipeline_uuid: str,
30
- current_error: str,
19
+ self,
20
+ pipeline_uuid: str,
21
+ current_error: str,
31
22
  suppression_hours: int = 1
32
23
  ) -> Tuple[bool, Dict[str, Any]]:
33
- """
34
- Decide whether to alert for ``current_error``.
35
-
36
- Rule: alert immediately on a first-ever or *changed* error; for an
37
- unchanged error, only re-alert once ``suppression_hours`` has elapsed.
38
- Returns ``(should_send, previous_state)``.
39
- """
24
+
40
25
  state = self.state_manager.load_alert_state(pipeline_uuid)
41
26
  last_error = state.get("last_error_message")
42
27
  last_alert_time_str = state.get("last_alert_time")
43
-
44
- # No prior alert on record — always send.
28
+
29
+ # If no previous alert, always send
45
30
  if not last_error or not last_alert_time_str:
46
31
  return True, {"last_error_message": None, "last_alert_time": None}
47
-
48
- # A different error than last time — always send.
32
+
33
+ # If error is different, always send
49
34
  if last_error != current_error:
50
35
  return True, {"last_error_message": last_error, "last_alert_time": last_alert_time_str}
51
-
52
- # Same error as last time — only re-send once the window has elapsed.
36
+
37
+ # Same error - check if enough time has passed
53
38
  try:
54
39
  last_alert_time = datetime.fromisoformat(last_alert_time_str)
55
40
  time_since_last_alert = datetime.now() - last_alert_time
56
-
41
+
57
42
  if time_since_last_alert >= timedelta(hours=suppression_hours):
58
43
  return True, {"last_error_message": last_error, "last_alert_time": last_alert_time_str}
59
44
  else:
60
45
  return False, {"last_error_message": last_error, "last_alert_time": last_alert_time_str}
61
46
  except (ValueError, TypeError):
62
- # Unparseable timestamp — fail open and alert rather than stay silent.
47
+ # If we can't parse the time, send the alert to be safe
63
48
  return True, {"last_error_message": last_error, "last_alert_time": last_alert_time_str}
64
49
 
65
50
  def save_alert_state(
@@ -22,11 +22,17 @@ class GitPullExecutor:
22
22
  def __init__(
23
23
  self,
24
24
  slack_webhook_url: Optional[str] = None,
25
+ slack_bot_token: Optional[str] = None,
26
+ slack_channel: Optional[str] = None,
25
27
  use_mage_ai: bool = False,
26
28
  state_manager: Optional[Any] = None
27
29
  ):
28
- # Slack sink for failure alerts.
29
- self.slack_notifier = SlackNotifier(webhook_url=slack_webhook_url)
30
+ # Slack sink for failure alerts (bot token + daily thread preferred).
31
+ self.slack_notifier = SlackNotifier(
32
+ webhook_url=slack_webhook_url,
33
+ bot_token=slack_bot_token,
34
+ channel=slack_channel,
35
+ )
30
36
  # Decides whether/when to alert (de-duplicates repeated identical errors).
31
37
  self.alert_manager = AlertManager(state_manager=state_manager, use_mage_ai=use_mage_ai)
32
38
 
@@ -264,7 +270,12 @@ class GitPullExecutor:
264
270
  ``suppression_hours`` caps how often the *same* error is alerted on,
265
271
  avoiding Slack spam when a broken state persists across many runs.
266
272
  """
267
- notifier = SlackNotifier(webhook_url=webhook_url) if webhook_url else self.slack_notifier
273
+ # webhook_url is a fallback only; CDM_PROD_SLACK_BOT_TOKEN still wins
274
+ # so Mage callers that pass CDM_SLACK_WEBHOOK_URL stay on the daily thread.
275
+ if webhook_url and not self.slack_notifier.bot_token:
276
+ notifier = SlackNotifier(webhook_url=webhook_url)
277
+ else:
278
+ notifier = self.slack_notifier
268
279
  # Derive a friendly repo name (e.g. "partner-mageai") for the alert.
269
280
  repo_name = git_url.split('/')[-1].replace('.git', '')
270
281
 
@@ -300,7 +311,11 @@ class GitPullExecutor:
300
311
 
301
312
  if should_alert:
302
313
  # Post to Slack (cap payload so we don't blow Slack's limits).
303
- notifier.send_alert(repo_name, git_output[:1500])
314
+ notifier.send_alert(
315
+ repo_name,
316
+ git_output[:1500],
317
+ workspace_name=workspace_name,
318
+ )
304
319
 
305
320
  # Record that we alerted so future identical errors are suppressed.
306
321
  self.alert_manager.save_alert_state(
@@ -0,0 +1,579 @@
1
+ import json
2
+ import os
3
+ import time
4
+ from datetime import datetime, timedelta
5
+ from typing import Any, Dict, List, Optional
6
+ from zoneinfo import ZoneInfo
7
+
8
+ import requests
9
+
10
+ SLACK_POST_MESSAGE_URL = "https://slack.com/api/chat.postMessage"
11
+ SLACK_CONVERSATIONS_HISTORY_URL = "https://slack.com/api/conversations.history"
12
+ DEFAULT_SLACK_CHANNEL = "C05MLHR55JT"
13
+
14
+ # Unified across every Mage deployment — do not prefix with MAGE_WORKSPACE_NAME.
15
+ # One hash per channel per day; field = gitpuller workspace, value = Slack thread_ts.
16
+ REDIS_THREAD_KEY_PREFIX = "gitpuller:slack_thread"
17
+ REDIS_CREATE_LOCK_TTL_SECONDS = 120
18
+ REDIS_ERROR_MAX_CHARS = 1500
19
+ REDIS_ERRORS_PER_WORKSPACE = 20
20
+
21
+ _redis_client = None
22
+ _redis_init_attempted = False
23
+
24
+
25
+ def _get_redis_client():
26
+ """Mage io_config.yaml first, then REDIS_HOST / REDIS_PORT / REDIS_PASSWORD."""
27
+ global _redis_client, _redis_init_attempted
28
+ if _redis_client is not None:
29
+ return _redis_client
30
+ if _redis_init_attempted:
31
+ return None
32
+ _redis_init_attempted = True
33
+
34
+ try:
35
+ import redis as redis_lib
36
+ except ImportError:
37
+ print("⚠️ redis package not installed; Slack thread_ts will not persist across pods")
38
+ return None
39
+
40
+ host = None
41
+ password = None
42
+ port = 6379
43
+ try:
44
+ from mage_ai.io.config import ConfigFileLoader
45
+ from mage_ai.settings.repo import get_repo_path
46
+
47
+ loader = ConfigFileLoader(os.path.join(get_repo_path(), "io_config.yaml"), "default")
48
+ host = loader["REDIS_HOST"]
49
+ password = loader["REDIS_PASSWORD"] or None
50
+ port = int(loader["REDIS_PORT"])
51
+ except Exception:
52
+ host = None
53
+
54
+ if not host:
55
+ host = (os.environ.get("REDIS_HOST") or "").strip() or None
56
+ password = (os.environ.get("REDIS_PASSWORD") or "").strip() or None
57
+ port = int(os.environ.get("REDIS_PORT") or 6379)
58
+
59
+ if not host:
60
+ print("⚠️ Redis is not configured; Slack thread_ts will not persist across pods")
61
+ return None
62
+
63
+ try:
64
+ _redis_client = redis_lib.Redis(
65
+ host=host,
66
+ password=password or None,
67
+ port=port,
68
+ db=0,
69
+ decode_responses=True,
70
+ socket_connect_timeout=5,
71
+ socket_timeout=5,
72
+ )
73
+ _redis_client.ping()
74
+ return _redis_client
75
+ except Exception as exc:
76
+ print(f"⚠️ Redis connection failed: {exc}")
77
+ _redis_client = None
78
+ _redis_init_attempted = False
79
+ return None
80
+
81
+
82
+ class SlackNotifier:
83
+ """Posts formatted failure messages to Slack.
84
+
85
+ Bot-token path (preferred): one parent message per workspace per calendar
86
+ day, each failure as a thread reply. Daily ``thread_ts`` values live in one
87
+ unified Redis hash (all Mage workspaces share it; TTL until midnight).
88
+ Webhook path: a standalone channel post.
89
+ """
90
+
91
+ def __init__(
92
+ self,
93
+ webhook_url: Optional[str] = None,
94
+ bot_token: Optional[str] = None,
95
+ channel: Optional[str] = None,
96
+ ):
97
+ self.bot_token = (bot_token or os.environ.get("CDM_PROD_SLACK_BOT_TOKEN") or "").strip() or None
98
+ self.channel = (
99
+ (channel or os.environ.get("CDM_PROD_DB_SLACK_CHANNEL") or "").strip()
100
+ or DEFAULT_SLACK_CHANNEL
101
+ )
102
+ self.webhook_url = (webhook_url or os.environ.get("CDM_SLACK_WEBHOOK_URL") or "").strip() or None
103
+ self._thread_ts_cache: Dict[str, str] = {}
104
+
105
+ if not self.bot_token and not self.webhook_url:
106
+ print(
107
+ "⚠️ Slack not configured; skipping alerts "
108
+ "(set CDM_PROD_SLACK_BOT_TOKEN, or CDM_SLACK_WEBHOOK_URL as fallback)"
109
+ )
110
+ elif not self.bot_token:
111
+ print(
112
+ "⚠️ CDM_PROD_SLACK_BOT_TOKEN is not set; using webhook "
113
+ "(alerts will not be threaded)"
114
+ )
115
+
116
+ @staticmethod
117
+ def _is_paused() -> bool:
118
+ flag = os.environ.get("CDM_PAUSE_SLACK_MESSAGES", "0").lower()
119
+ return flag in ("1", "true", "yes")
120
+
121
+ @staticmethod
122
+ def _thread_tz() -> str:
123
+ return os.environ.get("CDM_SLACK_THREAD_TZ", "UTC")
124
+
125
+ def _today_date_key(self) -> str:
126
+ return datetime.now(ZoneInfo(self._thread_tz())).strftime("%Y-%m-%d")
127
+
128
+ def _seconds_until_end_of_day(self) -> int:
129
+ tz = ZoneInfo(self._thread_tz())
130
+ now = datetime.now(tz)
131
+ start_next_day = (now + timedelta(days=1)).replace(
132
+ hour=0, minute=0, second=0, microsecond=0
133
+ )
134
+ return max(1, int((start_next_day - now).total_seconds()))
135
+
136
+ @staticmethod
137
+ def _workspace_label(workspace_name: Optional[str] = None) -> str:
138
+ return (
139
+ (workspace_name or "").strip()
140
+ or (os.environ.get("MAGE_WORKSPACE_NAME") or "").strip()
141
+ or "unknown"
142
+ )
143
+
144
+ def _cache_key(self, date_key: str, workspace: str) -> str:
145
+ return f"{date_key}:{workspace}"
146
+
147
+ def _parent_text(self, date_key: str, workspace: str) -> str:
148
+ return f"Git Pull Failures {workspace} — daily thread ({date_key})"
149
+
150
+ def _parent_header(self, date_key: str, workspace: str) -> str:
151
+ return f"🚨 Git Pull Failures {workspace} — {date_key}"
152
+
153
+ def _redis_daily_key(self, date_key: str) -> str:
154
+ """One hash for every gitpuller workspace on this channel today."""
155
+ return f"{REDIS_THREAD_KEY_PREFIX}:{self.channel}:{date_key}"
156
+
157
+ def _redis_create_lock_key(self, date_key: str, workspace: str) -> str:
158
+ return f"{self._redis_daily_key(date_key)}:{workspace}:creating"
159
+
160
+ def _remember_thread_ts(self, date_key: str, workspace: str, thread_ts: str) -> None:
161
+ self._thread_ts_cache[self._cache_key(date_key, workspace)] = thread_ts
162
+
163
+ @staticmethod
164
+ def _parse_workspace_record(raw: Any) -> Dict[str, Any]:
165
+ """Accept JSON records and the legacy plain thread_ts string."""
166
+ empty: Dict[str, Any] = {"workspace": None, "thread_ts": None, "errors": []}
167
+ if raw is None:
168
+ return empty
169
+ if isinstance(raw, dict):
170
+ data = raw
171
+ else:
172
+ text = str(raw).strip()
173
+ if not text:
174
+ return empty
175
+ if text.startswith("{"):
176
+ try:
177
+ data = json.loads(text)
178
+ except json.JSONDecodeError:
179
+ return {"workspace": None, "thread_ts": text, "errors": []}
180
+ else:
181
+ return {"workspace": None, "thread_ts": text, "errors": []}
182
+ if not isinstance(data, dict):
183
+ return empty
184
+ errors = data.get("errors") or []
185
+ if not isinstance(errors, list):
186
+ errors = []
187
+ thread_ts = data.get("thread_ts")
188
+ workspace_name = data.get("workspace")
189
+ return {
190
+ "workspace": str(workspace_name).strip() if workspace_name else None,
191
+ "thread_ts": str(thread_ts).strip() if thread_ts else None,
192
+ "errors": errors,
193
+ }
194
+
195
+ def _load_workspace_record(self, date_key: str, workspace: str) -> Dict[str, Any]:
196
+ client = _get_redis_client()
197
+ if not client:
198
+ return {"workspace": workspace, "thread_ts": None, "errors": []}
199
+ redis_key = self._redis_daily_key(date_key)
200
+ try:
201
+ raw = client.hget(redis_key, workspace)
202
+ except Exception as exc:
203
+ print(f"⚠️ Redis HGET failed for {redis_key} field={workspace}: {exc}")
204
+ return {"workspace": workspace, "thread_ts": None, "errors": []}
205
+ record = self._parse_workspace_record(raw)
206
+ record["workspace"] = record.get("workspace") or workspace
207
+ return record
208
+
209
+ def _save_workspace_record(
210
+ self,
211
+ date_key: str,
212
+ workspace: str,
213
+ record: Dict[str, Any],
214
+ log_message: Optional[str] = None,
215
+ ) -> None:
216
+ client = _get_redis_client()
217
+ if not client:
218
+ return
219
+ redis_key = self._redis_daily_key(date_key)
220
+ ttl_seconds = self._seconds_until_end_of_day()
221
+ record = dict(record)
222
+ record["workspace"] = workspace
223
+ payload = json.dumps(record, ensure_ascii=False)
224
+ try:
225
+ pipe = client.pipeline()
226
+ pipe.hset(redis_key, workspace, payload)
227
+ pipe.expire(redis_key, ttl_seconds)
228
+ pipe.execute()
229
+ print(
230
+ log_message
231
+ or (
232
+ f"✅ Saved gitpuller Slack record to Redis "
233
+ f"(key={redis_key}, workspace={workspace}, ttl={ttl_seconds}s)"
234
+ )
235
+ )
236
+ except Exception as exc:
237
+ print(f"⚠️ Redis HSET failed for {redis_key} field={workspace}: {exc}")
238
+
239
+ def _read_cached_thread_ts(self, date_key: str, workspace: str) -> Optional[str]:
240
+ cached = self._thread_ts_cache.get(self._cache_key(date_key, workspace))
241
+ if cached:
242
+ return cached
243
+
244
+ record = self._load_workspace_record(date_key, workspace)
245
+ thread_ts = record.get("thread_ts")
246
+ if thread_ts:
247
+ self._remember_thread_ts(date_key, workspace, thread_ts)
248
+ redis_key = self._redis_daily_key(date_key)
249
+ print(
250
+ f"ℹ️ Loaded daily Slack thread from Redis "
251
+ f"(key={redis_key}, workspace={workspace}, ts={thread_ts})"
252
+ )
253
+ return thread_ts or None
254
+
255
+ def _write_cached_thread_ts(self, date_key: str, workspace: str, thread_ts: str) -> None:
256
+ self._remember_thread_ts(date_key, workspace, thread_ts)
257
+ record = self._load_workspace_record(date_key, workspace)
258
+ record["workspace"] = workspace
259
+ record["thread_ts"] = thread_ts
260
+ redis_key = self._redis_daily_key(date_key)
261
+ ttl_seconds = self._seconds_until_end_of_day()
262
+ self._save_workspace_record(
263
+ date_key,
264
+ workspace,
265
+ record,
266
+ log_message=(
267
+ f"✅ Saved daily Slack thread to Redis "
268
+ f"(key={redis_key}, workspace={workspace}, ts={thread_ts}, ttl={ttl_seconds}s)"
269
+ ),
270
+ )
271
+
272
+ def _append_redis_error(
273
+ self,
274
+ workspace: str,
275
+ repo_name: str,
276
+ error_output: str,
277
+ thread_ts: Optional[str] = None,
278
+ reply_ts: Optional[str] = None,
279
+ ) -> None:
280
+ if not _get_redis_client():
281
+ return
282
+ date_key = self._today_date_key()
283
+ if thread_ts:
284
+ self._remember_thread_ts(date_key, workspace, thread_ts)
285
+ record = self._load_workspace_record(date_key, workspace)
286
+ record["workspace"] = workspace
287
+ if thread_ts:
288
+ record["thread_ts"] = thread_ts
289
+ error_entry = {
290
+ "workspace": workspace,
291
+ "repo": repo_name,
292
+ "error": (error_output or "")[:REDIS_ERROR_MAX_CHARS],
293
+ "at": datetime.now(ZoneInfo(self._thread_tz())).strftime("%Y-%m-%d %H:%M:%S %Z"),
294
+ }
295
+ if reply_ts:
296
+ error_entry["reply_ts"] = reply_ts
297
+ errors = record.get("errors") or []
298
+ errors.append(error_entry)
299
+ record["errors"] = errors[-REDIS_ERRORS_PER_WORKSPACE:]
300
+ redis_key = self._redis_daily_key(date_key)
301
+ self._save_workspace_record(
302
+ date_key,
303
+ workspace,
304
+ record,
305
+ log_message=(
306
+ f"✅ Saved git error to Redis "
307
+ f"(key={redis_key}, workspace={workspace}, repo={repo_name}, "
308
+ f"errors={len(record['errors'])})"
309
+ ),
310
+ )
311
+
312
+ def _acquire_create_lock(self, date_key: str, workspace: str) -> bool:
313
+ client = _get_redis_client()
314
+ if not client:
315
+ return True
316
+ lock_key = self._redis_create_lock_key(date_key, workspace)
317
+ try:
318
+ return bool(
319
+ client.set(lock_key, "1", nx=True, ex=REDIS_CREATE_LOCK_TTL_SECONDS)
320
+ )
321
+ except Exception as exc:
322
+ print(f"⚠️ Redis SET NX failed for {lock_key}: {exc}")
323
+ return True
324
+
325
+ def _release_create_lock(self, date_key: str, workspace: str) -> None:
326
+ client = _get_redis_client()
327
+ if not client:
328
+ return
329
+ lock_key = self._redis_create_lock_key(date_key, workspace)
330
+ try:
331
+ client.delete(lock_key)
332
+ except Exception as exc:
333
+ print(f"⚠️ Redis DELETE failed for {lock_key}: {exc}")
334
+
335
+ def _auth_headers(self) -> Dict[str, str]:
336
+ return {
337
+ "Authorization": f"Bearer {self.bot_token}",
338
+ "Content-Type": "application/json; charset=utf-8",
339
+ }
340
+
341
+ def _post_bot_message(
342
+ self,
343
+ text: str,
344
+ blocks: Optional[List[Dict[str, Any]]] = None,
345
+ thread_ts: Optional[str] = None,
346
+ ) -> Optional[str]:
347
+ payload: Dict[str, Any] = {
348
+ "channel": self.channel,
349
+ "text": text,
350
+ "mrkdwn": True,
351
+ }
352
+ if blocks:
353
+ payload["blocks"] = blocks
354
+ if thread_ts:
355
+ payload["thread_ts"] = thread_ts
356
+
357
+ response = requests.post(
358
+ SLACK_POST_MESSAGE_URL,
359
+ headers=self._auth_headers(),
360
+ json=payload,
361
+ timeout=10,
362
+ )
363
+ response.raise_for_status()
364
+ body = response.json()
365
+ if not body.get("ok"):
366
+ raise RuntimeError(body.get("error", "unknown Slack API error"))
367
+ return body.get("ts")
368
+
369
+ def _find_existing_parent_ts(self, date_key: str, workspace: str) -> Optional[str]:
370
+ """Look in recent channel history for today's gitpuller parent message."""
371
+ marker = self._parent_text(date_key, workspace)
372
+ try:
373
+ response = requests.get(
374
+ SLACK_CONVERSATIONS_HISTORY_URL,
375
+ headers=self._auth_headers(),
376
+ params={"channel": self.channel, "limit": 100},
377
+ timeout=10,
378
+ )
379
+ response.raise_for_status()
380
+ body = response.json()
381
+ if not body.get("ok"):
382
+ print(f"⚠️ Slack history lookup failed: {body.get('error')}")
383
+ return None
384
+ for message in body.get("messages") or []:
385
+ text = message.get("text") or ""
386
+ ts = message.get("ts")
387
+ thread_ts = message.get("thread_ts")
388
+ if marker in text and ts and thread_ts in (None, ts):
389
+ return ts
390
+ except Exception as exc:
391
+ print(f"⚠️ Slack history lookup failed: {exc}")
392
+ return None
393
+
394
+ def _create_daily_parent(self, date_key: str, workspace: str) -> Optional[str]:
395
+ parent_text = self._parent_text(date_key, workspace)
396
+ parent_blocks = [
397
+ {
398
+ "type": "header",
399
+ "text": {
400
+ "type": "plain_text",
401
+ "text": self._parent_header(date_key, workspace),
402
+ "emoji": True,
403
+ },
404
+ },
405
+ {
406
+ "type": "section",
407
+ "text": {
408
+ "type": "mrkdwn",
409
+ "text": (
410
+ f"All gitpuller failure alerts for `{workspace}` today "
411
+ "are posted in this thread."
412
+ ),
413
+ },
414
+ },
415
+ ]
416
+ return self._post_bot_message(parent_text, blocks=parent_blocks)
417
+
418
+ def _wait_for_redis_thread_ts(
419
+ self,
420
+ date_key: str,
421
+ workspace: str,
422
+ attempts: int = 10,
423
+ delay_seconds: float = 0.5,
424
+ ) -> Optional[str]:
425
+ if not _get_redis_client():
426
+ return None
427
+ for _ in range(attempts):
428
+ thread_ts = self._read_cached_thread_ts(date_key, workspace)
429
+ if thread_ts:
430
+ return thread_ts
431
+ time.sleep(delay_seconds)
432
+ return None
433
+
434
+ def _get_or_create_daily_thread_ts(self, workspace: str) -> Optional[str]:
435
+ date_key = self._today_date_key()
436
+ thread_ts = self._read_cached_thread_ts(date_key, workspace)
437
+ if thread_ts:
438
+ return thread_ts
439
+
440
+ if not self._acquire_create_lock(date_key, workspace):
441
+ thread_ts = self._wait_for_redis_thread_ts(date_key, workspace)
442
+ if thread_ts:
443
+ print(
444
+ f"ℹ️ Loaded daily Slack thread created by another run "
445
+ f"(workspace={workspace}, ts={thread_ts})"
446
+ )
447
+ return thread_ts
448
+ print(
449
+ f"❌ Timed out waiting for daily Slack thread in Redis "
450
+ f"(workspace={workspace})"
451
+ )
452
+ return None
453
+
454
+ try:
455
+ thread_ts = self._read_cached_thread_ts(date_key, workspace)
456
+ if thread_ts:
457
+ return thread_ts
458
+
459
+ thread_ts = self._find_existing_parent_ts(date_key, workspace)
460
+ if thread_ts:
461
+ self._write_cached_thread_ts(date_key, workspace, thread_ts)
462
+ print(
463
+ f"ℹ️ Reused existing daily Slack thread "
464
+ f"(workspace={workspace}, ts={thread_ts})"
465
+ )
466
+ return thread_ts
467
+
468
+ parent_ts = self._create_daily_parent(date_key, workspace)
469
+ if not parent_ts:
470
+ print("❌ Slack parent message did not return thread_ts")
471
+ return None
472
+
473
+ self._write_cached_thread_ts(date_key, workspace, parent_ts)
474
+ return parent_ts
475
+ finally:
476
+ self._release_create_lock(date_key, workspace)
477
+
478
+ def create_failure_payload(
479
+ self,
480
+ repo_name: str,
481
+ error_output: str,
482
+ workspace_name: Optional[str] = None,
483
+ ) -> Dict[str, Any]:
484
+ """Build the Slack Block Kit payload describing a pull failure."""
485
+ reported_at = datetime.now(ZoneInfo(self._thread_tz())).strftime(
486
+ "%Y-%m-%d %H:%M:%S %Z"
487
+ )
488
+ workspace = self._workspace_label(workspace_name)
489
+ details = [
490
+ f"*Workspace:* `{workspace}`",
491
+ f"*Repository:* `{repo_name}`",
492
+ f"*Reported at:* `{reported_at}`",
493
+ ]
494
+
495
+ payload: Dict[str, Any] = {
496
+ "text": f"Automate Git Pull Pipeline Failed — {workspace} / {repo_name}",
497
+ "blocks": [
498
+ {
499
+ "type": "header",
500
+ "text": {
501
+ "type": "plain_text",
502
+ "text": ":alert: Automate Git Pull Pipeline Failed :alert:",
503
+ },
504
+ },
505
+ {"type": "divider"},
506
+ {
507
+ "type": "section",
508
+ "text": {
509
+ "type": "mrkdwn",
510
+ "text": "\n".join(details),
511
+ },
512
+ },
513
+ ],
514
+ }
515
+
516
+ if error_output:
517
+ payload["blocks"].append({
518
+ "type": "section",
519
+ "text": {
520
+ "type": "mrkdwn",
521
+ "text": f"*Error Output:*\n```{error_output[:1500]}```",
522
+ },
523
+ })
524
+
525
+ return payload
526
+
527
+ def send_alert(
528
+ self,
529
+ repo_name: str,
530
+ error_output: str = "",
531
+ workspace_name: Optional[str] = None,
532
+ ) -> bool:
533
+ """Send the failure alert. Returns True on success, False on any error
534
+ (delivery failures are logged but never raised, so alerting can't mask
535
+ the original git error)."""
536
+ if self._is_paused():
537
+ print("ℹ️ Slack notifications paused via CDM_PAUSE_SLACK_MESSAGES")
538
+ return False
539
+
540
+ workspace = self._workspace_label(workspace_name)
541
+ payload = self.create_failure_payload(repo_name, error_output, workspace)
542
+ thread_ts: Optional[str] = None
543
+ reply_ts: Optional[str] = None
544
+ posted = False
545
+
546
+ try:
547
+ if self.bot_token:
548
+ thread_ts = self._get_or_create_daily_thread_ts(workspace)
549
+ if not thread_ts:
550
+ print("❌ Could not resolve daily Slack thread_ts")
551
+ else:
552
+ reply_ts = self._post_bot_message(
553
+ payload["text"],
554
+ blocks=payload["blocks"],
555
+ thread_ts=thread_ts,
556
+ )
557
+ print(
558
+ f"✅ Posted Slack alert as thread reply "
559
+ f"(thread_ts={thread_ts}, reply_ts={reply_ts})"
560
+ )
561
+ posted = True
562
+ elif self.webhook_url:
563
+ response = requests.post(self.webhook_url, json=payload, timeout=10)
564
+ response.raise_for_status()
565
+ print("✅ Alert sent to Slack (webhook, not threaded)")
566
+ posted = True
567
+ else:
568
+ print("⚠️ Slack not configured; skipping alert")
569
+ except Exception as exc:
570
+ print(f"⚠️ Failed to send Slack alert: {exc}")
571
+
572
+ self._append_redis_error(
573
+ workspace,
574
+ repo_name,
575
+ error_output,
576
+ thread_ts=thread_ts,
577
+ reply_ts=reply_ts,
578
+ )
579
+ return posted
@@ -1,16 +1,7 @@
1
- """Pluggable storage backends for alert-suppression state.
2
-
3
- ``InMemoryStateManager`` keeps state for the lifetime of the process; the Mage
4
- variant persists it across pipeline runs via Mage global variables.
5
- """
6
-
7
1
  from typing import Any, Dict, Optional
8
2
  from datetime import datetime
9
3
 
10
-
11
- class StateManager:
12
- """Abstract interface for persisting per-pipeline alert state."""
13
-
4
+ class StateManager:
14
5
  def load_alert_state(self, pipeline_uuid: str) -> Dict[str, Any]:
15
6
  """Load alert state for a pipeline."""
16
7
  raise NotImplementedError
@@ -29,11 +20,7 @@ class StateManager:
29
20
  """Clear alert state for a pipeline."""
30
21
  raise NotImplementedError
31
22
 
32
- class InMemoryStateManager(StateManager):
33
- """Process-local state. Note: state is lost when the process exits, so
34
- alert suppression only works within a single run unless a persistent
35
- backend (e.g. Mage) is used."""
36
-
23
+ class InMemoryStateManager(StateManager):
37
24
  def __init__(self):
38
25
  self._state: Dict[str, Dict[str, Any]] = {}
39
26
 
@@ -68,15 +55,11 @@ class InMemoryStateManager(StateManager):
68
55
  "pipeline_status": "success"
69
56
  }
70
57
 
71
- class MageAIStateManager(StateManager):
72
- """Persists alert state across runs using Mage AI global variables."""
73
-
58
+ class MageAIStateManager(StateManager):
74
59
  def __init__(self):
75
- # Import lazily so the package works outside a Mage environment;
76
- # AlertManager catches this ImportError to fall back to in-memory.
77
60
  try:
78
61
  from mage_ai.data_preparation.variable_manager import (
79
- set_global_variable,
62
+ set_global_variable,
80
63
  get_global_variable
81
64
  )
82
65
  self._set_global_variable = set_global_variable
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "salla_gitpuller"
7
- version = "1.1.2"
7
+ version = "1.2.0"
8
8
  description = "A lightweight utility to git pull a repository using SSH deploy keys stored in environment variables"
9
9
  readme = "README.md"
10
10
  authors = [
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: salla_gitpuller
3
- Version: 1.1.2
3
+ Version: 1.2.0
4
4
  Summary: A lightweight utility to git pull a repository using SSH deploy keys stored in environment variables
5
5
  Author-email: Mohammed Junaid <safijunaid.ss@gmail.com>, Muhammad Zahid <zahidmuhammad127@gmail.com>
6
6
  License: MIT
@@ -58,8 +58,9 @@ state to persist across pipeline runs (see [State management](#state-management)
58
58
  from gitpuller import GitPullExecutor
59
59
 
60
60
  executor = GitPullExecutor(
61
- slack_webhook_url="https://hooks.slack.com/services/XXX", # or set CDM_SLACK_WEBHOOK_URL
62
- use_mage_ai=True, # persist alert state via Mage
61
+ use_mage_ai=True, # persist alert state via Mage
62
+ # Slack: set CDM_PROD_SLACK_BOT_TOKEN (threaded daily alerts).
63
+ # Optional: CDM_PROD_DB_SLACK_CHANNEL (defaults to C05MLHR55JT).
63
64
  )
64
65
 
65
66
  result = executor.execute_with_alerting(
@@ -74,7 +75,8 @@ print(result["discarded_changes"]) # what local drift (if any) was wiped
74
75
  ```
75
76
 
76
77
  On failure, `execute_with_alerting` sends a Slack alert (subject to suppression)
77
- and then **re-raises**, so the Mage pipeline still fails loudly.
78
+ as a **reply in today's daily thread**, then **re-raises**, so the Mage pipeline
79
+ still fails loudly.
78
80
 
79
81
  ---
80
82
 
@@ -116,14 +118,56 @@ and exit code**, which becomes the Slack alert body and the pipeline error.
116
118
 
117
119
  ## API
118
120
 
119
- ### `GitPullExecutor(slack_webhook_url=None, use_mage_ai=False, state_manager=None)`
121
+ ### `GitPullExecutor(slack_webhook_url=None, slack_bot_token=None, slack_channel=None, use_mage_ai=False, state_manager=None)`
120
122
 
121
123
  | Param | Description |
122
124
  |-------|-------------|
123
- | `slack_webhook_url` | Slack incoming-webhook URL. Falls back to the `CDM_SLACK_WEBHOOK_URL` env var. Required (one of the two must be set). |
125
+ | `slack_bot_token` | Slack bot token (`xoxb-...`). Falls back to `CDM_PROD_SLACK_BOT_TOKEN`. Preferred — enables daily-thread replies. |
126
+ | `slack_channel` | Channel ID/name. Falls back to `CDM_PROD_DB_SLACK_CHANNEL`, then `C05MLHR55JT`. |
127
+ | `slack_webhook_url` | Incoming-webhook URL. Falls back to `CDM_SLACK_WEBHOOK_URL`. Used only when no bot token is set (no threading). |
124
128
  | `use_mage_ai` | If `True`, persist alert-suppression state via Mage global variables (falls back to in-memory if Mage isn't installed). |
125
129
  | `state_manager` | Inject a custom `StateManager`; overrides `use_mage_ai`. |
126
130
 
131
+ If neither a bot token nor a webhook is configured, gitpuller still runs and
132
+ prints a warning instead of failing construction.
133
+
134
+ ### Slack daily thread
135
+
136
+ When `CDM_PROD_SLACK_BOT_TOKEN` is set, failures are posted with `chat.postMessage`:
137
+
138
+ 1. Open (or reuse) one parent message **per workspace per calendar day**:
139
+ `🚨 Git Pull Failures {workspace_name} — YYYY-MM-DD`.
140
+ 2. Post each later failure for that workspace as a **thread reply**.
141
+
142
+ Daily `thread_ts` values are stored in **one unified Redis hash** shared by every
143
+ Mage workspace (the key is **not** prefixed with `MAGE_WORKSPACE_NAME`):
144
+
145
+ ```text
146
+ gitpuller:slack_thread:{channel}:{YYYY-MM-DD}
147
+ cloud_data → JSON { workspace, thread_ts, errors: [{workspace, repo, error, at, reply_ts}, ...] }
148
+ partner → JSON { workspace, thread_ts, errors: [...] }
149
+ TTL: until midnight (CDM_SLACK_THREAD_TZ, default UTC)
150
+ ```
151
+
152
+ Each workspace keeps the Slack parent `thread_ts` plus up to 20 git errors
153
+ from that day (error text capped at 1500 chars). Legacy plain `thread_ts`
154
+ strings are still read.
155
+
156
+ Redis comes from Mage `io_config.yaml` (`REDIS_HOST` / `REDIS_PORT` /
157
+ `REDIS_PASSWORD`), or those same env vars. If Redis is unavailable, gitpuller
158
+ falls back to Slack `conversations.history`.
159
+
160
+ | Env var | Role |
161
+ |---------|------|
162
+ | `CDM_PROD_SLACK_BOT_TOKEN` | Required for threading. |
163
+ | `CDM_PROD_DB_SLACK_CHANNEL` | Channel; default `C05MLHR55JT`. |
164
+ | `CDM_SLACK_THREAD_TZ` | Timezone for the daily parent date; default `UTC`. |
165
+ | `CDM_PAUSE_SLACK_MESSAGES` | Set to `1` to skip Slack. |
166
+ | `CDM_SLACK_WEBHOOK_URL` | Legacy fallback when no bot token is set. |
167
+
168
+ The bot must be in the channel (`chat:write`). History reuse also needs
169
+ `channels:history` (or `groups:history` for a private channel).
170
+
127
171
  ### `execute_with_alerting(...)` → `dict`
128
172
 
129
173
  Runs the sync and, on failure, alerts Slack (with suppression) then re-raises.
@@ -139,6 +183,7 @@ Runs the sync and, on failure, alerts Slack (with suppression) then re-raises.
139
183
  | `suppression_hours` | `1` | Don't re-alert on the *same* error within this many hours. |
140
184
  | `key_filename` | `None` | Override the on-disk key filename. |
141
185
  | `ssh_dir` | `"/home/src/.ssh"` | Directory to write the key into. |
186
+ | `webhook_url` | `None` | Legacy webhook override for this call. Ignored when a bot token is configured. |
142
187
 
143
188
  ### `execute_git_pull(...)` → `dict`
144
189
 
@@ -222,7 +267,21 @@ Keep the version in sync in **both** `pyproject.toml` and `gitpuller/__init__.py
222
267
 
223
268
  ## Changelog
224
269
 
225
- ### 1.1.0 (current)
270
+ ### 1.2.0 (current)
271
+
272
+ - **Threaded Slack alerts.** Failures post as replies under one daily parent
273
+ **per workspace** (`🚨 Git Pull Failures {workspace} — YYYY-MM-DD`) via
274
+ `CDM_PROD_SLACK_BOT_TOKEN` and `CDM_PROD_DB_SLACK_CHANNEL`
275
+ (default `C05MLHR55JT`). Daily `thread_ts` lives in one unified Redis hash
276
+ (`gitpuller:slack_thread:{channel}:{YYYY-MM-DD}`, field = workspace, TTL
277
+ until midnight) shared across all Mage workspaces. Each workspace field is
278
+ JSON: `thread_ts` plus that day's git errors.
279
+ - Incoming webhooks (`CDM_SLACK_WEBHOOK_URL` / `slack_webhook_url`) remain as a
280
+ non-threaded fallback when no bot token is set.
281
+ - Missing Slack credentials no longer raise on `GitPullExecutor` construction;
282
+ alerts are skipped with a warning.
283
+
284
+ ### 1.1.0
226
285
 
227
286
  Reliability and clarity overhaul.
228
287
 
@@ -1,21 +0,0 @@
1
- MIT License
2
-
3
- Copyright (c) 2025 Mohammed Junaid, Muhammad Zahid
4
-
5
- Permission is hereby granted, free of charge, to any person obtaining a copy
6
- of this software and associated documentation files (the "Software"), to deal
7
- in the Software without restriction, including without limitation the rights
8
- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
- copies of the Software, and to permit persons to whom the Software is
10
- furnished to do so, subject to the following conditions:
11
-
12
- The above copyright notice and this permission notice shall be included in all
13
- copies or substantial portions of the Software.
14
-
15
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
- SOFTWARE.
@@ -1,71 +0,0 @@
1
- """Slack incoming-webhook notifier for git-pull failures."""
2
-
3
- import os
4
- import requests
5
- from typing import Optional, Dict, Any
6
-
7
-
8
- class SlackNotifier:
9
- """Posts formatted failure messages to a Slack incoming webhook."""
10
-
11
- def __init__(self, webhook_url: Optional[str] = None):
12
- # Explicit arg wins; otherwise fall back to the shared env var.
13
- self.webhook_url = webhook_url or os.environ.get("CDM_SLACK_WEBHOOK_URL")
14
- if not self.webhook_url:
15
- raise ValueError(
16
- "Slack webhook URL is required. "
17
- "Provide it as parameter or set CDM_SLACK_WEBHOOK_URL environment variable."
18
- )
19
-
20
- def create_failure_payload(self, repo_name: str, error_output: str) -> Dict[str, Any]:
21
- """Build the Slack Block Kit payload describing a pull failure."""
22
- payload = {
23
- # "text" is the notification/fallback shown in previews.
24
- "text": "Automate Git Pull Pipeline Failed",
25
- "blocks": [
26
- {
27
- "type": "header",
28
- "text": {
29
- "type": "plain_text",
30
- "text": ":alert: Automate Git Pull Pipeline Failed :alert:"
31
- }
32
- },
33
- {
34
- "type": "divider"
35
- },
36
- {
37
- "type": "section",
38
- "text": {
39
- "type": "mrkdwn",
40
- "text": f"*Repository:* `{repo_name}`"
41
- }
42
- }
43
- ]
44
- }
45
-
46
- # Append the captured git error as a code block when present.
47
- if error_output:
48
- payload["blocks"].append({
49
- "type": "section",
50
- "text": {
51
- "type": "mrkdwn",
52
- # Cap length so we stay within Slack's 3000-char block limit.
53
- "text": f"*Error Output:*\n```{error_output[:1500]}```"
54
- }
55
- })
56
-
57
- return payload
58
-
59
- def send_alert(self, repo_name: str, error_output: str = "") -> bool:
60
- """Send the failure alert. Returns True on success, False on any error
61
- (delivery failures are logged but never raised, so alerting can't mask
62
- the original git error)."""
63
- try:
64
- payload = self.create_failure_payload(repo_name, error_output)
65
- response = requests.post(self.webhook_url, json=payload, timeout=10)
66
- response.raise_for_status()
67
- print("✅ Alert sent to Slack")
68
- return True
69
- except Exception as e:
70
- print(f"⚠️ Failed to send Slack alert: {e}")
71
- return False