PyPI - mage-ai - Versions diffs - 0.8.4__py3-none-any.whl → 0.8.6__py3-none-any.whl - Mend

mage-ai 0.8.4py3-none-any.whl → 0.8.6py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.

This version of mage-ai might be problematic. Click here for more details.

Files changed (94) hide show

mage_ai/server/frontend_dist/triggers.html CHANGED Viewed

@@ -1,4 +1,4 @@
-<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta content="width=device-width, initial-scale=1.0, maximum-scale=5.0, user-scalable=0" name="viewport"/><title>Triggers | Mage</title><meta name="next-head-count" content="3"/><link href="/favicon.ico" rel="icon"/><link rel="preload" href="/_next/static/css/d1e8e64d0b07af2f.css" as="style"/><link rel="stylesheet" href="/_next/static/css/d1e8e64d0b07af2f.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/_next/static/chunks/polyfills-5cd94c89d3acac5f.js"></script><script src="/_next/static/chunks/webpack-bc5e4eb2c1ff587c.js" defer=""></script><script src="/_next/static/chunks/framework-7c365855dab1bf41.js" defer=""></script><script src="/_next/static/chunks/main-bb0dd5375146d7fd.js" defer=""></script><script src="/_next/static/chunks/pages/_app-0aed65f2e085822e.js" defer=""></script><script src="/_next/static/chunks/3850-6395783d820def1c.js" defer=""></script><script src="/_next/static/chunks/9767-599ba5464d0bf65a.js" defer=""></script><script src="/_next/static/chunks/6579-0bf2380344587a96.js" defer=""></script><script src="/_next/static/chunks/434-69ddfacd3e93f2db.js" defer=""></script><script src="/_next/static/chunks/9898-91c6384c9bd33ca7.js" defer=""></script><script src="/_next/static/chunks/pages/triggers-dbce4f85a95ea336.js" defer=""></script><script src="/_next/static/0jln56azuIZflrR1CXt9U/_buildManifest.js" defer=""></script><script src="/_next/static/0jln56azuIZflrR1CXt9U/_ssgManifest.js" defer=""></script><script src="/_next/static/0jln56azuIZflrR1CXt9U/_middlewareManifest.js" defer=""></script><style data-styled="" data-styled-version="5.3.6">html{-webkit-box-sizing:border-box;box-sizing:border-box;-ms-overflow-style:scrollbar;}/*!sc*/
+<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta content="width=device-width, initial-scale=1.0, maximum-scale=5.0, user-scalable=0" name="viewport"/><title>Triggers | Mage</title><meta name="next-head-count" content="3"/><link href="/favicon.ico" rel="icon"/><link rel="preload" href="/_next/static/css/d1e8e64d0b07af2f.css" as="style"/><link rel="stylesheet" href="/_next/static/css/d1e8e64d0b07af2f.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/_next/static/chunks/polyfills-5cd94c89d3acac5f.js"></script><script src="/_next/static/chunks/webpack-bc5e4eb2c1ff587c.js" defer=""></script><script src="/_next/static/chunks/framework-7c365855dab1bf41.js" defer=""></script><script src="/_next/static/chunks/main-bb0dd5375146d7fd.js" defer=""></script><script src="/_next/static/chunks/pages/_app-0aed65f2e085822e.js" defer=""></script><script src="/_next/static/chunks/3850-6395783d820def1c.js" defer=""></script><script src="/_next/static/chunks/9767-599ba5464d0bf65a.js" defer=""></script><script src="/_next/static/chunks/6579-0bf2380344587a96.js" defer=""></script><script src="/_next/static/chunks/434-69ddfacd3e93f2db.js" defer=""></script><script src="/_next/static/chunks/9898-51ca6a904b7a2382.js" defer=""></script><script src="/_next/static/chunks/pages/triggers-e0172c422c95eda9.js" defer=""></script><script src="/_next/static/okm8eXXn0kUptL5A1B7a6/_buildManifest.js" defer=""></script><script src="/_next/static/okm8eXXn0kUptL5A1B7a6/_ssgManifest.js" defer=""></script><script src="/_next/static/okm8eXXn0kUptL5A1B7a6/_middlewareManifest.js" defer=""></script><style data-styled="" data-styled-version="5.3.6">html{-webkit-box-sizing:border-box;box-sizing:border-box;-ms-overflow-style:scrollbar;}/*!sc*/
 *,*::before,*::after{-webkit-box-sizing:inherit;box-sizing:inherit;}/*!sc*/
 data-styled.g4[id="sc-global-czSCUT1"]{content:"sc-global-czSCUT1,"}/*!sc*/
 .kOVcuR .Toastify__toast-container{margin-top:24px;padding:0 !important;width:500px !important;}/*!sc*/
@@ -17,4 +17,4 @@ data-styled.g72[id="indexstyle__HeaderStyle-sc-1bk8irg-0"]{content:"gbXfes,"}/*!
 data-styled.g74[id="indexstyle__ContainerStyle-sc-ecogjt-0"]{content:"ijwRXz,"}/*!sc*/
 .jMqDRN{padding:16px;background-color:#232429;border-right:1px solid #1C1C1C;}/*!sc*/
 data-styled.g75[id="indexstyle__VerticalNavigationStyle-sc-ecogjt-1"]{content:"jMqDRN,"}/*!sc*/
-</style></head><body><div id="__next"><div class="" style="position:fixed;top:0;left:0;height:2px;background:transparent;z-index:99999999999;width:100%"><div class="" style="height:100%;background:#FF144D;transition:all 500ms ease;width:0%"><div style="box-shadow:0 0 10px #FF144D, 0 0 10px #FF144D;width:5%;opacity:1;position:absolute;height:100%;transition:all 500ms ease;transform:rotate(3deg) translate(0px, -4px);left:-10rem"></div></div></div><div class="indexstyle__HeaderStyle-sc-1bk8irg-0 gbXfes"><div></div></div><div class="indexstyle__ContainerStyle-sc-ecogjt-0 ijwRXz"><div class="indexstyle__VerticalNavigationStyle-sc-ecogjt-1 jMqDRN"><div></div></div><div class="Flex-sc-sgfnl9-0 dKQluW"><div></div></div></div><div></div><div></div><div class="ToastWrapper-sc-1a33ph1-0 kOVcuR"><div class="Toastify"></div></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"auth":{"decodedToken":{"expires":0,"token":null}}},"currentTheme":{"accent":{"alert":"#F6540B","blue":"#4877FF","blueLight":"rgba(72, 119, 255, 0.5)","contentDefaultTransparent":"rgba(174, 174, 174, 0.5)","cyan":"#65E3FF","cyanTransparent":"rgba(101, 227, 255, 0.12)","dbt":"#fc6949","dbtLight":"rgba(252, 105, 73, 0.5)","info":"#00ABFF","infoTransparent":"rgba(0, 171, 255, 0.5)","negative":"#FF1E59","negativeTransparent":"rgba(255, 30, 89, 0.3)","pink":"#FF4FF8","pinkLight":"#FFB9FC","positive":"#00A81A","primaryTransparent":"rgba(155, 108, 167, 0.5)","purple":"#7D55EC","purpleLight":"rgba(125, 85, 236, 0.5)","teal":"#00B4CC","tealLight":"rgba(0, 180, 204, 0.5)","warning":"#DD9900","warningTransparent":"rgba(221, 153, 0, 0.5)","yellow":"#FFCC19","yellowLight":"rgba(255, 204, 25, 0.5)"},"background":{"chartBlock":"#2E3036","codeArea":"#1E1F24","codeTextarea":"#000000","content":"#1B1C20","danger":"#FFD0DB","dark":"#B1B8C3","header":"#1B1B1B","menu":"#0F4CFF","muted":"#F9FAFC","navigation":"#EDEDED","output":"#2E3036","page":"#1E1F24","panel":"#232429","popup":"#27292E","row":"#2C2C2C","row2":"#51535C","scrollbarThumb":"rgba(100, 100, 100, 0.5)","scrollbarThumbHover":"rgba(255, 255, 255, 0.3)","scrollbarTrack":"#2E3036","success":"#8ADE00","table":"#292A2F"},"borders":{"contrast":"#FFFFFF","danger":"#FF144D","dark":"#000000","info":"#FFCC19","light":"#2F3034","medium":"#1C1C1C","success":"#2FCB52"},"brand":{"earth100":"#C6EEDB","earth200":"#9DDFBF","earth300":"#6BBF96","earth400":"#37A46F","earth400Transparent":"rgba(55, 164, 111, 0.4)","earth500":"#00954C","energy100":"#FFF4BA","energy200":"#FFED92","energy300":"#FFE662","energy400":"#FFDA19","energy400Transparent":"rgba(255, 218, 25, 0.04)","energy500":"#F6C000","fire100":"#FFD7E0","fire200":"#FFA3B9","fire300":"#FF547D","fire400":"#FF144D","fire400Transparent":"rgba(255, 20, 77, 0.4)","fire500":"#EB0032","stone100":"#F3E6D7","stone200":"#E3D4C2","stone400":"#BFA78B","stone500":"#AF8859","water100":"#BDCEFF","water200":"#81A1FF","water300":"#517DFF","water400":"#2A60FE","water400Transparent":"rgba(42, 96, 254, 0.4)","water500":"#0F4CFF","wind100":"#EEEAFF","wind200":"#CCC1F4","wind300":"#A698DD","wind400":"#6B50D7","wind400SuperTransparent":"rgba(107, 80, 215, 0.12)","wind400Transparent":"rgba(107, 80, 215, 0.4)","wind500":"#4E32BC"},"chart":{"backgroundPrimary":"#7D55EC","backgroundSecondary":"#FF144D","backgroundTertiary":"#86E2FF","button1":"#4877FF","button2":"#FFCC19","button3":"#8ADE00","button4":"#FF4FF8","button5":"#B98D95","lines":"#9B6CA7","primary":"#6B50D7","secondary":"#FF144D","tertiary":"#2A60FE"},"content":{"active":"#FFFFFF","default":"#AEAEAE","disabled":"rgba(255, 255, 255, 0.3)","inverted":"#2C2C2C","muted":"#787A85"},"elevation":{"visualizationAccent":"#996CFF","visualizationAccentAlt":"#C1ACF7"},"feature":{"active":"rgba(250, 248, 254, 0.14)","disabled":"rgba(201, 206, 218, 0.12)"},"icons":{"neutral":"#787878"},"interactive":{"activeBorder":"#060606","checked":"#060606","dangerBorder":"#FF144D","defaultBackground":"#36383F","defaultBorder":"#1C1C1C","disabledBorder":"#B1B8C3","focusBackground":"#B1B8C3","focusBorder":"#86E2FF","hoverBackground":"#4E4E4E","hoverBorder":"#B9BFCA","hoverOverlay":"rgba(255, 255, 255, 0.1)","linkPrimary":"#1752FF","linkPrimaryHover":"#4877FF","linkSecondary":"#6B50D7","linkSecondaryDisabled":"#C4B9EF","rowHoverBackground":"rgba(0, 0, 0, 0.1)"},"loader":{"color":"#EB0032","colorInverted":"#8ADE00"},"logo":{"color":"#FFFFFF"},"monotone":{"black":"#060606","blackTransparent":"rgba(0, 0, 0, 0.6)","gray":"#B1B8C3","grey100":"#F2F2F2","grey200":"#D5D7DC","grey300":"#B4B8C0","grey400":"#70747C","grey500":"#51535C","purple":"#6B50D7","white":"#FFFFFF"},"neutral":{"n100":"#E7E8EA","n200":"#D8DADE","n300":"#CBCCD0","n400":"#BCBEC4","n500":"#AEB0B6"},"progress":{"negative":"#FF144D","positive":"#6B50D7"},"shadow":{"base":"12px 40px 120px rgba(106, 117, 139, 0.4)","menu":"4px 10px 20px rgba(6, 6, 6, 0.12)","popup":"10px 20px 40px rgba(0, 0, 0, 0.2)","small":"0px, 4px, rgba(0, 0, 0, 0.25)","window":"0px 10px 60px rgba(0, 0, 0, 0.7)"},"status":{"negative":"#FF144D","positive":"#24B400"},"text":{"fileBrowser":"#787A85"}}},"page":"/triggers","query":{},"buildId":"0jln56azuIZflrR1CXt9U","nextExport":true,"isFallback":false,"gip":true,"appGip":true,"scriptLoader":[]}</script></body></html>
+</style></head><body><div id="__next"><div class="" style="position:fixed;top:0;left:0;height:2px;background:transparent;z-index:99999999999;width:100%"><div class="" style="height:100%;background:#FF144D;transition:all 500ms ease;width:0%"><div style="box-shadow:0 0 10px #FF144D, 0 0 10px #FF144D;width:5%;opacity:1;position:absolute;height:100%;transition:all 500ms ease;transform:rotate(3deg) translate(0px, -4px);left:-10rem"></div></div></div><div class="indexstyle__HeaderStyle-sc-1bk8irg-0 gbXfes"><div></div></div><div class="indexstyle__ContainerStyle-sc-ecogjt-0 ijwRXz"><div class="indexstyle__VerticalNavigationStyle-sc-ecogjt-1 jMqDRN"><div></div></div><div class="Flex-sc-sgfnl9-0 dKQluW"><div></div></div></div><div></div><div></div><div class="ToastWrapper-sc-1a33ph1-0 kOVcuR"><div class="Toastify"></div></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"auth":{"decodedToken":{"expires":0,"token":null}}},"currentTheme":{"accent":{"alert":"#F6540B","blue":"#4877FF","blueLight":"rgba(72, 119, 255, 0.5)","contentDefaultTransparent":"rgba(174, 174, 174, 0.5)","cyan":"#65E3FF","cyanTransparent":"rgba(101, 227, 255, 0.12)","dbt":"#fc6949","dbtLight":"rgba(252, 105, 73, 0.5)","info":"#00ABFF","infoTransparent":"rgba(0, 171, 255, 0.5)","negative":"#FF1E59","negativeTransparent":"rgba(255, 30, 89, 0.3)","pink":"#FF4FF8","pinkLight":"#FFB9FC","positive":"#00A81A","primaryTransparent":"rgba(155, 108, 167, 0.5)","purple":"#7D55EC","purpleLight":"rgba(125, 85, 236, 0.5)","teal":"#00B4CC","tealLight":"rgba(0, 180, 204, 0.5)","warning":"#DD9900","warningTransparent":"rgba(221, 153, 0, 0.5)","yellow":"#FFCC19","yellowLight":"rgba(255, 204, 25, 0.5)"},"background":{"chartBlock":"#2E3036","codeArea":"#1E1F24","codeTextarea":"#000000","content":"#1B1C20","danger":"#FFD0DB","dark":"#B1B8C3","header":"#1B1B1B","menu":"#0F4CFF","muted":"#F9FAFC","navigation":"#EDEDED","output":"#2E3036","page":"#1E1F24","panel":"#232429","popup":"#27292E","row":"#2C2C2C","row2":"#51535C","scrollbarThumb":"rgba(100, 100, 100, 0.5)","scrollbarThumbHover":"rgba(255, 255, 255, 0.3)","scrollbarTrack":"#2E3036","success":"#8ADE00","table":"#292A2F"},"borders":{"contrast":"#FFFFFF","danger":"#FF144D","dark":"#000000","info":"#FFCC19","light":"#2F3034","medium":"#1C1C1C","success":"#2FCB52"},"brand":{"earth100":"#C6EEDB","earth200":"#9DDFBF","earth300":"#6BBF96","earth400":"#37A46F","earth400Transparent":"rgba(55, 164, 111, 0.4)","earth500":"#00954C","energy100":"#FFF4BA","energy200":"#FFED92","energy300":"#FFE662","energy400":"#FFDA19","energy400Transparent":"rgba(255, 218, 25, 0.04)","energy500":"#F6C000","fire100":"#FFD7E0","fire200":"#FFA3B9","fire300":"#FF547D","fire400":"#FF144D","fire400Transparent":"rgba(255, 20, 77, 0.4)","fire500":"#EB0032","stone100":"#F3E6D7","stone200":"#E3D4C2","stone400":"#BFA78B","stone500":"#AF8859","water100":"#BDCEFF","water200":"#81A1FF","water300":"#517DFF","water400":"#2A60FE","water400Transparent":"rgba(42, 96, 254, 0.4)","water500":"#0F4CFF","wind100":"#EEEAFF","wind200":"#CCC1F4","wind300":"#A698DD","wind400":"#6B50D7","wind400SuperTransparent":"rgba(107, 80, 215, 0.12)","wind400Transparent":"rgba(107, 80, 215, 0.4)","wind500":"#4E32BC"},"chart":{"backgroundPrimary":"#7D55EC","backgroundSecondary":"#FF144D","backgroundTertiary":"#86E2FF","button1":"#4877FF","button2":"#FFCC19","button3":"#8ADE00","button4":"#FF4FF8","button5":"#B98D95","lines":"#9B6CA7","primary":"#6B50D7","secondary":"#FF144D","tertiary":"#2A60FE"},"content":{"active":"#FFFFFF","default":"#AEAEAE","disabled":"rgba(255, 255, 255, 0.3)","inverted":"#2C2C2C","muted":"#787A85"},"elevation":{"visualizationAccent":"#996CFF","visualizationAccentAlt":"#C1ACF7"},"feature":{"active":"rgba(250, 248, 254, 0.14)","disabled":"rgba(201, 206, 218, 0.12)"},"icons":{"neutral":"#787878"},"interactive":{"activeBorder":"#060606","checked":"#060606","dangerBorder":"#FF144D","defaultBackground":"#36383F","defaultBorder":"#1C1C1C","disabledBorder":"#B1B8C3","focusBackground":"#B1B8C3","focusBorder":"#86E2FF","hoverBackground":"#4E4E4E","hoverBorder":"#B9BFCA","hoverOverlay":"rgba(255, 255, 255, 0.1)","linkPrimary":"#1752FF","linkPrimaryHover":"#4877FF","linkSecondary":"#6B50D7","linkSecondaryDisabled":"#C4B9EF","rowHoverBackground":"rgba(0, 0, 0, 0.1)"},"loader":{"color":"#EB0032","colorInverted":"#8ADE00"},"logo":{"color":"#FFFFFF"},"monotone":{"black":"#060606","blackTransparent":"rgba(0, 0, 0, 0.6)","gray":"#B1B8C3","grey100":"#F2F2F2","grey200":"#D5D7DC","grey300":"#B4B8C0","grey400":"#70747C","grey500":"#51535C","purple":"#6B50D7","white":"#FFFFFF"},"neutral":{"n100":"#E7E8EA","n200":"#D8DADE","n300":"#CBCCD0","n400":"#BCBEC4","n500":"#AEB0B6"},"progress":{"negative":"#FF144D","positive":"#6B50D7"},"shadow":{"base":"12px 40px 120px rgba(106, 117, 139, 0.4)","menu":"4px 10px 20px rgba(6, 6, 6, 0.12)","popup":"10px 20px 40px rgba(0, 0, 0, 0.2)","small":"0px, 4px, rgba(0, 0, 0, 0.25)","window":"0px 10px 60px rgba(0, 0, 0, 0.7)"},"status":{"negative":"#FF144D","positive":"#24B400"},"text":{"fileBrowser":"#787A85"}}},"page":"/triggers","query":{},"buildId":"okm8eXXn0kUptL5A1B7a6","nextExport":true,"isFallback":false,"gip":true,"appGip":true,"scriptLoader":[]}</script></body></html>

mage_ai/server/server.py CHANGED Viewed

@@ -148,6 +148,7 @@ def make_app():
         (r'/pipelines', MainPageHandler),
         (r'/pipelines/(.*)', MainPageHandler),
         (r'/pipeline-runs', PipelineRunsPageHandler),
+        (r'/settings', MainPageHandler),
         (r'/sign-in', MainPageHandler),
         (r'/terminal', MainPageHandler),
         (r'/manage', ManagePageHandler),

mage_ai/server/utils/output_display.py CHANGED Viewed

@@ -6,6 +6,7 @@ from mage_ai.data_preparation.models.constants import (
 from mage_ai.server.kernels import KernelName
 from mage_ai.shared.code import is_pyspark_code
 from typing import Dict, List
+import json
 import re
@@ -226,9 +227,12 @@ def add_execution_code(
     run_upstream: bool = False,
     update_status: bool = True,
     widget: bool = False,
+    run_settings: Dict = None,
 ) -> str:
     escaped_code = code.replace("'''", "\"\"\"")
+    run_settings_json = json.dumps(run_settings or {})
     magic_header = ''
     spark_session_init = ''
     if kernel_name == KernelName.PYSPARK:
@@ -252,8 +256,10 @@ from mage_ai.data_preparation.repo_manager import get_repo_path
 from mage_ai.orchestration.db import db_connection
 from mage_ai.shared.array import find
 import datetime
+import json
 import pandas as pd
 db_connection.start_session()
 {spark_session_init}
@@ -285,6 +291,7 @@ def execute_custom_code():
         custom_code=code,
         global_vars=global_vars,
         analyze_outputs={analyze_outputs},
+        run_settings=json.loads('{run_settings_json}'),
         update_status={update_status},
         test_execution=True,
     )

mage_ai/server/websocket_server.py CHANGED Viewed

@@ -175,8 +175,6 @@ class WebSocketServer(tornado.websocket.WebSocketHandler):
                         oauth_token.user and \
                         has_at_least_editor_role(oauth_token.user)
-                    print('WTFFFFFFFFFFFFFFFFFFFFFF', has_at_least_editor_role(oauth_token.user))
             if not valid:
                 return self.send_message(
                     dict(
@@ -299,6 +297,7 @@ class WebSocketServer(tornado.websocket.WebSocketHandler):
         run_downstream = message.get('run_downstream')
         run_tests = message.get('run_tests')
         run_upstream = message.get('run_upstream')
+        run_settings = message.get('run_settings')
         pipeline_uuid = pipeline.uuid
@@ -344,6 +343,7 @@ class WebSocketServer(tornado.websocket.WebSocketHandler):
                     run_upstream=run_upstream,
                     update_status=False if remote_execution else True,
                     widget=widget,
+                    run_settings=run_settings,
                 )
             msg_id = client.execute(add_internal_output_info(code))

mage_ai/services/datadog/__init__.py ADDED Viewed

@@ -0,0 +1,123 @@
+from datadog import initialize, api, statsd
+from datetime import datetime
+import os
+import time
+options = {
+    'api_key': os.getenv('DD_API_KEY'),
+    'app_key': os.getenv('DD_APP_KEY'),
+}
+initialize(**options)
+def create_event(title, text, tags={}):
+    return api.Event.create(
+        title=title,
+        text=text,
+        tags=tags,
+    )
+def gauge(metric, value, host='mage', tags={}):
+    return api.Metric.send(
+        host=host,
+        metric=metric,
+        points=[
+            (datetime.utcnow().timestamp(), value),
+        ],
+        tags=tags,
+        type='gauge',
+    )
+def increment(metric, tags={}, value=1):
+    return create_metrics([
+        (metric, value, tags),
+    ])
+def create_metrics(metrics, host='mage', metric_type='count'):
+    # Format of argument
+    # metrics = [
+    #     ('metric', 'value', 'tags'),
+    # ]
+    arr = [{
+        'host': host,
+        'metric': t[0],
+        'points': t[1],
+        'tags': t[2] if len(t) == 3 else {},
+        'type': metric_type,
+    } for t in metrics]
+    return api.Metric.send(metrics=arr)
+def create_metric(metric, value, host='mage', tags={}):
+    return api.Metric.send(
+        host=host,
+        metric=metric,
+        points=[
+            (datetime.utcnow().timestamp(), value),
+        ],
+        tags=tags,
+        type='count',
+    )
+def histogram(metric, value, host='mage', tags={}):
+    return api.Metric.send(
+        host=host,
+        metric=metric,
+        points=[
+            (datetime.utcnow().timestamp(), value),
+        ],
+        tags=tags,
+        type='histogram',
+    )
+def timing(metric, value, host='mage', tags={}):
+    return api.Metric.send(
+        host=host,
+        metric=metric,
+        points=[
+            (datetime.utcnow().timestamp(), value),
+        ],
+        tags=tags,
+        type='timer',
+    )
+class timed_decorator(object):
+    """
+    @timed_decorator('metric.metric', tags={ 'key': 'value' })
+    """
+    def __init__(self, metric, tags={}):
+        self.metric = metric
+        self.tags = tags
+    def __call__(self, f):
+        def wrapped_f(*args):
+            with statsd.timed(self.metric, tags=self.tags):
+                return f(*args)
+        return wrapped_f
+class timer(object):
+    """
+    with timer('metric.metric', tags={ 'key': 'value' }):
+        function()
+    """
+    def __init__(self, metric, tags={}):
+        self.metric = metric
+        self.start = None
+        self.tags = tags
+    def __enter__(self):
+        self.start = time.time()
+    def __exit__(self, type, value, traceback):
+        # Must convert to milliseconds, see details in
+        # https://statsd.readthedocs.io/en/v3.1/timing.html
+        dt = int((time.time() - self.start) * 1000)
+        return timing(self.metric, dt, tags=self.tags)

mage_ai/tests/data_preparation/test_templates.py CHANGED Viewed

@@ -36,8 +36,6 @@ class TemplateTest(TestCase):
         expected_string = """from mage_ai.data_cleaner.transformer_actions.base import BaseAction
 from mage_ai.data_cleaner.transformer_actions.constants import ActionType, Axis
 from mage_ai.data_cleaner.transformer_actions.utils import build_transformer_action
-from pandas import DataFrame
 if 'transformer' not in globals():
     from mage_ai.data_preparation.decorators import transformer
 if 'test' not in globals():
@@ -80,11 +78,11 @@ def remove_rows_with_missing_entries(df: DataFrame, *args, **kwargs) -> DataFram
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         new_string = build_template_from_suggestion(suggestion)
         self.assertEqual(expected_string, new_string)
@@ -102,7 +100,7 @@ def load_data(*args, **kwargs):
     Template code for loading data from any source.
     Returns:
-        Anything
+        Anything (e.g. data frame, dictionary, array, int, str, etc.)
     \"\"\"
     # Specify your data loading logic here
@@ -110,11 +108,11 @@ def load_data(*args, **kwargs):
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config1 = {'data_source': 'default'}
@@ -153,11 +151,11 @@ def load_data_from_redshift(*args, **kwargs):
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         s3_template = """from mage_ai.data_preparation.repo_manager import get_repo_path
 from mage_ai.io.config import ConfigFileLoader
@@ -190,11 +188,11 @@ def load_from_s3_bucket(*args, **kwargs):
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config1 = {'data_source': DataSource.REDSHIFT}
@@ -220,17 +218,17 @@ def load_data_from_api(*args, **kwargs):
     Template for loading data from API
     \"\"\"
     url = ''
     response = requests.get(url)
     return pd.read_csv(io.StringIO(response.text), sep=',')
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config = {'data_source': DataSource.API}
         api_template = fetch_template_source(BlockType.DATA_LOADER, config)
@@ -268,16 +266,14 @@ consumer_group: unique_consumer_group
         self.assertEqual(kafka_template, new_kafka_template)
     def test_template_generation_transformer_default(self):
-        expected_template = """from pandas import DataFrame
-if 'transformer' not in globals():
+        expected_template = """if 'transformer' not in globals():
     from mage_ai.data_preparation.decorators import transformer
 if 'test' not in globals():
     from mage_ai.data_preparation.decorators import test
 @transformer
-def transform_df(df: DataFrame, *args, **kwargs) -> DataFrame:
+def transform(*args, **kwargs):
     \"\"\"
     Template code for a transformer block.
@@ -285,68 +281,22 @@ def transform_df(df: DataFrame, *args, **kwargs) -> DataFrame:
     There should be one parameter for each output variable from each parent block.
     Args:
-        df (DataFrame): Data frame from parent block.
+        args: The input variables from upstream blocks
     Returns:
-        DataFrame: Transformed data frame
+        Anything (e.g. data frame, dictionary, array, int, str, etc.)
     \"\"\"
     # Specify your transformation logic here
-    return df
-@test
-def test_output(df, *args) -> None:
-    \"\"\"
-    Template code for testing the output of the block.
-    \"\"\"
-    assert df is not None, 'The output is undefined'
-"""
-        config1 = {'action_type': 'custom'}
-        config2 = {'axis': 'row'}
-        config3 = {}
-        new_template1 = fetch_template_source(BlockType.TRANSFORMER, config1)
-        new_template2 = fetch_template_source(BlockType.TRANSFORMER, config2)
-        new_template3 = fetch_template_source(BlockType.TRANSFORMER, config3)
-        self.assertEqual(expected_template, new_template1)
-        self.assertEqual(expected_template, new_template2)
-        self.assertEqual(expected_template, new_template3)
-    def test_template_generation_transformer_action_default(self):
-        expected_template = """from pandas import DataFrame
-if 'transformer' not in globals():
-    from mage_ai.data_preparation.decorators import transformer
-if 'test' not in globals():
-    from mage_ai.data_preparation.decorators import test
-@transformer
-def transform_df(df: DataFrame, *args, **kwargs) -> DataFrame:
-    \"\"\"
-    Template code for a transformer block.
-    Add more parameters to this function if this block has multiple parent blocks.
-    There should be one parameter for each output variable from each parent block.
-    Args:
-        df (DataFrame): Data frame from parent block.
-    Returns:
-        DataFrame: Transformed data frame
-    \"\"\"
-    # Specify your transformation logic here
-    return df
+    return {}
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config1 = {'action_type': 'custom'}
@@ -389,11 +339,11 @@ def execute_transformer_action(df: DataFrame, *args, **kwargs) -> DataFrame:
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config = {'action_type': ActionType.CLEAN_COLUMN_NAME, 'axis': Axis.COLUMN}
@@ -430,11 +380,11 @@ def execute_transformer_action(df: DataFrame, *args, **kwargs) -> DataFrame:
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config = {'action_type': ActionType.FILTER, 'axis': Axis.ROW}
@@ -472,11 +422,11 @@ def execute_transformer_action(df: DataFrame, *args, **kwargs) -> DataFrame:
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config = {'action_type': ActionType.REFORMAT, 'axis': Axis.COLUMN}
@@ -519,11 +469,11 @@ def execute_transformer_action(df: DataFrame, *args, **kwargs) -> DataFrame:
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config = {'action_type': ActionType.FIRST, 'axis': Axis.COLUMN}
@@ -531,19 +481,17 @@ def test_output(df, *args) -> None:
         self.assertEqual(expected_template, new_template)
     def test_template_generation_data_exporter_default(self):
-        expected_template = """from pandas import DataFrame
-if 'data_exporter' not in globals():
+        expected_template = """if 'data_exporter' not in globals():
     from mage_ai.data_preparation.decorators import data_exporter
 @data_exporter
-def export_data(df: DataFrame, **kwargs):
+def export_data(*args, **kwargs):
     \"\"\"
     Exports data to some source
     Args:
-        df (DataFrame): Data frame to export to
+        args: The input variables from upstream blocks
     Output (optional):
         Optionally return any object and it'll be logged and
@@ -690,11 +638,11 @@ def transform_in_postgres(*args, **kwargs) -> DataFrame:
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         bigquery_template = """from mage_ai.data_preparation.repo_manager import get_repo_path
@@ -732,11 +680,11 @@ def transform_in_bigquery(*args, **kwargs) -> DataFrame:
 @test
-def test_output(df, *args) -> None:
+def test_output(output, *args) -> None:
     \"\"\"
     Template code for testing the output of the block.
     \"\"\"
-    assert df is not None, 'The output is undefined'
+    assert output is not None, 'The output is undefined'
 """
         config1 = {'data_source': DataSource.POSTGRES}

mage_ai/tests/services/datadog/__init__.py ADDED Viewed

File without changes

mage_ai/tests/services/datadog/test_datadog.py ADDED Viewed

@@ -0,0 +1,69 @@
+from datetime import datetime
+from mage_ai.services import datadog as dd
+from mage_ai.tests.base_test import TestCase
+from unittest.mock import Mock, patch
+TEST_METRIC = 'mage.test'
+TAGS = dict(tag1='tag')
+class DatadogTests(TestCase):
+    @patch('datadog.api.Event.create')
+    def test_create_event(self, mock_event):
+        event_name = 'event name'
+        event_text = 'event text'
+        dd.create_event(event_name, event_text, tags=TAGS)
+        mock_event.assert_called_with(title=event_name, text=event_text, tags=TAGS)
+    @patch('datadog.api.Metric.send')
+    @patch('mage_ai.services.datadog.datetime')
+    def test_gauge(self, mock_dt, mock_metric):
+        dt = datetime(2023, 1, 1)
+        mock_dt.utcnow = Mock(return_value=dt)
+        dd.gauge(TEST_METRIC, 5, tags=TAGS)
+        mock_metric.assert_called_with(
+            host='mage',
+            metric=TEST_METRIC,
+            points=[(dt.timestamp(), 5)],
+            tags=TAGS,
+            type='gauge'
+        )
+    @patch('datadog.api.Metric.send')
+    def test_increment(self, mock_metric):
+        dd.increment(TEST_METRIC, tags=TAGS)
+        mock_metric.assert_called_with(metrics=[{
+            'host': 'mage',
+            'metric': TEST_METRIC,
+            'points': 1,
+            'tags': TAGS,
+            'type': 'count'
+        }])
+    @patch('datadog.api.Metric.send')
+    @patch('mage_ai.services.datadog.datetime')
+    def test_histogram(self, mock_dt, mock_metric):
+        dt = datetime(2023, 1, 1)
+        mock_dt.utcnow = Mock(return_value=dt)
+        dd.histogram(TEST_METRIC, 3, tags=TAGS)
+        mock_metric.assert_called_with(
+            host='mage',
+            metric=TEST_METRIC,
+            points=[(dt.timestamp(), 3)],
+            tags=TAGS,
+            type='histogram'
+        )
+    @patch('datadog.api.Metric.send')
+    @patch('mage_ai.services.datadog.datetime')
+    def test_timing(self, mock_dt, mock_metric):
+        dt = datetime(2023, 1, 1)
+        mock_dt.utcnow = Mock(return_value=dt)
+        dd.timing(TEST_METRIC, 100, tags=TAGS)
+        mock_metric.assert_called_with(
+            host='mage',
+            metric=TEST_METRIC,
+            points=[(dt.timestamp(), 100)],
+            tags=TAGS,
+            type='timer'
+        )

{mage_ai-0.8.4.dist-info → mage_ai-0.8.6.dist-info}/METADATA RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mage-ai
-Version: 0.8.4
+Version: 0.8.6
 Summary: Mage is a tool for building and deploying data pipelines.
 Home-page: https://github.com/mage-ai/mage-ai
 Author: Mage
@@ -20,6 +20,7 @@ Requires-Dist: bcrypt (==4.0.1)
 Requires-Dist: croniter (==1.3.7)
 Requires-Dist: cryptography (==36.0.2)
 Requires-Dist: dask (>=2022.2.0)
+Requires-Dist: datadog (==0.44.0)
 Requires-Dist: freezegun (==1.2.2)
 Requires-Dist: httpx (==0.23.1)
 Requires-Dist: inflection (==0.5.1)

mage-ai 0.8.4__py3-none-any.whl → 0.8.6__py3-none-any.whl

Potentially problematic release.

mage-ai 0.8.4py3-none-any.whl → 0.8.6py3-none-any.whl