PyPI - mage-ai - Versions diffs - 0.9.7__py3-none-any.whl → 0.9.9__py3-none-any.whl - Mend

mage-ai 0.9.7py3-none-any.whl → 0.9.9py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.

This version of mage-ai might be problematic. Click here for more details.

Files changed (161) hide show

mage_ai/server/frontend_dist/version-control.html CHANGED Viewed

@@ -1,4 +1,4 @@
-<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta content="width=device-width, initial-scale=1.0, maximum-scale=5.0, user-scalable=0" name="viewport"/><title>Version control | Mage</title><meta name="next-head-count" content="3"/><link href="/favicon.ico" rel="icon"/><link rel="preload" href="/_next/static/css/d1e8e64d0b07af2f.css" as="style"/><link rel="stylesheet" href="/_next/static/css/d1e8e64d0b07af2f.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/_next/static/chunks/polyfills-5cd94c89d3acac5f.js"></script><script src="/_next/static/chunks/webpack-8f51011a10b384a1.js" defer=""></script><script src="/_next/static/chunks/framework-7c365855dab1bf41.js" defer=""></script><script src="/_next/static/chunks/main-bb0dd5375146d7fd.js" defer=""></script><script src="/_next/static/chunks/pages/_app-c7557dc9e725825e.js" defer=""></script><script src="/_next/static/chunks/844-1e171f361e63b36d.js" defer=""></script><script src="/_next/static/chunks/9902-b7981346a40ecdc3.js" defer=""></script><script src="/_next/static/chunks/7011-81dd8269c4806d26.js" defer=""></script><script src="/_next/static/chunks/7768-2325216ed654ad05.js" defer=""></script><script src="/_next/static/chunks/1424-de97c5aa0477894b.js" defer=""></script><script src="/_next/static/chunks/1005-ce36fe7ed460d006.js" defer=""></script><script src="/_next/static/chunks/7815-6dc55b5fd5d1f48a.js" defer=""></script><script src="/_next/static/chunks/3654-bff4740b63cf280c.js" defer=""></script><script src="/_next/static/chunks/6299-97f283be01adf6a8.js" defer=""></script><script src="/_next/static/chunks/pages/version-control-2ecda1f1437013f2.js" defer=""></script><script src="/_next/static/tMZjCwLa1LRYRzhFRdKNf/_buildManifest.js" defer=""></script><script src="/_next/static/tMZjCwLa1LRYRzhFRdKNf/_ssgManifest.js" defer=""></script><script src="/_next/static/tMZjCwLa1LRYRzhFRdKNf/_middlewareManifest.js" defer=""></script><style data-styled="" data-styled-version="5.3.6">html{-webkit-box-sizing:border-box;box-sizing:border-box;-ms-overflow-style:scrollbar;}/*!sc*/
+<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta content="width=device-width, initial-scale=1.0, maximum-scale=5.0, user-scalable=0" name="viewport"/><title>Version control | Mage</title><meta name="next-head-count" content="3"/><link href="/favicon.ico" rel="icon"/><link rel="preload" href="/_next/static/css/d1e8e64d0b07af2f.css" as="style"/><link rel="stylesheet" href="/_next/static/css/d1e8e64d0b07af2f.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/_next/static/chunks/polyfills-5cd94c89d3acac5f.js"></script><script src="/_next/static/chunks/webpack-8f51011a10b384a1.js" defer=""></script><script src="/_next/static/chunks/framework-7c365855dab1bf41.js" defer=""></script><script src="/_next/static/chunks/main-bb0dd5375146d7fd.js" defer=""></script><script src="/_next/static/chunks/pages/_app-fa38a24707c76411.js" defer=""></script><script src="/_next/static/chunks/844-1e171f361e63b36d.js" defer=""></script><script src="/_next/static/chunks/9902-b7981346a40ecdc3.js" defer=""></script><script src="/_next/static/chunks/7011-81dd8269c4806d26.js" defer=""></script><script src="/_next/static/chunks/7768-2325216ed654ad05.js" defer=""></script><script src="/_next/static/chunks/1424-e22139f33196c5c1.js" defer=""></script><script src="/_next/static/chunks/1005-16c1a6678de5311e.js" defer=""></script><script src="/_next/static/chunks/7815-fbd99c8f259d9e8b.js" defer=""></script><script src="/_next/static/chunks/3654-573457b62efff6e2.js" defer=""></script><script src="/_next/static/chunks/6299-cf188c1b7a1bc33c.js" defer=""></script><script src="/_next/static/chunks/pages/version-control-a1cf445565b68809.js" defer=""></script><script src="/_next/static/OwvTROmputQCA1Z9HEGDv/_buildManifest.js" defer=""></script><script src="/_next/static/OwvTROmputQCA1Z9HEGDv/_ssgManifest.js" defer=""></script><script src="/_next/static/OwvTROmputQCA1Z9HEGDv/_middlewareManifest.js" defer=""></script><style data-styled="" data-styled-version="5.3.6">html{-webkit-box-sizing:border-box;box-sizing:border-box;-ms-overflow-style:scrollbar;}/*!sc*/
 *,*::before,*::after{-webkit-box-sizing:inherit;box-sizing:inherit;}/*!sc*/
 data-styled.g4[id="sc-global-czSCUT1"]{content:"sc-global-czSCUT1,"}/*!sc*/
 .kOVcuR .Toastify__toast-container{margin-top:24px;padding:0 !important;width:500px !important;}/*!sc*/
@@ -21,4 +21,4 @@ data-styled.g72[id="indexstyle__ContainerStyle-sc-ecogjt-0"]{content:"ijwRXz,"}/
 @-webkit-keyframes animate-in{0%{width:168px;}100%{width:272px;}}/*!sc*/
 @keyframes animate-in{0%{width:168px;}100%{width:272px;}}/*!sc*/
 data-styled.g73[id="indexstyle__VerticalNavigationStyleComponent-sc-ecogjt-1"]{content:"jQArIy,"}/*!sc*/
-</style></head><body><div id="__next"><div class="" style="position:fixed;top:0;left:0;height:2px;background:transparent;z-index:99999999999;width:100%"><div class="" style="height:100%;background:#FF144D;transition:all 500ms ease;width:0%"><div style="box-shadow:0 0 10px #FF144D, 0 0 10px #FF144D;width:5%;opacity:1;position:absolute;height:100%;transition:all 500ms ease;transform:rotate(3deg) translate(0px, -4px);left:-10rem"></div></div></div><div class="indexstyle__HeaderStyle-sc-1bk8irg-0 gbXfes"><div></div></div><div class="indexstyle__ContainerStyle-sc-ecogjt-0 ijwRXz"><div class="indexstyle__VerticalNavigationStyleComponent-sc-ecogjt-1 jQArIy"><div class="Spacing__SpacingStyle-sc-1mpmtgj-0 hDBiHu"><div></div></div></div><div class="Flex-sc-sgfnl9-0 dKQluW"><div></div></div></div><div></div><div></div><div></div><div class="ToastWrapper-sc-1a33ph1-0 kOVcuR"><div class="Toastify"></div></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"auth":{"decodedToken":{"expires":0,"token":null}}},"currentTheme":{"accent":{"alert":"#F6540B","blue":"#4877FF","blueLight":"rgba(72, 119, 255, 0.5)","contentDefaultTransparent":"rgba(174, 174, 174, 0.5)","cyan":"#65E3FF","cyanLight":"rgba(101, 227, 255, 0.3)","cyanTransparent":"rgba(101, 227, 255, 0.12)","dbt":"#fc6949","dbtLight":"rgba(252, 105, 73, 0.5)","info":"#00ABFF","infoTransparent":"rgba(0, 171, 255, 0.5)","negative":"#FF1E59","negativeTransparent":"rgba(255, 30, 89, 0.3)","pink":"#FF4FF8","pinkLight":"rgb(255, 79, 248, 0.5)","positive":"#00A81A","primaryTransparent":"rgba(155, 108, 167, 0.5)","purple":"#7D55EC","purpleLight":"rgba(125, 85, 236, 0.5)","rose":"#D1A2AB","roseLight":"rgba(209, 162, 171, 0.5)","sky":"#6AA1E0","skyLight":"rgba(106, 161, 224, 0.05)","teal":"#00B4CC","tealLight":"rgba(0, 180, 204, 0.5)","warning":"#DD9900","warningTransparent":"rgba(221, 153, 0, 0.5)","yellow":"#FFCC19","yellowLight":"rgba(255, 204, 25, 0.5)"},"background":{"chartBlock":"#2E3036","codeArea":"#1E1F24","codeTextarea":"#000000","content":"#1B1C20","danger":"#FFD0DB","dark":"#B1B8C3","dashboard":"#18181C","header":"#1B1B1B","menu":"#0F4CFF","muted":"#F9FAFC","navigation":"#EDEDED","output":"#2E3036","page":"#1E1F24","panel":"#232429","popup":"#27292E","row":"#2C2C2C","row2":"#51535C","scrollbarThumb":"rgba(100, 100, 100, 0.5)","scrollbarThumbHover":"rgba(255, 255, 255, 0.3)","scrollbarTrack":"#2E3036","success":"#8ADE00","successLight":"rgb(138, 222, 0, 0.3)","table":"#292A2F","tag":"#3A4550"},"borders":{"button":"#454850","contrast":"#FFFFFF","danger":"#FF144D","dark":"#000000","info":"#FFCC19","light":"#2F3034","medium":"#1C1C1C","medium2":"#141414","success":"#2FCB52"},"brand":{"earth100":"#C6EEDB","earth200":"#9DDFBF","earth300":"#6BBF96","earth400":"#37A46F","earth400Transparent":"rgba(55, 164, 111, 0.4)","earth500":"#00954C","energy100":"#FFF4BA","energy200":"#FFED92","energy300":"#FFE662","energy400":"#FFDA19","energy400Transparent":"rgba(255, 218, 25, 0.04)","energy500":"#F6C000","fire100":"#FFD7E0","fire200":"#FFA3B9","fire300":"#FF547D","fire400":"#FF144D","fire400Transparent":"rgba(255, 20, 77, 0.4)","fire500":"#EB0032","stone100":"#F3E6D7","stone200":"#E3D4C2","stone400":"#BFA78B","stone500":"#AF8859","water100":"#BDCEFF","water200":"#81A1FF","water300":"#517DFF","water400":"#2A60FE","water400Transparent":"rgba(42, 96, 254, 0.4)","water500":"#0F4CFF","wind100":"#EEEAFF","wind200":"#CCC1F4","wind300":"#A698DD","wind400":"#6B50D7","wind400SuperTransparent":"rgba(107, 80, 215, 0.12)","wind400Transparent":"rgba(107, 80, 215, 0.4)","wind500":"#4E32BC"},"chart":{"backgroundPrimary":"#7D55EC","backgroundSecondary":"#FF144D","backgroundTertiary":"#86E2FF","button1":"#4877FF","button2":"#FFCC19","button3":"#8ADE00","button4":"#FF4FF8","button5":"#B98D95","lines":"#9B6CA7","primary":"#6B50D7","secondary":"#FF144D","tertiary":"#2A60FE"},"content":{"active":"#FFFFFF","default":"#AEAEAE","disabled":"rgba(255, 255, 255, 0.3)","inverted":"#2C2C2C","muted":"#787A85"},"elevation":{"visualizationAccent":"#996CFF","visualizationAccentAlt":"#C1ACF7"},"feature":{"active":"rgba(250, 248, 254, 0.14)","disabled":"rgba(201, 206, 218, 0.12)"},"icons":{"neutral":"#787878"},"interactive":{"activeBorder":"#060606","checked":"#060606","dangerBorder":"#FF144D","defaultBackground":"#36383F","defaultBorder":"#2E3036","disabledBorder":"#B1B8C3","focusBackground":"#B1B8C3","focusBorder":"#86E2FF","hoverBackground":"#4E4E4E","hoverBorder":"#B9BFCA","hoverOverlay":"rgba(255, 255, 255, 0.1)","linkPrimary":"#1752FF","linkPrimaryHover":"#4877FF","linkPrimaryLight":"#5982ff","linkSecondary":"#6B50D7","linkSecondaryDisabled":"#C4B9EF","linkText":"#6AA1E0","rowHoverBackground":"rgba(0, 0, 0, 0.1)","transparent":"rgba(255, 255, 255, 0)"},"loader":{"color":"#EB0032","colorInverted":"#8ADE00"},"logo":{"color":"#FFFFFF"},"monotone":{"black":"#060606","blackTransparent":"rgba(0, 0, 0, 0.6)","gray":"#B1B8C3","grey100":"#F2F2F2","grey200":"#D5D7DC","grey300":"#B4B8C0","grey400":"#70747C","grey500":"#51535C","purple":"#6B50D7","white":"#FFFFFF"},"neutral":{"n100":"#E7E8EA","n200":"#D8DADE","n300":"#CBCCD0","n400":"#BCBEC4","n500":"#AEB0B6"},"progress":{"negative":"#FF144D","positive":"#6B50D7"},"shadow":{"base":"12px 40px 120px rgba(106, 117, 139, 0.4)","frame":"0px 10px 40px rgba(0, 0, 0, 0.26)","menu":"4px 10px 20px rgba(6, 6, 6, 0.12)","popup":"10px 20px 40px rgba(0, 0, 0, 0.2)","small":"0px, 4px, rgba(0, 0, 0, 0.25)","window":"0px 10px 60px rgba(0, 0, 0, 0.7)"},"status":{"negative":"#FF144D","positive":"#24B400"},"text":{"fileBrowser":"#787A85"}}},"page":"/version-control","query":{},"buildId":"tMZjCwLa1LRYRzhFRdKNf","nextExport":true,"isFallback":false,"gip":true,"appGip":true,"scriptLoader":[]}</script></body></html>
+</style></head><body><div id="__next"><div class="" style="position:fixed;top:0;left:0;height:2px;background:transparent;z-index:99999999999;width:100%"><div class="" style="height:100%;background:#FF144D;transition:all 500ms ease;width:0%"><div style="box-shadow:0 0 10px #FF144D, 0 0 10px #FF144D;width:5%;opacity:1;position:absolute;height:100%;transition:all 500ms ease;transform:rotate(3deg) translate(0px, -4px);left:-10rem"></div></div></div><div class="indexstyle__HeaderStyle-sc-1bk8irg-0 gbXfes"><div></div></div><div class="indexstyle__ContainerStyle-sc-ecogjt-0 ijwRXz"><div class="indexstyle__VerticalNavigationStyleComponent-sc-ecogjt-1 jQArIy"><div class="Spacing__SpacingStyle-sc-1mpmtgj-0 hDBiHu"><div></div></div></div><div class="Flex-sc-sgfnl9-0 dKQluW"><div></div></div></div><div></div><div></div><div></div><div class="ToastWrapper-sc-1a33ph1-0 kOVcuR"><div class="Toastify"></div></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"auth":{"decodedToken":{"expires":0,"token":null}}},"currentTheme":{"accent":{"alert":"#F6540B","blue":"#4877FF","blueLight":"rgba(72, 119, 255, 0.5)","contentDefaultTransparent":"rgba(174, 174, 174, 0.5)","cyan":"#65E3FF","cyanLight":"rgba(101, 227, 255, 0.3)","cyanTransparent":"rgba(101, 227, 255, 0.12)","dbt":"#fc6949","dbtLight":"rgba(252, 105, 73, 0.5)","info":"#00ABFF","infoTransparent":"rgba(0, 171, 255, 0.5)","negative":"#FF1E59","negativeTransparent":"rgba(255, 30, 89, 0.3)","pink":"#FF4FF8","pinkLight":"rgb(255, 79, 248, 0.5)","positive":"#00A81A","primaryTransparent":"rgba(155, 108, 167, 0.5)","purple":"#7D55EC","purpleLight":"rgba(125, 85, 236, 0.5)","rose":"#D1A2AB","roseLight":"rgba(209, 162, 171, 0.5)","sky":"#6AA1E0","skyLight":"rgba(106, 161, 224, 0.05)","teal":"#00B4CC","tealLight":"rgba(0, 180, 204, 0.5)","warning":"#DD9900","warningTransparent":"rgba(221, 153, 0, 0.5)","yellow":"#FFCC19","yellowLight":"rgba(255, 204, 25, 0.5)"},"background":{"chartBlock":"#2E3036","codeArea":"#1E1F24","codeTextarea":"#000000","content":"#1B1C20","danger":"#FFD0DB","dark":"#B1B8C3","dashboard":"#18181C","header":"#1B1B1B","menu":"#0F4CFF","muted":"#F9FAFC","navigation":"#EDEDED","output":"#2E3036","page":"#1E1F24","panel":"#232429","popup":"#27292E","row":"#2C2C2C","row2":"#51535C","scrollbarThumb":"rgba(100, 100, 100, 0.5)","scrollbarThumbHover":"rgba(255, 255, 255, 0.3)","scrollbarTrack":"#2E3036","success":"#8ADE00","successLight":"rgb(138, 222, 0, 0.3)","table":"#292A2F","tag":"#3A4550"},"borders":{"button":"#454850","contrast":"#FFFFFF","danger":"#FF144D","dark":"#000000","info":"#FFCC19","light":"#2F3034","medium":"#1C1C1C","medium2":"#141414","success":"#2FCB52"},"brand":{"earth100":"#C6EEDB","earth200":"#9DDFBF","earth300":"#6BBF96","earth400":"#37A46F","earth400Transparent":"rgba(55, 164, 111, 0.4)","earth500":"#00954C","energy100":"#FFF4BA","energy200":"#FFED92","energy300":"#FFE662","energy400":"#FFDA19","energy400Transparent":"rgba(255, 218, 25, 0.04)","energy500":"#F6C000","fire100":"#FFD7E0","fire200":"#FFA3B9","fire300":"#FF547D","fire400":"#FF144D","fire400Transparent":"rgba(255, 20, 77, 0.4)","fire500":"#EB0032","stone100":"#F3E6D7","stone200":"#E3D4C2","stone400":"#BFA78B","stone500":"#AF8859","water100":"#BDCEFF","water200":"#81A1FF","water300":"#517DFF","water400":"#2A60FE","water400Transparent":"rgba(42, 96, 254, 0.4)","water500":"#0F4CFF","wind100":"#EEEAFF","wind200":"#CCC1F4","wind300":"#A698DD","wind400":"#6B50D7","wind400SuperTransparent":"rgba(107, 80, 215, 0.12)","wind400Transparent":"rgba(107, 80, 215, 0.4)","wind500":"#4E32BC"},"chart":{"backgroundPrimary":"#7D55EC","backgroundSecondary":"#FF144D","backgroundTertiary":"#86E2FF","button1":"#4877FF","button2":"#FFCC19","button3":"#8ADE00","button4":"#FF4FF8","button5":"#B98D95","lines":"#9B6CA7","primary":"#6B50D7","secondary":"#FF144D","tertiary":"#2A60FE"},"content":{"active":"#FFFFFF","default":"#AEAEAE","disabled":"rgba(255, 255, 255, 0.3)","inverted":"#2C2C2C","muted":"#787A85"},"elevation":{"visualizationAccent":"#996CFF","visualizationAccentAlt":"#C1ACF7"},"feature":{"active":"rgba(250, 248, 254, 0.14)","disabled":"rgba(201, 206, 218, 0.12)"},"icons":{"neutral":"#787878"},"interactive":{"activeBorder":"#060606","checked":"#060606","dangerBorder":"#FF144D","defaultBackground":"#36383F","defaultBorder":"#2E3036","disabledBorder":"#B1B8C3","focusBackground":"#B1B8C3","focusBorder":"#86E2FF","hoverBackground":"#4E4E4E","hoverBorder":"#B9BFCA","hoverOverlay":"rgba(255, 255, 255, 0.1)","linkPrimary":"#1752FF","linkPrimaryHover":"#4877FF","linkPrimaryLight":"#5982ff","linkSecondary":"#6B50D7","linkSecondaryDisabled":"#C4B9EF","linkText":"#6AA1E0","rowHoverBackground":"rgba(0, 0, 0, 0.1)","transparent":"rgba(255, 255, 255, 0)"},"loader":{"color":"#EB0032","colorInverted":"#8ADE00"},"logo":{"color":"#FFFFFF"},"monotone":{"black":"#060606","blackTransparent":"rgba(0, 0, 0, 0.6)","gray":"#B1B8C3","grey100":"#F2F2F2","grey200":"#D5D7DC","grey300":"#B4B8C0","grey400":"#70747C","grey500":"#51535C","purple":"#6B50D7","white":"#FFFFFF"},"neutral":{"n100":"#E7E8EA","n200":"#D8DADE","n300":"#CBCCD0","n400":"#BCBEC4","n500":"#AEB0B6"},"progress":{"negative":"#FF144D","positive":"#6B50D7"},"shadow":{"base":"12px 40px 120px rgba(106, 117, 139, 0.4)","frame":"0px 10px 40px rgba(0, 0, 0, 0.26)","menu":"4px 10px 20px rgba(6, 6, 6, 0.12)","popup":"10px 20px 40px rgba(0, 0, 0, 0.2)","small":"0px, 4px, rgba(0, 0, 0, 0.25)","window":"0px 10px 60px rgba(0, 0, 0, 0.7)"},"status":{"negative":"#FF144D","positive":"#24B400"},"text":{"fileBrowser":"#787A85"}}},"page":"/version-control","query":{},"buildId":"OwvTROmputQCA1Z9HEGDv","nextExport":true,"isFallback":false,"gip":true,"appGip":true,"scriptLoader":[]}</script></body></html>

mage_ai/server/server.py CHANGED Viewed

@@ -15,6 +15,7 @@ from tornado.options import options
 from mage_ai.authentication.passwords import create_bcrypt_hash, generate_salt
 from mage_ai.cache.block import BlockCache
+from mage_ai.cache.block_action_object import BlockActionObjectCache
 from mage_ai.cache.tag import TagCache
 from mage_ai.data_preparation.preferences import get_preferences
 from mage_ai.data_preparation.repo_manager import (
@@ -300,9 +301,13 @@ async def main(
     print('Initializing block cache.')
     await BlockCache.initialize_cache(replace=True)
     print('Initializing tag cache.')
     await TagCache.initialize_cache(replace=True)
+    print('Initializing block action object cache.')
+    await BlockActionObjectCache.initialize_cache(replace=True)
     # Check scheduler status periodically
     periodic_callback = PeriodicCallback(
         check_scheduler_status,

mage_ai/server/websocket_server.py CHANGED Viewed

@@ -238,6 +238,7 @@ class WebSocketServer(tornado.websocket.WebSocketHandler):
             global_vars['interval_end_datetime'] = None
             global_vars['interval_seconds'] = None
             global_vars['interval_start_datetime'] = now
+            global_vars['interval_start_datetime_previous'] = None
         global_vars['event'] = dict()

mage_ai/services/search/__init__.py ADDED Viewed

File without changes

mage_ai/services/search/block_action_objects.py ADDED Viewed

@@ -0,0 +1,33 @@
+from thefuzz import fuzz
+from typing import List
+from mage_ai.cache.block_action_object import BlockActionObjectCache
+DEFAULT_RATIO = 80
+async def search(query: str, ratio: float = None, limit: int = None) -> List:
+    ratio_to_use = ratio or DEFAULT_RATIO
+    cache = await BlockActionObjectCache.initialize_cache()
+    results = []
+    for object_type, mapping in cache.load_all_data().items():
+        for uuid, block_action_object in mapping.items():
+            score = fuzz.partial_token_sort_ratio(query, uuid)
+            if score < ratio_to_use:
+                continue
+            results.append(dict(
+                block_action_object=block_action_object,
+                object_type=object_type,
+                score=score,
+                uuid=uuid,
+            ))
+    arr = sorted(results, key=lambda x: x['score'], reverse=True)
+    if limit:
+        arr = arr[:limit]
+    return arr

mage_ai/services/search/constants.py ADDED Viewed

	@@ -0,0 +1 @@
1	+ SEARCH_TYPE_BLOCK_ACTION_OBJECTS = 'block_action_objects'

mage_ai/shared/array.py CHANGED Viewed

@@ -24,6 +24,13 @@ def find(condition, arr, map=None):
         return None
+def find_index(condition, arr):
+    for idx, item in enumerate(arr):
+        if condition(item):
+            return idx
+    return -1
 def sample(arr):
     return arr[random.randrange(0, len(arr))]

mage_ai/shared/parsers.py CHANGED Viewed

@@ -1,5 +1,6 @@
 from datetime import datetime
 from json import JSONDecoder
 import numpy as np
 INTS = (
@@ -46,13 +47,15 @@ def encode_complex(obj):
     return obj
-def extract_json_objects(text, decoder=JSONDecoder()):
+def extract_json_objects(text, decoder=None):
     """Find JSON objects in text, and yield the decoded JSON data
     Does not attempt to look for JSON arrays, text, or other JSON types outside
     of a parent JSON object.
     """
+    if decoder is None:
+        decoder = JSONDecoder()
     pos = 0
     while True:
         match = text.find('{', pos)
@@ -69,7 +72,13 @@ def extract_json_objects(text, decoder=JSONDecoder()):
 def sample_output(obj):
     if isinstance(obj, list):
         sampled = len(obj) > MAX_ITEMS_IN_SAMPLE_OUTPUT
-        return obj[:MAX_ITEMS_IN_SAMPLE_OUTPUT], sampled
+        sampled_list = []
+        for item in obj[:MAX_ITEMS_IN_SAMPLE_OUTPUT]:
+            item, item_sampled = sample_output(item)
+            if item_sampled:
+                sampled = True
+            sampled_list.append(item)
+        return sampled_list, sampled
     elif isinstance(obj, dict):
         sampled = False
         output = dict()

mage_ai/tests/shared/test_parsers.py ADDED Viewed

@@ -0,0 +1,73 @@
+from mage_ai.shared.parsers import sample_output
+from mage_ai.tests.base_test import TestCase
+class ParsersTests(TestCase):
+    def test_sample_output_empty_input(self):
+        self.assertEqual(sample_output([]), ([], False))
+        self.assertEqual(sample_output({}), ({}, False))
+    def test_sample_output_no_sampling(self):
+        input_data = [1, 2, 3]
+        self.assertEqual(sample_output(input_data), (input_data, False))
+        input_data = {'key': 'value', 'nested': {'inner': 'data'}}
+        self.assertEqual(sample_output(input_data), (input_data, False))
+    def test_sample_output_sampling_list(self):
+        # A nested list with more than 20 items, should be sampled
+        input_data = [i for i in range(30)]
+        expected_output = ([i for i in range(20)], True)
+        self.assertEqual(sample_output(input_data), expected_output)
+    def test_sample_output_sampling_dict(self):
+        input_data = {'key{}'.format(i): i for i in range(30)}
+        expected_output = ({'key{}'.format(i): i for i in range(30)}, False)
+        self.assertEqual(sample_output(input_data), expected_output)
+    def test_sample_output_sampling_mixed(self):
+        # A mixed structure with a nested list and dictionary, both exceeding the threshold
+        input_data = {'key1': [i for i in range(30)], 'key2': {'inner_key': [i for i in range(30)]}}
+        expected_output = (
+            {
+                'key1': [i for i in range(20)],
+                'key2': {'inner_key': [i for i in range(20)]}
+            },
+            True,
+        )
+        self.assertEqual(sample_output(input_data), expected_output)
+        input_data2 = {
+            'key1': [
+                {
+                    'subkey_1': [i for i in range(30)],
+                    'subkey_2': {
+                        'subkey_21': {
+                            'subkey_211': [i for i in range(30)]
+                        }
+                    },
+                },
+                {
+                    'subkey_3': [1, 2, 3],
+                }
+            ]
+        }
+        expected_output2 = (
+            {
+                'key1': [
+                    {
+                        'subkey_1': [i for i in range(20)],
+                        'subkey_2': {
+                            'subkey_21': {
+                                'subkey_211': [i for i in range(20)]
+                            },
+                        },
+                    },
+                    {
+                        'subkey_3': [1, 2, 3],
+                    }
+                ]
+            },
+            True,
+        )
+        self.assertEqual(sample_output(input_data2), expected_output2)

mage_ai/usage_statistics/constants.py CHANGED Viewed

@@ -10,6 +10,7 @@ class EventActionType(str, enum.Enum):
 class EventObjectType(str, enum.Enum):
     ERROR = 'error'
+    FEATURE = 'feature'
     PIPELINE = 'pipeline'
     PIPELINE_RUN = 'pipeline_run'
     PROJECT = 'project'

mage_ai/usage_statistics/logger.py CHANGED Viewed

@@ -22,7 +22,25 @@ class UsageStatisticLogger():
         if not self.help_improve_mage:
             return False
-        return await self.__send_message(EventObjectType.PROJECT, EventActionType.IMPRESSION)
+        features = {}
+        for k, v in (self.project.features or {}).items():
+            features[k] = 1 if v else 0
+        if self.project.repo_config.openai_api_key and \
+                len(self.project.repo_config.openai_api_key) >= 1:
+            features['openai'] = 1
+        else:
+            features['openai'] = 0
+        return await self.__send_message(
+            EventObjectType.PROJECT,
+            EventActionType.IMPRESSION,
+            dict(
+                features=features,
+            ),
+        )
     @safe_db_query
     async def pipeline_runs_impression(self, count_func: Callable) -> bool:

{mage_ai-0.9.7.dist-info → mage_ai-0.9.9.dist-info}/METADATA RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mage-ai
-Version: 0.9.7
+Version: 0.9.9
 Summary: Mage is a tool for building and deploying data pipelines.
 Home-page: https://github.com/mage-ai/mage-ai
 Author: Mage
@@ -53,6 +53,7 @@ Requires-Dist: simplejson
 Requires-Dist: six (>=1.15.0)
 Requires-Dist: sqlalchemy (<2.0.0,>=1.4.20)
 Requires-Dist: terminado (==0.17.1)
+Requires-Dist: thefuzz[speedup] (==0.19.0)
 Requires-Dist: tornado (==6.1)
 Requires-Dist: typer[all] (==0.7.0)
 Requires-Dist: typing-extensions (==4.5.0)
@@ -73,9 +74,9 @@ Requires-Dist: botocore (==1.29.60) ; extra == 'all'
 Requires-Dist: clickhouse-connect (==0.5.20) ; extra == 'all'
 Requires-Dist: confluent-avro ; extra == 'all'
 Requires-Dist: db-dtypes (==1.0.5) ; extra == 'all'
-Requires-Dist: dbt-bigquery (==1.4.0) ; extra == 'all'
-Requires-Dist: dbt-core (==1.4.0) ; extra == 'all'
-Requires-Dist: dbt-postgres (==1.4.0) ; extra == 'all'
+Requires-Dist: dbt-bigquery (==1.4.3) ; extra == 'all'
+Requires-Dist: dbt-core (==1.4.6) ; extra == 'all'
+Requires-Dist: dbt-postgres (==1.4.6) ; extra == 'all'
 Requires-Dist: dbt-redshift (==1.4.0) ; extra == 'all'
 Requires-Dist: dbt-snowflake (==1.4.0) ; extra == 'all'
 Requires-Dist: dbt-spark (==1.4.0) ; extra == 'all'
@@ -106,6 +107,7 @@ Requires-Dist: redshift-connector (==2.0.909) ; extra == 'all'
 Requires-Dist: requests-aws4auth (==1.1.2) ; extra == 'all'
 Requires-Dist: snowflake-connector-python (==2.7.9) ; extra == 'all'
 Requires-Dist: sshtunnel (==0.4.0) ; extra == 'all'
+Requires-Dist: thefuzz[speedup] (==0.19.0) ; extra == 'all'
 Provides-Extra: azure
 Requires-Dist: azure-eventhub (==5.11.2) ; extra == 'azure'
 Requires-Dist: azure-identity (==1.11.0) ; extra == 'azure'
@@ -119,9 +121,9 @@ Requires-Dist: db-dtypes (==1.0.5) ; extra == 'bigquery'
 Provides-Extra: clickhouse
 Requires-Dist: clickhouse-connect (==0.5.20) ; extra == 'clickhouse'
 Provides-Extra: dbt
-Requires-Dist: dbt-bigquery (==1.4.0) ; extra == 'dbt'
-Requires-Dist: dbt-core (==1.4.0) ; extra == 'dbt'
-Requires-Dist: dbt-postgres (==1.4.0) ; extra == 'dbt'
+Requires-Dist: dbt-bigquery (==1.4.3) ; extra == 'dbt'
+Requires-Dist: dbt-core (==1.4.6) ; extra == 'dbt'
+Requires-Dist: dbt-postgres (==1.4.6) ; extra == 'dbt'
 Requires-Dist: dbt-redshift (==1.4.0) ; extra == 'dbt'
 Requires-Dist: dbt-snowflake (==1.4.0) ; extra == 'dbt'
 Requires-Dist: dbt-spark (==1.4.0) ; extra == 'dbt'

mage-ai 0.9.7__py3-none-any.whl → 0.9.9__py3-none-any.whl

Potentially problematic release.

mage-ai 0.9.7py3-none-any.whl → 0.9.9py3-none-any.whl