pycarlo 0.10.227__tar.gz → 0.12.33__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of pycarlo might be problematic. Click here for more details.

Files changed (112) hide show
  1. {pycarlo-0.10.227 → pycarlo-0.12.33}/.circleci/config.yml +10 -19
  2. pycarlo-0.12.33/.circleci/scripts/clean-generated-types-prs.sh +171 -0
  3. {pycarlo-0.10.227 → pycarlo-0.12.33}/.gitignore +7 -1
  4. {pycarlo-0.10.227 → pycarlo-0.12.33}/.pre-commit-config.yaml +3 -0
  5. pycarlo-0.12.33/CONTRIBUTING.md +47 -0
  6. pycarlo-0.12.33/Makefile +135 -0
  7. {pycarlo-0.10.227 → pycarlo-0.12.33}/PKG-INFO +47 -30
  8. {pycarlo-0.10.227 → pycarlo-0.12.33}/README.md +46 -29
  9. pycarlo-0.12.33/pycarlo/features/metadata/__init__.py +32 -0
  10. pycarlo-0.12.33/pycarlo/features/metadata/asset_allow_block_list.py +22 -0
  11. pycarlo-0.12.33/pycarlo/features/metadata/asset_filters_container.py +79 -0
  12. pycarlo-0.12.33/pycarlo/features/metadata/base_allow_block_list.py +137 -0
  13. pycarlo-0.12.33/pycarlo/features/metadata/metadata_allow_block_list.py +94 -0
  14. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/metadata/metadata_filters_container.py +25 -16
  15. pycarlo-0.12.33/pycarlo/lib/README.md +35 -0
  16. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/lib/schema.json +6455 -441
  17. pycarlo-0.12.33/pycarlo/lib/schema.py +82753 -0
  18. pycarlo-0.12.33/pycarlo/lib/types.py +68 -0
  19. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo.egg-info/PKG-INFO +47 -30
  20. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo.egg-info/SOURCES.txt +15 -1
  21. {pycarlo-0.10.227 → pycarlo-0.12.33}/pyproject.toml +2 -0
  22. pycarlo-0.12.33/tests/data/README.md +17 -0
  23. pycarlo-0.10.227/pycarlo/lib/schema.py → pycarlo-0.12.33/tests/data/schema_original.py +3052 -727
  24. pycarlo-0.12.33/tests/features/metadata/test_asset_filtering.py +416 -0
  25. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/metadata/test_dataset_filtering.py +60 -38
  26. pycarlo-0.12.33/tests/lib/test_enum.py +166 -0
  27. pycarlo-0.12.33/tests/lib/test_enum_integration.py +88 -0
  28. pycarlo-0.12.33/tests/lib/test_schema_generation.py +182 -0
  29. pycarlo-0.12.33/tests/test_forgiving_enums_published.sh +148 -0
  30. pycarlo-0.12.33/tests/verify_forgiving_enums_published.py +285 -0
  31. pycarlo-0.12.33/utils/env.sh +23 -0
  32. {pycarlo-0.10.227 → pycarlo-0.12.33}/utils/generate.py +1 -1
  33. pycarlo-0.10.227/Makefile +0 -80
  34. pycarlo-0.10.227/pycarlo/features/metadata/__init__.py +0 -15
  35. pycarlo-0.10.227/pycarlo/features/metadata/allow_block_list.py +0 -159
  36. pycarlo-0.10.227/pycarlo/lib/README.md +0 -3
  37. pycarlo-0.10.227/utils/env.sh +0 -12
  38. {pycarlo-0.10.227 → pycarlo-0.12.33}/.circleci/README.md +0 -0
  39. {pycarlo-0.10.227 → pycarlo-0.12.33}/.coveragerc +0 -0
  40. {pycarlo-0.10.227 → pycarlo-0.12.33}/.github/workflows/release.yml +0 -0
  41. {pycarlo-0.10.227 → pycarlo-0.12.33}/LICENSE +0 -0
  42. {pycarlo-0.10.227 → pycarlo-0.12.33}/examples/sample_circuit_breaker.py +0 -0
  43. {pycarlo-0.10.227 → pycarlo-0.12.33}/examples/sample_insight_upload.py +0 -0
  44. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/__init__.py +0 -0
  45. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/common/__init__.py +0 -0
  46. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/common/errors.py +0 -0
  47. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/common/files.py +0 -0
  48. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/common/http.py +0 -0
  49. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/common/mcon.py +0 -0
  50. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/common/retries.py +0 -0
  51. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/common/settings.py +0 -0
  52. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/common/utils.py +0 -0
  53. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/core/__init__.py +0 -0
  54. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/core/client.py +0 -0
  55. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/core/endpoint.py +0 -0
  56. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/core/operations.py +0 -0
  57. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/core/session.py +0 -0
  58. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/__init__.py +0 -0
  59. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/circuit_breakers/__init__.py +0 -0
  60. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/circuit_breakers/exceptions.py +0 -0
  61. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/circuit_breakers/service.py +0 -0
  62. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/dbt/__init__.py +0 -0
  63. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/dbt/dbt_importer.py +0 -0
  64. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/dbt/queries.py +0 -0
  65. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/exceptions.py +0 -0
  66. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/pii/__init__.py +0 -0
  67. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/pii/constants.py +0 -0
  68. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/pii/pii_filterer.py +0 -0
  69. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/pii/queries.py +0 -0
  70. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/pii/service.py +0 -0
  71. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/user/__init__.py +0 -0
  72. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/user/exceptions.py +0 -0
  73. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/user/models.py +0 -0
  74. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/user/queries.py +0 -0
  75. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/features/user/service.py +0 -0
  76. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo/lib/__init__.py +0 -0
  77. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo.egg-info/dependency_links.txt +0 -0
  78. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo.egg-info/requires.txt +0 -0
  79. {pycarlo-0.10.227 → pycarlo-0.12.33}/pycarlo.egg-info/top_level.txt +0 -0
  80. {pycarlo-0.10.227 → pycarlo-0.12.33}/requirements-ci.txt +0 -0
  81. {pycarlo-0.10.227 → pycarlo-0.12.33}/requirements-dev.txt +0 -0
  82. {pycarlo-0.10.227 → pycarlo-0.12.33}/requirements.txt +0 -0
  83. {pycarlo-0.10.227 → pycarlo-0.12.33}/setup.cfg +0 -0
  84. {pycarlo-0.10.227 → pycarlo-0.12.33}/setup.py +0 -0
  85. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/__init__.py +0 -0
  86. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/common/__init__.py +0 -0
  87. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/common/data.json +0 -0
  88. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/common/test_files.py +0 -0
  89. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/common/test_http.py +0 -0
  90. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/common/test_mcon.py +0 -0
  91. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/common/test_retries.py +0 -0
  92. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/common/test_utils.py +0 -0
  93. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/__init__.py +0 -0
  94. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/circuit_breakers/__init__.py +0 -0
  95. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/circuit_breakers/test_service.py +0 -0
  96. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/dbt/__init__.py +0 -0
  97. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/dbt/sample_logs.txt +0 -0
  98. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/dbt/sample_manifest.json +0 -0
  99. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/dbt/sample_run_results.json +0 -0
  100. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/dbt/test_dbt_importer.py +0 -0
  101. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/pii/sample_events/sample_md_events_01.json +0 -0
  102. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/pii/sample_events/sample_md_events_02.json +0 -0
  103. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/pii/test_pii_filtering.py +0 -0
  104. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/user/__init__.py +0 -0
  105. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/features/user/test_user_service.py +0 -0
  106. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/test_client.py +0 -0
  107. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/test_operations.py +0 -0
  108. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/test_retry_decorator.py +0 -0
  109. {pycarlo-0.10.227 → pycarlo-0.12.33}/tests/test_session.py +0 -0
  110. {pycarlo-0.10.227 → pycarlo-0.12.33}/utils/sample.env +0 -0
  111. {pycarlo-0.10.227 → pycarlo-0.12.33}/utils/sanity.py +0 -0
  112. {pycarlo-0.10.227 → pycarlo-0.12.33}/utils/vars.py +0 -0
@@ -1,7 +1,7 @@
1
1
  version: 2.1
2
2
 
3
- deploy_filter: &deploy_filter
4
- # YAML reference for tag based filtering on deployments
3
+ deploy_filter:
4
+ &deploy_filter # YAML reference for tag based filtering on deployments
5
5
  filters:
6
6
  tags:
7
7
  only: /^v.*/
@@ -15,18 +15,14 @@ test_steps: &test_steps
15
15
  name: Update pip
16
16
  command: pip install --upgrade pip
17
17
  - run:
18
- name: Install dependencies
19
- command: pip install virtualenv==20.26.1
20
- - run:
21
- name: Run unit tests
22
- command: make install-with-tests
18
+ name: Run tests and checks
19
+ command: |
20
+ make install
21
+ make test
23
22
 
24
23
  sanity_check: &sanity_check_steps
25
24
  steps:
26
25
  - checkout
27
- - run:
28
- name: Install dependencies
29
- command: pip install virtualenv==20.26.1
30
26
  - run:
31
27
  name: Make Sanity Check
32
28
  command: |
@@ -67,22 +63,17 @@ jobs:
67
63
  - image: python:3.9
68
64
  <<: *sanity_check_steps
69
65
 
70
- deploy:
66
+ deploy: # cci-ignore
71
67
  docker:
72
68
  - image: python:3.8.6
73
69
 
74
70
  steps:
75
71
  - checkout
76
- - run:
77
- name: Install dependencies
78
- command: pip install virtualenv==20.26.1
79
72
  - run:
80
73
  name: Deploy to PyPI
81
74
  command: make distribute
82
75
 
83
76
  workflows:
84
- version: 2
85
-
86
77
  build:
87
78
  jobs:
88
79
  - run-test-3-8
@@ -91,9 +82,9 @@ workflows:
91
82
  - run-test-3-11
92
83
  - run-test-3-12
93
84
  - run-sanity-check:
94
- context: mc-prod
85
+ context: mc-prod # cci-ignore
95
86
 
96
- deploy:
87
+ deploy: # cci-ignore
97
88
  jobs:
98
89
  - run-test-3-8:
99
90
  <<: *deploy_filter
@@ -106,7 +97,7 @@ workflows:
106
97
  - run-test-3-12:
107
98
  <<: *deploy_filter
108
99
  - run-sanity-check:
109
- context: mc-prod
100
+ context: mc-prod # cci-ignore
110
101
  <<: *deploy_filter
111
102
  - deploy:
112
103
  requires:
@@ -0,0 +1,171 @@
1
+ #!/bin/bash
2
+ set -euo pipefail
3
+
4
+ # Colors for output
5
+ RED='\033[1;31m'
6
+ GREEN='\033[1;32m'
7
+ YELLOW='\033[1;33m'
8
+ BLUE='\033[0;36m' # Light cyan/blue
9
+ NC='\033[0m' # No Color
10
+
11
+ # Print functions
12
+ info() {
13
+ echo -e "${BLUE}$1${NC}" >&2
14
+ }
15
+
16
+ success() {
17
+ echo -e "${GREEN}$1${NC}" >&2
18
+ }
19
+
20
+ warn() {
21
+ echo -e "${YELLOW}[WARNING]${NC} $1" >&2
22
+ }
23
+
24
+ error() {
25
+ echo -e "${RED}[ERROR]${NC} $1" >&2
26
+ }
27
+
28
+ # Function to trigger type generation workflow for a given monolith PR
29
+ rerun_generation() {
30
+ local monolith_pr=$1
31
+ info "Triggering type generation workflow for monolith PR #$monolith_pr..."
32
+
33
+ # Get the branch name for the monolith PR
34
+ local monolith_branch
35
+ monolith_branch=$(gh pr view "$monolith_pr" --repo monte-carlo-data/monolith-django --json headRefName --jq '.headRefName' 2>/dev/null)
36
+
37
+ if [ -z "$monolith_branch" ]; then
38
+ error "Failed to get branch name for monolith PR #$monolith_pr"
39
+ return 1
40
+ fi
41
+
42
+ info "Found monolith branch: $monolith_branch"
43
+
44
+ # Get the latest workflow run for this branch
45
+ local run_id
46
+ run_id=$(gh run list --repo monte-carlo-data/monolith-django \
47
+ --workflow handle-pycarlo-schema-changes.yml \
48
+ --branch "$monolith_branch" \
49
+ --limit 1 \
50
+ --json databaseId \
51
+ --jq '.[0].databaseId' 2>/dev/null)
52
+
53
+ if [ -z "$run_id" ]; then
54
+ error "No workflow runs found for monolith PR #$monolith_pr (branch: $monolith_branch)"
55
+ return 1
56
+ fi
57
+
58
+ info "Found workflow run ID: $run_id"
59
+
60
+ # Rerun the workflow
61
+ if gh run rerun "$run_id" --repo monte-carlo-data/monolith-django; then
62
+ success "Successfully triggered workflow rerun for monolith PR #$monolith_pr"
63
+ return 0
64
+ else
65
+ error "Failed to rerun workflow for monolith PR #$monolith_pr"
66
+ return 1
67
+ fi
68
+ }
69
+
70
+ info "Checking for existing PRs with same changes as main..."
71
+
72
+ # Check for clean working directory
73
+ if ! git diff --quiet || ! git diff --cached --quiet; then
74
+ error "Working directory has uncommitted changes. Please commit or stash them before running this script."
75
+ exit 1
76
+ fi
77
+
78
+ # Get all open PRs that match our branch naming pattern
79
+ OPEN_PRS=$(gh pr list --base main --search "update-types-from-monolith- in:title is:open" --json number,headRefName,url --jq '.[]')
80
+
81
+ git fetch -q origin main
82
+ for pr in $(echo "${OPEN_PRS}" | jq -c '.'); do
83
+ PR_NUMBER=$(echo "$pr" | jq -r '.number')
84
+ PR_BRANCH=$(echo "$pr" | jq -r '.headRefName')
85
+ PR_URL=$(echo "$pr" | jq -r '.url')
86
+ MONOLITH_PR=$(echo "$PR_BRANCH" | sed -n 's/.*update-types-from-monolith-\([0-9]*\).*/\1/p')
87
+
88
+ info "Comparing current branch with PR #$PR_NUMBER (branch: $PR_BRANCH)..."
89
+
90
+ # Check if the related monolith PR is closed/merged
91
+ # If so, we will go ahead and close the SDK PR because
92
+ # it's more likely than not no longer needed and any changes
93
+ # should already be in main or will get into main via one of the open ones
94
+ if [ -n "$MONOLITH_PR" ]; then
95
+ info "Checking status of monolith PR #$MONOLITH_PR..."
96
+ MONOLITH_STATUS=$(gh pr view "$MONOLITH_PR" --repo monte-carlo-data/monolith-django --json state --jq '.state' 2>/dev/null || echo "NOT_FOUND")
97
+
98
+ if [ "$MONOLITH_STATUS" = "CLOSED" ] || [ "$MONOLITH_STATUS" = "MERGED" ]; then
99
+ warn "Monolith PR #$MONOLITH_PR is $MONOLITH_STATUS. Closing SDK PR #$PR_NUMBER..."
100
+ gh pr close "$PR_NUMBER" --delete-branch --comment "Closing this PR as the related monolith PR #$MONOLITH_PR has been $MONOLITH_STATUS."
101
+ echo
102
+ continue
103
+ elif [ "$MONOLITH_STATUS" = "NOT_FOUND" ]; then
104
+ warn "Could not find monolith PR #$MONOLITH_PR"
105
+ else
106
+ info "Monolith PR #$MONOLITH_PR is still open ($MONOLITH_STATUS)"
107
+ fi
108
+ fi
109
+
110
+ # Fetch the PR branch and main
111
+ git fetch -q origin "$PR_BRANCH"
112
+
113
+ # Track whether PR should be closed
114
+ is_same_as_main=false
115
+
116
+ # Fetch the PR branch
117
+ if ! git fetch -q origin "$PR_BRANCH"; then
118
+ error "Failed to fetch branch $PR_BRANCH for PR #$PR_NUMBER. Skipping..."
119
+ continue
120
+ fi
121
+
122
+ # Compare PR branch with main
123
+ if git diff --quiet origin/"$PR_BRANCH" origin/main; then
124
+ info "Existing PR #$PR_NUMBER is same as main: $PR_URL"
125
+ is_same_as_main=true
126
+ else
127
+ # Create a temporary branch from the PR branch
128
+ if ! git checkout -q -b temp_"$PR_BRANCH" origin/"$PR_BRANCH" 2>/dev/null; then
129
+ error "Failed to checkout branch $PR_BRANCH for PR #$PR_NUMBER. Skipping..."
130
+ continue
131
+ fi
132
+
133
+ # Attempt to merge latest from main into the temporary branch
134
+ if git merge --quiet origin/main --no-ff -m "Merging latest from main" > /dev/null 2>&1; then
135
+ # Now compare with main
136
+ if git diff --quiet origin/main; then
137
+ info "PR #$PR_NUMBER is now same as main after merging: $PR_URL"
138
+ is_same_as_main=true
139
+ else
140
+ info "PR #$PR_NUMBER still has changes compared to main: $PR_URL"
141
+ fi
142
+ else
143
+ warn "Merge conflict detected for PR #$PR_NUMBER. Closing to allow fresh regeneration..."
144
+ # Clean up: abort the merge
145
+ git merge --abort
146
+ # Switch back to original branch and delete the temporary branch
147
+ git checkout -q -
148
+ git branch -q -D temp_"$PR_BRANCH"
149
+ # Close the PR
150
+ info "API repo PR: https://github.com/monte-carlo-data/monolith-django/pull/$MONOLITH_PR"
151
+ gh pr close "$PR_NUMBER" --delete-branch --comment "Closing this PR due to merge conflicts with main. Types will be regenerated on the next update."
152
+
153
+ # Rerun the workflow for the monolith PR
154
+ rerun_generation "$MONOLITH_PR"
155
+
156
+ continue
157
+ fi
158
+
159
+ # Switch back to original branch and delete the temporary branch
160
+ git checkout -q -
161
+ git branch -q -D temp_"$PR_BRANCH"
162
+ fi
163
+
164
+ # Close PR if it's the same as main
165
+ if [ "$is_same_as_main" = true ]; then
166
+ info "API repo PR: https://github.com/monte-carlo-data/monolith-django/pull/$MONOLITH_PR"
167
+ warn "Closing PR #$PR_NUMBER..."
168
+ gh pr close "$PR_NUMBER" --delete-branch --comment "Closing this PR as there are no remaining changes compared to current main."
169
+ fi
170
+ echo
171
+ done
@@ -14,8 +14,9 @@ dist/
14
14
  downloads/
15
15
  eggs/
16
16
  .eggs/
17
- lib/
17
+ /lib/
18
18
  !pycarlo/lib/
19
+ !tests/lib/
19
20
  lib64/
20
21
  parts/
21
22
  sdist/
@@ -132,3 +133,8 @@ dmypy.json
132
133
  .idea/
133
134
 
134
135
  .DS_Store
136
+
137
+ .ruff_cache/
138
+
139
+ # used in tests/test_forgiving_enums_published.sh
140
+ test-forgiving-enums-venv
@@ -8,6 +8,7 @@ repos:
8
8
  types: [python]
9
9
  pass_filenames: true
10
10
  args: [format]
11
+ exclude: ^(pycarlo/lib/schema\.py|tests/data/schema_original\.py)$
11
12
  - repo: local
12
13
  hooks:
13
14
  - id: ruff-check
@@ -17,6 +18,7 @@ repos:
17
18
  types: [python]
18
19
  pass_filenames: true
19
20
  args: [check, --fix, --exit-non-zero-on-fix]
21
+ exclude: ^(pycarlo/lib/schema\.py|tests/data/schema_original\.py)$
20
22
  - repo: local
21
23
  hooks:
22
24
  - id: pyright
@@ -26,3 +28,4 @@ repos:
26
28
  types: [python]
27
29
  pass_filenames: true
28
30
  args: [--level, error]
31
+ exclude: ^(pycarlo/lib/schema\.py|tests/data/schema_original\.py)$
@@ -0,0 +1,47 @@
1
+ # Contribution Guidelines
2
+
3
+ This is primarily for Monte Carlo developers. External contributions are welcome, but you may want
4
+ to reach out though support first to discuss your proposed changes.
5
+
6
+ ## Setting up for development
7
+
8
+ ```sh
9
+ make install-dev
10
+ ```
11
+
12
+ This will create a virtual environment, install dependencies, and set up pre-commit hooks.
13
+
14
+ ## Updating Typed Schema to Match GraphQL API
15
+
16
+ The schema is automatically kept up-to-date by our CI pipeline when changes are merged to the API
17
+ repo. You typically don't need to run `make generate` unless you're testing or debugging an issue
18
+ locally.
19
+
20
+ If you do need to manually update the schema for testing on your development machine, use
21
+ `make generate`. This will fetch the latest schema from the API and regenerate the Python types,
22
+ including any customizations.
23
+
24
+ ## Schema Customizations
25
+
26
+ The generated `schema.py` is automatically modified during the build process to apply the following
27
+ customizations. See [pycarlo/lib/README.md](pycarlo/lib/README.md) for more details. If you need to
28
+ make additional customizations, you need to do so through that process.
29
+
30
+ ## Testing
31
+
32
+ `make test` can be used to run all tests locally. CircleCI manages testing for deployment.
33
+
34
+ ## Releases
35
+
36
+ Our CI pipeline is configured to automatically update the schema and run tests when changes are
37
+ merged to the default branch in our API repo. It will also automatically deploy a new versioned
38
+ release.
39
+
40
+ ### Custom Releases
41
+
42
+ If you need to make a custom release, create a PR against `main`.
43
+
44
+ When ready to release after merging, create a new
45
+ [Github release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository)
46
+ with a tag using semantic versioning (e.g. `v0.42.0`) and CircleCI will test and publish to PyPI.
47
+ Note that an existing version will not be deployed.
@@ -0,0 +1,135 @@
1
+ .PHONY: default clean install install-dev pip-install generate generate-local retrieve-schema transform-schema format-schema update-test-schema customize-schema env-check-venv test sanity-check
2
+ .SILENT: generate generate-local # Prevent echoing of any tokens
3
+
4
+ PACKAGE_NAME=pycarlo
5
+ ENVIRONMENT_NAME=venv
6
+ MCD_URL=https://api.getmontecarlo.com/graphql
7
+
8
+ # Schema generator util
9
+ SCHEMA_GEN_UTIL=utils/generate.py
10
+ SANITY_CHECK_UTIL=utils/sanity.py
11
+ ENV_UTIL=utils/env.sh
12
+ ENV_FILE=utils/.env
13
+ SAMPLE_ENV_FILE=utils/sample.env
14
+
15
+ # Generated schema destinations
16
+ SCHEMA_FROM_INTROSPECTION=$(PACKAGE_NAME)/lib/schema.json
17
+ SCHEMA_PY=$(PACKAGE_NAME)/lib/schema.py
18
+ SCHEMA_TEST_DATA=tests/data/schema_original.py
19
+
20
+ default:
21
+ @echo "Refer to CONTRIBUTING.md for details on common tasks."
22
+
23
+ clean:
24
+ @echo "Cleaning up existing build artifacts..."; \
25
+ rm -rf $(ENVIRONMENT_NAME) build dist $(PACKAGE_NAME).egg-info .coverage nosetests.xml
26
+
27
+ pip-install:
28
+ @echo "\nInstalling Python package in editable mode..."; \
29
+ pip install --editable .
30
+
31
+ install: clean
32
+ ifeq ("$(wildcard $(ENV_FILE))","")
33
+ @echo "\nCreating env file from sample..."; \
34
+ cp $(SAMPLE_ENV_FILE) $(ENV_FILE)
35
+ endif
36
+ @echo "\nCreating virtual environment..."; \
37
+ python3 -m venv $(ENVIRONMENT_NAME);
38
+ @echo "\nInstalling dependencies..."; \
39
+ . $(ENVIRONMENT_NAME)/bin/activate; \
40
+ pip install -r requirements-dev.txt; \
41
+ $(MAKE) pip-install; \
42
+ pip show $(PACKAGE_NAME)
43
+
44
+ # Full development setup = install + pre-commit hooks
45
+ install-dev: install
46
+ @echo "\nInstalling pre-commit hooks..."; \
47
+ . $(ENVIRONMENT_NAME)/bin/activate; \
48
+ pre-commit install; \
49
+ echo ""; \
50
+ echo "\033[0;32m✅ Development environment ready!\033[0m"; \
51
+ echo "\nActivate with: . $(ENVIRONMENT_NAME)/bin/activate"
52
+
53
+ sanity-check: env-check-venv
54
+ @echo "\nRunning sanity checks..."; \
55
+ . $(ENVIRONMENT_NAME)/bin/activate; \
56
+ sh $(ENV_UTIL) python $(SANITY_CHECK_UTIL); \
57
+
58
+ # Requires .env or profile credentials setup before usage.
59
+ retrieve-schema:
60
+ @echo "\nRetrieving latest schema..."; \
61
+ . $(ENVIRONMENT_NAME)/bin/activate; \
62
+ sh $(ENV_UTIL) python $(SCHEMA_GEN_UTIL) $(MCD_URL) $(SCHEMA_FROM_INTROSPECTION);
63
+
64
+ # Expects $(SCHEMA_FROM_INTROSPECTION) to already exist.
65
+ transform-schema:
66
+ @echo "\nGenerating Python types from schema.json..."; \
67
+ . $(ENVIRONMENT_NAME)/bin/activate; \
68
+ sgqlc-codegen schema --docstrings $(SCHEMA_FROM_INTROSPECTION) $(SCHEMA_PY);
69
+
70
+ format-schema:
71
+ @echo "\nFormatting Python code..."; \
72
+ . $(ENVIRONMENT_NAME)/bin/activate; \
73
+ ruff format $(SCHEMA_PY);
74
+
75
+ update-test-schema:
76
+ @echo "\nUpdating test data with original schema..."; \
77
+ mkdir -p tests/data; \
78
+ cp $(SCHEMA_PY) $(SCHEMA_TEST_DATA);
79
+
80
+ customize-schema:
81
+ @echo "\nApplying schema customizations..."; \
82
+ if [ "$$(uname)" = "Darwin" ]; then \
83
+ sed -i '' 's/class Connection(sgqlc.types.relay.Connection):/class Connection(sgqlc.types.Type):/g' $(SCHEMA_PY); \
84
+ sed -i '' 's/import sgqlc.types$$/import sgqlc.types\nimport pycarlo.lib.types/g' $(SCHEMA_PY); \
85
+ sed -i '' 's/class \([A-Za-z0-9_]*\)(sgqlc.types.Enum):/class \1(pycarlo.lib.types.Enum):/g' $(SCHEMA_PY); \
86
+ else \
87
+ sed -i 's/class Connection(sgqlc.types.relay.Connection):/class Connection(sgqlc.types.Type):/g' $(SCHEMA_PY); \
88
+ sed -i 's/import sgqlc.types$$/import sgqlc.types\nimport pycarlo.lib.types/g' $(SCHEMA_PY); \
89
+ sed -i 's/class \([A-Za-z0-9_]*\)(sgqlc.types.Enum):/class \1(pycarlo.lib.types.Enum):/g' $(SCHEMA_PY); \
90
+ fi; \
91
+ echo "*************"; \
92
+ echo "The following schema customizations have been applied:"; \
93
+ echo " 1. Changed 'class Connection(sgqlc.types.relay.Connection)' to 'class Connection(sgqlc.types.Type)'"; \
94
+ echo " 2. Changed all Enum classes to use 'pycarlo.lib.types.Enum' for backward compatibility"; \
95
+ echo "*************"; \
96
+
97
+ env-check-venv:
98
+ @if [ ! -d "$(ENVIRONMENT_NAME)" ]; then \
99
+ echo "\033[0;31m❌ Error: Virtual environment not found. Please run 'make install' first.\033[0m"; \
100
+ exit 1; \
101
+ fi
102
+
103
+ # Generate schema from existing schema.json (skip retrieval).
104
+ # Used by CI when schema.json is already provided.
105
+ generate-local: env-check-venv transform-schema format-schema update-test-schema customize-schema
106
+ @echo "\n\033[0;32m✅ Schema generation complete!\033[0m";
107
+
108
+ # Requires .env or profile credentials setup before usage.
109
+ generate: env-check-venv retrieve-schema generate-local
110
+
111
+ test: env-check-venv
112
+ @echo "\nRunning tests..."; \
113
+ . $(ENVIRONMENT_NAME)/bin/activate; \
114
+ export DEBUG=True; \
115
+ pytest --durations=5 ./tests; \
116
+ echo ""; \
117
+ echo "Linting and formatting..."; \
118
+ ruff check .; \
119
+ ruff format --check .; \
120
+ echo ""; \
121
+ echo "Type checking..."; \
122
+ pyright .
123
+ @echo "\n\033[0;32m✅ All tests and checks passed!\033[0m";
124
+
125
+ # this is only used by CI. See circleci/config.yml:
126
+ distribute: install
127
+ @echo "\nInstalling dependencies..."; \
128
+ . $(ENVIRONMENT_NAME)/bin/activate; \
129
+ pip install -r requirements-ci.txt; \
130
+ echo "\nRunning setup..."; \
131
+ python setup.py sdist bdist_wheel; \
132
+ echo "\nChecking distribution..."; \
133
+ twine check dist/*; \
134
+ echo "\nUploading distribution to PyPI..."; \
135
+ twine upload --non-interactive dist/*
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: pycarlo
3
- Version: 0.10.227
3
+ Version: 0.12.33
4
4
  Summary: Monte Carlo's Python SDK
5
5
  Home-page: https://www.montecarlodata.com/
6
6
  Author: Monte Carlo Data, Inc
@@ -25,9 +25,7 @@ Requires-Dist: requests<3.0.0,>=2.0.0
25
25
  Requires-Dist: responses>=0.20.0
26
26
  Requires-Dist: sgqlc<17.0,>=14.1
27
27
 
28
- # Pycarlo
29
-
30
- Monte Carlo's Alpha Python SDK!
28
+ # Pycarlo - Monte Carlo's Python SDK
31
29
 
32
30
  ## Installation
33
31
 
@@ -40,23 +38,22 @@ virtualenv venv
40
38
  pip install -U pycarlo
41
39
  ```
42
40
 
43
- Developers of the SDK can use:
44
-
45
- ```shell
46
- make install-with-tests
47
- . venv/bin/activate
48
- pre-commit install
49
- ```
50
-
51
41
  ## Overview
52
42
 
53
43
  Pycarlo comprises two components: `core` and `features`.
54
44
 
55
- All Monte Carlo API queries and mutations that you could execute via the API are supported via the `core` library. Operations can be executed as first class objects, using [sgqlc](https://github.com/profusion/sgqlc), or as raw GQL with variables. In both cases, a consistent object where fields can be referenced by dot notation and the more pythonic snake_case is returned for ease of use.
45
+ All Monte Carlo API queries and mutations that you could execute via the API are supported via the
46
+ `core` library. Operations can be executed as first class objects, using
47
+ [sgqlc](https://github.com/profusion/sgqlc), or as raw GQL with variables. In both cases, a
48
+ consistent object where fields can be referenced by dot notation and the more pythonic snake_case is
49
+ returned for ease of use.
56
50
 
57
- The `features` library provides additional convenience for performing common operations like with dbt, circuit breaking, and pii filtering.
51
+ The `features` library provides additional convenience for performing common operations like with
52
+ dbt, circuit breaking, and pii filtering.
58
53
 
59
- Note that an API Key is required to use the SDK. See [here](https://docs.getmontecarlo.com/docs/developer-resources#creating-an-api-key) for details on how to generate one.
54
+ Note that an API Key is required to use the SDK. See
55
+ [our docs on generating API keys](https://docs.getmontecarlo.com/docs/developer-resources#creating-an-api-key)
56
+ for details.
60
57
 
61
58
  ## Basic usage
62
59
 
@@ -144,13 +141,20 @@ print(client(mutation))
144
141
  # ]
145
142
  ```
146
143
 
147
- Note that you can find Monte Carlo's API reference [here](https://apidocs.getmontecarlo.com/).
144
+ ### Examples
145
+
146
+ We have [a few examples here you can reference](./examples).
148
147
 
149
- For details and additional examples on how to map (convert) GraphQL queries to `sgqlc` operations please refer to the docs [here](https://sgqlc.readthedocs.io/en/latest/sgqlc.operation.html).
148
+ See [Monte Carlo's API reference](https://apidocs.getmontecarlo.com/) for all supported queries and
149
+ mutations.
150
+
151
+ For details and additional examples on how to map (convert) GraphQL queries to `sgqlc` operations
152
+ please refer to [the sgqlc docs](https://sgqlc.readthedocs.io/en/latest/sgqlc.operation.html).
150
153
 
151
154
  ### Features
152
155
 
153
- You can use [pydoc](https://docs.python.org/3.8/library/pydoc.html) to retrieve documentation on any feature packages (`pydoc pycarlo.features`).
156
+ You can use [pydoc](https://docs.python.org/library/pydoc.html) to retrieve documentation on any
157
+ feature packages (`pydoc pycarlo.features`).
154
158
 
155
159
  For instance for [circuit breakers](https://docs.getmontecarlo.com/docs/circuit-breakers):
156
160
 
@@ -160,9 +164,13 @@ pydoc pycarlo.features.circuit_breakers.service
160
164
 
161
165
  ## Session configuration
162
166
 
163
- By default, when creating a client the `default` profile from `~/.mcd/profiles.ini` is used. This file created via [montecarlo configure](https://docs.getmontecarlo.com/docs/using-the-cli#setting-up-the-cli) on the CLI. Note that you can find Monte Carlo's CLI reference [here](https://clidocs.getmontecarlo.com/).
167
+ By default, when creating a client the `default` profile from `~/.mcd/profiles.ini` is used. This
168
+ file created via
169
+ [montecarlo configure](https://docs.getmontecarlo.com/docs/using-the-cli#setting-up-the-cli) on the
170
+ CLI. See [Monte Carlo's CLI reference](https://clidocs.getmontecarlo.com/) for more details.
164
171
 
165
- You can override this usage by creating a custom `Session`. For instance, if you want to pass the ID and Token:
172
+ You can override this usage by creating a custom `Session`. For instance, if you want to pass the ID
173
+ and Token:
166
174
 
167
175
  ```python
168
176
  from pycarlo.core import Client, Session
@@ -174,7 +182,8 @@ Sessions support the following params:
174
182
 
175
183
  - `mcd_id`: API Key ID.
176
184
  - `mcd_token`: API secret.
177
- - `mcd_profile`: Named profile containing credentials. This is created via the CLI (e.g. `montecarlo configure --profile-name zeus`).
185
+ - `mcd_profile`: Named profile containing credentials. This is created via the CLI (e.g.
186
+ `montecarlo configure --profile-name zeus`).
178
187
  - `mcd_config_path`: Path to file containing credentials. Defaults to `~/.mcd/`.
179
188
 
180
189
  You can also specify the API Key, secret or profile name using the following environment variables:
@@ -183,7 +192,8 @@ You can also specify the API Key, secret or profile name using the following env
183
192
  - `MCD_DEFAULT_API_TOKEN`
184
193
  - `MCD_DEFAULT_PROFILE`
185
194
 
186
- When creating a session any explicitly passed `mcd_id` and `mcd_token` params take precedence, followed by environmental variables and then any config-file options.
195
+ When creating a session any explicitly passed `mcd_id` and `mcd_token` params take precedence,
196
+ followed by environmental variables and then any config-file options.
187
197
 
188
198
  Environment variables can be mixed with passed credentials, but not the config-file profile.
189
199
 
@@ -191,9 +201,11 @@ Environment variables can be mixed with passed credentials, but not the config-f
191
201
 
192
202
  ## Integration Gateway API
193
203
 
194
- There are features that require the Integration Gateway API instead of the regular GraphQL Application API, for example Airflow Callbacks invoked by the `airflow-mcd` library.
204
+ There are features that require the Integration Gateway API instead of the regular GraphQL
205
+ Application API, for example Airflow Callbacks invoked by the `airflow-mcd` library.
195
206
 
196
- To use the Gateway you need to initialize the `Session` object passing a `scope` parameter and then use `make_request` to invoke Gateway endpoints:
207
+ To use the Gateway you need to initialize the `Session` object passing a `scope` parameter and then
208
+ use `make_request` to invoke Gateway endpoints:
197
209
 
198
210
  ```python
199
211
  from pycarlo.core import Client, Session
@@ -211,17 +223,22 @@ The following values also be set by the environment:
211
223
  - `MCD_VERBOSE_ERRORS`: Enable logging. This includes a trace ID for each session and request.
212
224
  - `MCD_API_ENDPOINT`: Customize the endpoint where queries and mutations are executed.
213
225
 
214
- ## Tests and releases
226
+ ## Enum Backward Compatibility
227
+
228
+ Unlike the baseline `sgqlc` behavior, this SDK is designed to maintain backward compatibility when
229
+ new enum values are added to the Monte Carlo API. If the API returns an enum value that doesn't
230
+ exist in your SDK version, it will be returned as a string with a warning logged, rather than
231
+ raising an error. This allows older SDK versions to continue working when new features are added.
215
232
 
216
- To update queries and mutations via introspection, use `make generate`.
233
+ To avoid warnings and ensure full feature support, keep your SDK updated to the latest version.
217
234
 
218
- `make test` can be used to run all tests locally. CircleCI manages all testing for deployment. When ready for a review, create a PR against `main`.
235
+ ## Contributing
219
236
 
220
- When ready to release, create a new [Github release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository) with a tag using semantic versioning (e.g. `v0.42.0`) and CircleCI will test and publish to PyPI. Note that an existing version will not be deployed.
237
+ See [CONTRIBUTING.md](./CONTRIBUTING.md) for details.
221
238
 
222
239
  ## References
223
240
 
224
- - Dashboard: <https://getmontecarlo.com>
241
+ - Monte Carlo App: <https://getmontecarlo.com>
225
242
  - Product docs: <https://docs.getmontecarlo.com>
226
243
  - Status page: <https://status.getmontecarlo.com>
227
244
  - API (and SDK): <https://apidocs.getmontecarlo.com>
@@ -229,4 +246,4 @@ When ready to release, create a new [Github release](https://docs.github.com/en/
229
246
 
230
247
  ## License
231
248
 
232
- Apache 2.0 - See the [LICENSE](http://www.apache.org/licenses/LICENSE-2.0) for more information.
249
+ Apache 2.0 - See the [LICENSE](./LICENSE) for more information.