datadoc-cli 0.3.0__tar.gz → 0.6.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (313) hide show
  1. datadoc_cli-0.6.0/.gitignore +65 -0
  2. datadoc_cli-0.6.0/CHANGELOG.md +80 -0
  3. datadoc_cli-0.6.0/MIGRATION.md +16 -0
  4. datadoc_cli-0.6.0/PKG-INFO +286 -0
  5. datadoc_cli-0.6.0/README.md +238 -0
  6. datadoc_cli-0.6.0/RELEASE_CHECKLIST.md +74 -0
  7. datadoc_cli-0.6.0/datadoc/__init__.py +5 -0
  8. datadoc_cli-0.6.0/datadoc/cli/app.py +1367 -0
  9. datadoc_cli-0.6.0/datadoc/cli/ui_server.py +300 -0
  10. datadoc_cli-0.6.0/datadoc/core/__init__.py +25 -0
  11. datadoc_cli-0.6.0/datadoc/core/compare.py +428 -0
  12. datadoc_cli-0.6.0/datadoc/core/pipeline.py +1244 -0
  13. datadoc_cli-0.6.0/datadoc/core/report.py +668 -0
  14. {datadoc_cli-0.3.0 → datadoc_cli-0.6.0}/datadoc/plugins/base.py +1 -23
  15. {datadoc_cli-0.3.0 → datadoc_cli-0.6.0}/datadoc/plugins/datetime_feat.py +16 -37
  16. datadoc_cli-0.6.0/datadoc/plugins/duplicates.py +51 -0
  17. {datadoc_cli-0.3.0 → datadoc_cli-0.6.0}/datadoc/plugins/encoders.py +13 -25
  18. {datadoc_cli-0.3.0 → datadoc_cli-0.6.0}/datadoc/plugins/missing_values.py +4 -16
  19. {datadoc_cli-0.3.0 → datadoc_cli-0.6.0}/datadoc/plugins/outliers.py +9 -20
  20. datadoc_cli-0.6.0/datadoc/plugins/polynomial.py +111 -0
  21. datadoc_cli-0.6.0/datadoc/plugins/rare.py +95 -0
  22. datadoc_cli-0.6.0/datadoc/plugins/registry.py +97 -0
  23. {datadoc_cli-0.3.0 → datadoc_cli-0.6.0}/datadoc/plugins/scaling.py +16 -24
  24. datadoc_cli-0.6.0/datadoc/plugins/target_encoder.py +156 -0
  25. datadoc_cli-0.6.0/pyproject.toml +125 -0
  26. datadoc_cli-0.6.0/web/README.md +23 -0
  27. datadoc_cli-0.6.0/web/dist/assets/index-BPDnt264.js +17 -0
  28. datadoc_cli-0.6.0/web/dist/assets/index-C_j7uPW0.css +2 -0
  29. datadoc_cli-0.6.0/web/dist/favicon.svg +1 -0
  30. datadoc_cli-0.6.0/web/dist/icons.svg +24 -0
  31. datadoc_cli-0.6.0/web/dist/index.html +17 -0
  32. datadoc_cli-0.6.0/web/node_modules/@babel/runtime/LICENSE +22 -0
  33. datadoc_cli-0.6.0/web/node_modules/@babel/runtime/README.md +19 -0
  34. datadoc_cli-0.6.0/web/node_modules/@emnapi/core/LICENSE +21 -0
  35. datadoc_cli-0.6.0/web/node_modules/@emnapi/core/README.md +1 -0
  36. datadoc_cli-0.6.0/web/node_modules/@emnapi/runtime/LICENSE +21 -0
  37. datadoc_cli-0.6.0/web/node_modules/@emnapi/runtime/README.md +1 -0
  38. datadoc_cli-0.6.0/web/node_modules/@emnapi/wasi-threads/LICENSE +21 -0
  39. datadoc_cli-0.6.0/web/node_modules/@emnapi/wasi-threads/README.md +203 -0
  40. datadoc_cli-0.6.0/web/node_modules/@hello-pangea/dnd/LICENSE +15 -0
  41. datadoc_cli-0.6.0/web/node_modules/@hello-pangea/dnd/README.md +172 -0
  42. datadoc_cli-0.6.0/web/node_modules/@jridgewell/gen-mapping/LICENSE +19 -0
  43. datadoc_cli-0.6.0/web/node_modules/@jridgewell/gen-mapping/README.md +227 -0
  44. datadoc_cli-0.6.0/web/node_modules/@jridgewell/remapping/LICENSE +19 -0
  45. datadoc_cli-0.6.0/web/node_modules/@jridgewell/remapping/README.md +218 -0
  46. datadoc_cli-0.6.0/web/node_modules/@jridgewell/resolve-uri/LICENSE +19 -0
  47. datadoc_cli-0.6.0/web/node_modules/@jridgewell/resolve-uri/README.md +40 -0
  48. datadoc_cli-0.6.0/web/node_modules/@jridgewell/sourcemap-codec/LICENSE +19 -0
  49. datadoc_cli-0.6.0/web/node_modules/@jridgewell/sourcemap-codec/README.md +264 -0
  50. datadoc_cli-0.6.0/web/node_modules/@jridgewell/trace-mapping/LICENSE +19 -0
  51. datadoc_cli-0.6.0/web/node_modules/@jridgewell/trace-mapping/README.md +348 -0
  52. datadoc_cli-0.6.0/web/node_modules/@oxc-project/types/LICENSE +22 -0
  53. datadoc_cli-0.6.0/web/node_modules/@oxc-project/types/README.md +3 -0
  54. datadoc_cli-0.6.0/web/node_modules/@oxlint/binding-win32-x64-msvc/README.md +3 -0
  55. datadoc_cli-0.6.0/web/node_modules/@reduxjs/toolkit/LICENSE +21 -0
  56. datadoc_cli-0.6.0/web/node_modules/@reduxjs/toolkit/README.md +112 -0
  57. datadoc_cli-0.6.0/web/node_modules/@rolldown/binding-win32-x64-msvc/README.md +3 -0
  58. datadoc_cli-0.6.0/web/node_modules/@rolldown/pluginutils/LICENSE +21 -0
  59. datadoc_cli-0.6.0/web/node_modules/@rolldown/pluginutils/README.md +145 -0
  60. datadoc_cli-0.6.0/web/node_modules/@standard-schema/spec/LICENSE +21 -0
  61. datadoc_cli-0.6.0/web/node_modules/@standard-schema/spec/README.md +198 -0
  62. datadoc_cli-0.6.0/web/node_modules/@standard-schema/utils/LICENSE +21 -0
  63. datadoc_cli-0.6.0/web/node_modules/@standard-schema/utils/README.md +61 -0
  64. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/node/LICENSE +21 -0
  65. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/node/README.md +36 -0
  66. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/node/node_modules/lightningcss/LICENSE +373 -0
  67. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/node/node_modules/lightningcss/README.md +105 -0
  68. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/node/node_modules/lightningcss-win32-x64-msvc/LICENSE +373 -0
  69. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/node/node_modules/lightningcss-win32-x64-msvc/README.md +1 -0
  70. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/oxide/LICENSE +21 -0
  71. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/oxide-win32-x64-msvc/LICENSE +21 -0
  72. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/oxide-win32-x64-msvc/README.md +3 -0
  73. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/vite/LICENSE +21 -0
  74. datadoc_cli-0.6.0/web/node_modules/@tailwindcss/vite/README.md +76 -0
  75. datadoc_cli-0.6.0/web/node_modules/@tybys/wasm-util/README.md +193 -0
  76. datadoc_cli-0.6.0/web/node_modules/@types/d3-array/LICENSE +21 -0
  77. datadoc_cli-0.6.0/web/node_modules/@types/d3-array/README.md +15 -0
  78. datadoc_cli-0.6.0/web/node_modules/@types/d3-color/LICENSE +21 -0
  79. datadoc_cli-0.6.0/web/node_modules/@types/d3-color/README.md +15 -0
  80. datadoc_cli-0.6.0/web/node_modules/@types/d3-ease/LICENSE +21 -0
  81. datadoc_cli-0.6.0/web/node_modules/@types/d3-ease/README.md +15 -0
  82. datadoc_cli-0.6.0/web/node_modules/@types/d3-interpolate/LICENSE +21 -0
  83. datadoc_cli-0.6.0/web/node_modules/@types/d3-interpolate/README.md +15 -0
  84. datadoc_cli-0.6.0/web/node_modules/@types/d3-path/LICENSE +21 -0
  85. datadoc_cli-0.6.0/web/node_modules/@types/d3-path/README.md +15 -0
  86. datadoc_cli-0.6.0/web/node_modules/@types/d3-scale/LICENSE +21 -0
  87. datadoc_cli-0.6.0/web/node_modules/@types/d3-scale/README.md +15 -0
  88. datadoc_cli-0.6.0/web/node_modules/@types/d3-shape/LICENSE +21 -0
  89. datadoc_cli-0.6.0/web/node_modules/@types/d3-shape/README.md +15 -0
  90. datadoc_cli-0.6.0/web/node_modules/@types/d3-time/LICENSE +21 -0
  91. datadoc_cli-0.6.0/web/node_modules/@types/d3-time/README.md +15 -0
  92. datadoc_cli-0.6.0/web/node_modules/@types/d3-timer/LICENSE +21 -0
  93. datadoc_cli-0.6.0/web/node_modules/@types/d3-timer/README.md +15 -0
  94. datadoc_cli-0.6.0/web/node_modules/@types/react/LICENSE +21 -0
  95. datadoc_cli-0.6.0/web/node_modules/@types/react/README.md +15 -0
  96. datadoc_cli-0.6.0/web/node_modules/@types/react-dom/LICENSE +21 -0
  97. datadoc_cli-0.6.0/web/node_modules/@types/react-dom/README.md +16 -0
  98. datadoc_cli-0.6.0/web/node_modules/@types/use-sync-external-store/LICENSE +21 -0
  99. datadoc_cli-0.6.0/web/node_modules/@types/use-sync-external-store/README.md +15 -0
  100. datadoc_cli-0.6.0/web/node_modules/@vitejs/plugin-react/LICENSE +21 -0
  101. datadoc_cli-0.6.0/web/node_modules/@vitejs/plugin-react/README.md +169 -0
  102. datadoc_cli-0.6.0/web/node_modules/agent-base/README.md +145 -0
  103. datadoc_cli-0.6.0/web/node_modules/asynckit/LICENSE +21 -0
  104. datadoc_cli-0.6.0/web/node_modules/asynckit/README.md +233 -0
  105. datadoc_cli-0.6.0/web/node_modules/axios/CHANGELOG.md +1872 -0
  106. datadoc_cli-0.6.0/web/node_modules/axios/LICENSE +7 -0
  107. datadoc_cli-0.6.0/web/node_modules/axios/README.md +2559 -0
  108. datadoc_cli-0.6.0/web/node_modules/axios/lib/adapters/README.md +36 -0
  109. datadoc_cli-0.6.0/web/node_modules/axios/lib/core/README.md +8 -0
  110. datadoc_cli-0.6.0/web/node_modules/axios/lib/helpers/README.md +7 -0
  111. datadoc_cli-0.6.0/web/node_modules/call-bind-apply-helpers/CHANGELOG.md +30 -0
  112. datadoc_cli-0.6.0/web/node_modules/call-bind-apply-helpers/LICENSE +21 -0
  113. datadoc_cli-0.6.0/web/node_modules/call-bind-apply-helpers/README.md +62 -0
  114. datadoc_cli-0.6.0/web/node_modules/css-box-model/LICENSE +21 -0
  115. datadoc_cli-0.6.0/web/node_modules/css-box-model/README.md +290 -0
  116. datadoc_cli-0.6.0/web/node_modules/csstype/LICENSE +19 -0
  117. datadoc_cli-0.6.0/web/node_modules/csstype/README.md +291 -0
  118. datadoc_cli-0.6.0/web/node_modules/d3-array/LICENSE +13 -0
  119. datadoc_cli-0.6.0/web/node_modules/d3-array/README.md +954 -0
  120. datadoc_cli-0.6.0/web/node_modules/d3-color/LICENSE +13 -0
  121. datadoc_cli-0.6.0/web/node_modules/d3-color/README.md +203 -0
  122. datadoc_cli-0.6.0/web/node_modules/d3-ease/LICENSE +28 -0
  123. datadoc_cli-0.6.0/web/node_modules/d3-ease/README.md +253 -0
  124. datadoc_cli-0.6.0/web/node_modules/d3-format/LICENSE +13 -0
  125. datadoc_cli-0.6.0/web/node_modules/d3-format/README.md +12 -0
  126. datadoc_cli-0.6.0/web/node_modules/d3-interpolate/LICENSE +13 -0
  127. datadoc_cli-0.6.0/web/node_modules/d3-interpolate/README.md +268 -0
  128. datadoc_cli-0.6.0/web/node_modules/d3-path/LICENSE +13 -0
  129. datadoc_cli-0.6.0/web/node_modules/d3-path/README.md +94 -0
  130. datadoc_cli-0.6.0/web/node_modules/d3-scale/LICENSE +13 -0
  131. datadoc_cli-0.6.0/web/node_modules/d3-scale/README.md +1003 -0
  132. datadoc_cli-0.6.0/web/node_modules/d3-shape/LICENSE +13 -0
  133. datadoc_cli-0.6.0/web/node_modules/d3-shape/README.md +1227 -0
  134. datadoc_cli-0.6.0/web/node_modules/d3-time/LICENSE +13 -0
  135. datadoc_cli-0.6.0/web/node_modules/d3-time/README.md +380 -0
  136. datadoc_cli-0.6.0/web/node_modules/d3-time-format/LICENSE +13 -0
  137. datadoc_cli-0.6.0/web/node_modules/d3-time-format/README.md +209 -0
  138. datadoc_cli-0.6.0/web/node_modules/d3-timer/LICENSE +13 -0
  139. datadoc_cli-0.6.0/web/node_modules/d3-timer/README.md +87 -0
  140. datadoc_cli-0.6.0/web/node_modules/debug/LICENSE +20 -0
  141. datadoc_cli-0.6.0/web/node_modules/debug/README.md +481 -0
  142. datadoc_cli-0.6.0/web/node_modules/decimal.js-light/CHANGELOG.md +87 -0
  143. datadoc_cli-0.6.0/web/node_modules/decimal.js-light/README.md +92 -0
  144. datadoc_cli-0.6.0/web/node_modules/detect-libc/LICENSE +201 -0
  145. datadoc_cli-0.6.0/web/node_modules/detect-libc/README.md +163 -0
  146. datadoc_cli-0.6.0/web/node_modules/dunder-proto/CHANGELOG.md +24 -0
  147. datadoc_cli-0.6.0/web/node_modules/dunder-proto/LICENSE +21 -0
  148. datadoc_cli-0.6.0/web/node_modules/dunder-proto/README.md +54 -0
  149. datadoc_cli-0.6.0/web/node_modules/enhanced-resolve/LICENSE +20 -0
  150. datadoc_cli-0.6.0/web/node_modules/enhanced-resolve/README.md +862 -0
  151. datadoc_cli-0.6.0/web/node_modules/es-define-property/CHANGELOG.md +29 -0
  152. datadoc_cli-0.6.0/web/node_modules/es-define-property/LICENSE +21 -0
  153. datadoc_cli-0.6.0/web/node_modules/es-define-property/README.md +49 -0
  154. datadoc_cli-0.6.0/web/node_modules/es-errors/CHANGELOG.md +40 -0
  155. datadoc_cli-0.6.0/web/node_modules/es-errors/LICENSE +21 -0
  156. datadoc_cli-0.6.0/web/node_modules/es-errors/README.md +55 -0
  157. datadoc_cli-0.6.0/web/node_modules/es-object-atoms/CHANGELOG.md +44 -0
  158. datadoc_cli-0.6.0/web/node_modules/es-object-atoms/LICENSE +21 -0
  159. datadoc_cli-0.6.0/web/node_modules/es-object-atoms/README.md +63 -0
  160. datadoc_cli-0.6.0/web/node_modules/es-set-tostringtag/CHANGELOG.md +67 -0
  161. datadoc_cli-0.6.0/web/node_modules/es-set-tostringtag/LICENSE +21 -0
  162. datadoc_cli-0.6.0/web/node_modules/es-set-tostringtag/README.md +53 -0
  163. datadoc_cli-0.6.0/web/node_modules/es-toolkit/CHANGELOG.md +936 -0
  164. datadoc_cli-0.6.0/web/node_modules/es-toolkit/LICENSE +21 -0
  165. datadoc_cli-0.6.0/web/node_modules/es-toolkit/README.md +69 -0
  166. datadoc_cli-0.6.0/web/node_modules/eventemitter3/LICENSE +21 -0
  167. datadoc_cli-0.6.0/web/node_modules/eventemitter3/README.md +85 -0
  168. datadoc_cli-0.6.0/web/node_modules/fdir/LICENSE +7 -0
  169. datadoc_cli-0.6.0/web/node_modules/fdir/README.md +91 -0
  170. datadoc_cli-0.6.0/web/node_modules/follow-redirects/LICENSE +18 -0
  171. datadoc_cli-0.6.0/web/node_modules/follow-redirects/README.md +157 -0
  172. datadoc_cli-0.6.0/web/node_modules/form-data/CHANGELOG.md +686 -0
  173. datadoc_cli-0.6.0/web/node_modules/form-data/README.md +355 -0
  174. datadoc_cli-0.6.0/web/node_modules/function-bind/CHANGELOG.md +136 -0
  175. datadoc_cli-0.6.0/web/node_modules/function-bind/LICENSE +20 -0
  176. datadoc_cli-0.6.0/web/node_modules/function-bind/README.md +46 -0
  177. datadoc_cli-0.6.0/web/node_modules/get-intrinsic/CHANGELOG.md +186 -0
  178. datadoc_cli-0.6.0/web/node_modules/get-intrinsic/LICENSE +21 -0
  179. datadoc_cli-0.6.0/web/node_modules/get-intrinsic/README.md +71 -0
  180. datadoc_cli-0.6.0/web/node_modules/get-proto/CHANGELOG.md +21 -0
  181. datadoc_cli-0.6.0/web/node_modules/get-proto/LICENSE +21 -0
  182. datadoc_cli-0.6.0/web/node_modules/get-proto/README.md +50 -0
  183. datadoc_cli-0.6.0/web/node_modules/gopd/CHANGELOG.md +45 -0
  184. datadoc_cli-0.6.0/web/node_modules/gopd/LICENSE +21 -0
  185. datadoc_cli-0.6.0/web/node_modules/gopd/README.md +40 -0
  186. datadoc_cli-0.6.0/web/node_modules/graceful-fs/LICENSE +15 -0
  187. datadoc_cli-0.6.0/web/node_modules/graceful-fs/README.md +143 -0
  188. datadoc_cli-0.6.0/web/node_modules/has-symbols/CHANGELOG.md +91 -0
  189. datadoc_cli-0.6.0/web/node_modules/has-symbols/LICENSE +21 -0
  190. datadoc_cli-0.6.0/web/node_modules/has-symbols/README.md +46 -0
  191. datadoc_cli-0.6.0/web/node_modules/has-tostringtag/CHANGELOG.md +42 -0
  192. datadoc_cli-0.6.0/web/node_modules/has-tostringtag/LICENSE +21 -0
  193. datadoc_cli-0.6.0/web/node_modules/has-tostringtag/README.md +46 -0
  194. datadoc_cli-0.6.0/web/node_modules/hasown/CHANGELOG.md +58 -0
  195. datadoc_cli-0.6.0/web/node_modules/hasown/LICENSE +21 -0
  196. datadoc_cli-0.6.0/web/node_modules/hasown/README.md +40 -0
  197. datadoc_cli-0.6.0/web/node_modules/https-proxy-agent/README.md +137 -0
  198. datadoc_cli-0.6.0/web/node_modules/immer/LICENSE +21 -0
  199. datadoc_cli-0.6.0/web/node_modules/internmap/LICENSE +13 -0
  200. datadoc_cli-0.6.0/web/node_modules/internmap/README.md +94 -0
  201. datadoc_cli-0.6.0/web/node_modules/jiti/LICENSE +21 -0
  202. datadoc_cli-0.6.0/web/node_modules/jiti/README.md +258 -0
  203. datadoc_cli-0.6.0/web/node_modules/lightningcss/LICENSE +373 -0
  204. datadoc_cli-0.6.0/web/node_modules/lightningcss/README.md +105 -0
  205. datadoc_cli-0.6.0/web/node_modules/lightningcss-win32-x64-msvc/LICENSE +373 -0
  206. datadoc_cli-0.6.0/web/node_modules/lightningcss-win32-x64-msvc/README.md +1 -0
  207. datadoc_cli-0.6.0/web/node_modules/lucide-react/LICENSE +43 -0
  208. datadoc_cli-0.6.0/web/node_modules/lucide-react/README.md +87 -0
  209. datadoc_cli-0.6.0/web/node_modules/magic-string/LICENSE +7 -0
  210. datadoc_cli-0.6.0/web/node_modules/magic-string/README.md +325 -0
  211. datadoc_cli-0.6.0/web/node_modules/math-intrinsics/CHANGELOG.md +24 -0
  212. datadoc_cli-0.6.0/web/node_modules/math-intrinsics/LICENSE +21 -0
  213. datadoc_cli-0.6.0/web/node_modules/math-intrinsics/README.md +50 -0
  214. datadoc_cli-0.6.0/web/node_modules/mime-db/LICENSE +23 -0
  215. datadoc_cli-0.6.0/web/node_modules/mime-db/README.md +100 -0
  216. datadoc_cli-0.6.0/web/node_modules/mime-types/LICENSE +23 -0
  217. datadoc_cli-0.6.0/web/node_modules/mime-types/README.md +113 -0
  218. datadoc_cli-0.6.0/web/node_modules/nanoid/LICENSE +20 -0
  219. datadoc_cli-0.6.0/web/node_modules/nanoid/README.md +36 -0
  220. datadoc_cli-0.6.0/web/node_modules/oxlint/LICENSE +22 -0
  221. datadoc_cli-0.6.0/web/node_modules/oxlint/README.md +58 -0
  222. datadoc_cli-0.6.0/web/node_modules/picocolors/LICENSE +15 -0
  223. datadoc_cli-0.6.0/web/node_modules/picocolors/README.md +21 -0
  224. datadoc_cli-0.6.0/web/node_modules/picomatch/LICENSE +21 -0
  225. datadoc_cli-0.6.0/web/node_modules/picomatch/README.md +743 -0
  226. datadoc_cli-0.6.0/web/node_modules/postcss/LICENSE +20 -0
  227. datadoc_cli-0.6.0/web/node_modules/postcss/README.md +24 -0
  228. datadoc_cli-0.6.0/web/node_modules/proxy-from-env/LICENSE +20 -0
  229. datadoc_cli-0.6.0/web/node_modules/proxy-from-env/README.md +163 -0
  230. datadoc_cli-0.6.0/web/node_modules/raf-schd/CHANGELOG.md +4 -0
  231. datadoc_cli-0.6.0/web/node_modules/raf-schd/LICENSE +21 -0
  232. datadoc_cli-0.6.0/web/node_modules/raf-schd/README.md +154 -0
  233. datadoc_cli-0.6.0/web/node_modules/react/LICENSE +21 -0
  234. datadoc_cli-0.6.0/web/node_modules/react/README.md +37 -0
  235. datadoc_cli-0.6.0/web/node_modules/react-dom/LICENSE +21 -0
  236. datadoc_cli-0.6.0/web/node_modules/react-dom/README.md +60 -0
  237. datadoc_cli-0.6.0/web/node_modules/react-is/LICENSE +21 -0
  238. datadoc_cli-0.6.0/web/node_modules/react-is/README.md +103 -0
  239. datadoc_cli-0.6.0/web/node_modules/react-redux/README.md +63 -0
  240. datadoc_cli-0.6.0/web/node_modules/recharts/CHANGELOG.md +1965 -0
  241. datadoc_cli-0.6.0/web/node_modules/recharts/LICENSE +22 -0
  242. datadoc_cli-0.6.0/web/node_modules/recharts/README.md +98 -0
  243. datadoc_cli-0.6.0/web/node_modules/redux/README.md +157 -0
  244. datadoc_cli-0.6.0/web/node_modules/redux-thunk/README.md +403 -0
  245. datadoc_cli-0.6.0/web/node_modules/reselect/LICENSE +21 -0
  246. datadoc_cli-0.6.0/web/node_modules/reselect/README.md +206 -0
  247. datadoc_cli-0.6.0/web/node_modules/rolldown/LICENSE +25 -0
  248. datadoc_cli-0.6.0/web/node_modules/rolldown/README.md +11 -0
  249. datadoc_cli-0.6.0/web/node_modules/scheduler/LICENSE +21 -0
  250. datadoc_cli-0.6.0/web/node_modules/scheduler/README.md +9 -0
  251. datadoc_cli-0.6.0/web/node_modules/source-map-js/LICENSE +28 -0
  252. datadoc_cli-0.6.0/web/node_modules/source-map-js/README.md +765 -0
  253. datadoc_cli-0.6.0/web/node_modules/tailwindcss/LICENSE +21 -0
  254. datadoc_cli-0.6.0/web/node_modules/tailwindcss/README.md +36 -0
  255. datadoc_cli-0.6.0/web/node_modules/tapable/LICENSE +21 -0
  256. datadoc_cli-0.6.0/web/node_modules/tapable/README.md +715 -0
  257. datadoc_cli-0.6.0/web/node_modules/tiny-invariant/LICENSE +21 -0
  258. datadoc_cli-0.6.0/web/node_modules/tiny-invariant/README.md +109 -0
  259. datadoc_cli-0.6.0/web/node_modules/tinyglobby/LICENSE +21 -0
  260. datadoc_cli-0.6.0/web/node_modules/tinyglobby/README.md +25 -0
  261. datadoc_cli-0.6.0/web/node_modules/tslib/README.md +164 -0
  262. datadoc_cli-0.6.0/web/node_modules/use-sync-external-store/LICENSE +21 -0
  263. datadoc_cli-0.6.0/web/node_modules/use-sync-external-store/README.md +5 -0
  264. datadoc_cli-0.6.0/web/node_modules/victory-vendor/CHANGELOG.md +117 -0
  265. datadoc_cli-0.6.0/web/node_modules/victory-vendor/README.md +70 -0
  266. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-array/LICENSE +13 -0
  267. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-color/LICENSE +13 -0
  268. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-ease/LICENSE +28 -0
  269. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-format/LICENSE +13 -0
  270. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-interpolate/LICENSE +13 -0
  271. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-path/LICENSE +13 -0
  272. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-scale/LICENSE +13 -0
  273. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-shape/LICENSE +13 -0
  274. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-time/LICENSE +13 -0
  275. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-time-format/LICENSE +13 -0
  276. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-timer/LICENSE +13 -0
  277. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/d3-voronoi/LICENSE +50 -0
  278. datadoc_cli-0.6.0/web/node_modules/victory-vendor/lib-vendor/internmap/LICENSE +13 -0
  279. datadoc_cli-0.6.0/web/node_modules/vite/README.md +20 -0
  280. datadoc_cli-0.3.0/.github/workflows/ci.yml +0 -52
  281. datadoc_cli-0.3.0/.gitignore +0 -37
  282. datadoc_cli-0.3.0/CONTRIBUTING.md +0 -98
  283. datadoc_cli-0.3.0/FULL_DOCUMENTATION.md +0 -144
  284. datadoc_cli-0.3.0/PKG-INFO +0 -210
  285. datadoc_cli-0.3.0/README.md +0 -176
  286. datadoc_cli-0.3.0/comprehensive_test.csv +0 -11
  287. datadoc_cli-0.3.0/datadoc/cli/app.py +0 -805
  288. datadoc_cli-0.3.0/datadoc/core/__init__.py +0 -0
  289. datadoc_cli-0.3.0/datadoc/core/engine.py +0 -353
  290. datadoc_cli-0.3.0/datadoc/plugins/__init__.py +0 -0
  291. datadoc_cli-0.3.0/diagnose.py +0 -97
  292. datadoc_cli-0.3.0/diagnose_output.txt +0 -130
  293. datadoc_cli-0.3.0/diagnose_output2.txt +0 -76
  294. datadoc_cli-0.3.0/docs/api-docs.html +0 -140
  295. datadoc_cli-0.3.0/docs/contribute.html +0 -128
  296. datadoc_cli-0.3.0/docs/css/style.css +0 -522
  297. datadoc_cli-0.3.0/docs/how-it-works.html +0 -127
  298. datadoc_cli-0.3.0/docs/index.html +0 -259
  299. datadoc_cli-0.3.0/docs/license.html +0 -106
  300. datadoc_cli-0.3.0/docs/sdk.html +0 -137
  301. datadoc_cli-0.3.0/docs/setup.html +0 -160
  302. datadoc_cli-0.3.0/generate_heavy_test.py +0 -113
  303. datadoc_cli-0.3.0/heavy_test.csv +0 -10001
  304. datadoc_cli-0.3.0/heavy_test_output.txt +0 -52
  305. datadoc_cli-0.3.0/pyproject.toml +0 -61
  306. datadoc_cli-0.3.0/test.csv +0 -6
  307. datadoc_cli-0.3.0/test_output.txt +0 -7
  308. datadoc_cli-0.3.0/tests/__init__.py +0 -1
  309. datadoc_cli-0.3.0/tests/test_core.py +0 -390
  310. datadoc_cli-0.3.0/uv.lock +0 -3940
  311. {datadoc_cli-0.3.0 → datadoc_cli-0.6.0}/LICENSE +0 -0
  312. {datadoc_cli-0.3.0/datadoc → datadoc_cli-0.6.0/datadoc/cli}/__init__.py +0 -0
  313. {datadoc_cli-0.3.0/datadoc/cli → datadoc_cli-0.6.0/datadoc/plugins}/__init__.py +0 -0
@@ -0,0 +1,65 @@
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+ *.egg-info/
6
+ dist/
7
+ build/
8
+ *.egg
9
+
10
+ # Virtual environments
11
+ .venv/
12
+ venv/
13
+ env/
14
+
15
+ # IDE
16
+ .vscode/
17
+ .idea/
18
+ *.swp
19
+ *.swo
20
+
21
+ # OS
22
+ .DS_Store
23
+ Thumbs.db
24
+
25
+ # DATADOC generated outputs
26
+ clean_*.csv
27
+ pipeline_*.py
28
+ report_*.md
29
+ .release-verify/
30
+ datadoc-run/
31
+ full-run/
32
+ titanic-run/
33
+ telco-run/
34
+ datadoc.toml
35
+
36
+ # Local audit datasets (re-download from Kaggle; never ship)
37
+ # NOTE: demo.csv and docs/demo.csv are TRACKED fixtures and stay tracked.
38
+ /california_housing.csv
39
+ /telco_churn.csv
40
+ /titanic.csv
41
+ /titanic-audit/*.csv
42
+ /titanic-audit/*.joblib
43
+ /titanic-audit/*.parquet
44
+ /titanic-audit/*.txt
45
+ /titanic-audit/full-run/
46
+ /titanic-audit/wizard-run/
47
+ /titanic-test/
48
+
49
+ # Scratch / debug (dev-only, never ship)
50
+ scratch_*
51
+ diagnose*
52
+ changes.diff
53
+ *_test.csv
54
+ *_output*.txt
55
+ heavy_test.csv
56
+
57
+ # Testing
58
+ .pytest_cache/
59
+ .playwright-cli/
60
+ htmlcov/
61
+ .coverage
62
+
63
+ # Environment & Secrets
64
+ .env
65
+ .env.*
@@ -0,0 +1,80 @@
1
+ # Changelog
2
+
3
+ All notable changes to DATADOC are documented here. Format follows Keep a Changelog and SemVer.
4
+
5
+ ## [0.6.0] - 2026-09-13
6
+ ### Added
7
+ - `datadoc report`: Automated, standalone, high-fidelity HTML report generation (`datadoc report <dataset> [--target] [--output] [--title] [--preset]`). 100% self-contained single-file HTML with embedded responsive CSS, health grade scoring (A+ to F), column role classification, distribution statistics, and transformation plan.
8
+ - `datadoc compare`: Visual side-by-side dataset comparison command (`datadoc compare <raw> <transformed> [--target] [--html] [--json]`). Computes dimension deltas, missing cell reduction (100% resolution tracking), column lineage/lifecycle (retained, dropped, engineered), and numeric distribution shifts (imputation and scaling effects) with color-coded terminal tables and interactive HTML export.
9
+ - `TargetEncoderPlugin` (Priority 41): Empirical Bayes smoothed target encoding for high-cardinality categoricals: `y_hat = (n * cat_mean + m * global_mean) / (n + m)` to prevent target leakage and dimension explosion.
10
+ - `PolynomialFeaturesPlugin` (Priority 44): Degree-2 interactions (`x1 * x2`) and squared features (`x^2`) for numeric columns to capture non-linear signal for linear models.
11
+ - Dashboard enhancements: Added "View HTML Report" direct action in the header, navigation sidebar, and artifact actions panel.
12
+ - UI Server API: Added `GET /api/dataset/report` (standalone HTML audit report) and `GET /api/dataset/compare` (side-by-side comparison metrics).
13
+ - Standalone execution: Added `if __name__ == "__main__": app()` to `datadoc.cli.app`.
14
+
15
+ ## [0.5.0] - 2026-09-14
16
+ ### Added
17
+ - `datadoc.toml` / `pyproject.toml [tool.datadoc]` config file support with flag overrides.
18
+ - `datadoc wizard` guided TUI for `target`, `preset`, `drop_identifiers`, `deduplicate`, `scaling`, `clip_outliers`, `rare_frequency`; writes `datadoc.toml` + run dir.
19
+ - Presets `--preset quick|balanced|linear|tree|time|robust` mapping to `scaling`, `clip_outliers`, `estimator_family`.
20
+ - Short aliases `-t` / `-o` / `-p` / `-f` for `--target` / `--output` / `--pipeline` / `--format`.
21
+ - Shell completion via Typer (`add_completion=True`) — `datadoc --install-completion`.
22
+ - `datadoc diff` for `profile`/`plan`/`pipeline` JSON comparison (incl. ops added/removed).
23
+ - `datadoc lint` leakage lint (target duplication, null target, infinities, duplicates, profile findings).
24
+ - `datadoc plugins list|show` registry introspection, `datadoc.plugins` entry-points auto-discovery.
25
+ - New deterministic plugins: `DuplicateRemoverPlugin` (P05), `RareCategoryPlugin` (P42); `CategoricalEncoderPlugin` threshold reconciled 10 → 20.
26
+ - Pipeline artifact v2 with `provenance` (schema/columns hash, rows, dedup count, version); v1 artifacts still load.
27
+ - `plan` now surfaces `deduplicate`, `outlier_clipping`, `scaling`, rare-grouping details; new `explain_plan()`.
28
+ - `fit`/`run` flags: `--deduplicate`, `--rare-frequency`, `--cyclical`, `--no-hour`.
29
+ - Repeatable `--identifier-column` / `--ignore-column` on `plan`/`fit`/`run` (also via `datadoc.toml` lists), reaching `PipelineConfig.identifier_columns` / `ignored_columns` which previously had no CLI path.
30
+ - `transform --validate` via `drift_report()` (schema + median-shift drift).
31
+ - `evaluate --ablation` via `evaluate_ablation()` (full / no_clip / no_scaling / minimal).
32
+ - Notebook widgets: `profile_to_html()` + `_repr_html_()` for Jupyter/marimo.
33
+ - `export --format python|joblib` via `export_sklearn_artifact()` (joblib needs `ml` extra).
34
+ - Dashboard: Ctrl+K/Cmd+K palette, lineage panel (`/api/pipeline/lineage`), drift endpoint, new config knobs.
35
+ - PyPI-ready wheel: `web/dist` force-included as `datadoc/_webui`, so `datadoc ui` serves the React app from a plain pip install (verified in a clean venv); `ui_server` falls back to the checkout path in dev.
36
+ - `tomli>=2.0.0` dependency on Python < 3.11 for real TOML parsing.
37
+ - New `RELEASE_CHECKLIST.md` (referenced by `docs/production.html`); `uv.lock` re-synced; `datadoc.__version__` added as the code-level version source.
38
+ ### Changed
39
+ - `scikit-learn` moved back to optional `ml` extra for lighter core install.
40
+ - `PipelineConfig` docs clarified `resolved_scaling` and `plan` exposure.
41
+ - `run` now parities all `fit` flags.
42
+ ### Fixed
43
+ - Removed `product_specification.md` (Retrod PMS orphan).
44
+ - Fixed `CONTRIBUTING.md` pandas/engine refs.
45
+ - `plan` no longer lets `datadoc.toml` silently override explicit `--drop-identifiers`/`--task` flags.
46
+ - `wizard` crashed when delegating to `run` (unpassed params hit raw `typer.Option` defaults); all params are now explicit.
47
+ - `diff`/`plan --diff` labels corrected to "Only in A/B" (were backwards "added/removed").
48
+ - `estimator_family="both"` now truly fits linear + tree and keeps the stronger validation score (was silently tree-only); `resolved_scaling` maps `both` → `standard`.
49
+ - Dashboard export template now emits an executable script (`__main__` + argparse) and restores `train_provenance_`.
50
+ - Commented `datadoc.toml` (including the file `datadoc init` itself writes) crashed config loading on Python 3.10 without `tomli` — the fallback parser now strips inline comments and parses lists/quoted strings.
51
+ - Fixed installed-dashboard path resolution (`datadoc/cli/_webui` → `datadoc/_webui`).
52
+ - `strict_schema` now exposed via CLI/config.
53
+ - CI re-added.
54
+
55
+ ## [0.4.0] - 2026-08-09
56
+ ### Added
57
+ - Leakage-safe `DataDocPipeline` (`fit`/`transform`/`save`/`load`/`evaluate`/`export_python`) in `datadoc/core/pipeline.py`.
58
+ - `PipelineConfig` with 17 knobs (`target`, `task`, `drop_identifiers`, `scaling`, `clip_outliers`, ...).
59
+ - FastAPI dashboard `datadoc ui` + `web/` Vite+React.
60
+ - Optional extras `ai` (litellm), `ml`, `ui`.
61
+ ### Changed
62
+ - CLI narrowed to 7 commands `profile,plan,fit,transform,evaluate,export,run`; legacy `engine.py` removed.
63
+ - Polars as primary backend.
64
+
65
+ ## [0.3.0] - 2026-07-28
66
+ ### Added
67
+ - Agentic `agent.py` + `chat` command.
68
+ - FastAPI `ui` via `fastapi/uvicorn/multipart`.
69
+
70
+ ## [0.2.0] - 2026-07-26
71
+ ### Added
72
+ - Phase 2 AI planner `litellm`, `pydantic`, `dotenv`.
73
+ - `chat` + `engineer --ai`.
74
+
75
+ ## [0.1.0] - 2026-07-26
76
+ - Initial scaffold `typer/rich/pandas` + `MissingValuePlugin`.
77
+
78
+ ## Roadmap
79
+ - Export targets `dbt` / Airflow (see README).
80
+ - Text embeddings optional plugin.
@@ -0,0 +1,16 @@
1
+ # Migrating to DATADOC's fitted pipeline API
2
+
3
+ Version 0.3 keeps the original `DATADOC` facade for compatibility, but new projects should use `DataDocPipeline`.
4
+
5
+ ```python
6
+ from datadoc import DataDocPipeline, PipelineConfig
7
+
8
+ pipeline = DataDocPipeline(PipelineConfig(target="label"))
9
+ pipeline.fit(train_df)
10
+ validation_features = pipeline.transform(validation_df)
11
+ pipeline.save("pipeline.json")
12
+ ```
13
+
14
+ `DATADOC.engineer()` fits and transforms the same file and is therefore only suitable for exploratory cleaning. It emits a deprecation warning for supervised work. Use `fit()` on a training split and `transform()` on validation, test, and inference data instead.
15
+
16
+ The legacy AI code-execution path has been removed. AI planning, when installed with `datadoc-cli[ai]`, is limited to registered transformations and never executes provider-generated Python.
@@ -0,0 +1,286 @@
1
+ Metadata-Version: 2.3
2
+ Name: datadoc-cli
3
+ Version: 0.6.0
4
+ Summary: Leakage-safe, local-first preparation pipelines for tabular machine learning.
5
+ Project-URL: Homepage, https://github.com/narain-karti/DATADOC
6
+ Project-URL: Repository, https://github.com/narain-karti/DATADOC
7
+ Project-URL: Issues, https://github.com/narain-karti/DATADOC/issues
8
+ Author: narain-karti
9
+ License: MIT
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: Intended Audience :: Science/Research
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.10
16
+ Classifier: Programming Language :: Python :: 3.11
17
+ Classifier: Programming Language :: Python :: 3.12
18
+ Classifier: Topic :: Scientific/Engineering
19
+ Requires-Python: >=3.10
20
+ Requires-Dist: plotext>=5.2.8
21
+ Requires-Dist: polars>=0.20.0
22
+ Requires-Dist: pydantic>=2.0.0
23
+ Requires-Dist: python-dotenv>=1.0.0
24
+ Requires-Dist: rich>=13.7.0
25
+ Requires-Dist: tomli>=2.0.0; python_version < '3.11'
26
+ Requires-Dist: typer>=0.12.0
27
+ Provides-Extra: ai
28
+ Requires-Dist: litellm>=1.40.0; extra == 'ai'
29
+ Provides-Extra: all
30
+ Requires-Dist: fastapi>=0.115.0; extra == 'all'
31
+ Requires-Dist: litellm>=1.40.0; extra == 'all'
32
+ Requires-Dist: python-multipart>=0.0.20; extra == 'all'
33
+ Requires-Dist: scikit-learn>=1.7.2; extra == 'all'
34
+ Requires-Dist: uvicorn>=0.30.0; extra == 'all'
35
+ Provides-Extra: dev
36
+ Requires-Dist: build>=1.2.0; extra == 'dev'
37
+ Requires-Dist: hatchling>=1.25.0; extra == 'dev'
38
+ Requires-Dist: pytest>=8.0.0; extra == 'dev'
39
+ Requires-Dist: ruff>=0.4.0; extra == 'dev'
40
+ Requires-Dist: scikit-learn>=1.7.2; extra == 'dev'
41
+ Provides-Extra: ml
42
+ Requires-Dist: scikit-learn>=1.7.2; extra == 'ml'
43
+ Provides-Extra: ui
44
+ Requires-Dist: fastapi>=0.115.0; extra == 'ui'
45
+ Requires-Dist: python-multipart>=0.0.20; extra == 'ui'
46
+ Requires-Dist: uvicorn>=0.30.0; extra == 'ui'
47
+ Description-Content-Type: text/markdown
48
+
49
+ <p align="center">
50
+ <pre align="center">
51
+ ____ _ _____ _ ____ ___ ____
52
+ | _ \ / \|_ _|/ \ | _ \ / _ \ / ___|
53
+ | | | |/ _ \ | | / _ \ | | | | | | | |
54
+ | |_| / ___ \| |/ ___ \| |_| | |_| | |___
55
+ |____/_/ \_\_/_/ \_\____/ \___/ \____|
56
+ </pre>
57
+ </p>
58
+
59
+ <h3 align="center">The Open Source Operating System for Dataset Engineering.</h3>
60
+
61
+ <p align="center">
62
+ <a href="https://narain-karti.github.io/DATADOC/"><b>&#x1F4D6; View Official Documentation Website</b></a>
63
+ </p>
64
+
65
+ <p align="center">
66
+ <a href="https://pypi.org/project/datadoc-cli/"><img alt="PyPI version" src="https://img.shields.io/pypi/v/datadoc-cli.svg"></a>
67
+ <a href="https://pypi.org/project/datadoc-cli/"><img alt="Python Versions" src="https://img.shields.io/pypi/pyversions/datadoc-cli.svg"></a>
68
+ <a href="https://github.com/narain-karti/DATADOC/blob/main/LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/License-MIT-yellow.svg"></a>
69
+ </p>
70
+
71
+ <p align="center">
72
+ <a href="#installation"><b>Install</b></a> &bull;
73
+ <a href="#why-datadoc"><b>Why DATADOC?</b></a> &bull;
74
+ <a href="#quick-start"><b>Quick Start</b></a> &bull;
75
+ <a href="#cli-commands"><b>CLI Commands</b></a> &bull;
76
+ <a href="#architecture--plugins"><b>Architecture</b></a>
77
+ </p>
78
+
79
+ <hr>
80
+
81
+ ## 🚀 What is DATADOC?
82
+
83
+ **DATADOC** is a local-first CLI and Python library for preparing tabular data for machine learning. It profiles dataset risks, creates explainable transformation plans, and saves fitted pipelines that apply the same training-derived rules to validation, test, and inference data.
84
+
85
+ Powered by **Polars**, DATADOC reads CSV and Parquet files, diagnoses missing values, identifiers, schema issues, duplicates, constants, and unsafe feature types. It does not promise model improvement: optional evaluation reports the observed result against a baseline under a reproducible split.
86
+
87
+ **DATADOC is NOT just another EDA (Exploratory Data Analysis) tool.** It profiles data quality, lets you review a plan, fits transformations from training data, and hands you a portable artifact and Python wrapper for reuse.
88
+
89
+ ### ⚡ The Impact: Why Industry Professionals Use DATADOC
90
+
91
+ Data Scientists and ML Engineers repeatedly rebuild the same preparation steps across projects.
92
+ DATADOC turns those steps into a reviewable, reusable pipeline.
93
+
94
+ - **Save boilerplate:** Review recommendations for imputing nulls, encoding categories, and optional scaling or clipping before applying them.
95
+ - **Explainable by default:** The deterministic core records roles, findings, operations, protected columns, and fitted statistics in an inspectable artifact.
96
+ - **Local-first:** The core package works offline. Optional ML, UI, and AI features are separate extras.
97
+ - **Optional AI planning:** AI can help explain or rank a constrained plan; it is never allowed to execute arbitrary generated code.
98
+ - **Leakage-safe workflows:** Fitted statistics for imputation, categorical vocabularies, clipping, and scaling are learned from training data and saved as an artifact.
99
+
100
+ ### 📊 Empirical Proof: Does DATADOC Actually Improve Models?
101
+
102
+ Yes. We benchmarked naive manual preparation vs. DATADOC automated preprocessing on Kaggle's Titanic dataset under **5-fold Stratified Cross-Validation**:
103
+
104
+ | Model | Baseline (Naive Prep) | DATADOC Cleaned | Accuracy Δ | Relative Lift |
105
+ |:---|:---:|:---:|:---:|:---:|
106
+ | **Logistic Regression** | 78.90% ± 0.99% | **79.91% ± 1.90%** | **+1.01%** | **+1.28%** |
107
+ | **Random Forest** | 82.15% ± 2.45% | **82.82% ± 2.40%** | **+0.67%** | **+0.82%** |
108
+
109
+ *Why?* DATADOC extracts informative missingness indicators (`Age__missing`, `Cabin__missing`), frequency-encodes high-cardinality features (`Ticket`, `Cabin`), applies standard scaling, and isolates all statistics strictly to training splits to eliminate target leakage.
110
+
111
+ ---
112
+
113
+ ## 📦 Installation
114
+
115
+ DATADOC is published on PyPI. You can install it globally via `pip` or `uv`:
116
+
117
+ ```bash
118
+ pip install datadoc-cli
119
+ ```
120
+
121
+ *(Requires Python 3.10+)*
122
+
123
+ ---
124
+
125
+ ## 🛠️ Quick Start (CLI)
126
+
127
+ You don't need to write a single line of Python to clean your data. Just use the CLI.
128
+ New here? Run the guided wizard — it asks for target + preset and runs everything:
129
+
130
+ ```bash
131
+ datadoc wizard train.csv
132
+ ```
133
+
134
+ Or run the one-shot happy path (profile → plan → fit → transform + manifest):
135
+
136
+ ```bash
137
+ datadoc run train.csv --target churn --preset balanced --evaluate
138
+ ```
139
+
140
+ Full step-by-step (auditable) workflow:
141
+
142
+ ```bash
143
+ # 0. Optional: save repeatable settings (target, preset, scaling, ...)
144
+ datadoc init --preset balanced # writes datadoc.toml
145
+
146
+ # 1. Inspect data-quality findings and column roles
147
+ datadoc profile raw_data.csv --target churn --explain --output profile.json
148
+
149
+ # 2. Review the proposed transformations before applying them
150
+ datadoc plan raw_data.csv --target churn --explain --output plan.json
151
+
152
+ # 3. Fit only on a training dataset, then save a reusable artifact
153
+ datadoc fit train.csv --target churn --preset balanced --rare-frequency 0.02 --output artifacts/churn-pipeline.json
154
+
155
+ # 4. Apply the fitted artifact to validation, test, or new data
156
+ datadoc transform validation.csv --pipeline artifacts/churn-pipeline.json --output validation-features.parquet --validate
157
+
158
+ # 5. Optionally benchmark a safe candidate pipeline against a baseline
159
+ pip install "datadoc-cli[ml]"
160
+ datadoc evaluate train.csv --target churn --task classification --ablation
161
+
162
+ # 6. Export a small executable wrapper around the fitted artifact
163
+ datadoc export --pipeline artifacts/churn-pipeline.json --output pipeline.py
164
+ # or: datadoc export --pipeline artifacts/churn-pipeline.json --format joblib --output pipeline.joblib
165
+
166
+ # 7. Generate a standalone, shareable HTML audit report
167
+ datadoc report train.csv --target churn --output report.html
168
+
169
+ # 8. Visually compare raw vs transformed datasets side-by-side
170
+ datadoc compare train.csv validation-features.parquet --target churn --html compare.html
171
+
172
+ # 9. Lint for leakage risks / diff two plans
173
+ datadoc lint train.csv --target churn
174
+ datadoc diff plan-v1.json plan-v2.json
175
+ ```
176
+
177
+ ### 🖥️ Web dashboard (same pipeline, visual)
178
+
179
+ ```bash
180
+ pip install "datadoc-cli[ui]"
181
+ datadoc ui train.csv --port 8000
182
+ ```
183
+
184
+ The local dashboard calls the same `DataDocPipeline` behind the CLI: profile findings and roles, preparation settings (target, scaling, identifiers, dedup, clipping, cyclical datetime, rare frequency), reviewable plan, fit with output-schema preview, lineage/provenance panel, transformed-CSV download, and an executable Python export. Press `Ctrl+K`/`Cmd+K` for the command palette. Full guide: [docs/ui.html](https://narain-karti.github.io/DATADOC/ui.html).
185
+
186
+ ---
187
+
188
+ ## 🐍 Python SDK (Library Usage)
189
+
190
+ DATADOC is also a Python library. The stable workflow is `profile → plan → fit → transform`; the same fitted artifact can be used in notebooks, services, and batch jobs:
191
+
192
+ ```python
193
+ from datadoc import DataDocPipeline, PipelineConfig
194
+ import polars as pl
195
+
196
+ # Fit only on the training split. The target is protected from feature transforms.
197
+ train_df = pl.read_csv("train.csv")
198
+ pipeline = DataDocPipeline(PipelineConfig(target="churn")).fit(train_df)
199
+ pipeline.save("artifacts/churn-pipeline.json")
200
+
201
+ # Transform data that was never used to fit statistics.
202
+ validation_df = pl.read_csv("validation.csv")
203
+ validation_features = pipeline.transform(validation_df)
204
+ ```
205
+
206
+ For an observed model comparison, install the optional ML extra and call `pipeline.evaluate(train_df)` or `datadoc evaluate`. Evaluation is evidence for the declared task and split strategy; it is not a promise that cleaning always improves a model.
207
+
208
+ ---
209
+
210
+ ## 💻 CLI Commands Reference
211
+
212
+ | Command | Description |
213
+ |---------|-------------|
214
+ | `datadoc wizard <file>` | Guided TUI: asks target/preset/scaling, writes `datadoc.toml`, runs pipeline |
215
+ | `datadoc init` | Writes a starter `datadoc.toml` (or `pyproject.toml [tool.datadoc]`) config |
216
+ | `datadoc profile <file>` | Data-quality report + roles (`--explain`, `--compare profile2.json`) |
217
+ | `datadoc plan <file>` | Explainable plan (`--explain`, `--diff plan2.json`) |
218
+ | `datadoc fit <train>` | Learns pipeline on train only (`--preset`, `--deduplicate`, `--rare-frequency`, `--cyclical`, `--no-hour`, repeatable `--identifier-column` / `--ignore-column`) |
219
+ | `datadoc transform <file>` | Applies saved artifact (`--validate` for schema + drift checks) |
220
+ | `datadoc evaluate <file>` | Candidate vs baseline (`--ablation` for per-component deltas) |
221
+ | `datadoc export` | Wrapper for artifact (`--format python\|joblib`) |
222
+ | `datadoc run <file>` | One-shot profile→plan→fit→transform + `manifest.json` (+ `--evaluate --ablation`) |
223
+ | `datadoc report <file>` | Generates a standalone, shareable HTML data health and preparation audit report |
224
+ | `datadoc compare <raw> <trans>` | Visual side-by-side dataset comparison (terminal table + HTML export) |
225
+ | `datadoc lint <file>` | Leakage/pitfall lint (target duplication, nulls, infinities, duplicates) |
226
+ | `datadoc diff <a.json> <b.json>` | Diff profile/plan/pipeline artifacts |
227
+ | `datadoc plugins list` | Lists 9 registered plugins (priorities, entry-points) |
228
+ | `datadoc ui <file>` | Local FastAPI dashboard (Ctrl+K palette, lineage panel, HTML report export) |
229
+
230
+ Short aliases: `-t/--target`, `-o/--output`, `-p/--pipeline`, `-f/--format`.
231
+ Presets: `--preset quick|balanced|linear|tree|time|robust`. Shell completion: `datadoc --install-completion`.
232
+
233
+
234
+
235
+ ---
236
+
237
+ ## 🧩 Architecture & Plugins
238
+
239
+ DATADOC operates as a fitted pipeline. Every transformation learns state only from training data, saves that state to JSON, and reuses it unchanged for later datasets.
240
+
241
+ | Priority | Plugin | Action Performed |
242
+ |----------|--------|-------------|
243
+ | 5 | **DuplicateRemoverPlugin** | Detects duplicate rows; `deduplicate=True` drops them at fit (train-only) |
244
+ | 10 | **MissingValuePlugin** | Imputes missing numeric values with median, categorical with mode |
245
+ | 20 | **OutlierPlugin** | Offers optional IQR clipping; clipping is not forced by default |
246
+ | 30 | **DatetimePlugin** | Detects date strings and extracts year, month, day, day_of_week (+hour when time present, optional cyclical sin/cos) |
247
+ | 40 | **CategoricalEncoderPlugin** | Encodes categories using training vocabularies (threshold 20) and handles unseen values |
248
+ | 41 | **TargetEncoderPlugin** | Empirical Bayes smoothed target encoding: `(n * cat_mean + m * global_mean) / (n + m)` |
249
+ | 42 | **RareCategoryPlugin** | Groups rare categories (< `rare_category_min_frequency`) into `__RARE__` |
250
+ | 44 | **PolynomialFeaturesPlugin** | Generates degree-2 interaction terms (`x1 * x2`) and squared terms (`x^2`) |
251
+ | 45 | **ScalingPlugin** | Applies configured standard or robust scaling, fit on training data only |
252
+
253
+ The fitted pipeline is the production source of truth. Plugin work should follow the lifecycle `analyze → recommend → apply`, with fitted state (`median`, `clip`, `vocabularies`, `rare maps`, `hour flags`, `center/spread`) serializable in `pipeline.json` (artifact v2 with `provenance`). External plugins auto-register via `datadoc.plugins` entry-points.
254
+
255
+ Want to build your own? See [CONTRIBUTING.md](CONTRIBUTING.md) to learn how to create and register custom plugins!
256
+
257
+ ---
258
+
259
+ ## 🗺️ Roadmap
260
+
261
+ - [x] Core Engine with plugin orchestration
262
+ - [x] 9 Built-in deterministic plugins (duplicate, missing, outlier, datetime, encoder, target encoder, rare, polynomial, scaling)
263
+ - [x] Stunning Rich Terminal UI (wizard, presets, `datadoc.toml`, completion)
264
+ - [x] Pipeline export capability (`python` + `joblib`)
265
+ - [x] Polars backend and local-first pipeline artifacts (v2 + provenance)
266
+ - [x] PyPI Release (`pip install datadoc-cli`)
267
+ - [x] Constrained optional AI planning path
268
+ - [x] Session-scoped local FastAPI dashboard (Ctrl+K palette, lineage, drift)
269
+ - [x] Addictive loop: `profile --compare`, `plan --explain/--diff`, `transform --validate`, `evaluate --ablation`
270
+ - [x] Standalone HTML audit reports (`datadoc report`)
271
+ - [x] Visual dataset comparison engine (`datadoc compare`)
272
+ - [x] Notebook widgets (`profile_to_html`, `_repr_html_`)
273
+ - [ ] Export targets for `dbt` and Apache Airflow
274
+ - [x] Local FastAPI dashboard/API companion
275
+
276
+ ---
277
+
278
+ ## ⚖️ License
279
+
280
+ This project is licensed under the MIT License. See [LICENSE](LICENSE) for details.
281
+
282
+ ## 🤝 Contributing
283
+
284
+ We welcome contributions from the community! If you'd like to add a new plugin or improve the core engine, please see [CONTRIBUTING.md](CONTRIBUTING.md).
285
+
286
+ See [CHANGELOG.md](CHANGELOG.md) for the 0.6.0 release notes.