arctic-platform 0.1.1.dev0__tar.gz → 0.2.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (776) hide show
  1. arctic_platform-0.2.1/.gitignore +190 -0
  2. arctic_platform-0.2.1/PKG-INFO +451 -0
  3. arctic_platform-0.2.1/README.md +145 -0
  4. arctic_platform-0.2.1/arctic_platform/__init__.py +14 -0
  5. arctic_platform-0.2.1/arctic_platform/_dependency_groups.py +91 -0
  6. arctic_platform-0.2.1/arctic_platform/client/UNIFICATION_NOTES.md +145 -0
  7. arctic_platform-0.2.1/arctic_platform/client/__init__.py +51 -0
  8. arctic_platform-0.2.1/arctic_platform/client/base.py +659 -0
  9. arctic_platform-0.2.1/arctic_platform/client/config.py +452 -0
  10. arctic_platform-0.2.1/arctic_platform/client/requests.py +289 -0
  11. arctic_platform-0.2.1/arctic_platform/client/rl.py +42 -0
  12. arctic_platform-0.2.1/arctic_platform/client/sft.py +107 -0
  13. arctic_platform-0.2.1/arctic_platform/client/transport.py +192 -0
  14. arctic_platform-0.2.1/arctic_platform/client/transports/__init__.py +14 -0
  15. arctic_platform-0.2.1/arctic_platform/client/transports/cortex.py +903 -0
  16. arctic_platform-0.2.1/arctic_platform/client/transports/onprem.py +223 -0
  17. arctic_platform-0.2.1/arctic_platform/common/__init__.py +61 -0
  18. arctic_platform-0.2.1/arctic_platform/common/balance_metadata.py +177 -0
  19. arctic_platform-0.2.1/arctic_platform/common/callbacks.py +217 -0
  20. arctic_platform-0.2.1/arctic_platform/common/chunked_cross_entropy.py +92 -0
  21. arctic_platform-0.2.1/arctic_platform/common/config.py +93 -0
  22. arctic_platform-0.2.1/arctic_platform/common/cpu_adam.py +135 -0
  23. arctic_platform-0.2.1/arctic_platform/common/deepspeed_engine.py +532 -0
  24. arctic_platform-0.2.1/arctic_platform/common/deepspeed_worker.py +1267 -0
  25. arctic_platform-0.2.1/arctic_platform/common/flops_counter.py +452 -0
  26. arctic_platform-0.2.1/arctic_platform/common/model_options.py +260 -0
  27. arctic_platform-0.2.1/arctic_platform/common/optimizer.py +88 -0
  28. arctic_platform-0.2.1/arctic_platform/common/option_registry.py +552 -0
  29. arctic_platform-0.2.1/arctic_platform/common/packing/__init__.py +114 -0
  30. arctic_platform-0.2.1/arctic_platform/common/packing/groups.py +235 -0
  31. arctic_platform-0.2.1/arctic_platform/common/packing/pack.py +223 -0
  32. arctic_platform-0.2.1/arctic_platform/common/packing/walk.py +50 -0
  33. arctic_platform-0.2.1/arctic_platform/common/packing/windows.py +158 -0
  34. arctic_platform-0.2.1/arctic_platform/common/peft.py +109 -0
  35. arctic_platform-0.2.1/arctic_platform/common/profiles/__init__.py +56 -0
  36. arctic_platform-0.2.1/arctic_platform/common/profiles/deepseek_v4.py +37 -0
  37. arctic_platform-0.2.1/arctic_platform/common/profiles/deepseek_v41.py +37 -0
  38. arctic_platform-0.2.1/arctic_platform/common/profiles/dflash2.py +68 -0
  39. arctic_platform-0.2.1/arctic_platform/common/profiles/generic_moe.py +64 -0
  40. arctic_platform-0.2.1/arctic_platform/common/profiles/glm5_next.py +38 -0
  41. arctic_platform-0.2.1/arctic_platform/common/profiles/glm_moe_dsa.py +62 -0
  42. arctic_platform-0.2.1/arctic_platform/common/profiles/huggingface.py +81 -0
  43. arctic_platform-0.2.1/arctic_platform/common/profiles/inkling.py +33 -0
  44. arctic_platform-0.2.1/arctic_platform/common/profiles/qwen3_5_moe.py +64 -0
  45. arctic_platform-0.2.1/arctic_platform/common/profiles/qwen4_exp.py +41 -0
  46. {arctic_platform-0.1.1.dev0/arctic_platform/rl → arctic_platform-0.2.1/arctic_platform/common}/ray_cluster.py +78 -22
  47. arctic_platform-0.2.1/arctic_platform/common/ray_server.py +1650 -0
  48. arctic_platform-0.2.1/arctic_platform/common/registry.py +211 -0
  49. arctic_platform-0.2.1/arctic_platform/common/server.py +22 -0
  50. arctic_platform-0.2.1/arctic_platform/common/text_config_overrides.py +48 -0
  51. arctic_platform-0.2.1/arctic_platform/common/utils/__init__.py +68 -0
  52. arctic_platform-0.2.1/arctic_platform/common/utils/balanced_batch.py +421 -0
  53. arctic_platform-0.2.1/arctic_platform/common/utils/batch.py +712 -0
  54. arctic_platform-0.2.1/arctic_platform/common/utils/bf16_zero_norm.py +49 -0
  55. arctic_platform-0.2.1/arctic_platform/common/utils/checkpoint.py +62 -0
  56. {arctic_platform-0.1.1.dev0/arctic_platform/rl → arctic_platform-0.2.1/arctic_platform/common}/utils/debug.py +23 -35
  57. {arctic_platform-0.1.1.dev0/arctic_platform/rl → arctic_platform-0.2.1/arctic_platform/common}/utils/ray_pg.py +39 -17
  58. arctic_platform-0.2.1/arctic_platform/common/utils/record_replay.py +190 -0
  59. arctic_platform-0.2.1/arctic_platform/common/utils/server_models.py +172 -0
  60. arctic_platform-0.2.1/arctic_platform/common/utils/tiled_logits.py +429 -0
  61. arctic_platform-0.2.1/arctic_platform/common/utils/weight_sync.py +222 -0
  62. arctic_platform-0.2.1/arctic_platform/common/weight_checksum.py +103 -0
  63. arctic_platform-0.2.1/arctic_platform/correctness/__init__.py +26 -0
  64. arctic_platform-0.2.1/arctic_platform/correctness/__main__.py +23 -0
  65. arctic_platform-0.2.1/arctic_platform/correctness/api.py +256 -0
  66. arctic_platform-0.2.1/arctic_platform/correctness/checks/__init__.py +35 -0
  67. arctic_platform-0.2.1/arctic_platform/correctness/checks/checkpoint_resume.py +305 -0
  68. arctic_platform-0.2.1/arctic_platform/correctness/checks/e2e_sft_train_validate.py +867 -0
  69. arctic_platform-0.2.1/arctic_platform/correctness/checks/fwd_bwd.py +179 -0
  70. arctic_platform-0.2.1/arctic_platform/correctness/checks/fwd_bwd_step.py +150 -0
  71. arctic_platform-0.2.1/arctic_platform/correctness/checks/inference_checkpoint_loss.py +1153 -0
  72. arctic_platform-0.2.1/arctic_platform/correctness/checks/optimizer_state.py +425 -0
  73. arctic_platform-0.2.1/arctic_platform/correctness/checks/rl_router_replay.py +568 -0
  74. arctic_platform-0.2.1/arctic_platform/correctness/checks/rl_weight_sync.py +518 -0
  75. arctic_platform-0.2.1/arctic_platform/correctness/config-layout.json +14 -0
  76. arctic_platform-0.2.1/arctic_platform/correctness/diagnostics/__main__.py +60 -0
  77. arctic_platform-0.2.1/arctic_platform/correctness/diagnostics/lm_head_chunking.py +71 -0
  78. arctic_platform-0.2.1/arctic_platform/correctness/docs/code-inventory.md +31 -0
  79. arctic_platform-0.2.1/arctic_platform/correctness/docs/getting-started.md +260 -0
  80. arctic_platform-0.2.1/arctic_platform/correctness/docs/hf-reference-onboarding.md +133 -0
  81. arctic_platform-0.2.1/arctic_platform/correctness/docs/running-tests.md +392 -0
  82. arctic_platform-0.2.1/arctic_platform/correctness/docs/tests.md +186 -0
  83. arctic_platform-0.2.1/arctic_platform/correctness/harness/__init__.py +16 -0
  84. arctic_platform-0.2.1/arctic_platform/correctness/harness/config.py +1017 -0
  85. arctic_platform-0.2.1/arctic_platform/correctness/harness/cortex.py +1175 -0
  86. arctic_platform-0.2.1/arctic_platform/correctness/harness/engine_admission.py +570 -0
  87. arctic_platform-0.2.1/arctic_platform/correctness/harness/execution.py +1521 -0
  88. arctic_platform-0.2.1/arctic_platform/correctness/harness/gateway.py +2662 -0
  89. arctic_platform-0.2.1/arctic_platform/correctness/harness/inference.py +1895 -0
  90. arctic_platform-0.2.1/arctic_platform/correctness/harness/lora.py +2081 -0
  91. arctic_platform-0.2.1/arctic_platform/correctness/harness/native_optimizer.py +336 -0
  92. arctic_platform-0.2.1/arctic_platform/correctness/harness/optimizer.py +4626 -0
  93. arctic_platform-0.2.1/arctic_platform/correctness/harness/registry.py +174 -0
  94. arctic_platform-0.2.1/arctic_platform/correctness/harness/report.py +741 -0
  95. arctic_platform-0.2.1/arctic_platform/correctness/harness/runner.py +1384 -0
  96. arctic_platform-0.2.1/arctic_platform/correctness/harness/sft.py +460 -0
  97. arctic_platform-0.2.1/arctic_platform/correctness/harness/storage.py +1361 -0
  98. arctic_platform-0.2.1/arctic_platform/correctness/models/__init__.py +20 -0
  99. arctic_platform-0.2.1/arctic_platform/correctness/models/original.py +294 -0
  100. arctic_platform-0.2.1/arctic_platform/correctness/onboarding/__init__.py +21 -0
  101. arctic_platform-0.2.1/arctic_platform/correctness/onboarding/calibration.py +215 -0
  102. arctic_platform-0.2.1/arctic_platform/correctness/onboarding/cortex_fixture_upload.py +82 -0
  103. arctic_platform-0.2.1/arctic_platform/correctness/onboarding/onboard.py +3553 -0
  104. arctic_platform-0.2.1/arctic_platform/correctness/onboarding/synth_model.py +537 -0
  105. arctic_platform-0.2.1/arctic_platform/correctness/reference/__init__.py +16 -0
  106. arctic_platform-0.2.1/arctic_platform/correctness/reference/__main__.py +144 -0
  107. arctic_platform-0.2.1/arctic_platform/correctness/reference/adamw_trajectory.py +245 -0
  108. arctic_platform-0.2.1/arctic_platform/correctness/reference/hf_single_gpu.py +596 -0
  109. arctic_platform-0.2.1/arctic_platform/correctness/reference/model_features.py +32 -0
  110. arctic_platform-0.2.1/arctic_platform/correctness/reference/optimizer_step.py +271 -0
  111. arctic_platform-0.2.1/arctic_platform/correctness/reference/process_determinism.py +83 -0
  112. arctic_platform-0.2.1/arctic_platform/correctness/reference/qwen4_reference.py +91 -0
  113. arctic_platform-0.2.1/arctic_platform/correctness/reference/sft_trajectory.py +361 -0
  114. arctic_platform-0.2.1/arctic_platform/correctness/selftest/__init__.py +14 -0
  115. arctic_platform-0.2.1/arctic_platform/correctness/selftest/check_suite.py +38 -0
  116. arctic_platform-0.2.1/arctic_platform/correctness/selftest/library.py +31664 -0
  117. arctic_platform-0.2.1/arctic_platform/debug/training.py +373 -0
  118. arctic_platform-0.2.1/arctic_platform/debug/weight_sync.py +1629 -0
  119. arctic_platform-0.2.1/arctic_platform/inference/INTERNAL_COMMIT.txt +1 -0
  120. arctic_platform-0.2.1/arctic_platform/inference/__init__.py +15 -0
  121. arctic_platform-0.2.1/arctic_platform/inference/common/__init__.py +15 -0
  122. arctic_platform-0.2.1/arctic_platform/inference/common/swiftkv/__init__.py +18 -0
  123. arctic_platform-0.2.1/arctic_platform/inference/common/swiftkv/configs.py +37 -0
  124. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/CMakeLists.txt +40 -0
  125. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/attention_generic.cuh +59 -0
  126. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/custom_ops.h +41 -0
  127. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/dispatch_utils.h +69 -0
  128. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/dtype_common.cuh +72 -0
  129. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/dtype_fp8.cuh +37 -0
  130. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/quant_utils.cuh +571 -0
  131. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/reshape_and_cache_flash_bulk.cu +144 -0
  132. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/reshape_and_cache_flash_fp4.cu +338 -0
  133. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/speculator_ln.cu +252 -0
  134. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/sum_lstm.cu +473 -0
  135. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/torch_bindings.cpp +53 -0
  136. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/vectorization.cuh +29 -0
  137. arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/vectorization_utils.cuh +176 -0
  138. arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/CMakeLists.txt +38 -0
  139. arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/bindings.cc +103 -0
  140. arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/int32_map.h +449 -0
  141. arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/suffix_tree.cc +933 -0
  142. arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/suffix_tree.h +179 -0
  143. arctic_platform-0.2.1/arctic_platform/inference/dynasor/__init__.py +0 -0
  144. arctic_platform-0.2.1/arctic_platform/inference/dynasor/cot.py +190 -0
  145. arctic_platform-0.2.1/arctic_platform/inference/dynasor/entropy.py +250 -0
  146. arctic_platform-0.2.1/arctic_platform/inference/dynasor/evaluator.py +852 -0
  147. arctic_platform-0.2.1/arctic_platform/inference/dynasor/openai_server.py +482 -0
  148. arctic_platform-0.2.1/arctic_platform/inference/dynasor/util.py +70 -0
  149. arctic_platform-0.2.1/arctic_platform/inference/dynasor/vllm_server.py +130 -0
  150. arctic_platform-0.2.1/arctic_platform/inference/embedding/README.md +169 -0
  151. arctic_platform-0.2.1/arctic_platform/inference/embedding/client.py +306 -0
  152. arctic_platform-0.2.1/arctic_platform/inference/embedding/generate_proto.py +79 -0
  153. arctic_platform-0.2.1/arctic_platform/inference/embedding/proto/inference.proto +91 -0
  154. arctic_platform-0.2.1/arctic_platform/inference/embedding/replica.py +509 -0
  155. arctic_platform-0.2.1/arctic_platform/inference/embedding/replica_manager.py +509 -0
  156. arctic_platform-0.2.1/arctic_platform/inference/envs.py +75 -0
  157. arctic_platform-0.2.1/arctic_platform/inference/op_builder/__init__.py +0 -0
  158. arctic_platform-0.2.1/arctic_platform/inference/op_builder/builder.py +545 -0
  159. arctic_platform-0.2.1/arctic_platform/inference/op_builder/swiftkv_ops_builder.py +29 -0
  160. arctic_platform-0.2.1/arctic_platform/inference/patching.py +139 -0
  161. arctic_platform-0.2.1/arctic_platform/inference/py_custom_ops.py +117 -0
  162. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/README.md +179 -0
  163. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/__init__.py +47 -0
  164. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/_semip_worker.py +224 -0
  165. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/abstract.py +565 -0
  166. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/assets/figure1-sleep-wake-latency.png +0 -0
  167. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/ca_graph_rebind.py +4661 -0
  168. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/client.py +827 -0
  169. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/dashboard.py +2594 -0
  170. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/demuxer.py +224 -0
  171. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/instance.py +1153 -0
  172. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/matched_baseline.py +86 -0
  173. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/mq_plane.py +371 -0
  174. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/multinode.py +107 -0
  175. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/nccl_bootstrap.py +185 -0
  176. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/README.md +324 -0
  177. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/__init__.py +16 -0
  178. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/book.py +319 -0
  179. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/cleanup.py +80 -0
  180. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/client.py +69 -0
  181. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/common.py +127 -0
  182. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/correctness.py +244 -0
  183. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/jobs.py +264 -0
  184. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/backends.json +12 -0
  185. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/glm53-tp16.json +20 -0
  186. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/glm53-tp8.json +20 -0
  187. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen36-35b-a3b-tp1.json +18 -0
  188. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen36-35b-a3b-tp1x8.json +18 -0
  189. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen36-35b-a3b-tp2x4.json +18 -0
  190. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen36-35b-a3b-tp4x2.json +18 -0
  191. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen38-27b-tp1.json +18 -0
  192. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen38-27b-tp1x8.json +18 -0
  193. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen38-27b-tp2x4.json +18 -0
  194. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen38-flash-next-tp8.json +22 -0
  195. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/onboard.py +458 -0
  196. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/perf.py +326 -0
  197. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/publish.py +261 -0
  198. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/report.py +278 -0
  199. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/rereport.py +126 -0
  200. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/startup.py +301 -0
  201. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/tests/test_onboarding.py +582 -0
  202. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/verify.py +102 -0
  203. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/orch_server.py +55 -0
  204. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/orchestrator.py +1942 -0
  205. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/pipeline.py +1570 -0
  206. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/reproduce/README.md +19 -0
  207. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/reproduce/example_full.py +165 -0
  208. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/imgdiff.py +309 -0
  209. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/main_test.py +141 -0
  210. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/pidcheck.py +326 -0
  211. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/pidref_test.py +189 -0
  212. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/register.py +72 -0
  213. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/semip_publish.py +1474 -0
  214. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_copy.py +97 -0
  215. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_env.py +121 -0
  216. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_generate.py +66 -0
  217. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_image.py +40 -0
  218. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_tp2.py +86 -0
  219. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_weights.py +77 -0
  220. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/semip_logging.py +439 -0
  221. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/CRIU_PLUMBING.md +2163 -0
  222. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/CROSS_NODE_RESTORE.md +476 -0
  223. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/IMAGE_CACHE.md +1099 -0
  224. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/INSTALL.md +317 -0
  225. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/MULTINODE_TP16.md +324 -0
  226. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/SKILL.md +480 -0
  227. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/TEARDOWN_SCOPING.md +559 -0
  228. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/async_generate_DETAILS.md +1195 -0
  229. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/client_DESIGN.md +306 -0
  230. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/dss_dump.md +819 -0
  231. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/dss_integration.md +1620 -0
  232. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/instance_DESIGN.md +925 -0
  233. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/orchestrator_DESIGN.md +2312 -0
  234. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/pipeline_DESIGN.md +985 -0
  235. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/reference.md +395 -0
  236. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/semi-p_DESIGN.md +256 -0
  237. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/slots_DESIGN.md +373 -0
  238. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/tp_DESIGN.md +573 -0
  239. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/slots.py +325 -0
  240. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/state_server.py +394 -0
  241. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_allreduce_pinning.py +203 -0
  242. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_communicator_checkpoint_hooks.py +1054 -0
  243. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_device_visibility.py +309 -0
  244. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_double_buffer.py +142 -0
  245. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_dump_publish.py +394 -0
  246. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_dump_slots.py +84 -0
  247. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_graph_discovery.py +1006 -0
  248. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_image_cache_key.py +1449 -0
  249. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_layout_names.py +294 -0
  250. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_listener_close.py +479 -0
  251. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_lora_host_copy.py +164 -0
  252. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_matched_baseline.py +166 -0
  253. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_multinode.py +507 -0
  254. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_parallel_state_reset.py +176 -0
  255. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_pid_floor.py +495 -0
  256. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_pipeline.py +1204 -0
  257. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_pod_log.py +345 -0
  258. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_publish_layout.py +968 -0
  259. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_restore_nccl_env.py +94 -0
  260. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_restore_quiescence.py +304 -0
  261. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_semip_frontend.py +752 -0
  262. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_serve_core.py +409 -0
  263. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_sleep_guard.py +120 -0
  264. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_slots.py +212 -0
  265. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_worker_initialize.py +267 -0
  266. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/vllm_child.py +5292 -0
  267. arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/worker.py +2816 -0
  268. arctic_platform-0.2.1/arctic_platform/inference/server/__init__.py +16 -0
  269. arctic_platform-0.2.1/arctic_platform/inference/server/action_mask_replay.py +727 -0
  270. arctic_platform-0.2.1/arctic_platform/inference/server/action_masks.py +131 -0
  271. arctic_platform-0.2.1/arctic_platform/inference/server/api.py +184 -0
  272. arctic_platform-0.2.1/arctic_platform/inference/server/chat.py +834 -0
  273. arctic_platform-0.2.1/arctic_platform/inference/server/cli.py +30 -0
  274. arctic_platform-0.2.1/arctic_platform/inference/server/config.py +234 -0
  275. arctic_platform-0.2.1/arctic_platform/inference/server/metrics.py +284 -0
  276. arctic_platform-0.2.1/arctic_platform/inference/server/multi_model.py +414 -0
  277. arctic_platform-0.2.1/arctic_platform/inference/server/pipeline.py +156 -0
  278. arctic_platform-0.2.1/arctic_platform/inference/server/replica_pool.py +2309 -0
  279. arctic_platform-0.2.1/arctic_platform/inference/server/router_replay/__init__.py +34 -0
  280. arctic_platform-0.2.1/arctic_platform/inference/server/router_replay/all2all.py +613 -0
  281. arctic_platform-0.2.1/arctic_platform/inference/server/router_replay/cache.py +601 -0
  282. arctic_platform-0.2.1/arctic_platform/inference/server/router_replay/shm.py +180 -0
  283. arctic_platform-0.2.1/arctic_platform/inference/server/scheduler.py +573 -0
  284. arctic_platform-0.2.1/arctic_platform/inference/server/semip_agent.py +312 -0
  285. arctic_platform-0.2.1/arctic_platform/inference/server/semip_asyncllm.py +169 -0
  286. arctic_platform-0.2.1/arctic_platform/inference/server/semip_engine.py +3336 -0
  287. arctic_platform-0.2.1/arctic_platform/inference/server/semip_frontend.py +318 -0
  288. arctic_platform-0.2.1/arctic_platform/inference/server/streaming.py +1448 -0
  289. arctic_platform-0.2.1/arctic_platform/inference/server/testing.py +271 -0
  290. arctic_platform-0.2.1/arctic_platform/inference/server/token_prompts.py +49 -0
  291. arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/__init__.py +40 -0
  292. arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/broadcast.py +29 -0
  293. arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/engine.py +633 -0
  294. arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/receiver.py +1407 -0
  295. arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/schedule.py +159 -0
  296. arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/sender.py +287 -0
  297. arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/utils.py +876 -0
  298. arctic_platform-0.2.1/arctic_platform/inference/server/worker.py +1984 -0
  299. arctic_platform-0.2.1/arctic_platform/inference/setup.py +194 -0
  300. arctic_platform-0.2.1/arctic_platform/inference/suffix_decoding/__init__.py +18 -0
  301. arctic_platform-0.2.1/arctic_platform/inference/suffix_decoding/cache.py +357 -0
  302. arctic_platform-0.2.1/arctic_platform/inference/suffix_decoding/simulator.py +594 -0
  303. arctic_platform-0.2.1/arctic_platform/inference/utils.py +66 -0
  304. arctic_platform-0.2.1/arctic_platform/inference/vllm/__init__.py +15 -0
  305. arctic_platform-0.2.1/arctic_platform/inference/vllm/args.py +195 -0
  306. arctic_platform-0.2.1/arctic_platform/inference/vllm/attention/README.md +193 -0
  307. arctic_platform-0.2.1/arctic_platform/inference/vllm/attention/__init__.py +65 -0
  308. arctic_platform-0.2.1/arctic_platform/inference/vllm/attention/flash_attn_forest_cascade.py +1951 -0
  309. arctic_platform-0.2.1/arctic_platform/inference/vllm/config.py +208 -0
  310. arctic_platform-0.2.1/arctic_platform/inference/vllm/deepseek_v41_lora.py +39 -0
  311. arctic_platform-0.2.1/arctic_platform/inference/vllm/deepseek_v41_lora_model.py +34 -0
  312. arctic_platform-0.2.1/arctic_platform/inference/vllm/dense_prompt_logprobs.py +360 -0
  313. arctic_platform-0.2.1/arctic_platform/inference/vllm/fp32_lm_head.py +194 -0
  314. arctic_platform-0.2.1/arctic_platform/inference/vllm/glm5next_lora.py +19 -0
  315. arctic_platform-0.2.1/arctic_platform/inference/vllm/glm5next_lora_model.py +11 -0
  316. arctic_platform-0.2.1/arctic_platform/inference/vllm/hybrid_state_fixes.py +273 -0
  317. arctic_platform-0.2.1/arctic_platform/inference/vllm/kv_group_size.py +142 -0
  318. arctic_platform-0.2.1/arctic_platform/inference/vllm/kvcached/__init__.py +2 -0
  319. arctic_platform-0.2.1/arctic_platform/inference/vllm/kvcached/patches.py +68 -0
  320. arctic_platform-0.2.1/arctic_platform/inference/vllm/kvcached/prefix_block_pool.py +490 -0
  321. arctic_platform-0.2.1/arctic_platform/inference/vllm/mamba_completion_refresh.py +140 -0
  322. arctic_platform-0.2.1/arctic_platform/inference/vllm/model_runner.py +2059 -0
  323. arctic_platform-0.2.1/arctic_platform/inference/vllm/mrope_yarn.py +74 -0
  324. arctic_platform-0.2.1/arctic_platform/inference/vllm/patches.py +516 -0
  325. arctic_platform-0.2.1/arctic_platform/inference/vllm/plugin.py +57 -0
  326. arctic_platform-0.2.1/arctic_platform/inference/vllm/required_patches.py +44 -0
  327. arctic_platform-0.2.1/arctic_platform/inference/vllm/router_replay.py +111 -0
  328. arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/__init__.py +14 -0
  329. arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/arctic_proposer.py +301 -0
  330. arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/arctic_speculator.py +1067 -0
  331. arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/config.py +40 -0
  332. arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/fp8.py +364 -0
  333. arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/logits_processor_opt.py +107 -0
  334. arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/vocab_parallel_embedding.py +501 -0
  335. arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_decode_grammar.py +232 -0
  336. arctic_platform-0.2.1/arctic_platform/inference/vllm/startup_error.py +109 -0
  337. arctic_platform-0.2.1/arctic_platform/inference/vllm/stats.py +67 -0
  338. arctic_platform-0.2.1/arctic_platform/inference/vllm/structured_output.py +39 -0
  339. arctic_platform-0.2.1/arctic_platform/inference/vllm/swiftkv/__init__.py +18 -0
  340. arctic_platform-0.2.1/arctic_platform/inference/vllm/swiftkv/llama_swiftkv.py +889 -0
  341. arctic_platform-0.2.1/arctic_platform/inference/vllm/tp_spec_decode_sync.py +230 -0
  342. arctic_platform-0.2.1/arctic_platform/inference/vllm/ulysses.py +765 -0
  343. arctic_platform-0.2.1/arctic_platform/inference/vllm/xgrammar_stop_mask.py +38 -0
  344. arctic_platform-0.2.1/arctic_platform/integrations/__init__.py +31 -0
  345. arctic_platform-0.2.1/arctic_platform/integrations/_cortex_dispatch.py +218 -0
  346. arctic_platform-0.2.1/arctic_platform/integrations/_cortex_shared.py +427 -0
  347. arctic_platform-0.2.1/arctic_platform/integrations/harbor/__init__.py +22 -0
  348. arctic_platform-0.2.1/arctic_platform/integrations/harbor/adapter.py +452 -0
  349. arctic_platform-0.2.1/arctic_platform/integrations/harbor/env.py +387 -0
  350. arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/__init__.py +36 -0
  351. arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/dockerfile_replay.py +290 -0
  352. arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/environment.py +976 -0
  353. arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/images.py +315 -0
  354. arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/session.py +169 -0
  355. arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/shell.py +189 -0
  356. arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/transfer.py +258 -0
  357. arctic_platform-0.2.1/arctic_platform/integrations/harness/__init__.py +24 -0
  358. arctic_platform-0.2.1/arctic_platform/integrations/harness/episodes.py +291 -0
  359. arctic_platform-0.2.1/arctic_platform/integrations/harness/generator.py +284 -0
  360. arctic_platform-0.2.1/arctic_platform/integrations/harness/proxy.py +302 -0
  361. arctic_platform-0.2.1/arctic_platform/integrations/harness/tokens.py +45 -0
  362. arctic_platform-0.2.1/arctic_platform/integrations/harness/tool_parser.py +89 -0
  363. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/README.md +86 -0
  364. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/__init__.py +41 -0
  365. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/_grading_startup.py +203 -0
  366. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/config.py +223 -0
  367. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/examples/qwen3.5-4b/math-i3.toml +67 -0
  368. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/examples/qwen3.5-4b/swe-grade.toml +72 -0
  369. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/jobs.py +236 -0
  370. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/loss.py +45 -0
  371. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/payload.py +135 -0
  372. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/rl.py +196 -0
  373. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/worker.py +589 -0
  374. arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/zz_arctic_prime_rl_grading.pth +1 -0
  375. arctic_platform-0.2.1/arctic_platform/integrations/skyrl/__init__.py +37 -0
  376. arctic_platform-0.2.1/arctic_platform/integrations/skyrl/entrypoint.py +222 -0
  377. arctic_platform-0.2.1/arctic_platform/integrations/tinker/README.md +349 -0
  378. arctic_platform-0.2.1/arctic_platform/integrations/tinker/__init__.py +595 -0
  379. arctic_platform-0.2.1/arctic_platform/integrations/tinker/convert.py +438 -0
  380. arctic_platform-0.2.1/arctic_platform/integrations/tinker/cortex.py +381 -0
  381. arctic_platform-0.2.1/arctic_platform/integrations/tinker/job.py +217 -0
  382. arctic_platform-0.2.1/arctic_platform/integrations/tinker/run.py +63 -0
  383. arctic_platform-0.2.1/arctic_platform/integrations/tinker/snowflake_profile.py +129 -0
  384. arctic_platform-0.2.1/arctic_platform/integrations/trl/README.md +77 -0
  385. arctic_platform-0.2.1/arctic_platform/integrations/trl/__init__.py +60 -0
  386. arctic_platform-0.2.1/arctic_platform/integrations/trl/client.py +732 -0
  387. arctic_platform-0.2.1/arctic_platform/integrations/trl/loss.py +71 -0
  388. arctic_platform-0.2.1/arctic_platform/integrations/trl/rollout.py +400 -0
  389. arctic_platform-0.2.1/arctic_platform/integrations/trl/trainer.py +402 -0
  390. arctic_platform-0.2.1/arctic_platform/integrations/trl/weights.py +49 -0
  391. arctic_platform-0.2.1/arctic_platform/integrations/verl/README.md +143 -0
  392. arctic_platform-0.2.1/arctic_platform/integrations/verl/__init__.py +34 -0
  393. arctic_platform-0.2.1/arctic_platform/integrations/verl/adapter.py +749 -0
  394. arctic_platform-0.2.1/arctic_platform/integrations/verl/config/remote_backend/arctic.yaml +124 -0
  395. arctic_platform-0.2.1/arctic_platform/integrations/verl/examples/run_bird_grpo_arl.sh +107 -0
  396. arctic_platform-0.2.1/arctic_platform/integrations/verl/examples/run_gsm8k_grpo_arl.sh +108 -0
  397. arctic_platform-0.1.1.dev0/arctic_platform/rl/processors/verl_grpo.py → arctic_platform-0.2.1/arctic_platform/integrations/verl/grpo_loss.py +81 -35
  398. arctic_platform-0.2.1/arctic_platform/integrations/verl/register.py +73 -0
  399. arctic_platform-0.2.1/arctic_platform/integrations/verl/rollout.py +404 -0
  400. arctic_platform-0.2.1/arctic_platform/integrations/verl/worker.py +291 -0
  401. arctic_platform-0.2.1/arctic_platform/model/__init__.py +157 -0
  402. arctic_platform-0.2.1/arctic_platform/model/config.py +281 -0
  403. arctic_platform-0.2.1/arctic_platform/model/convert_hf_to_primerl.py +364 -0
  404. arctic_platform-0.2.1/arctic_platform/model/dense_spec.py +151 -0
  405. arctic_platform-0.2.1/arctic_platform/model/factory.py +40 -0
  406. arctic_platform-0.2.1/arctic_platform/model/implementations/NOTICE +3 -0
  407. arctic_platform-0.2.1/arctic_platform/model/implementations/__init__.py +15 -0
  408. arctic_platform-0.2.1/arctic_platform/model/implementations/afmoe/__init__.py +24 -0
  409. arctic_platform-0.2.1/arctic_platform/model/implementations/afmoe/configuration_afmoe.py +116 -0
  410. arctic_platform-0.2.1/arctic_platform/model/implementations/afmoe/converting_afmoe.py +147 -0
  411. arctic_platform-0.2.1/arctic_platform/model/implementations/afmoe/modeling_afmoe.py +640 -0
  412. arctic_platform-0.2.1/arctic_platform/model/implementations/debug/__init__.py +14 -0
  413. arctic_platform-0.2.1/arctic_platform/model/implementations/debug/config.py +117 -0
  414. arctic_platform-0.2.1/arctic_platform/model/implementations/debug/determinism.py +268 -0
  415. arctic_platform-0.2.1/arctic_platform/model/implementations/debug/gpu_memory.py +69 -0
  416. arctic_platform-0.2.1/arctic_platform/model/implementations/debug/row_invariant_projection.py +514 -0
  417. arctic_platform-0.2.1/arctic_platform/model/implementations/debug/token_combine.py +199 -0
  418. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/__init__.py +7 -0
  419. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/attention.py +833 -0
  420. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/configuration_deepseek_v4.py +371 -0
  421. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/conversion_ops.py +348 -0
  422. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/converting_deepseek_v4.py +170 -0
  423. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/cp.py +46 -0
  424. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/deepspeed_integration.py +139 -0
  425. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/dequantize.py +118 -0
  426. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/hyperconnections.py +113 -0
  427. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/modeling_deepseek_v4.py +406 -0
  428. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/moe.py +236 -0
  429. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/rotary.py +131 -0
  430. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/sequence.py +52 -0
  431. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/__init__.py +22 -0
  432. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/attention.py +573 -0
  433. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/configuration_deepseek_v41.py +247 -0
  434. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/converting_deepseek_v41.py +173 -0
  435. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/deepspeed_integration.py +174 -0
  436. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/dequantize.py +73 -0
  437. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/engram.py +535 -0
  438. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/hyperconnections.py +121 -0
  439. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/modeling_deepseek_v41.py +456 -0
  440. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/moe.py +147 -0
  441. arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/streaming_conversion.py +339 -0
  442. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/NOTICE +20 -0
  443. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/__init__.py +21 -0
  444. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/config.py +83 -0
  445. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/online.py +287 -0
  446. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/__init__.py +1 -0
  447. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/algorithms/__init__.py +0 -0
  448. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/algorithms/common/__init__.py +0 -0
  449. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/algorithms/common/dflash_family_model.py +1822 -0
  450. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/algorithms/common/dflash_metrics.py +65 -0
  451. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/core/__init__.py +0 -0
  452. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/core/chunking.py +109 -0
  453. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/core/dflash_head_triton.py +522 -0
  454. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/export/__init__.py +0 -0
  455. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/export/checkpoint_io.py +85 -0
  456. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/__init__.py +1 -0
  457. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/__init__.py +1 -0
  458. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/dflash.py +933 -0
  459. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/dflash2.py +487 -0
  460. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/dflash2_conv_triton.py +359 -0
  461. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/dflash_kernels.py +58 -0
  462. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/flex_attention_backend.py +30 -0
  463. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/registry.py +67 -0
  464. arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/torch_compat.py +63 -0
  465. arctic_platform-0.2.1/arctic_platform/model/implementations/fp8.py +872 -0
  466. arctic_platform-0.2.1/arctic_platform/model/implementations/glm4_moe/__init__.py +24 -0
  467. arctic_platform-0.2.1/arctic_platform/model/implementations/glm4_moe/configuration_glm4_moe.py +215 -0
  468. arctic_platform-0.2.1/arctic_platform/model/implementations/glm4_moe/converting_glm4_moe.py +151 -0
  469. arctic_platform-0.2.1/arctic_platform/model/implementations/glm4_moe/modeling_glm4_moe.py +336 -0
  470. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/INTERNAL_COMMIT.txt +1 -0
  471. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/NOTICE +12 -0
  472. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/__init__.py +16 -0
  473. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/config.py +58 -0
  474. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/config_validation.py +1 -0
  475. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/deepspeed_integration.py +172 -0
  476. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/model_builder.py +448 -0
  477. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/__init__.py +44 -0
  478. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/fp8.py +12 -0
  479. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/__init__.py +13 -0
  480. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/configuration_glm_moe_dsa.py +261 -0
  481. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/converting_glm_moe_dsa.py +442 -0
  482. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/debug_overrides.py +126 -0
  483. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/modeling_glm_moe_dsa.py +498 -0
  484. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/sparse_mla_attention.py +328 -0
  485. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/__init__.py +0 -0
  486. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/fp8_indexer.py +317 -0
  487. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/sparse_mla_bwd.py +375 -0
  488. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/sparse_mla_flashmla.py +466 -0
  489. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/sparse_mla_fwd.py +198 -0
  490. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/layers/__init__.py +0 -0
  491. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/layers/mlp.py +38 -0
  492. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/layers/norms.py +79 -0
  493. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/layers/rotary_emb.py +142 -0
  494. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/utils/__init__.py +0 -0
  495. arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/utils/cp.py +22 -0
  496. arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/__init__.py +28 -0
  497. arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/context_parallel.py +1013 -0
  498. arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/converting_glm5_next.py +275 -0
  499. arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/deepspeed_integration.py +138 -0
  500. arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/fused_norms.py +79 -0
  501. arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/hyper_connections.py +556 -0
  502. arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/modeling_glm5_next.py +289 -0
  503. arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/vllm_weights.py +156 -0
  504. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/__init__.py +14 -0
  505. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/action_masks.py +638 -0
  506. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/activation_offload.py +576 -0
  507. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/grouped_lm_head.py +206 -0
  508. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/lm_head.py +816 -0
  509. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/packing.py +29 -0
  510. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/pinned_staging_cache.py +450 -0
  511. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/qwen3_5_dense_parity.py +476 -0
  512. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/router_replay_recompute.py +198 -0
  513. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/sp/__init__.py +14 -0
  514. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/sp/collectives.py +101 -0
  515. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/sp/gated_delta_net.py +913 -0
  516. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/sp/transformers.py +30 -0
  517. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/tiled_mlp.py +312 -0
  518. arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/vllm_rounding.py +279 -0
  519. arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/__init__.py +16 -0
  520. arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/attention.py +258 -0
  521. arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/deepspeed_integration.py +97 -0
  522. arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/modeling_inkling.py +313 -0
  523. arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/sequence_parallel.py +80 -0
  524. arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/weights.py +191 -0
  525. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/__init__.py +14 -0
  526. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/__init__.py +7 -0
  527. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_kv_fake_quant.py +111 -0
  528. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_mhc.py +646 -0
  529. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_rope.py +201 -0
  530. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_sparse_attn.py +434 -0
  531. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_sparse_attn_bwd.py +334 -0
  532. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_sparse_attn_fwd.py +289 -0
  533. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/fp8_block_dequant.py +106 -0
  534. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/fp8_indexer.py +248 -0
  535. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/fp8_quant.py +160 -0
  536. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/fused_swiglu.py +105 -0
  537. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/scored_swiglu_down.py +188 -0
  538. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonic_experts.py +137 -0
  539. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/LICENSE +201 -0
  540. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/__init__.py +25 -0
  541. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/enums.py +46 -0
  542. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/__init__.py +689 -0
  543. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/backward.py +583 -0
  544. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/forward.py +157 -0
  545. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/reduction_over_k_gather.py +176 -0
  546. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/topk.py +355 -0
  547. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/triton_kernels/README.md +1 -0
  548. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/triton_kernels/TRITON_LICENSE +23 -0
  549. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/triton_kernels/__init__.py +370 -0
  550. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/triton_kernels/bitmatrix.py +162 -0
  551. arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/utils.py +139 -0
  552. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/__init__.py +1 -0
  553. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/attn.py +288 -0
  554. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/cp_mamba.py +290 -0
  555. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/grouped_linear.py +66 -0
  556. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/mlp.py +29 -0
  557. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/norms.py +83 -0
  558. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/rms_norm.py +3 -0
  559. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/rotary_emb.py +115 -0
  560. arctic_platform-0.2.1/arctic_platform/model/implementations/layers/sequence.py +14 -0
  561. arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/__init__.py +24 -0
  562. arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/configuration_minimax_m2.py +172 -0
  563. arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/converting_minimax_m2.py +101 -0
  564. arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/modeling_minimax_m2.py +281 -0
  565. arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/vllm_weights.py +57 -0
  566. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/INTERNAL_COMMIT.txt +1 -0
  567. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/NOTICE +16 -0
  568. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/__init__.py +1 -0
  569. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/base.py +69 -0
  570. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/config.py +4 -0
  571. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/config_validation.py +18 -0
  572. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/conversion_cache.py +226 -0
  573. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/deepspeed_integration.py +482 -0
  574. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/__init__.py +2 -0
  575. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/combine_replay.py +181 -0
  576. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/deepep.py +373 -0
  577. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/ep_backend.py +27 -0
  578. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/expert_parallel.py +79 -0
  579. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/ncclep.py +154 -0
  580. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/token_permute.py +58 -0
  581. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/uccl_ep/__init__.py +6 -0
  582. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/uccl_ep/bootstrap.py +357 -0
  583. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/uccl_ep/buffer.py +1463 -0
  584. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/ucclep.py +430 -0
  585. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/hf_loader.py +279 -0
  586. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/layers/__init__.py +1 -0
  587. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/layers/checkpointing.py +154 -0
  588. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/layers/lm_head.py +653 -0
  589. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/layers/moe.py +1526 -0
  590. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/logging_utils.py +26 -0
  591. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/parallel_dims.py +251 -0
  592. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/router_stats.py +177 -0
  593. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/routing_weights.py +200 -0
  594. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/sequence_parallel.py +180 -0
  595. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/token_combine.py +109 -0
  596. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/vllm_weights.py +270 -0
  597. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/vlm.py +121 -0
  598. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/weights.py +140 -0
  599. arctic_platform-0.2.1/arctic_platform/model/implementations/moe/world.py +46 -0
  600. arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/__init__.py +24 -0
  601. arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/configuration_nemotron_h.py +227 -0
  602. arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/converting_nemotron_h.py +176 -0
  603. arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/modeling_nemotron_h.py +595 -0
  604. arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/vllm_weights.py +71 -0
  605. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/INTERNAL_COMMIT.txt +1 -0
  606. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/NOTICE +15 -0
  607. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/__init__.py +19 -0
  608. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/config.py +66 -0
  609. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/config_validation.py +1 -0
  610. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/conversion_cache.py +2 -0
  611. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/deepspeed_integration.py +422 -0
  612. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/__init__.py +3 -0
  613. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/deepep.py +1 -0
  614. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/ep_backend.py +1 -0
  615. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/expert_parallel.py +1 -0
  616. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/token_permute.py +1 -0
  617. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/uccl_ep/__init__.py +7 -0
  618. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/uccl_ep/bootstrap.py +1 -0
  619. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/uccl_ep/buffer.py +2 -0
  620. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/ucclep.py +1 -0
  621. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/logging_utils.py +1 -0
  622. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/model_builder.py +754 -0
  623. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/__init__.py +133 -0
  624. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/base.py +1 -0
  625. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/__init__.py +0 -0
  626. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/checkpointing.py +1 -0
  627. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/lm_head.py +2 -0
  628. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/moe.py +2 -0
  629. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/rotary_emb.py +114 -0
  630. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/qwen3_5_moe/__init__.py +13 -0
  631. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/qwen3_5_moe/configuration_qwen3_5_moe.py +113 -0
  632. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/qwen3_5_moe/converting_qwen3_5_moe.py +119 -0
  633. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/qwen3_5_moe/modeling_qwen3_5_moe.py +1217 -0
  634. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/parallel_dims.py +1 -0
  635. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/sequence_parallel.py +167 -0
  636. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/vlm.py +1 -0
  637. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/weights.py +1 -0
  638. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/world.py +1 -0
  639. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/__init__.py +27 -0
  640. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/context_parallel.py +335 -0
  641. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/converting_qwen4_exp.py +192 -0
  642. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/deepspeed_integration.py +141 -0
  643. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/hyper_connection.py +467 -0
  644. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/modeling_qwen4_exp.py +500 -0
  645. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/qsa_flex.py +654 -0
  646. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/qsa_index_scores.py +125 -0
  647. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/qsa_sparse_gqa.py +288 -0
  648. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen3_moe/__init__.py +24 -0
  649. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen3_moe/configuration_qwen3_moe.py +210 -0
  650. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen3_moe/converting_qwen3_moe.py +100 -0
  651. arctic_platform-0.2.1/arctic_platform/model/implementations/qwen3_moe/modeling_qwen3_moe.py +441 -0
  652. arctic_platform-0.2.1/arctic_platform/model/loader.py +312 -0
  653. arctic_platform-0.2.1/arctic_platform/model/loaders/__init__.py +25 -0
  654. arctic_platform-0.2.1/arctic_platform/model/loaders/deepseek_v4.py +108 -0
  655. arctic_platform-0.2.1/arctic_platform/model/loaders/deepseek_v41.py +111 -0
  656. arctic_platform-0.2.1/arctic_platform/model/loaders/dflash2.py +122 -0
  657. arctic_platform-0.2.1/arctic_platform/model/loaders/flash_moe.py +178 -0
  658. arctic_platform-0.2.1/arctic_platform/model/loaders/generic_moe.py +123 -0
  659. arctic_platform-0.2.1/arctic_platform/model/loaders/glm_moe_dsa.py +137 -0
  660. arctic_platform-0.2.1/arctic_platform/model/loaders/huggingface.py +157 -0
  661. arctic_platform-0.2.1/arctic_platform/model/loaders/inkling.py +112 -0
  662. arctic_platform-0.2.1/arctic_platform/model/loaders/qwen3_5_moe.py +150 -0
  663. arctic_platform-0.2.1/arctic_platform/model/moe_spec.py +194 -0
  664. arctic_platform-0.2.1/arctic_platform/model/parallel/__init__.py +15 -0
  665. arctic_platform-0.2.1/arctic_platform/model/parallel/groups.py +369 -0
  666. arctic_platform-0.2.1/arctic_platform/model/parallel/topology.py +277 -0
  667. arctic_platform-0.2.1/arctic_platform/model/parallel/validation.py +101 -0
  668. arctic_platform-0.2.1/arctic_platform/model/patch.py +83 -0
  669. arctic_platform-0.2.1/arctic_platform/model/patches/__init__.py +24 -0
  670. arctic_platform-0.2.1/arctic_platform/model/patches/activation_offload.py +40 -0
  671. arctic_platform-0.2.1/arctic_platform/model/patches/compile.py +32 -0
  672. arctic_platform-0.2.1/arctic_platform/model/patches/gradient_checkpointing.py +67 -0
  673. arctic_platform-0.2.1/arctic_platform/model/patches/liger.py +37 -0
  674. arctic_platform-0.2.1/arctic_platform/model/patches/lm_head.py +42 -0
  675. arctic_platform-0.2.1/arctic_platform/model/patches/peft.py +191 -0
  676. arctic_platform-0.2.1/arctic_platform/model/patches/tiled_mlp.py +247 -0
  677. arctic_platform-0.2.1/arctic_platform/model/patches/utils.py +28 -0
  678. arctic_platform-0.2.1/arctic_platform/model/patches/zorro_train.py +57 -0
  679. arctic_platform-0.2.1/arctic_platform/model/recompute.py +30 -0
  680. arctic_platform-0.2.1/arctic_platform/model/router_stats.py +26 -0
  681. arctic_platform-0.2.1/arctic_platform/model/weight_export.py +749 -0
  682. arctic_platform-0.2.1/arctic_platform/registry.py +111 -0
  683. arctic_platform-0.2.1/arctic_platform/rl/__init__.py +79 -0
  684. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/client.py +3 -19
  685. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/config.py +29 -48
  686. arctic_platform-0.2.1/arctic_platform/rl/deepspeed_worker.py +18 -0
  687. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/examples/inference_example.py +2 -1
  688. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/examples/local_server_example.py +10 -5
  689. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/examples/weight_sync_example.py +2 -1
  690. arctic_platform-0.2.1/arctic_platform/rl/processors/__init__.py +240 -0
  691. arctic_platform-0.2.1/arctic_platform/rl/processors/base_loss.py +278 -0
  692. arctic_platform-0.2.1/arctic_platform/rl/processors/causal_cross_entropy.py +244 -0
  693. arctic_platform-0.2.1/arctic_platform/rl/processors/compute_logprobs.py +123 -0
  694. arctic_platform-0.2.1/arctic_platform/rl/processors/cortex_grpo.py +127 -0
  695. arctic_platform-0.2.1/arctic_platform/rl/processors/dflash2.py +149 -0
  696. arctic_platform-0.2.1/arctic_platform/rl/processors/functional.py +1337 -0
  697. arctic_platform-0.2.1/arctic_platform/rl/processors/grouped_distillation.py +1040 -0
  698. arctic_platform-0.2.1/arctic_platform/rl/processors/grpo.py +1766 -0
  699. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/processors/microbatch.py +49 -9
  700. arctic_platform-0.2.1/arctic_platform/rl/processors/packed_reduction.py +239 -0
  701. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/processors/packing.py +24 -3
  702. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/processors/pipeline.py +428 -275
  703. arctic_platform-0.2.1/arctic_platform/rl/processors/sft.py +18 -0
  704. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/processors/stats_tracker.py +25 -3
  705. arctic_platform-0.2.1/arctic_platform/rl/processors/verl_grpo.py +31 -0
  706. arctic_platform-0.2.1/arctic_platform/rl/processors/weighted_cross_entropy.py +190 -0
  707. arctic_platform-0.2.1/arctic_platform/rl/processors/weighted_logprob.py +82 -0
  708. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/ray_client.py +136 -81
  709. arctic_platform-0.2.1/arctic_platform/rl/ray_cluster.py +18 -0
  710. arctic_platform-0.2.1/arctic_platform/rl/ray_server.py +18 -0
  711. arctic_platform-0.2.1/arctic_platform/rl/server.py +18 -0
  712. arctic_platform-0.2.1/arctic_platform/rl/utils/__init__.py +50 -0
  713. arctic_platform-0.2.1/arctic_platform/rl/utils/batch.py +18 -0
  714. arctic_platform-0.2.1/arctic_platform/rl/utils/cuda_ipc.py +18 -0
  715. arctic_platform-0.2.1/arctic_platform/rl/utils/debug.py +18 -0
  716. arctic_platform-0.2.1/arctic_platform/rl/utils/ray_pg.py +18 -0
  717. arctic_platform-0.2.1/arctic_platform/rl/utils/record_replay.py +18 -0
  718. arctic_platform-0.2.1/arctic_platform/rl/utils/server_models.py +18 -0
  719. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/weight_sync.py +7 -3
  720. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/README.md +127 -68
  721. arctic_platform-0.2.1/arctic_platform/rl/zorro_train/__init__.py +34 -0
  722. arctic_platform-0.2.1/arctic_platform/rl/zorro_train/actor.py +252 -0
  723. arctic_platform-0.2.1/arctic_platform/rl/zorro_train/demo.py +128 -0
  724. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/module_patcher.py +23 -7
  725. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/qwen_attention_patcher.py +610 -281
  726. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/qwen_model_patcher.py +384 -416
  727. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/seqlen_balancing.py +69 -287
  728. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/test_perf.py +148 -87
  729. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/tests.py +347 -309
  730. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/zorro_train.py +502 -708
  731. arctic_platform-0.2.1/arctic_platform/sft/__init__.py +47 -0
  732. arctic_platform-0.2.1/arctic_platform/sft/examples/__init__.py +14 -0
  733. arctic_platform-0.2.1/arctic_platform/sft/examples/run_inkling_sft.py +105 -0
  734. arctic_platform-0.2.1/arctic_platform/sft/examples/run_sft_ckpt_resume_demo.py +158 -0
  735. arctic_platform-0.2.1/arctic_platform/sft/examples/run_sft_demo.py +216 -0
  736. arctic_platform-0.2.1/arctic_platform/sft/examples/run_sft_generate_demo.py +115 -0
  737. arctic_platform-0.2.1/arctic_platform/sft/examples/sft_example.py +146 -0
  738. arctic_platform-0.2.1/arctic_platform/sft/processor.py +399 -0
  739. arctic_platform-0.2.1/arctic_platform/sft_profile.py +110 -0
  740. arctic_platform-0.2.1/arctic_platform/testing_utils.py +1275 -0
  741. arctic_platform-0.2.1/arctic_platform/wire.py +449 -0
  742. arctic_platform-0.2.1/hatch_build.py +51 -0
  743. arctic_platform-0.2.1/pyproject.toml +281 -0
  744. arctic_platform-0.1.1.dev0/PKG-INFO +0 -84
  745. arctic_platform-0.1.1.dev0/README.md +0 -23
  746. arctic_platform-0.1.1.dev0/arctic_platform/rl/README.md +0 -7
  747. arctic_platform-0.1.1.dev0/arctic_platform/rl/__init__.py +0 -27
  748. arctic_platform-0.1.1.dev0/arctic_platform/rl/deepspeed_worker.py +0 -905
  749. arctic_platform-0.1.1.dev0/arctic_platform/rl/http_client.py +0 -705
  750. arctic_platform-0.1.1.dev0/arctic_platform/rl/http_server.py +0 -1166
  751. arctic_platform-0.1.1.dev0/arctic_platform/rl/processors/__init__.py +0 -166
  752. arctic_platform-0.1.1.dev0/arctic_platform/rl/processors/functional.py +0 -257
  753. arctic_platform-0.1.1.dev0/arctic_platform/rl/processors/grpo.py +0 -330
  754. arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/long_context_qa/README.md +0 -87
  755. arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/long_context_qa/download_data.py +0 -120
  756. arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/long_context_qa/run_qwen3_32b_longcontext_grpo_arl_zorro_yes_kl.sh +0 -221
  757. arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/README.md +0 -176
  758. arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/bird_reward.py +0 -272
  759. arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/preprocess_bird.py +0 -684
  760. arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/run_qwen3_32b_bird_grpo_arl_zorro_yes.sh +0 -251
  761. arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/run_qwen3_32b_bird_grpo_arl_zorro_yes_kl.sh +0 -31
  762. arctic_platform-0.1.1.dev0/arctic_platform/rl/ray_server.py +0 -1475
  763. arctic_platform-0.1.1.dev0/arctic_platform/rl/server.py +0 -6
  764. arctic_platform-0.1.1.dev0/arctic_platform/rl/utils/__init__.py +0 -16
  765. arctic_platform-0.1.1.dev0/arctic_platform/rl/utils/batch.py +0 -434
  766. arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/__init__.py +0 -23
  767. arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/actor.py +0 -271
  768. arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/demo.py +0 -130
  769. arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/test.py +0 -7
  770. arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/test_actor_demo.py +0 -68
  771. arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/test_forward_and_backward.py +0 -492
  772. arctic_platform-0.1.1.dev0/pyproject.toml +0 -106
  773. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/LICENSE +0 -0
  774. {arctic_platform-0.1.1.dev0/arctic_platform/rl → arctic_platform-0.2.1/arctic_platform/common}/utils/cuda_ipc.py +0 -0
  775. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/test_perf-g1.sh +0 -0
  776. {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/test_perf-g8.sh +0 -0
@@ -0,0 +1,190 @@
1
+ # Byte-compiled / optimized / DLL files
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+
6
+ # C extensions
7
+ *.so
8
+
9
+ # Distribution / packaging
10
+ .Python
11
+ build/
12
+ develop-eggs/
13
+ dist/
14
+ downloads/
15
+ eggs/
16
+ .eggs/
17
+ lib/
18
+ lib64/
19
+ parts/
20
+ sdist/
21
+ var/
22
+ wheels/
23
+ share/python-wheels/
24
+ *.egg-info/
25
+ .installed.cfg
26
+ *.egg
27
+ MANIFEST
28
+
29
+ # PyInstaller
30
+ # Usually these files are written by a python script from a template
31
+ # before PyInstaller builds the exe, so as to inject date/other infos into it.
32
+ *.manifest
33
+ *.spec
34
+
35
+ # Installer logs
36
+ pip-log.txt
37
+ pip-delete-this-directory.txt
38
+
39
+ # Unit test / coverage reports
40
+ htmlcov/
41
+ .tox/
42
+ .nox/
43
+ .coverage
44
+ .coverage.*
45
+ .cache
46
+ nosetests.xml
47
+ coverage.xml
48
+ *.cover
49
+ *.py,cover
50
+ .hypothesis/
51
+ .pytest_cache/
52
+ cover/
53
+
54
+ # Translations
55
+ *.mo
56
+ *.pot
57
+
58
+ # Django stuff:
59
+ *.log
60
+ local_settings.py
61
+ db.sqlite3
62
+ db.sqlite3-journal
63
+
64
+ # Flask stuff:
65
+ instance/
66
+ .webassets-cache
67
+
68
+ # Scrapy stuff:
69
+ .scrapy
70
+
71
+ # Sphinx documentation
72
+ docs/_build/
73
+
74
+ # PyBuilder
75
+ .pybuilder/
76
+ target/
77
+
78
+ # Jupyter Notebook
79
+ .ipynb_checkpoints
80
+
81
+ # IPython
82
+ profile_default/
83
+ ipython_config.py
84
+
85
+ # pyenv
86
+ # For a library or package, you might want to ignore these files since the code is
87
+ # intended to run in multiple environments; otherwise, check them in:
88
+ # .python-version
89
+
90
+ # pipenv
91
+ # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
92
+ # However, in case of collaboration, if having platform-specific dependencies or dependencies
93
+ # having no cross-platform support, pipenv may install dependencies that don't work, or not
94
+ # install all needed dependencies.
95
+ #Pipfile.lock
96
+
97
+ # poetry
98
+ # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
99
+ # This is especially recommended for binary packages to ensure reproducibility, and is more
100
+ # commonly ignored for libraries.
101
+ # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
102
+ #poetry.lock
103
+
104
+ # pdm
105
+ # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
106
+ #pdm.lock
107
+ # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
108
+ # in version control.
109
+ # https://pdm.fming.dev/latest/usage/project/#working-with-version-control
110
+ .pdm.toml
111
+ .pdm-python
112
+ .pdm-build/
113
+
114
+ # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
115
+ __pypackages__/
116
+
117
+ # Celery stuff
118
+ celerybeat-schedule
119
+ celerybeat.pid
120
+
121
+ # SageMath parsed files
122
+ *.sage.py
123
+
124
+ # Environments
125
+ .env
126
+ .venv
127
+ env/
128
+ venv/
129
+ ENV/
130
+ env.bak/
131
+ venv.bak/
132
+
133
+ # Spyder project settings
134
+ .spyderproject
135
+ .spyproject
136
+
137
+ # Rope project settings
138
+ .ropeproject
139
+
140
+ # mkdocs documentation
141
+ /site
142
+
143
+ # mypy
144
+ .mypy_cache/
145
+ .dmypy.json
146
+ dmypy.json
147
+
148
+ # Pyre type checker
149
+ .pyre/
150
+
151
+ # pytype static type analyzer
152
+ .pytype/
153
+
154
+ # Cython debug symbols
155
+ cython_debug/
156
+
157
+ # PyCharm
158
+ # JetBrains specific template is maintained in a separate JetBrains.gitignore that can
159
+ # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
160
+ # and can be added to the global gitignore or merged into this file. For a more nuclear
161
+ # option (not recommended) you can uncomment the following to ignore the entire idea folder.
162
+ #.idea/
163
+
164
+ # Arctic Synth work directory
165
+ batch_work_dir/
166
+
167
+ # gRPC code generated by arctic_platform/inference/embedding/generate_proto.py
168
+ arctic_platform/inference/embedding/proto/python/
169
+
170
+ # Hydra run artifacts from recipe launchers. Hydra writes to the process's cwd,
171
+ # which is the repo root when a launcher is invoked by path (the documented way
172
+ # to run the Cortex recipes) -- so the recipes/ pattern alone misses them.
173
+ recipes/**/outputs/
174
+ /outputs/
175
+
176
+ # Client recipe connection configs — these hold a Snowflake PAT. The checked-in
177
+ # config.json.template is not matched (it does not end in .json).
178
+ recipes/*.json
179
+
180
+ # Semi-persistence onboarding reports and job ledgers, written under the cwd.
181
+ onboarding_reports/
182
+
183
+ # Local correctness measurements and execution state; recipes and references are selected separately.
184
+ /work/
185
+ /.correctness-tmp/
186
+ /.codex-run-state/
187
+ /.agent-work/
188
+ /correctness-progress.md
189
+ /local-correctness-records/
190
+ /arctic_platform/correctness/specs/
@@ -0,0 +1,451 @@
1
+ Metadata-Version: 2.5
2
+ Name: arctic_platform
3
+ Version: 0.2.1
4
+ Summary: Snowflake LLM training library w/ RL
5
+ Project-URL: Homepage, https://github.com/Snowflake-AI-Research/Arctic-Platform
6
+ Project-URL: Repository, https://github.com/Snowflake-AI-Research/Arctic-Platform
7
+ Project-URL: Issues, https://github.com/Snowflake-AI-Research/Arctic-Platform/issues
8
+ Author-email: Tunji Ruwase <tunji.ruwase@snowflake.com>, Stas Bekman <stas.bekman@snowflake.com>, Michael Wyatt <michael.wyatt@snowflake.com>, Jeff Rasley <jeff.rasley@snowflake.com>, Samyam Rajbhandari <samyam.rajbhandari@snowflake.com>
9
+ License-Expression: Apache-2.0
10
+ License-File: LICENSE
11
+ Keywords: llm,rl,training
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3.12
15
+ Requires-Python: >=3.12
16
+ Requires-Dist: pydantic-settings>=2.0
17
+ Requires-Dist: pydantic>=2.10
18
+ Requires-Dist: typing-extensions
19
+ Provides-Extra: all
20
+ Requires-Dist: accelerate; extra == 'all'
21
+ Requires-Dist: aiohttp; extra == 'all'
22
+ Requires-Dist: autoflake; extra == 'all'
23
+ Requires-Dist: certifi; extra == 'all'
24
+ Requires-Dist: codetiming; extra == 'all'
25
+ Requires-Dist: deepspeed>=0.19.9; extra == 'all'
26
+ Requires-Dist: dockerfile-parse>=2.0.1; extra == 'all'
27
+ Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'all'
28
+ Requires-Dist: fastapi; extra == 'all'
29
+ Requires-Dist: harbor==0.25.0; extra == 'all'
30
+ Requires-Dist: httpx; extra == 'all'
31
+ Requires-Dist: huggingface-hub; extra == 'all'
32
+ Requires-Dist: huggingface-hub<3,>=2.0; extra == 'all'
33
+ Requires-Dist: kvcached==0.1.4; extra == 'all'
34
+ Requires-Dist: liger-kernel>=0.8.5; extra == 'all'
35
+ Requires-Dist: numpy; extra == 'all'
36
+ Requires-Dist: nvidia-ml-py; extra == 'all'
37
+ Requires-Dist: omegaconf; extra == 'all'
38
+ Requires-Dist: openhands-sdk==1.7.1; extra == 'all'
39
+ Requires-Dist: openhands-tools==1.7.1; extra == 'all'
40
+ Requires-Dist: orjson; extra == 'all'
41
+ Requires-Dist: parameterized; extra == 'all'
42
+ Requires-Dist: pathspec>=1.0.3; extra == 'all'
43
+ Requires-Dist: peft<0.22,>=0.21.2; extra == 'all'
44
+ Requires-Dist: pre-commit; extra == 'all'
45
+ Requires-Dist: psutil; extra == 'all'
46
+ Requires-Dist: pydantic>=2.0; extra == 'all'
47
+ Requires-Dist: pytest; extra == 'all'
48
+ Requires-Dist: pytest-asyncio; extra == 'all'
49
+ Requires-Dist: pytest-flakefinder; extra == 'all'
50
+ Requires-Dist: pytest-instafail; extra == 'all'
51
+ Requires-Dist: pytest-sugar; extra == 'all'
52
+ Requires-Dist: pytest-timeout; extra == 'all'
53
+ Requires-Dist: pytest-xdist; extra == 'all'
54
+ Requires-Dist: quack-kernels>=0.4.0; extra == 'all'
55
+ Requires-Dist: ray; extra == 'all'
56
+ Requires-Dist: requests; extra == 'all'
57
+ Requires-Dist: safetensors; extra == 'all'
58
+ Requires-Dist: snowflake-sandbox-python==0.2.2a5; extra == 'all'
59
+ Requires-Dist: tenacity; extra == 'all'
60
+ Requires-Dist: tensordict; extra == 'all'
61
+ Requires-Dist: tinker==0.25.0; extra == 'all'
62
+ Requires-Dist: torch; extra == 'all'
63
+ Requires-Dist: torch>=2.9; extra == 'all'
64
+ Requires-Dist: tqdm; extra == 'all'
65
+ Requires-Dist: transformers; extra == 'all'
66
+ Requires-Dist: transformers==5.20.2; extra == 'all'
67
+ Requires-Dist: urllib3; extra == 'all'
68
+ Requires-Dist: uvicorn; extra == 'all'
69
+ Requires-Dist: vllm==0.33.0; extra == 'all'
70
+ Requires-Dist: xgrammar==0.2.8; extra == 'all'
71
+ Provides-Extra: correctness
72
+ Requires-Dist: aiohttp; extra == 'correctness'
73
+ Requires-Dist: deepspeed>=0.19.9; extra == 'correctness'
74
+ Requires-Dist: huggingface-hub; extra == 'correctness'
75
+ Requires-Dist: huggingface-hub<3,>=2.0; extra == 'correctness'
76
+ Requires-Dist: liger-kernel>=0.8.5; extra == 'correctness'
77
+ Requires-Dist: numpy; extra == 'correctness'
78
+ Requires-Dist: nvidia-ml-py; extra == 'correctness'
79
+ Requires-Dist: peft<0.22,>=0.21.2; extra == 'correctness'
80
+ Requires-Dist: psutil; extra == 'correctness'
81
+ Requires-Dist: pytest; extra == 'correctness'
82
+ Requires-Dist: pytest-instafail; extra == 'correctness'
83
+ Requires-Dist: quack-kernels>=0.4.0; extra == 'correctness'
84
+ Requires-Dist: ray; extra == 'correctness'
85
+ Requires-Dist: requests; extra == 'correctness'
86
+ Requires-Dist: safetensors; extra == 'correctness'
87
+ Requires-Dist: tenacity; extra == 'correctness'
88
+ Requires-Dist: tensordict; extra == 'correctness'
89
+ Requires-Dist: torch; extra == 'correctness'
90
+ Requires-Dist: torch>=2.9; extra == 'correctness'
91
+ Requires-Dist: tqdm; extra == 'correctness'
92
+ Requires-Dist: transformers==5.20.2; extra == 'correctness'
93
+ Requires-Dist: urllib3; extra == 'correctness'
94
+ Provides-Extra: cortex
95
+ Requires-Dist: aiohttp; extra == 'cortex'
96
+ Requires-Dist: numpy; extra == 'cortex'
97
+ Requires-Dist: requests; extra == 'cortex'
98
+ Requires-Dist: safetensors; extra == 'cortex'
99
+ Requires-Dist: tenacity; extra == 'cortex'
100
+ Requires-Dist: torch; extra == 'cortex'
101
+ Requires-Dist: urllib3; extra == 'cortex'
102
+ Provides-Extra: dev
103
+ Requires-Dist: aiohttp; extra == 'dev'
104
+ Requires-Dist: autoflake; extra == 'dev'
105
+ Requires-Dist: certifi; extra == 'dev'
106
+ Requires-Dist: dockerfile-parse>=2.0.1; extra == 'dev'
107
+ Requires-Dist: harbor==0.25.0; extra == 'dev'
108
+ Requires-Dist: httpx; extra == 'dev'
109
+ Requires-Dist: numpy; extra == 'dev'
110
+ Requires-Dist: openhands-sdk==1.7.1; extra == 'dev'
111
+ Requires-Dist: openhands-tools==1.7.1; extra == 'dev'
112
+ Requires-Dist: parameterized; extra == 'dev'
113
+ Requires-Dist: pathspec>=1.0.3; extra == 'dev'
114
+ Requires-Dist: pre-commit; extra == 'dev'
115
+ Requires-Dist: pytest; extra == 'dev'
116
+ Requires-Dist: pytest-asyncio; extra == 'dev'
117
+ Requires-Dist: pytest-flakefinder; extra == 'dev'
118
+ Requires-Dist: pytest-instafail; extra == 'dev'
119
+ Requires-Dist: pytest-sugar; extra == 'dev'
120
+ Requires-Dist: pytest-timeout; extra == 'dev'
121
+ Requires-Dist: pytest-xdist; extra == 'dev'
122
+ Requires-Dist: requests; extra == 'dev'
123
+ Requires-Dist: safetensors; extra == 'dev'
124
+ Requires-Dist: snowflake-sandbox-python==0.2.2a5; extra == 'dev'
125
+ Requires-Dist: tenacity; extra == 'dev'
126
+ Requires-Dist: tinker==0.25.0; extra == 'dev'
127
+ Requires-Dist: torch; extra == 'dev'
128
+ Requires-Dist: transformers; extra == 'dev'
129
+ Requires-Dist: urllib3; extra == 'dev'
130
+ Provides-Extra: formatting
131
+ Requires-Dist: autoflake; extra == 'formatting'
132
+ Requires-Dist: pre-commit; extra == 'formatting'
133
+ Provides-Extra: gdn
134
+ Requires-Dist: causal-conv1d; extra == 'gdn'
135
+ Requires-Dist: flash-linear-attention; extra == 'gdn'
136
+ Provides-Extra: harbor
137
+ Requires-Dist: certifi; extra == 'harbor'
138
+ Requires-Dist: dockerfile-parse>=2.0.1; extra == 'harbor'
139
+ Requires-Dist: harbor==0.25.0; extra == 'harbor'
140
+ Requires-Dist: pathspec>=1.0.3; extra == 'harbor'
141
+ Requires-Dist: snowflake-sandbox-python==0.2.2a5; extra == 'harbor'
142
+ Provides-Extra: inference
143
+ Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'inference'
144
+ Requires-Dist: fastapi; extra == 'inference'
145
+ Requires-Dist: kvcached==0.1.4; extra == 'inference'
146
+ Requires-Dist: pydantic>=2.0; extra == 'inference'
147
+ Requires-Dist: ray; extra == 'inference'
148
+ Requires-Dist: tqdm; extra == 'inference'
149
+ Requires-Dist: uvicorn; extra == 'inference'
150
+ Requires-Dist: vllm==0.33.0; extra == 'inference'
151
+ Requires-Dist: xgrammar==0.2.8; extra == 'inference'
152
+ Provides-Extra: onprem
153
+ Requires-Dist: deepspeed>=0.19.9; extra == 'onprem'
154
+ Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'onprem'
155
+ Requires-Dist: fastapi; extra == 'onprem'
156
+ Requires-Dist: huggingface-hub<3,>=2.0; extra == 'onprem'
157
+ Requires-Dist: kvcached==0.1.4; extra == 'onprem'
158
+ Requires-Dist: liger-kernel>=0.8.5; extra == 'onprem'
159
+ Requires-Dist: numpy; extra == 'onprem'
160
+ Requires-Dist: nvidia-ml-py; extra == 'onprem'
161
+ Requires-Dist: peft<0.22,>=0.21.2; extra == 'onprem'
162
+ Requires-Dist: psutil; extra == 'onprem'
163
+ Requires-Dist: pydantic>=2.0; extra == 'onprem'
164
+ Requires-Dist: quack-kernels>=0.4.0; extra == 'onprem'
165
+ Requires-Dist: ray; extra == 'onprem'
166
+ Requires-Dist: safetensors; extra == 'onprem'
167
+ Requires-Dist: tensordict; extra == 'onprem'
168
+ Requires-Dist: torch>=2.9; extra == 'onprem'
169
+ Requires-Dist: tqdm; extra == 'onprem'
170
+ Requires-Dist: transformers==5.20.2; extra == 'onprem'
171
+ Requires-Dist: uvicorn; extra == 'onprem'
172
+ Requires-Dist: vllm==0.33.0; extra == 'onprem'
173
+ Requires-Dist: xgrammar==0.2.8; extra == 'onprem'
174
+ Provides-Extra: prime-rl
175
+ Requires-Dist: aiohttp; extra == 'prime-rl'
176
+ Requires-Dist: httpx; extra == 'prime-rl'
177
+ Requires-Dist: huggingface-hub; extra == 'prime-rl'
178
+ Requires-Dist: numpy; extra == 'prime-rl'
179
+ Requires-Dist: orjson; extra == 'prime-rl'
180
+ Requires-Dist: requests; extra == 'prime-rl'
181
+ Requires-Dist: safetensors; extra == 'prime-rl'
182
+ Requires-Dist: tenacity; extra == 'prime-rl'
183
+ Requires-Dist: torch; extra == 'prime-rl'
184
+ Requires-Dist: urllib3; extra == 'prime-rl'
185
+ Provides-Extra: rl
186
+ Requires-Dist: deepspeed>=0.19.9; extra == 'rl'
187
+ Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'rl'
188
+ Requires-Dist: fastapi; extra == 'rl'
189
+ Requires-Dist: huggingface-hub<3,>=2.0; extra == 'rl'
190
+ Requires-Dist: kvcached==0.1.4; extra == 'rl'
191
+ Requires-Dist: liger-kernel>=0.8.5; extra == 'rl'
192
+ Requires-Dist: numpy; extra == 'rl'
193
+ Requires-Dist: nvidia-ml-py; extra == 'rl'
194
+ Requires-Dist: peft<0.22,>=0.21.2; extra == 'rl'
195
+ Requires-Dist: psutil; extra == 'rl'
196
+ Requires-Dist: pydantic>=2.0; extra == 'rl'
197
+ Requires-Dist: quack-kernels>=0.4.0; extra == 'rl'
198
+ Requires-Dist: ray; extra == 'rl'
199
+ Requires-Dist: safetensors; extra == 'rl'
200
+ Requires-Dist: tensordict; extra == 'rl'
201
+ Requires-Dist: torch>=2.9; extra == 'rl'
202
+ Requires-Dist: tqdm; extra == 'rl'
203
+ Requires-Dist: transformers==5.20.2; extra == 'rl'
204
+ Requires-Dist: uvicorn; extra == 'rl'
205
+ Requires-Dist: vllm==0.33.0; extra == 'rl'
206
+ Requires-Dist: xgrammar==0.2.8; extra == 'rl'
207
+ Provides-Extra: sft
208
+ Requires-Dist: deepspeed>=0.19.9; extra == 'sft'
209
+ Requires-Dist: huggingface-hub<3,>=2.0; extra == 'sft'
210
+ Requires-Dist: liger-kernel>=0.8.5; extra == 'sft'
211
+ Requires-Dist: numpy; extra == 'sft'
212
+ Requires-Dist: nvidia-ml-py; extra == 'sft'
213
+ Requires-Dist: peft<0.22,>=0.21.2; extra == 'sft'
214
+ Requires-Dist: psutil; extra == 'sft'
215
+ Requires-Dist: quack-kernels>=0.4.0; extra == 'sft'
216
+ Requires-Dist: ray; extra == 'sft'
217
+ Requires-Dist: safetensors; extra == 'sft'
218
+ Requires-Dist: tensordict; extra == 'sft'
219
+ Requires-Dist: torch>=2.9; extra == 'sft'
220
+ Requires-Dist: transformers==5.20.2; extra == 'sft'
221
+ Provides-Extra: sonicmoe
222
+ Requires-Dist: quack-kernels>=0.4.0; extra == 'sonicmoe'
223
+ Provides-Extra: testing
224
+ Requires-Dist: aiohttp; extra == 'testing'
225
+ Requires-Dist: certifi; extra == 'testing'
226
+ Requires-Dist: dockerfile-parse>=2.0.1; extra == 'testing'
227
+ Requires-Dist: harbor==0.25.0; extra == 'testing'
228
+ Requires-Dist: httpx; extra == 'testing'
229
+ Requires-Dist: numpy; extra == 'testing'
230
+ Requires-Dist: openhands-sdk==1.7.1; extra == 'testing'
231
+ Requires-Dist: openhands-tools==1.7.1; extra == 'testing'
232
+ Requires-Dist: parameterized; extra == 'testing'
233
+ Requires-Dist: pathspec>=1.0.3; extra == 'testing'
234
+ Requires-Dist: pytest; extra == 'testing'
235
+ Requires-Dist: pytest-asyncio; extra == 'testing'
236
+ Requires-Dist: pytest-flakefinder; extra == 'testing'
237
+ Requires-Dist: pytest-instafail; extra == 'testing'
238
+ Requires-Dist: pytest-sugar; extra == 'testing'
239
+ Requires-Dist: pytest-timeout; extra == 'testing'
240
+ Requires-Dist: pytest-xdist; extra == 'testing'
241
+ Requires-Dist: requests; extra == 'testing'
242
+ Requires-Dist: safetensors; extra == 'testing'
243
+ Requires-Dist: snowflake-sandbox-python==0.2.2a5; extra == 'testing'
244
+ Requires-Dist: tenacity; extra == 'testing'
245
+ Requires-Dist: tinker==0.25.0; extra == 'testing'
246
+ Requires-Dist: torch; extra == 'testing'
247
+ Requires-Dist: transformers; extra == 'testing'
248
+ Requires-Dist: urllib3; extra == 'testing'
249
+ Provides-Extra: tinker
250
+ Requires-Dist: aiohttp; extra == 'tinker'
251
+ Requires-Dist: numpy; extra == 'tinker'
252
+ Requires-Dist: requests; extra == 'tinker'
253
+ Requires-Dist: safetensors; extra == 'tinker'
254
+ Requires-Dist: tenacity; extra == 'tinker'
255
+ Requires-Dist: torch; extra == 'tinker'
256
+ Requires-Dist: transformers; extra == 'tinker'
257
+ Requires-Dist: urllib3; extra == 'tinker'
258
+ Provides-Extra: trl
259
+ Requires-Dist: accelerate; extra == 'trl'
260
+ Requires-Dist: deepspeed>=0.19.9; extra == 'trl'
261
+ Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'trl'
262
+ Requires-Dist: fastapi; extra == 'trl'
263
+ Requires-Dist: huggingface-hub<3,>=2.0; extra == 'trl'
264
+ Requires-Dist: kvcached==0.1.4; extra == 'trl'
265
+ Requires-Dist: liger-kernel>=0.8.5; extra == 'trl'
266
+ Requires-Dist: numpy; extra == 'trl'
267
+ Requires-Dist: nvidia-ml-py; extra == 'trl'
268
+ Requires-Dist: peft<0.22,>=0.21.2; extra == 'trl'
269
+ Requires-Dist: psutil; extra == 'trl'
270
+ Requires-Dist: pydantic>=2.0; extra == 'trl'
271
+ Requires-Dist: quack-kernels>=0.4.0; extra == 'trl'
272
+ Requires-Dist: ray; extra == 'trl'
273
+ Requires-Dist: safetensors; extra == 'trl'
274
+ Requires-Dist: tensordict; extra == 'trl'
275
+ Requires-Dist: torch>=2.9; extra == 'trl'
276
+ Requires-Dist: tqdm; extra == 'trl'
277
+ Requires-Dist: transformers==5.20.2; extra == 'trl'
278
+ Requires-Dist: uvicorn; extra == 'trl'
279
+ Requires-Dist: vllm==0.33.0; extra == 'trl'
280
+ Requires-Dist: xgrammar==0.2.8; extra == 'trl'
281
+ Provides-Extra: verl
282
+ Requires-Dist: codetiming; extra == 'verl'
283
+ Requires-Dist: deepspeed>=0.19.9; extra == 'verl'
284
+ Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'verl'
285
+ Requires-Dist: fastapi; extra == 'verl'
286
+ Requires-Dist: huggingface-hub<3,>=2.0; extra == 'verl'
287
+ Requires-Dist: kvcached==0.1.4; extra == 'verl'
288
+ Requires-Dist: liger-kernel>=0.8.5; extra == 'verl'
289
+ Requires-Dist: numpy; extra == 'verl'
290
+ Requires-Dist: nvidia-ml-py; extra == 'verl'
291
+ Requires-Dist: omegaconf; extra == 'verl'
292
+ Requires-Dist: peft<0.22,>=0.21.2; extra == 'verl'
293
+ Requires-Dist: psutil; extra == 'verl'
294
+ Requires-Dist: pydantic>=2.0; extra == 'verl'
295
+ Requires-Dist: quack-kernels>=0.4.0; extra == 'verl'
296
+ Requires-Dist: ray; extra == 'verl'
297
+ Requires-Dist: safetensors; extra == 'verl'
298
+ Requires-Dist: tensordict; extra == 'verl'
299
+ Requires-Dist: torch>=2.9; extra == 'verl'
300
+ Requires-Dist: tqdm; extra == 'verl'
301
+ Requires-Dist: transformers==5.20.2; extra == 'verl'
302
+ Requires-Dist: uvicorn; extra == 'verl'
303
+ Requires-Dist: vllm==0.33.0; extra == 'verl'
304
+ Requires-Dist: xgrammar==0.2.8; extra == 'verl'
305
+ Description-Content-Type: text/markdown
306
+
307
+ [![License Apache 2.0](https://badgen.net/badge/license/apache2.0/blue)](https://github.com/Snowflake-AI-Research/Arctic-Platform/blob/main/LICENSE)
308
+ [![PyPI version](https://badge.fury.io/py/arctic-platform.svg)](https://pypi.org/project/arctic-platform/)
309
+
310
+ <h3 align="center">
311
+ <img src="docs/images/arctic_platform_logo.svg" width=500px><br>
312
+ </h3>
313
+
314
+ # Arctic Platform: Simplifying and Accelerating Post-Training for LLMs
315
+
316
+ Arctic Platform is a framework for addressing challenges in current post-training frameworks, such as limited support for rapid prototyping and the lack of native data generation tools, by offering modularity across training and inference components, simplified code structures, and integrated pipelines for creating and cleaning synthetic data. These features enable users to enhance LLM capabilities, like code generation and complex reasoning, with greater efficiency and flexibility.
317
+
318
+ For example in the RL world if one were to compare various frameworks, one will find that each framework implements its own backends, adds fragmented system optimizations and the overall performance isn't quite portable from framework to framework. Thus the intention is to provide consistent building blocks that can be easily integrated into multiple frameworks while providing:
319
+
320
+ - Unified GPU Backends, e.g. RL on-prem or remote backends
321
+ - Unified System Optimizations leading to High Performance Portability
322
+
323
+ This is a work in progress, starting with the RL components, later integrating more training and inference components.
324
+
325
+ # Project Scope
326
+
327
+ Arctic Platform aims to cover the full post-training stack for LLMs behind a small, composable API. The codebase is being built out incrementally.
328
+
329
+ What is available today:
330
+
331
+ * [**Arctic Reinforcement Learning**](#arctic-reinforcement-learning) — a high-throughput RL training/inference backend that plugs into existing RL frameworks ([docs/rl.md](docs/rl.md)).
332
+ * [**ZoRRo Train**](#zorro-train) — a prompt-deduplication optimization that removes redundant prompt computation during RL training ([docs/rl.md#zorro-train](docs/rl.md#zorro-train)).
333
+ * [**ZoRRo Inference**](#zorro-inference) — forest cascade attention for efficient rollout step that eliminates redundant memory accesses via grouping ([docs/rl.md#zorro-inference-forest-cascade-attention](docs/rl.md#zorro-inference-forest-cascade-attention)).
334
+
335
+ Full documentation index: [docs/index.md](docs/index.md) (RL, shared server infra; SFT docs forthcoming).
336
+
337
+ What's coming next:
338
+
339
+ * additional trainers (SFT/distillation), synthetic data generation and cleaning pipelines, and tighter inference integration.
340
+
341
+ ## Arctic Reinforcement Learning
342
+
343
+ Arctic RL is designed to **integrate into existing RL frameworks** rather than replace them. The RL framework keeps ownership of the training loop, rollouts, rewards, and advantage estimation; Arctic Platform provides the heavy compute engines behind a thin client:
344
+
345
+ * **Training engine** — a DeepSpeed engine that runs forward/backward and the optimizer step.
346
+ * **Log-prob / reference engine** — a forward-only DeepSpeed engine for reference / old log-prob computation.
347
+ * **Sampling engine** — a vLLM engine with ArcticInference for fast rollouts.
348
+
349
+ These engines are orchestrated over [Ray](https://www.ray.io/), can be **colocated** on shared GPUs (via fractional Ray resources) or split across separate GPUs, and keep the sampler in sync with the trainer through NCCL or CUDA-IPC weight transfer. The RL framework interacts with the compute engines over Ray. ArcticInference extends the core optimizations with multi-replica scheduling, load-balancing, weight-sync, and router-replay.
350
+
351
+ An RL framework integrates this module by constructing a client and driving the standard operations (generate, forward/backward, optimizer step, sync_weights, and wake/sleep for memory management). For example, on a single 8-GPU node one could do:
352
+
353
+ ```python
354
+ from arctic_platform.rl import ArcticRLClientConfig, create_arctic_rl_client
355
+
356
+ config = ArcticRLClientConfig(
357
+ model_name="Qwen/Qwen3-4B",
358
+ training_gpus=8, #
359
+ sampling_gpus=8,
360
+ log_prob_gpus=0,
361
+ colocate=True,
362
+ )
363
+ client = create_arctic_rl_client(config)
364
+ ```
365
+
366
+ ### Integrated
367
+
368
+ 1. [SkyRL](https://github.com/NovaSky-AI/SkyRL/pull/1837). [Documentation](https://github.com/NovaSky-AI/SkyRL/tree/main/integrations/arctic_rl)
369
+ 2. TRL on Cortex. GSM8K GRPO from a CPU driver: [recipes/rl/trl/gsm8k_cortex](recipes/rl/trl/gsm8k_cortex).
370
+
371
+ ### Integration is done but not yet merged
372
+
373
+ 1. Verl. The reference integration is [verl](https://github.com/verl-project/verl) ([https://github.com/verl-project/verl/pull/6422](https://github.com/verl-project/verl/pull/6422)), which drives Arctic RL from its PPO/GRPO trainer. End-to-end recipes live under [recipes/rl/verl](recipes/rl/verl/), including a [simple single-GPU GSM8K](https://github.com/Snowflake-AI-Research/Arctic-Platform/tree/main/recipes/rl/verl/simple) starter, [Txt2SQL](https://github.com/Snowflake-AI-Research/Arctic-Platform/tree/main/recipes/rl/verl/txt2sql) and [long-context QA](https://github.com/Snowflake-AI-Research/Arctic-Platform/tree/main/recipes/rl/verl/long_context_qa).
374
+ 2. TRL. The adapter lives under [`arctic_platform/integrations/trl/`](arctic_platform/integrations/trl/) and drives Arctic RL from TRL's `AsyncGRPOTrainer`. End-to-end recipes live under [recipes/rl/trl](recipes/rl/trl/), including a [GSM8K](recipes/rl/trl/gsm8k) starter and [Txt2SQL (BIRD)](recipes/rl/trl/txt2sql).
375
+
376
+ ### Upcoming integrations
377
+
378
+ Multiple additional frameworks integrations are in the works and will be added here once available. We are in various stages of planning or working on integration into Axolotl, unsloth, PrimeRL and potentially others.
379
+
380
+ ### ZoRRo Train
381
+
382
+ ZoRRo stands for Zero Redundancy Rollouts.
383
+
384
+ In RL training (PPO/GRPO) the same prompt is sampled many times to explore different responses, therefore for long sequence tasks **80–95% of the tokens in a batch are redundant prompt tokens** — and with transformer attention’s O(n²) cost, recomputing those shared prompts dominates the bill for long-context RL.
385
+
386
+ **ZoRRo Train** eliminates that waste with automatic **prompt deduplication** at all levels: it detects sequences that share a prompt, packs each unique prompt once, runs the model a single time over the deduplicated sequence, and transparently reconstructs per-response logprobs/entropy in the original sample order. The result is mathematically equivalent to the naive forward/backward (gradients match the baseline within numerical precision) while substantially cutting memory use and increasing throughput — the longer and more-shared the prompts, the larger the win.
387
+
388
+ It is installed transparently by the DeepSpeed training/log-prob engines and toggled per run via `ds_worker_config.zorro_train_enable` (verl yaml: `remote_backend.train.zorro_train.enable`). See [docs/rl.md § ZoRRo Train](docs/rl.md#zorro-train).
389
+
390
+ The supported model families span dense and MoE models - see the list [here](arctic_platform/rl/zorro_train/README.md#supported-models).
391
+
392
+ See [arctic_platform/rl/zorro_train/README.md](arctic_platform/rl/zorro_train/README.md) for the full design, the deduplication/attention internals, and benchmarks.
393
+
394
+ ### ZoRRo Inference
395
+
396
+ During RL rollouts many sequences are generated from the same prompt. In the decode step, standard attention re-reads the KV cache of those shared prefixes **once per request**, so the sampler spends most of its memory bandwidth fetching identical keys and values over and over.
397
+
398
+ **ZoRRo Inference** removes that waste with **Forest Cascade Attention (FCA)**, which deduplicates shared KV reads at the attention layer of the sampling engine. For each decode batch, FCA discovers groups of requests that share a KV-cache prefix. Then it splits each attention call into a single grouped pass over the shared prefix blocks plus a per-request pass over the unique suffix blocks. FCA then reduces the two partial results with rigorous weighting. This reads each shared prefix block once per *group* instead of once per *request*, cutting redundant memory accesses while remaining mathematically equivalent to standard attention — the longer and more-shared the prefixes, the larger the win.
399
+
400
+ It is implemented in the vLLM sampling engine and activates transparently for decode-heavy batches with shared prefixes.
401
+
402
+ See the [Forest Cascade Attention README](https://github.com/snowflakedb/ArcticInference/tree/main/arctic_inference/vllm/attention) in Arctic Inference for the full design, the grouping/attention internals, and the tuning knobs.
403
+
404
+ # Quickstart
405
+
406
+ To get started training a model with Arctic Platform, first [install the package](#installation), then follow [the recipes](recipes/rl/).
407
+
408
+ # Installation
409
+
410
+ ## From PyPI
411
+
412
+ The base install carries the config models only. Pick the extra for the backend you are training against, from [PyPI](https://pypi.org/project/arctic-platform/):
413
+
414
+ ```shell
415
+ pip install "arctic-platform[cortex]" # drive Cortex training over SnowAPI
416
+ pip install "arctic-platform[sft]" # run a local training-only server
417
+ pip install "arctic-platform[rl]" # ...plus the sampling stack (arctic-inference, vLLM)
418
+ pip install "arctic-platform[verl]" # the verl adapter
419
+ pip install "arctic-platform[trl]" # the TRL adapter
420
+ ```
421
+
422
+ `[cortex]` is by far the lightest: it skips DeepSpeed, transformers, Ray and vLLM entirely. Importing on-prem code without the matching extra raises an error naming the one to install.
423
+
424
+ While this project is being very actively developed it's probably better to install directly from [git](#from-source-git).
425
+
426
+ ## From source (git)
427
+
428
+ To get the latest development version (or to contribute), clone the repository and install it in editable mode:
429
+
430
+ ```shell
431
+ git clone https://github.com/Snowflake-AI-Research/Arctic-Platform.git
432
+ cd Arctic-Platform
433
+ pip install -e ".[rl]"
434
+ ```
435
+
436
+ # Citation
437
+
438
+ If you use Arctic Platform in your research, please consider citing:
439
+
440
+ ```bibtex
441
+ @misc{arctic_platform_2025,
442
+ title={Arctic Platform: Simplifying and Accelerating Post-Training for LLMs},
443
+ author={Snowflake AI Research},
444
+ year={2025},
445
+ howpublished={\url{https://github.com/Snowflake-AI-Research/Arctic-Platform}}
446
+ }
447
+ ```
448
+
449
+ # License
450
+
451
+ Apache License 2.0. See [LICENSE](https://github.com/Snowflake-AI-Research/Arctic-Platform/blob/main/LICENSE).