arctic-platform 0.1.1.dev0__tar.gz → 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- arctic_platform-0.2.1/.gitignore +190 -0
- arctic_platform-0.2.1/PKG-INFO +451 -0
- arctic_platform-0.2.1/README.md +145 -0
- arctic_platform-0.2.1/arctic_platform/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/_dependency_groups.py +91 -0
- arctic_platform-0.2.1/arctic_platform/client/UNIFICATION_NOTES.md +145 -0
- arctic_platform-0.2.1/arctic_platform/client/__init__.py +51 -0
- arctic_platform-0.2.1/arctic_platform/client/base.py +659 -0
- arctic_platform-0.2.1/arctic_platform/client/config.py +452 -0
- arctic_platform-0.2.1/arctic_platform/client/requests.py +289 -0
- arctic_platform-0.2.1/arctic_platform/client/rl.py +42 -0
- arctic_platform-0.2.1/arctic_platform/client/sft.py +107 -0
- arctic_platform-0.2.1/arctic_platform/client/transport.py +192 -0
- arctic_platform-0.2.1/arctic_platform/client/transports/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/client/transports/cortex.py +903 -0
- arctic_platform-0.2.1/arctic_platform/client/transports/onprem.py +223 -0
- arctic_platform-0.2.1/arctic_platform/common/__init__.py +61 -0
- arctic_platform-0.2.1/arctic_platform/common/balance_metadata.py +177 -0
- arctic_platform-0.2.1/arctic_platform/common/callbacks.py +217 -0
- arctic_platform-0.2.1/arctic_platform/common/chunked_cross_entropy.py +92 -0
- arctic_platform-0.2.1/arctic_platform/common/config.py +93 -0
- arctic_platform-0.2.1/arctic_platform/common/cpu_adam.py +135 -0
- arctic_platform-0.2.1/arctic_platform/common/deepspeed_engine.py +532 -0
- arctic_platform-0.2.1/arctic_platform/common/deepspeed_worker.py +1267 -0
- arctic_platform-0.2.1/arctic_platform/common/flops_counter.py +452 -0
- arctic_platform-0.2.1/arctic_platform/common/model_options.py +260 -0
- arctic_platform-0.2.1/arctic_platform/common/optimizer.py +88 -0
- arctic_platform-0.2.1/arctic_platform/common/option_registry.py +552 -0
- arctic_platform-0.2.1/arctic_platform/common/packing/__init__.py +114 -0
- arctic_platform-0.2.1/arctic_platform/common/packing/groups.py +235 -0
- arctic_platform-0.2.1/arctic_platform/common/packing/pack.py +223 -0
- arctic_platform-0.2.1/arctic_platform/common/packing/walk.py +50 -0
- arctic_platform-0.2.1/arctic_platform/common/packing/windows.py +158 -0
- arctic_platform-0.2.1/arctic_platform/common/peft.py +109 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/__init__.py +56 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/deepseek_v4.py +37 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/deepseek_v41.py +37 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/dflash2.py +68 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/generic_moe.py +64 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/glm5_next.py +38 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/glm_moe_dsa.py +62 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/huggingface.py +81 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/inkling.py +33 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/qwen3_5_moe.py +64 -0
- arctic_platform-0.2.1/arctic_platform/common/profiles/qwen4_exp.py +41 -0
- {arctic_platform-0.1.1.dev0/arctic_platform/rl → arctic_platform-0.2.1/arctic_platform/common}/ray_cluster.py +78 -22
- arctic_platform-0.2.1/arctic_platform/common/ray_server.py +1650 -0
- arctic_platform-0.2.1/arctic_platform/common/registry.py +211 -0
- arctic_platform-0.2.1/arctic_platform/common/server.py +22 -0
- arctic_platform-0.2.1/arctic_platform/common/text_config_overrides.py +48 -0
- arctic_platform-0.2.1/arctic_platform/common/utils/__init__.py +68 -0
- arctic_platform-0.2.1/arctic_platform/common/utils/balanced_batch.py +421 -0
- arctic_platform-0.2.1/arctic_platform/common/utils/batch.py +712 -0
- arctic_platform-0.2.1/arctic_platform/common/utils/bf16_zero_norm.py +49 -0
- arctic_platform-0.2.1/arctic_platform/common/utils/checkpoint.py +62 -0
- {arctic_platform-0.1.1.dev0/arctic_platform/rl → arctic_platform-0.2.1/arctic_platform/common}/utils/debug.py +23 -35
- {arctic_platform-0.1.1.dev0/arctic_platform/rl → arctic_platform-0.2.1/arctic_platform/common}/utils/ray_pg.py +39 -17
- arctic_platform-0.2.1/arctic_platform/common/utils/record_replay.py +190 -0
- arctic_platform-0.2.1/arctic_platform/common/utils/server_models.py +172 -0
- arctic_platform-0.2.1/arctic_platform/common/utils/tiled_logits.py +429 -0
- arctic_platform-0.2.1/arctic_platform/common/utils/weight_sync.py +222 -0
- arctic_platform-0.2.1/arctic_platform/common/weight_checksum.py +103 -0
- arctic_platform-0.2.1/arctic_platform/correctness/__init__.py +26 -0
- arctic_platform-0.2.1/arctic_platform/correctness/__main__.py +23 -0
- arctic_platform-0.2.1/arctic_platform/correctness/api.py +256 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/__init__.py +35 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/checkpoint_resume.py +305 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/e2e_sft_train_validate.py +867 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/fwd_bwd.py +179 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/fwd_bwd_step.py +150 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/inference_checkpoint_loss.py +1153 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/optimizer_state.py +425 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/rl_router_replay.py +568 -0
- arctic_platform-0.2.1/arctic_platform/correctness/checks/rl_weight_sync.py +518 -0
- arctic_platform-0.2.1/arctic_platform/correctness/config-layout.json +14 -0
- arctic_platform-0.2.1/arctic_platform/correctness/diagnostics/__main__.py +60 -0
- arctic_platform-0.2.1/arctic_platform/correctness/diagnostics/lm_head_chunking.py +71 -0
- arctic_platform-0.2.1/arctic_platform/correctness/docs/code-inventory.md +31 -0
- arctic_platform-0.2.1/arctic_platform/correctness/docs/getting-started.md +260 -0
- arctic_platform-0.2.1/arctic_platform/correctness/docs/hf-reference-onboarding.md +133 -0
- arctic_platform-0.2.1/arctic_platform/correctness/docs/running-tests.md +392 -0
- arctic_platform-0.2.1/arctic_platform/correctness/docs/tests.md +186 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/__init__.py +16 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/config.py +1017 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/cortex.py +1175 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/engine_admission.py +570 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/execution.py +1521 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/gateway.py +2662 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/inference.py +1895 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/lora.py +2081 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/native_optimizer.py +336 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/optimizer.py +4626 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/registry.py +174 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/report.py +741 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/runner.py +1384 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/sft.py +460 -0
- arctic_platform-0.2.1/arctic_platform/correctness/harness/storage.py +1361 -0
- arctic_platform-0.2.1/arctic_platform/correctness/models/__init__.py +20 -0
- arctic_platform-0.2.1/arctic_platform/correctness/models/original.py +294 -0
- arctic_platform-0.2.1/arctic_platform/correctness/onboarding/__init__.py +21 -0
- arctic_platform-0.2.1/arctic_platform/correctness/onboarding/calibration.py +215 -0
- arctic_platform-0.2.1/arctic_platform/correctness/onboarding/cortex_fixture_upload.py +82 -0
- arctic_platform-0.2.1/arctic_platform/correctness/onboarding/onboard.py +3553 -0
- arctic_platform-0.2.1/arctic_platform/correctness/onboarding/synth_model.py +537 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/__init__.py +16 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/__main__.py +144 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/adamw_trajectory.py +245 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/hf_single_gpu.py +596 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/model_features.py +32 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/optimizer_step.py +271 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/process_determinism.py +83 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/qwen4_reference.py +91 -0
- arctic_platform-0.2.1/arctic_platform/correctness/reference/sft_trajectory.py +361 -0
- arctic_platform-0.2.1/arctic_platform/correctness/selftest/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/correctness/selftest/check_suite.py +38 -0
- arctic_platform-0.2.1/arctic_platform/correctness/selftest/library.py +31664 -0
- arctic_platform-0.2.1/arctic_platform/debug/training.py +373 -0
- arctic_platform-0.2.1/arctic_platform/debug/weight_sync.py +1629 -0
- arctic_platform-0.2.1/arctic_platform/inference/INTERNAL_COMMIT.txt +1 -0
- arctic_platform-0.2.1/arctic_platform/inference/__init__.py +15 -0
- arctic_platform-0.2.1/arctic_platform/inference/common/__init__.py +15 -0
- arctic_platform-0.2.1/arctic_platform/inference/common/swiftkv/__init__.py +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/common/swiftkv/configs.py +37 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/CMakeLists.txt +40 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/attention_generic.cuh +59 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/custom_ops.h +41 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/dispatch_utils.h +69 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/dtype_common.cuh +72 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/dtype_fp8.cuh +37 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/quant_utils.cuh +571 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/reshape_and_cache_flash_bulk.cu +144 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/reshape_and_cache_flash_fp4.cu +338 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/speculator_ln.cu +252 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/sum_lstm.cu +473 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/torch_bindings.cpp +53 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/vectorization.cuh +29 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/custom_ops/vectorization_utils.cuh +176 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/CMakeLists.txt +38 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/bindings.cc +103 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/int32_map.h +449 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/suffix_tree.cc +933 -0
- arctic_platform-0.2.1/arctic_platform/inference/csrc/suffix_decoding/suffix_tree.h +179 -0
- arctic_platform-0.2.1/arctic_platform/inference/dynasor/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/inference/dynasor/cot.py +190 -0
- arctic_platform-0.2.1/arctic_platform/inference/dynasor/entropy.py +250 -0
- arctic_platform-0.2.1/arctic_platform/inference/dynasor/evaluator.py +852 -0
- arctic_platform-0.2.1/arctic_platform/inference/dynasor/openai_server.py +482 -0
- arctic_platform-0.2.1/arctic_platform/inference/dynasor/util.py +70 -0
- arctic_platform-0.2.1/arctic_platform/inference/dynasor/vllm_server.py +130 -0
- arctic_platform-0.2.1/arctic_platform/inference/embedding/README.md +169 -0
- arctic_platform-0.2.1/arctic_platform/inference/embedding/client.py +306 -0
- arctic_platform-0.2.1/arctic_platform/inference/embedding/generate_proto.py +79 -0
- arctic_platform-0.2.1/arctic_platform/inference/embedding/proto/inference.proto +91 -0
- arctic_platform-0.2.1/arctic_platform/inference/embedding/replica.py +509 -0
- arctic_platform-0.2.1/arctic_platform/inference/embedding/replica_manager.py +509 -0
- arctic_platform-0.2.1/arctic_platform/inference/envs.py +75 -0
- arctic_platform-0.2.1/arctic_platform/inference/op_builder/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/inference/op_builder/builder.py +545 -0
- arctic_platform-0.2.1/arctic_platform/inference/op_builder/swiftkv_ops_builder.py +29 -0
- arctic_platform-0.2.1/arctic_platform/inference/patching.py +139 -0
- arctic_platform-0.2.1/arctic_platform/inference/py_custom_ops.py +117 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/README.md +179 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/__init__.py +47 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/_semip_worker.py +224 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/abstract.py +565 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/assets/figure1-sleep-wake-latency.png +0 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/ca_graph_rebind.py +4661 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/client.py +827 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/dashboard.py +2594 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/demuxer.py +224 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/instance.py +1153 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/matched_baseline.py +86 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/mq_plane.py +371 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/multinode.py +107 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/nccl_bootstrap.py +185 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/README.md +324 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/__init__.py +16 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/book.py +319 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/cleanup.py +80 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/client.py +69 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/common.py +127 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/correctness.py +244 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/jobs.py +264 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/backends.json +12 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/glm53-tp16.json +20 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/glm53-tp8.json +20 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen36-35b-a3b-tp1.json +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen36-35b-a3b-tp1x8.json +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen36-35b-a3b-tp2x4.json +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen36-35b-a3b-tp4x2.json +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen38-27b-tp1.json +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen38-27b-tp1x8.json +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen38-27b-tp2x4.json +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/models/qwen38-flash-next-tp8.json +22 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/onboard.py +458 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/perf.py +326 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/publish.py +261 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/report.py +278 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/rereport.py +126 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/startup.py +301 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/tests/test_onboarding.py +582 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/onboarding/verify.py +102 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/orch_server.py +55 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/orchestrator.py +1942 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/pipeline.py +1570 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/reproduce/README.md +19 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/reproduce/example_full.py +165 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/imgdiff.py +309 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/main_test.py +141 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/pidcheck.py +326 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/pidref_test.py +189 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/register.py +72 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/semip_publish.py +1474 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_copy.py +97 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_env.py +121 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_generate.py +66 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_image.py +40 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_tp2.py +86 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/scripts/test_weights.py +77 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/semip_logging.py +439 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/CRIU_PLUMBING.md +2163 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/CROSS_NODE_RESTORE.md +476 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/IMAGE_CACHE.md +1099 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/INSTALL.md +317 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/MULTINODE_TP16.md +324 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/SKILL.md +480 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/TEARDOWN_SCOPING.md +559 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/async_generate_DETAILS.md +1195 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/client_DESIGN.md +306 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/dss_dump.md +819 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/dss_integration.md +1620 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/instance_DESIGN.md +925 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/orchestrator_DESIGN.md +2312 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/pipeline_DESIGN.md +985 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/reference.md +395 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/semi-p_DESIGN.md +256 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/slots_DESIGN.md +373 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/skills/tp_DESIGN.md +573 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/slots.py +325 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/state_server.py +394 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_allreduce_pinning.py +203 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_communicator_checkpoint_hooks.py +1054 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_device_visibility.py +309 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_double_buffer.py +142 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_dump_publish.py +394 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_dump_slots.py +84 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_graph_discovery.py +1006 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_image_cache_key.py +1449 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_layout_names.py +294 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_listener_close.py +479 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_lora_host_copy.py +164 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_matched_baseline.py +166 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_multinode.py +507 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_parallel_state_reset.py +176 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_pid_floor.py +495 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_pipeline.py +1204 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_pod_log.py +345 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_publish_layout.py +968 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_restore_nccl_env.py +94 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_restore_quiescence.py +304 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_semip_frontend.py +752 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_serve_core.py +409 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_sleep_guard.py +120 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_slots.py +212 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/tests/test_worker_initialize.py +267 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/vllm_child.py +5292 -0
- arctic_platform-0.2.1/arctic_platform/inference/semi_persistence/worker.py +2816 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/__init__.py +16 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/action_mask_replay.py +727 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/action_masks.py +131 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/api.py +184 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/chat.py +834 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/cli.py +30 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/config.py +234 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/metrics.py +284 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/multi_model.py +414 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/pipeline.py +156 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/replica_pool.py +2309 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/router_replay/__init__.py +34 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/router_replay/all2all.py +613 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/router_replay/cache.py +601 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/router_replay/shm.py +180 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/scheduler.py +573 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/semip_agent.py +312 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/semip_asyncllm.py +169 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/semip_engine.py +3336 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/semip_frontend.py +318 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/streaming.py +1448 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/testing.py +271 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/token_prompts.py +49 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/__init__.py +40 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/broadcast.py +29 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/engine.py +633 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/receiver.py +1407 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/schedule.py +159 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/sender.py +287 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/weight_sync/utils.py +876 -0
- arctic_platform-0.2.1/arctic_platform/inference/server/worker.py +1984 -0
- arctic_platform-0.2.1/arctic_platform/inference/setup.py +194 -0
- arctic_platform-0.2.1/arctic_platform/inference/suffix_decoding/__init__.py +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/suffix_decoding/cache.py +357 -0
- arctic_platform-0.2.1/arctic_platform/inference/suffix_decoding/simulator.py +594 -0
- arctic_platform-0.2.1/arctic_platform/inference/utils.py +66 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/__init__.py +15 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/args.py +195 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/attention/README.md +193 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/attention/__init__.py +65 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/attention/flash_attn_forest_cascade.py +1951 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/config.py +208 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/deepseek_v41_lora.py +39 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/deepseek_v41_lora_model.py +34 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/dense_prompt_logprobs.py +360 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/fp32_lm_head.py +194 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/glm5next_lora.py +19 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/glm5next_lora_model.py +11 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/hybrid_state_fixes.py +273 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/kv_group_size.py +142 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/kvcached/__init__.py +2 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/kvcached/patches.py +68 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/kvcached/prefix_block_pool.py +490 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/mamba_completion_refresh.py +140 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/model_runner.py +2059 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/mrope_yarn.py +74 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/patches.py +516 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/plugin.py +57 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/required_patches.py +44 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/router_replay.py +111 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/arctic_proposer.py +301 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/arctic_speculator.py +1067 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/config.py +40 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/fp8.py +364 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/logits_processor_opt.py +107 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_dec/vocab_parallel_embedding.py +501 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/spec_decode_grammar.py +232 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/startup_error.py +109 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/stats.py +67 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/structured_output.py +39 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/swiftkv/__init__.py +18 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/swiftkv/llama_swiftkv.py +889 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/tp_spec_decode_sync.py +230 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/ulysses.py +765 -0
- arctic_platform-0.2.1/arctic_platform/inference/vllm/xgrammar_stop_mask.py +38 -0
- arctic_platform-0.2.1/arctic_platform/integrations/__init__.py +31 -0
- arctic_platform-0.2.1/arctic_platform/integrations/_cortex_dispatch.py +218 -0
- arctic_platform-0.2.1/arctic_platform/integrations/_cortex_shared.py +427 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/__init__.py +22 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/adapter.py +452 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/env.py +387 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/__init__.py +36 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/dockerfile_replay.py +290 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/environment.py +976 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/images.py +315 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/session.py +169 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/shell.py +189 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harbor/snowflake_sandbox/transfer.py +258 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harness/__init__.py +24 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harness/episodes.py +291 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harness/generator.py +284 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harness/proxy.py +302 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harness/tokens.py +45 -0
- arctic_platform-0.2.1/arctic_platform/integrations/harness/tool_parser.py +89 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/README.md +86 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/__init__.py +41 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/_grading_startup.py +203 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/config.py +223 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/examples/qwen3.5-4b/math-i3.toml +67 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/examples/qwen3.5-4b/swe-grade.toml +72 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/jobs.py +236 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/loss.py +45 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/payload.py +135 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/rl.py +196 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/worker.py +589 -0
- arctic_platform-0.2.1/arctic_platform/integrations/prime_rl/zz_arctic_prime_rl_grading.pth +1 -0
- arctic_platform-0.2.1/arctic_platform/integrations/skyrl/__init__.py +37 -0
- arctic_platform-0.2.1/arctic_platform/integrations/skyrl/entrypoint.py +222 -0
- arctic_platform-0.2.1/arctic_platform/integrations/tinker/README.md +349 -0
- arctic_platform-0.2.1/arctic_platform/integrations/tinker/__init__.py +595 -0
- arctic_platform-0.2.1/arctic_platform/integrations/tinker/convert.py +438 -0
- arctic_platform-0.2.1/arctic_platform/integrations/tinker/cortex.py +381 -0
- arctic_platform-0.2.1/arctic_platform/integrations/tinker/job.py +217 -0
- arctic_platform-0.2.1/arctic_platform/integrations/tinker/run.py +63 -0
- arctic_platform-0.2.1/arctic_platform/integrations/tinker/snowflake_profile.py +129 -0
- arctic_platform-0.2.1/arctic_platform/integrations/trl/README.md +77 -0
- arctic_platform-0.2.1/arctic_platform/integrations/trl/__init__.py +60 -0
- arctic_platform-0.2.1/arctic_platform/integrations/trl/client.py +732 -0
- arctic_platform-0.2.1/arctic_platform/integrations/trl/loss.py +71 -0
- arctic_platform-0.2.1/arctic_platform/integrations/trl/rollout.py +400 -0
- arctic_platform-0.2.1/arctic_platform/integrations/trl/trainer.py +402 -0
- arctic_platform-0.2.1/arctic_platform/integrations/trl/weights.py +49 -0
- arctic_platform-0.2.1/arctic_platform/integrations/verl/README.md +143 -0
- arctic_platform-0.2.1/arctic_platform/integrations/verl/__init__.py +34 -0
- arctic_platform-0.2.1/arctic_platform/integrations/verl/adapter.py +749 -0
- arctic_platform-0.2.1/arctic_platform/integrations/verl/config/remote_backend/arctic.yaml +124 -0
- arctic_platform-0.2.1/arctic_platform/integrations/verl/examples/run_bird_grpo_arl.sh +107 -0
- arctic_platform-0.2.1/arctic_platform/integrations/verl/examples/run_gsm8k_grpo_arl.sh +108 -0
- arctic_platform-0.1.1.dev0/arctic_platform/rl/processors/verl_grpo.py → arctic_platform-0.2.1/arctic_platform/integrations/verl/grpo_loss.py +81 -35
- arctic_platform-0.2.1/arctic_platform/integrations/verl/register.py +73 -0
- arctic_platform-0.2.1/arctic_platform/integrations/verl/rollout.py +404 -0
- arctic_platform-0.2.1/arctic_platform/integrations/verl/worker.py +291 -0
- arctic_platform-0.2.1/arctic_platform/model/__init__.py +157 -0
- arctic_platform-0.2.1/arctic_platform/model/config.py +281 -0
- arctic_platform-0.2.1/arctic_platform/model/convert_hf_to_primerl.py +364 -0
- arctic_platform-0.2.1/arctic_platform/model/dense_spec.py +151 -0
- arctic_platform-0.2.1/arctic_platform/model/factory.py +40 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/NOTICE +3 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/__init__.py +15 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/afmoe/__init__.py +24 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/afmoe/configuration_afmoe.py +116 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/afmoe/converting_afmoe.py +147 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/afmoe/modeling_afmoe.py +640 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/debug/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/debug/config.py +117 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/debug/determinism.py +268 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/debug/gpu_memory.py +69 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/debug/row_invariant_projection.py +514 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/debug/token_combine.py +199 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/__init__.py +7 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/attention.py +833 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/configuration_deepseek_v4.py +371 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/conversion_ops.py +348 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/converting_deepseek_v4.py +170 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/cp.py +46 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/deepspeed_integration.py +139 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/dequantize.py +118 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/hyperconnections.py +113 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/modeling_deepseek_v4.py +406 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/moe.py +236 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/rotary.py +131 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v4/sequence.py +52 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/__init__.py +22 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/attention.py +573 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/configuration_deepseek_v41.py +247 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/converting_deepseek_v41.py +173 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/deepspeed_integration.py +174 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/dequantize.py +73 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/engram.py +535 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/hyperconnections.py +121 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/modeling_deepseek_v41.py +456 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/moe.py +147 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/deepseek_v41/streaming_conversion.py +339 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/NOTICE +20 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/__init__.py +21 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/config.py +83 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/online.py +287 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/__init__.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/algorithms/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/algorithms/common/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/algorithms/common/dflash_family_model.py +1822 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/algorithms/common/dflash_metrics.py +65 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/core/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/core/chunking.py +109 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/core/dflash_head_triton.py +522 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/export/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/export/checkpoint_io.py +85 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/__init__.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/__init__.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/dflash.py +933 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/dflash2.py +487 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/dflash2_conv_triton.py +359 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/dflash_kernels.py +58 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/flex_attention_backend.py +30 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/modeling/draft/registry.py +67 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/dflash2/specforge/torch_compat.py +63 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/fp8.py +872 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm4_moe/__init__.py +24 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm4_moe/configuration_glm4_moe.py +215 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm4_moe/converting_glm4_moe.py +151 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm4_moe/modeling_glm4_moe.py +336 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/INTERNAL_COMMIT.txt +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/NOTICE +12 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/__init__.py +16 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/config.py +58 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/config_validation.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/deepspeed_integration.py +172 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/model_builder.py +448 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/__init__.py +44 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/fp8.py +12 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/__init__.py +13 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/configuration_glm_moe_dsa.py +261 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/converting_glm_moe_dsa.py +442 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/debug_overrides.py +126 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/modeling_glm_moe_dsa.py +498 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/glm_moe_dsa/sparse_mla_attention.py +328 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/fp8_indexer.py +317 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/sparse_mla_bwd.py +375 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/sparse_mla_flashmla.py +466 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/kernels/sparse_mla_fwd.py +198 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/layers/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/layers/mlp.py +38 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/layers/norms.py +79 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/models/layers/rotary_emb.py +142 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/utils/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm52/utils/cp.py +22 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/__init__.py +28 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/context_parallel.py +1013 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/converting_glm5_next.py +275 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/deepspeed_integration.py +138 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/fused_norms.py +79 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/hyper_connections.py +556 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/modeling_glm5_next.py +289 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/glm53/vllm_weights.py +156 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/action_masks.py +638 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/activation_offload.py +576 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/grouped_lm_head.py +206 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/lm_head.py +816 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/packing.py +29 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/pinned_staging_cache.py +450 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/qwen3_5_dense_parity.py +476 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/router_replay_recompute.py +198 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/sp/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/sp/collectives.py +101 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/sp/gated_delta_net.py +913 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/sp/transformers.py +30 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/tiled_mlp.py +312 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/gpu/vllm_rounding.py +279 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/__init__.py +16 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/attention.py +258 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/deepspeed_integration.py +97 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/modeling_inkling.py +313 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/sequence_parallel.py +80 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/inkling/weights.py +191 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/__init__.py +7 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_kv_fake_quant.py +111 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_mhc.py +646 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_rope.py +201 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_sparse_attn.py +434 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_sparse_attn_bwd.py +334 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/deepseek_v4/dsv4_sparse_attn_fwd.py +289 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/fp8_block_dequant.py +106 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/fp8_indexer.py +248 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/fp8_quant.py +160 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/fused_swiglu.py +105 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/scored_swiglu_down.py +188 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonic_experts.py +137 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/LICENSE +201 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/__init__.py +25 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/enums.py +46 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/__init__.py +689 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/backward.py +583 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/forward.py +157 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/reduction_over_k_gather.py +176 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/topk.py +355 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/triton_kernels/README.md +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/triton_kernels/TRITON_LICENSE +23 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/triton_kernels/__init__.py +370 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/functional/triton_kernels/bitmatrix.py +162 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/kernels/sonicmoe/utils.py +139 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/__init__.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/attn.py +288 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/cp_mamba.py +290 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/grouped_linear.py +66 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/mlp.py +29 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/norms.py +83 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/rms_norm.py +3 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/rotary_emb.py +115 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/layers/sequence.py +14 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/__init__.py +24 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/configuration_minimax_m2.py +172 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/converting_minimax_m2.py +101 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/modeling_minimax_m2.py +281 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/minimax_m2/vllm_weights.py +57 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/INTERNAL_COMMIT.txt +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/NOTICE +16 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/__init__.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/base.py +69 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/config.py +4 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/config_validation.py +18 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/conversion_cache.py +226 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/deepspeed_integration.py +482 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/__init__.py +2 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/combine_replay.py +181 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/deepep.py +373 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/ep_backend.py +27 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/expert_parallel.py +79 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/ncclep.py +154 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/token_permute.py +58 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/uccl_ep/__init__.py +6 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/uccl_ep/bootstrap.py +357 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/uccl_ep/buffer.py +1463 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/distributed/ucclep.py +430 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/hf_loader.py +279 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/layers/__init__.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/layers/checkpointing.py +154 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/layers/lm_head.py +653 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/layers/moe.py +1526 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/logging_utils.py +26 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/parallel_dims.py +251 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/router_stats.py +177 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/routing_weights.py +200 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/sequence_parallel.py +180 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/token_combine.py +109 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/vllm_weights.py +270 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/vlm.py +121 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/weights.py +140 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/moe/world.py +46 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/__init__.py +24 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/configuration_nemotron_h.py +227 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/converting_nemotron_h.py +176 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/modeling_nemotron_h.py +595 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/nemotron_h/vllm_weights.py +71 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/INTERNAL_COMMIT.txt +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/NOTICE +15 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/__init__.py +19 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/config.py +66 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/config_validation.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/conversion_cache.py +2 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/deepspeed_integration.py +422 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/__init__.py +3 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/deepep.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/ep_backend.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/expert_parallel.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/token_permute.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/uccl_ep/__init__.py +7 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/uccl_ep/bootstrap.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/uccl_ep/buffer.py +2 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/distributed/ucclep.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/logging_utils.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/model_builder.py +754 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/__init__.py +133 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/base.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/__init__.py +0 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/checkpointing.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/lm_head.py +2 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/moe.py +2 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/layers/rotary_emb.py +114 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/qwen3_5_moe/__init__.py +13 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/qwen3_5_moe/configuration_qwen3_5_moe.py +113 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/qwen3_5_moe/converting_qwen3_5_moe.py +119 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/models/qwen3_5_moe/modeling_qwen3_5_moe.py +1217 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/parallel_dims.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/sequence_parallel.py +167 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/vlm.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/weights.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen35/world.py +1 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/__init__.py +27 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/context_parallel.py +335 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/converting_qwen4_exp.py +192 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/deepspeed_integration.py +141 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/hyper_connection.py +467 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/modeling_qwen4_exp.py +500 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/qsa_flex.py +654 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/qsa_index_scores.py +125 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen38/qsa_sparse_gqa.py +288 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen3_moe/__init__.py +24 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen3_moe/configuration_qwen3_moe.py +210 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen3_moe/converting_qwen3_moe.py +100 -0
- arctic_platform-0.2.1/arctic_platform/model/implementations/qwen3_moe/modeling_qwen3_moe.py +441 -0
- arctic_platform-0.2.1/arctic_platform/model/loader.py +312 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/__init__.py +25 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/deepseek_v4.py +108 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/deepseek_v41.py +111 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/dflash2.py +122 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/flash_moe.py +178 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/generic_moe.py +123 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/glm_moe_dsa.py +137 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/huggingface.py +157 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/inkling.py +112 -0
- arctic_platform-0.2.1/arctic_platform/model/loaders/qwen3_5_moe.py +150 -0
- arctic_platform-0.2.1/arctic_platform/model/moe_spec.py +194 -0
- arctic_platform-0.2.1/arctic_platform/model/parallel/__init__.py +15 -0
- arctic_platform-0.2.1/arctic_platform/model/parallel/groups.py +369 -0
- arctic_platform-0.2.1/arctic_platform/model/parallel/topology.py +277 -0
- arctic_platform-0.2.1/arctic_platform/model/parallel/validation.py +101 -0
- arctic_platform-0.2.1/arctic_platform/model/patch.py +83 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/__init__.py +24 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/activation_offload.py +40 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/compile.py +32 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/gradient_checkpointing.py +67 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/liger.py +37 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/lm_head.py +42 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/peft.py +191 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/tiled_mlp.py +247 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/utils.py +28 -0
- arctic_platform-0.2.1/arctic_platform/model/patches/zorro_train.py +57 -0
- arctic_platform-0.2.1/arctic_platform/model/recompute.py +30 -0
- arctic_platform-0.2.1/arctic_platform/model/router_stats.py +26 -0
- arctic_platform-0.2.1/arctic_platform/model/weight_export.py +749 -0
- arctic_platform-0.2.1/arctic_platform/registry.py +111 -0
- arctic_platform-0.2.1/arctic_platform/rl/__init__.py +79 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/client.py +3 -19
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/config.py +29 -48
- arctic_platform-0.2.1/arctic_platform/rl/deepspeed_worker.py +18 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/examples/inference_example.py +2 -1
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/examples/local_server_example.py +10 -5
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/examples/weight_sync_example.py +2 -1
- arctic_platform-0.2.1/arctic_platform/rl/processors/__init__.py +240 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/base_loss.py +278 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/causal_cross_entropy.py +244 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/compute_logprobs.py +123 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/cortex_grpo.py +127 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/dflash2.py +149 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/functional.py +1337 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/grouped_distillation.py +1040 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/grpo.py +1766 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/processors/microbatch.py +49 -9
- arctic_platform-0.2.1/arctic_platform/rl/processors/packed_reduction.py +239 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/processors/packing.py +24 -3
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/processors/pipeline.py +428 -275
- arctic_platform-0.2.1/arctic_platform/rl/processors/sft.py +18 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/processors/stats_tracker.py +25 -3
- arctic_platform-0.2.1/arctic_platform/rl/processors/verl_grpo.py +31 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/weighted_cross_entropy.py +190 -0
- arctic_platform-0.2.1/arctic_platform/rl/processors/weighted_logprob.py +82 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/ray_client.py +136 -81
- arctic_platform-0.2.1/arctic_platform/rl/ray_cluster.py +18 -0
- arctic_platform-0.2.1/arctic_platform/rl/ray_server.py +18 -0
- arctic_platform-0.2.1/arctic_platform/rl/server.py +18 -0
- arctic_platform-0.2.1/arctic_platform/rl/utils/__init__.py +50 -0
- arctic_platform-0.2.1/arctic_platform/rl/utils/batch.py +18 -0
- arctic_platform-0.2.1/arctic_platform/rl/utils/cuda_ipc.py +18 -0
- arctic_platform-0.2.1/arctic_platform/rl/utils/debug.py +18 -0
- arctic_platform-0.2.1/arctic_platform/rl/utils/ray_pg.py +18 -0
- arctic_platform-0.2.1/arctic_platform/rl/utils/record_replay.py +18 -0
- arctic_platform-0.2.1/arctic_platform/rl/utils/server_models.py +18 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/weight_sync.py +7 -3
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/README.md +127 -68
- arctic_platform-0.2.1/arctic_platform/rl/zorro_train/__init__.py +34 -0
- arctic_platform-0.2.1/arctic_platform/rl/zorro_train/actor.py +252 -0
- arctic_platform-0.2.1/arctic_platform/rl/zorro_train/demo.py +128 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/module_patcher.py +23 -7
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/qwen_attention_patcher.py +610 -281
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/qwen_model_patcher.py +384 -416
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/seqlen_balancing.py +69 -287
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/test_perf.py +148 -87
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/tests.py +347 -309
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/zorro_train.py +502 -708
- arctic_platform-0.2.1/arctic_platform/sft/__init__.py +47 -0
- arctic_platform-0.2.1/arctic_platform/sft/examples/__init__.py +14 -0
- arctic_platform-0.2.1/arctic_platform/sft/examples/run_inkling_sft.py +105 -0
- arctic_platform-0.2.1/arctic_platform/sft/examples/run_sft_ckpt_resume_demo.py +158 -0
- arctic_platform-0.2.1/arctic_platform/sft/examples/run_sft_demo.py +216 -0
- arctic_platform-0.2.1/arctic_platform/sft/examples/run_sft_generate_demo.py +115 -0
- arctic_platform-0.2.1/arctic_platform/sft/examples/sft_example.py +146 -0
- arctic_platform-0.2.1/arctic_platform/sft/processor.py +399 -0
- arctic_platform-0.2.1/arctic_platform/sft_profile.py +110 -0
- arctic_platform-0.2.1/arctic_platform/testing_utils.py +1275 -0
- arctic_platform-0.2.1/arctic_platform/wire.py +449 -0
- arctic_platform-0.2.1/hatch_build.py +51 -0
- arctic_platform-0.2.1/pyproject.toml +281 -0
- arctic_platform-0.1.1.dev0/PKG-INFO +0 -84
- arctic_platform-0.1.1.dev0/README.md +0 -23
- arctic_platform-0.1.1.dev0/arctic_platform/rl/README.md +0 -7
- arctic_platform-0.1.1.dev0/arctic_platform/rl/__init__.py +0 -27
- arctic_platform-0.1.1.dev0/arctic_platform/rl/deepspeed_worker.py +0 -905
- arctic_platform-0.1.1.dev0/arctic_platform/rl/http_client.py +0 -705
- arctic_platform-0.1.1.dev0/arctic_platform/rl/http_server.py +0 -1166
- arctic_platform-0.1.1.dev0/arctic_platform/rl/processors/__init__.py +0 -166
- arctic_platform-0.1.1.dev0/arctic_platform/rl/processors/functional.py +0 -257
- arctic_platform-0.1.1.dev0/arctic_platform/rl/processors/grpo.py +0 -330
- arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/long_context_qa/README.md +0 -87
- arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/long_context_qa/download_data.py +0 -120
- arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/long_context_qa/run_qwen3_32b_longcontext_grpo_arl_zorro_yes_kl.sh +0 -221
- arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/README.md +0 -176
- arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/bird_reward.py +0 -272
- arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/preprocess_bird.py +0 -684
- arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/run_qwen3_32b_bird_grpo_arl_zorro_yes.sh +0 -251
- arctic_platform-0.1.1.dev0/arctic_platform/rl/projects/txt2sql/run_qwen3_32b_bird_grpo_arl_zorro_yes_kl.sh +0 -31
- arctic_platform-0.1.1.dev0/arctic_platform/rl/ray_server.py +0 -1475
- arctic_platform-0.1.1.dev0/arctic_platform/rl/server.py +0 -6
- arctic_platform-0.1.1.dev0/arctic_platform/rl/utils/__init__.py +0 -16
- arctic_platform-0.1.1.dev0/arctic_platform/rl/utils/batch.py +0 -434
- arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/__init__.py +0 -23
- arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/actor.py +0 -271
- arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/demo.py +0 -130
- arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/test.py +0 -7
- arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/test_actor_demo.py +0 -68
- arctic_platform-0.1.1.dev0/arctic_platform/rl/zorro_train/test_forward_and_backward.py +0 -492
- arctic_platform-0.1.1.dev0/pyproject.toml +0 -106
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/LICENSE +0 -0
- {arctic_platform-0.1.1.dev0/arctic_platform/rl → arctic_platform-0.2.1/arctic_platform/common}/utils/cuda_ipc.py +0 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/test_perf-g1.sh +0 -0
- {arctic_platform-0.1.1.dev0 → arctic_platform-0.2.1}/arctic_platform/rl/zorro_train/test_perf-g8.sh +0 -0
|
@@ -0,0 +1,190 @@
|
|
|
1
|
+
# Byte-compiled / optimized / DLL files
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*$py.class
|
|
5
|
+
|
|
6
|
+
# C extensions
|
|
7
|
+
*.so
|
|
8
|
+
|
|
9
|
+
# Distribution / packaging
|
|
10
|
+
.Python
|
|
11
|
+
build/
|
|
12
|
+
develop-eggs/
|
|
13
|
+
dist/
|
|
14
|
+
downloads/
|
|
15
|
+
eggs/
|
|
16
|
+
.eggs/
|
|
17
|
+
lib/
|
|
18
|
+
lib64/
|
|
19
|
+
parts/
|
|
20
|
+
sdist/
|
|
21
|
+
var/
|
|
22
|
+
wheels/
|
|
23
|
+
share/python-wheels/
|
|
24
|
+
*.egg-info/
|
|
25
|
+
.installed.cfg
|
|
26
|
+
*.egg
|
|
27
|
+
MANIFEST
|
|
28
|
+
|
|
29
|
+
# PyInstaller
|
|
30
|
+
# Usually these files are written by a python script from a template
|
|
31
|
+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
|
32
|
+
*.manifest
|
|
33
|
+
*.spec
|
|
34
|
+
|
|
35
|
+
# Installer logs
|
|
36
|
+
pip-log.txt
|
|
37
|
+
pip-delete-this-directory.txt
|
|
38
|
+
|
|
39
|
+
# Unit test / coverage reports
|
|
40
|
+
htmlcov/
|
|
41
|
+
.tox/
|
|
42
|
+
.nox/
|
|
43
|
+
.coverage
|
|
44
|
+
.coverage.*
|
|
45
|
+
.cache
|
|
46
|
+
nosetests.xml
|
|
47
|
+
coverage.xml
|
|
48
|
+
*.cover
|
|
49
|
+
*.py,cover
|
|
50
|
+
.hypothesis/
|
|
51
|
+
.pytest_cache/
|
|
52
|
+
cover/
|
|
53
|
+
|
|
54
|
+
# Translations
|
|
55
|
+
*.mo
|
|
56
|
+
*.pot
|
|
57
|
+
|
|
58
|
+
# Django stuff:
|
|
59
|
+
*.log
|
|
60
|
+
local_settings.py
|
|
61
|
+
db.sqlite3
|
|
62
|
+
db.sqlite3-journal
|
|
63
|
+
|
|
64
|
+
# Flask stuff:
|
|
65
|
+
instance/
|
|
66
|
+
.webassets-cache
|
|
67
|
+
|
|
68
|
+
# Scrapy stuff:
|
|
69
|
+
.scrapy
|
|
70
|
+
|
|
71
|
+
# Sphinx documentation
|
|
72
|
+
docs/_build/
|
|
73
|
+
|
|
74
|
+
# PyBuilder
|
|
75
|
+
.pybuilder/
|
|
76
|
+
target/
|
|
77
|
+
|
|
78
|
+
# Jupyter Notebook
|
|
79
|
+
.ipynb_checkpoints
|
|
80
|
+
|
|
81
|
+
# IPython
|
|
82
|
+
profile_default/
|
|
83
|
+
ipython_config.py
|
|
84
|
+
|
|
85
|
+
# pyenv
|
|
86
|
+
# For a library or package, you might want to ignore these files since the code is
|
|
87
|
+
# intended to run in multiple environments; otherwise, check them in:
|
|
88
|
+
# .python-version
|
|
89
|
+
|
|
90
|
+
# pipenv
|
|
91
|
+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
|
|
92
|
+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
|
|
93
|
+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
|
|
94
|
+
# install all needed dependencies.
|
|
95
|
+
#Pipfile.lock
|
|
96
|
+
|
|
97
|
+
# poetry
|
|
98
|
+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
|
|
99
|
+
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
|
100
|
+
# commonly ignored for libraries.
|
|
101
|
+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
|
|
102
|
+
#poetry.lock
|
|
103
|
+
|
|
104
|
+
# pdm
|
|
105
|
+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
|
|
106
|
+
#pdm.lock
|
|
107
|
+
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
|
|
108
|
+
# in version control.
|
|
109
|
+
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
|
|
110
|
+
.pdm.toml
|
|
111
|
+
.pdm-python
|
|
112
|
+
.pdm-build/
|
|
113
|
+
|
|
114
|
+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
|
|
115
|
+
__pypackages__/
|
|
116
|
+
|
|
117
|
+
# Celery stuff
|
|
118
|
+
celerybeat-schedule
|
|
119
|
+
celerybeat.pid
|
|
120
|
+
|
|
121
|
+
# SageMath parsed files
|
|
122
|
+
*.sage.py
|
|
123
|
+
|
|
124
|
+
# Environments
|
|
125
|
+
.env
|
|
126
|
+
.venv
|
|
127
|
+
env/
|
|
128
|
+
venv/
|
|
129
|
+
ENV/
|
|
130
|
+
env.bak/
|
|
131
|
+
venv.bak/
|
|
132
|
+
|
|
133
|
+
# Spyder project settings
|
|
134
|
+
.spyderproject
|
|
135
|
+
.spyproject
|
|
136
|
+
|
|
137
|
+
# Rope project settings
|
|
138
|
+
.ropeproject
|
|
139
|
+
|
|
140
|
+
# mkdocs documentation
|
|
141
|
+
/site
|
|
142
|
+
|
|
143
|
+
# mypy
|
|
144
|
+
.mypy_cache/
|
|
145
|
+
.dmypy.json
|
|
146
|
+
dmypy.json
|
|
147
|
+
|
|
148
|
+
# Pyre type checker
|
|
149
|
+
.pyre/
|
|
150
|
+
|
|
151
|
+
# pytype static type analyzer
|
|
152
|
+
.pytype/
|
|
153
|
+
|
|
154
|
+
# Cython debug symbols
|
|
155
|
+
cython_debug/
|
|
156
|
+
|
|
157
|
+
# PyCharm
|
|
158
|
+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
|
|
159
|
+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
|
|
160
|
+
# and can be added to the global gitignore or merged into this file. For a more nuclear
|
|
161
|
+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
|
162
|
+
#.idea/
|
|
163
|
+
|
|
164
|
+
# Arctic Synth work directory
|
|
165
|
+
batch_work_dir/
|
|
166
|
+
|
|
167
|
+
# gRPC code generated by arctic_platform/inference/embedding/generate_proto.py
|
|
168
|
+
arctic_platform/inference/embedding/proto/python/
|
|
169
|
+
|
|
170
|
+
# Hydra run artifacts from recipe launchers. Hydra writes to the process's cwd,
|
|
171
|
+
# which is the repo root when a launcher is invoked by path (the documented way
|
|
172
|
+
# to run the Cortex recipes) -- so the recipes/ pattern alone misses them.
|
|
173
|
+
recipes/**/outputs/
|
|
174
|
+
/outputs/
|
|
175
|
+
|
|
176
|
+
# Client recipe connection configs — these hold a Snowflake PAT. The checked-in
|
|
177
|
+
# config.json.template is not matched (it does not end in .json).
|
|
178
|
+
recipes/*.json
|
|
179
|
+
|
|
180
|
+
# Semi-persistence onboarding reports and job ledgers, written under the cwd.
|
|
181
|
+
onboarding_reports/
|
|
182
|
+
|
|
183
|
+
# Local correctness measurements and execution state; recipes and references are selected separately.
|
|
184
|
+
/work/
|
|
185
|
+
/.correctness-tmp/
|
|
186
|
+
/.codex-run-state/
|
|
187
|
+
/.agent-work/
|
|
188
|
+
/correctness-progress.md
|
|
189
|
+
/local-correctness-records/
|
|
190
|
+
/arctic_platform/correctness/specs/
|
|
@@ -0,0 +1,451 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: arctic_platform
|
|
3
|
+
Version: 0.2.1
|
|
4
|
+
Summary: Snowflake LLM training library w/ RL
|
|
5
|
+
Project-URL: Homepage, https://github.com/Snowflake-AI-Research/Arctic-Platform
|
|
6
|
+
Project-URL: Repository, https://github.com/Snowflake-AI-Research/Arctic-Platform
|
|
7
|
+
Project-URL: Issues, https://github.com/Snowflake-AI-Research/Arctic-Platform/issues
|
|
8
|
+
Author-email: Tunji Ruwase <tunji.ruwase@snowflake.com>, Stas Bekman <stas.bekman@snowflake.com>, Michael Wyatt <michael.wyatt@snowflake.com>, Jeff Rasley <jeff.rasley@snowflake.com>, Samyam Rajbhandari <samyam.rajbhandari@snowflake.com>
|
|
9
|
+
License-Expression: Apache-2.0
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Keywords: llm,rl,training
|
|
12
|
+
Classifier: Development Status :: 4 - Beta
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Requires-Python: >=3.12
|
|
16
|
+
Requires-Dist: pydantic-settings>=2.0
|
|
17
|
+
Requires-Dist: pydantic>=2.10
|
|
18
|
+
Requires-Dist: typing-extensions
|
|
19
|
+
Provides-Extra: all
|
|
20
|
+
Requires-Dist: accelerate; extra == 'all'
|
|
21
|
+
Requires-Dist: aiohttp; extra == 'all'
|
|
22
|
+
Requires-Dist: autoflake; extra == 'all'
|
|
23
|
+
Requires-Dist: certifi; extra == 'all'
|
|
24
|
+
Requires-Dist: codetiming; extra == 'all'
|
|
25
|
+
Requires-Dist: deepspeed>=0.19.9; extra == 'all'
|
|
26
|
+
Requires-Dist: dockerfile-parse>=2.0.1; extra == 'all'
|
|
27
|
+
Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'all'
|
|
28
|
+
Requires-Dist: fastapi; extra == 'all'
|
|
29
|
+
Requires-Dist: harbor==0.25.0; extra == 'all'
|
|
30
|
+
Requires-Dist: httpx; extra == 'all'
|
|
31
|
+
Requires-Dist: huggingface-hub; extra == 'all'
|
|
32
|
+
Requires-Dist: huggingface-hub<3,>=2.0; extra == 'all'
|
|
33
|
+
Requires-Dist: kvcached==0.1.4; extra == 'all'
|
|
34
|
+
Requires-Dist: liger-kernel>=0.8.5; extra == 'all'
|
|
35
|
+
Requires-Dist: numpy; extra == 'all'
|
|
36
|
+
Requires-Dist: nvidia-ml-py; extra == 'all'
|
|
37
|
+
Requires-Dist: omegaconf; extra == 'all'
|
|
38
|
+
Requires-Dist: openhands-sdk==1.7.1; extra == 'all'
|
|
39
|
+
Requires-Dist: openhands-tools==1.7.1; extra == 'all'
|
|
40
|
+
Requires-Dist: orjson; extra == 'all'
|
|
41
|
+
Requires-Dist: parameterized; extra == 'all'
|
|
42
|
+
Requires-Dist: pathspec>=1.0.3; extra == 'all'
|
|
43
|
+
Requires-Dist: peft<0.22,>=0.21.2; extra == 'all'
|
|
44
|
+
Requires-Dist: pre-commit; extra == 'all'
|
|
45
|
+
Requires-Dist: psutil; extra == 'all'
|
|
46
|
+
Requires-Dist: pydantic>=2.0; extra == 'all'
|
|
47
|
+
Requires-Dist: pytest; extra == 'all'
|
|
48
|
+
Requires-Dist: pytest-asyncio; extra == 'all'
|
|
49
|
+
Requires-Dist: pytest-flakefinder; extra == 'all'
|
|
50
|
+
Requires-Dist: pytest-instafail; extra == 'all'
|
|
51
|
+
Requires-Dist: pytest-sugar; extra == 'all'
|
|
52
|
+
Requires-Dist: pytest-timeout; extra == 'all'
|
|
53
|
+
Requires-Dist: pytest-xdist; extra == 'all'
|
|
54
|
+
Requires-Dist: quack-kernels>=0.4.0; extra == 'all'
|
|
55
|
+
Requires-Dist: ray; extra == 'all'
|
|
56
|
+
Requires-Dist: requests; extra == 'all'
|
|
57
|
+
Requires-Dist: safetensors; extra == 'all'
|
|
58
|
+
Requires-Dist: snowflake-sandbox-python==0.2.2a5; extra == 'all'
|
|
59
|
+
Requires-Dist: tenacity; extra == 'all'
|
|
60
|
+
Requires-Dist: tensordict; extra == 'all'
|
|
61
|
+
Requires-Dist: tinker==0.25.0; extra == 'all'
|
|
62
|
+
Requires-Dist: torch; extra == 'all'
|
|
63
|
+
Requires-Dist: torch>=2.9; extra == 'all'
|
|
64
|
+
Requires-Dist: tqdm; extra == 'all'
|
|
65
|
+
Requires-Dist: transformers; extra == 'all'
|
|
66
|
+
Requires-Dist: transformers==5.20.2; extra == 'all'
|
|
67
|
+
Requires-Dist: urllib3; extra == 'all'
|
|
68
|
+
Requires-Dist: uvicorn; extra == 'all'
|
|
69
|
+
Requires-Dist: vllm==0.33.0; extra == 'all'
|
|
70
|
+
Requires-Dist: xgrammar==0.2.8; extra == 'all'
|
|
71
|
+
Provides-Extra: correctness
|
|
72
|
+
Requires-Dist: aiohttp; extra == 'correctness'
|
|
73
|
+
Requires-Dist: deepspeed>=0.19.9; extra == 'correctness'
|
|
74
|
+
Requires-Dist: huggingface-hub; extra == 'correctness'
|
|
75
|
+
Requires-Dist: huggingface-hub<3,>=2.0; extra == 'correctness'
|
|
76
|
+
Requires-Dist: liger-kernel>=0.8.5; extra == 'correctness'
|
|
77
|
+
Requires-Dist: numpy; extra == 'correctness'
|
|
78
|
+
Requires-Dist: nvidia-ml-py; extra == 'correctness'
|
|
79
|
+
Requires-Dist: peft<0.22,>=0.21.2; extra == 'correctness'
|
|
80
|
+
Requires-Dist: psutil; extra == 'correctness'
|
|
81
|
+
Requires-Dist: pytest; extra == 'correctness'
|
|
82
|
+
Requires-Dist: pytest-instafail; extra == 'correctness'
|
|
83
|
+
Requires-Dist: quack-kernels>=0.4.0; extra == 'correctness'
|
|
84
|
+
Requires-Dist: ray; extra == 'correctness'
|
|
85
|
+
Requires-Dist: requests; extra == 'correctness'
|
|
86
|
+
Requires-Dist: safetensors; extra == 'correctness'
|
|
87
|
+
Requires-Dist: tenacity; extra == 'correctness'
|
|
88
|
+
Requires-Dist: tensordict; extra == 'correctness'
|
|
89
|
+
Requires-Dist: torch; extra == 'correctness'
|
|
90
|
+
Requires-Dist: torch>=2.9; extra == 'correctness'
|
|
91
|
+
Requires-Dist: tqdm; extra == 'correctness'
|
|
92
|
+
Requires-Dist: transformers==5.20.2; extra == 'correctness'
|
|
93
|
+
Requires-Dist: urllib3; extra == 'correctness'
|
|
94
|
+
Provides-Extra: cortex
|
|
95
|
+
Requires-Dist: aiohttp; extra == 'cortex'
|
|
96
|
+
Requires-Dist: numpy; extra == 'cortex'
|
|
97
|
+
Requires-Dist: requests; extra == 'cortex'
|
|
98
|
+
Requires-Dist: safetensors; extra == 'cortex'
|
|
99
|
+
Requires-Dist: tenacity; extra == 'cortex'
|
|
100
|
+
Requires-Dist: torch; extra == 'cortex'
|
|
101
|
+
Requires-Dist: urllib3; extra == 'cortex'
|
|
102
|
+
Provides-Extra: dev
|
|
103
|
+
Requires-Dist: aiohttp; extra == 'dev'
|
|
104
|
+
Requires-Dist: autoflake; extra == 'dev'
|
|
105
|
+
Requires-Dist: certifi; extra == 'dev'
|
|
106
|
+
Requires-Dist: dockerfile-parse>=2.0.1; extra == 'dev'
|
|
107
|
+
Requires-Dist: harbor==0.25.0; extra == 'dev'
|
|
108
|
+
Requires-Dist: httpx; extra == 'dev'
|
|
109
|
+
Requires-Dist: numpy; extra == 'dev'
|
|
110
|
+
Requires-Dist: openhands-sdk==1.7.1; extra == 'dev'
|
|
111
|
+
Requires-Dist: openhands-tools==1.7.1; extra == 'dev'
|
|
112
|
+
Requires-Dist: parameterized; extra == 'dev'
|
|
113
|
+
Requires-Dist: pathspec>=1.0.3; extra == 'dev'
|
|
114
|
+
Requires-Dist: pre-commit; extra == 'dev'
|
|
115
|
+
Requires-Dist: pytest; extra == 'dev'
|
|
116
|
+
Requires-Dist: pytest-asyncio; extra == 'dev'
|
|
117
|
+
Requires-Dist: pytest-flakefinder; extra == 'dev'
|
|
118
|
+
Requires-Dist: pytest-instafail; extra == 'dev'
|
|
119
|
+
Requires-Dist: pytest-sugar; extra == 'dev'
|
|
120
|
+
Requires-Dist: pytest-timeout; extra == 'dev'
|
|
121
|
+
Requires-Dist: pytest-xdist; extra == 'dev'
|
|
122
|
+
Requires-Dist: requests; extra == 'dev'
|
|
123
|
+
Requires-Dist: safetensors; extra == 'dev'
|
|
124
|
+
Requires-Dist: snowflake-sandbox-python==0.2.2a5; extra == 'dev'
|
|
125
|
+
Requires-Dist: tenacity; extra == 'dev'
|
|
126
|
+
Requires-Dist: tinker==0.25.0; extra == 'dev'
|
|
127
|
+
Requires-Dist: torch; extra == 'dev'
|
|
128
|
+
Requires-Dist: transformers; extra == 'dev'
|
|
129
|
+
Requires-Dist: urllib3; extra == 'dev'
|
|
130
|
+
Provides-Extra: formatting
|
|
131
|
+
Requires-Dist: autoflake; extra == 'formatting'
|
|
132
|
+
Requires-Dist: pre-commit; extra == 'formatting'
|
|
133
|
+
Provides-Extra: gdn
|
|
134
|
+
Requires-Dist: causal-conv1d; extra == 'gdn'
|
|
135
|
+
Requires-Dist: flash-linear-attention; extra == 'gdn'
|
|
136
|
+
Provides-Extra: harbor
|
|
137
|
+
Requires-Dist: certifi; extra == 'harbor'
|
|
138
|
+
Requires-Dist: dockerfile-parse>=2.0.1; extra == 'harbor'
|
|
139
|
+
Requires-Dist: harbor==0.25.0; extra == 'harbor'
|
|
140
|
+
Requires-Dist: pathspec>=1.0.3; extra == 'harbor'
|
|
141
|
+
Requires-Dist: snowflake-sandbox-python==0.2.2a5; extra == 'harbor'
|
|
142
|
+
Provides-Extra: inference
|
|
143
|
+
Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'inference'
|
|
144
|
+
Requires-Dist: fastapi; extra == 'inference'
|
|
145
|
+
Requires-Dist: kvcached==0.1.4; extra == 'inference'
|
|
146
|
+
Requires-Dist: pydantic>=2.0; extra == 'inference'
|
|
147
|
+
Requires-Dist: ray; extra == 'inference'
|
|
148
|
+
Requires-Dist: tqdm; extra == 'inference'
|
|
149
|
+
Requires-Dist: uvicorn; extra == 'inference'
|
|
150
|
+
Requires-Dist: vllm==0.33.0; extra == 'inference'
|
|
151
|
+
Requires-Dist: xgrammar==0.2.8; extra == 'inference'
|
|
152
|
+
Provides-Extra: onprem
|
|
153
|
+
Requires-Dist: deepspeed>=0.19.9; extra == 'onprem'
|
|
154
|
+
Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'onprem'
|
|
155
|
+
Requires-Dist: fastapi; extra == 'onprem'
|
|
156
|
+
Requires-Dist: huggingface-hub<3,>=2.0; extra == 'onprem'
|
|
157
|
+
Requires-Dist: kvcached==0.1.4; extra == 'onprem'
|
|
158
|
+
Requires-Dist: liger-kernel>=0.8.5; extra == 'onprem'
|
|
159
|
+
Requires-Dist: numpy; extra == 'onprem'
|
|
160
|
+
Requires-Dist: nvidia-ml-py; extra == 'onprem'
|
|
161
|
+
Requires-Dist: peft<0.22,>=0.21.2; extra == 'onprem'
|
|
162
|
+
Requires-Dist: psutil; extra == 'onprem'
|
|
163
|
+
Requires-Dist: pydantic>=2.0; extra == 'onprem'
|
|
164
|
+
Requires-Dist: quack-kernels>=0.4.0; extra == 'onprem'
|
|
165
|
+
Requires-Dist: ray; extra == 'onprem'
|
|
166
|
+
Requires-Dist: safetensors; extra == 'onprem'
|
|
167
|
+
Requires-Dist: tensordict; extra == 'onprem'
|
|
168
|
+
Requires-Dist: torch>=2.9; extra == 'onprem'
|
|
169
|
+
Requires-Dist: tqdm; extra == 'onprem'
|
|
170
|
+
Requires-Dist: transformers==5.20.2; extra == 'onprem'
|
|
171
|
+
Requires-Dist: uvicorn; extra == 'onprem'
|
|
172
|
+
Requires-Dist: vllm==0.33.0; extra == 'onprem'
|
|
173
|
+
Requires-Dist: xgrammar==0.2.8; extra == 'onprem'
|
|
174
|
+
Provides-Extra: prime-rl
|
|
175
|
+
Requires-Dist: aiohttp; extra == 'prime-rl'
|
|
176
|
+
Requires-Dist: httpx; extra == 'prime-rl'
|
|
177
|
+
Requires-Dist: huggingface-hub; extra == 'prime-rl'
|
|
178
|
+
Requires-Dist: numpy; extra == 'prime-rl'
|
|
179
|
+
Requires-Dist: orjson; extra == 'prime-rl'
|
|
180
|
+
Requires-Dist: requests; extra == 'prime-rl'
|
|
181
|
+
Requires-Dist: safetensors; extra == 'prime-rl'
|
|
182
|
+
Requires-Dist: tenacity; extra == 'prime-rl'
|
|
183
|
+
Requires-Dist: torch; extra == 'prime-rl'
|
|
184
|
+
Requires-Dist: urllib3; extra == 'prime-rl'
|
|
185
|
+
Provides-Extra: rl
|
|
186
|
+
Requires-Dist: deepspeed>=0.19.9; extra == 'rl'
|
|
187
|
+
Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'rl'
|
|
188
|
+
Requires-Dist: fastapi; extra == 'rl'
|
|
189
|
+
Requires-Dist: huggingface-hub<3,>=2.0; extra == 'rl'
|
|
190
|
+
Requires-Dist: kvcached==0.1.4; extra == 'rl'
|
|
191
|
+
Requires-Dist: liger-kernel>=0.8.5; extra == 'rl'
|
|
192
|
+
Requires-Dist: numpy; extra == 'rl'
|
|
193
|
+
Requires-Dist: nvidia-ml-py; extra == 'rl'
|
|
194
|
+
Requires-Dist: peft<0.22,>=0.21.2; extra == 'rl'
|
|
195
|
+
Requires-Dist: psutil; extra == 'rl'
|
|
196
|
+
Requires-Dist: pydantic>=2.0; extra == 'rl'
|
|
197
|
+
Requires-Dist: quack-kernels>=0.4.0; extra == 'rl'
|
|
198
|
+
Requires-Dist: ray; extra == 'rl'
|
|
199
|
+
Requires-Dist: safetensors; extra == 'rl'
|
|
200
|
+
Requires-Dist: tensordict; extra == 'rl'
|
|
201
|
+
Requires-Dist: torch>=2.9; extra == 'rl'
|
|
202
|
+
Requires-Dist: tqdm; extra == 'rl'
|
|
203
|
+
Requires-Dist: transformers==5.20.2; extra == 'rl'
|
|
204
|
+
Requires-Dist: uvicorn; extra == 'rl'
|
|
205
|
+
Requires-Dist: vllm==0.33.0; extra == 'rl'
|
|
206
|
+
Requires-Dist: xgrammar==0.2.8; extra == 'rl'
|
|
207
|
+
Provides-Extra: sft
|
|
208
|
+
Requires-Dist: deepspeed>=0.19.9; extra == 'sft'
|
|
209
|
+
Requires-Dist: huggingface-hub<3,>=2.0; extra == 'sft'
|
|
210
|
+
Requires-Dist: liger-kernel>=0.8.5; extra == 'sft'
|
|
211
|
+
Requires-Dist: numpy; extra == 'sft'
|
|
212
|
+
Requires-Dist: nvidia-ml-py; extra == 'sft'
|
|
213
|
+
Requires-Dist: peft<0.22,>=0.21.2; extra == 'sft'
|
|
214
|
+
Requires-Dist: psutil; extra == 'sft'
|
|
215
|
+
Requires-Dist: quack-kernels>=0.4.0; extra == 'sft'
|
|
216
|
+
Requires-Dist: ray; extra == 'sft'
|
|
217
|
+
Requires-Dist: safetensors; extra == 'sft'
|
|
218
|
+
Requires-Dist: tensordict; extra == 'sft'
|
|
219
|
+
Requires-Dist: torch>=2.9; extra == 'sft'
|
|
220
|
+
Requires-Dist: transformers==5.20.2; extra == 'sft'
|
|
221
|
+
Provides-Extra: sonicmoe
|
|
222
|
+
Requires-Dist: quack-kernels>=0.4.0; extra == 'sonicmoe'
|
|
223
|
+
Provides-Extra: testing
|
|
224
|
+
Requires-Dist: aiohttp; extra == 'testing'
|
|
225
|
+
Requires-Dist: certifi; extra == 'testing'
|
|
226
|
+
Requires-Dist: dockerfile-parse>=2.0.1; extra == 'testing'
|
|
227
|
+
Requires-Dist: harbor==0.25.0; extra == 'testing'
|
|
228
|
+
Requires-Dist: httpx; extra == 'testing'
|
|
229
|
+
Requires-Dist: numpy; extra == 'testing'
|
|
230
|
+
Requires-Dist: openhands-sdk==1.7.1; extra == 'testing'
|
|
231
|
+
Requires-Dist: openhands-tools==1.7.1; extra == 'testing'
|
|
232
|
+
Requires-Dist: parameterized; extra == 'testing'
|
|
233
|
+
Requires-Dist: pathspec>=1.0.3; extra == 'testing'
|
|
234
|
+
Requires-Dist: pytest; extra == 'testing'
|
|
235
|
+
Requires-Dist: pytest-asyncio; extra == 'testing'
|
|
236
|
+
Requires-Dist: pytest-flakefinder; extra == 'testing'
|
|
237
|
+
Requires-Dist: pytest-instafail; extra == 'testing'
|
|
238
|
+
Requires-Dist: pytest-sugar; extra == 'testing'
|
|
239
|
+
Requires-Dist: pytest-timeout; extra == 'testing'
|
|
240
|
+
Requires-Dist: pytest-xdist; extra == 'testing'
|
|
241
|
+
Requires-Dist: requests; extra == 'testing'
|
|
242
|
+
Requires-Dist: safetensors; extra == 'testing'
|
|
243
|
+
Requires-Dist: snowflake-sandbox-python==0.2.2a5; extra == 'testing'
|
|
244
|
+
Requires-Dist: tenacity; extra == 'testing'
|
|
245
|
+
Requires-Dist: tinker==0.25.0; extra == 'testing'
|
|
246
|
+
Requires-Dist: torch; extra == 'testing'
|
|
247
|
+
Requires-Dist: transformers; extra == 'testing'
|
|
248
|
+
Requires-Dist: urllib3; extra == 'testing'
|
|
249
|
+
Provides-Extra: tinker
|
|
250
|
+
Requires-Dist: aiohttp; extra == 'tinker'
|
|
251
|
+
Requires-Dist: numpy; extra == 'tinker'
|
|
252
|
+
Requires-Dist: requests; extra == 'tinker'
|
|
253
|
+
Requires-Dist: safetensors; extra == 'tinker'
|
|
254
|
+
Requires-Dist: tenacity; extra == 'tinker'
|
|
255
|
+
Requires-Dist: torch; extra == 'tinker'
|
|
256
|
+
Requires-Dist: transformers; extra == 'tinker'
|
|
257
|
+
Requires-Dist: urllib3; extra == 'tinker'
|
|
258
|
+
Provides-Extra: trl
|
|
259
|
+
Requires-Dist: accelerate; extra == 'trl'
|
|
260
|
+
Requires-Dist: deepspeed>=0.19.9; extra == 'trl'
|
|
261
|
+
Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'trl'
|
|
262
|
+
Requires-Dist: fastapi; extra == 'trl'
|
|
263
|
+
Requires-Dist: huggingface-hub<3,>=2.0; extra == 'trl'
|
|
264
|
+
Requires-Dist: kvcached==0.1.4; extra == 'trl'
|
|
265
|
+
Requires-Dist: liger-kernel>=0.8.5; extra == 'trl'
|
|
266
|
+
Requires-Dist: numpy; extra == 'trl'
|
|
267
|
+
Requires-Dist: nvidia-ml-py; extra == 'trl'
|
|
268
|
+
Requires-Dist: peft<0.22,>=0.21.2; extra == 'trl'
|
|
269
|
+
Requires-Dist: psutil; extra == 'trl'
|
|
270
|
+
Requires-Dist: pydantic>=2.0; extra == 'trl'
|
|
271
|
+
Requires-Dist: quack-kernels>=0.4.0; extra == 'trl'
|
|
272
|
+
Requires-Dist: ray; extra == 'trl'
|
|
273
|
+
Requires-Dist: safetensors; extra == 'trl'
|
|
274
|
+
Requires-Dist: tensordict; extra == 'trl'
|
|
275
|
+
Requires-Dist: torch>=2.9; extra == 'trl'
|
|
276
|
+
Requires-Dist: tqdm; extra == 'trl'
|
|
277
|
+
Requires-Dist: transformers==5.20.2; extra == 'trl'
|
|
278
|
+
Requires-Dist: uvicorn; extra == 'trl'
|
|
279
|
+
Requires-Dist: vllm==0.33.0; extra == 'trl'
|
|
280
|
+
Requires-Dist: xgrammar==0.2.8; extra == 'trl'
|
|
281
|
+
Provides-Extra: verl
|
|
282
|
+
Requires-Dist: codetiming; extra == 'verl'
|
|
283
|
+
Requires-Dist: deepspeed>=0.19.9; extra == 'verl'
|
|
284
|
+
Requires-Dist: exceptiongroup>=1.0; (python_version < '3.11') and extra == 'verl'
|
|
285
|
+
Requires-Dist: fastapi; extra == 'verl'
|
|
286
|
+
Requires-Dist: huggingface-hub<3,>=2.0; extra == 'verl'
|
|
287
|
+
Requires-Dist: kvcached==0.1.4; extra == 'verl'
|
|
288
|
+
Requires-Dist: liger-kernel>=0.8.5; extra == 'verl'
|
|
289
|
+
Requires-Dist: numpy; extra == 'verl'
|
|
290
|
+
Requires-Dist: nvidia-ml-py; extra == 'verl'
|
|
291
|
+
Requires-Dist: omegaconf; extra == 'verl'
|
|
292
|
+
Requires-Dist: peft<0.22,>=0.21.2; extra == 'verl'
|
|
293
|
+
Requires-Dist: psutil; extra == 'verl'
|
|
294
|
+
Requires-Dist: pydantic>=2.0; extra == 'verl'
|
|
295
|
+
Requires-Dist: quack-kernels>=0.4.0; extra == 'verl'
|
|
296
|
+
Requires-Dist: ray; extra == 'verl'
|
|
297
|
+
Requires-Dist: safetensors; extra == 'verl'
|
|
298
|
+
Requires-Dist: tensordict; extra == 'verl'
|
|
299
|
+
Requires-Dist: torch>=2.9; extra == 'verl'
|
|
300
|
+
Requires-Dist: tqdm; extra == 'verl'
|
|
301
|
+
Requires-Dist: transformers==5.20.2; extra == 'verl'
|
|
302
|
+
Requires-Dist: uvicorn; extra == 'verl'
|
|
303
|
+
Requires-Dist: vllm==0.33.0; extra == 'verl'
|
|
304
|
+
Requires-Dist: xgrammar==0.2.8; extra == 'verl'
|
|
305
|
+
Description-Content-Type: text/markdown
|
|
306
|
+
|
|
307
|
+
[](https://github.com/Snowflake-AI-Research/Arctic-Platform/blob/main/LICENSE)
|
|
308
|
+
[](https://pypi.org/project/arctic-platform/)
|
|
309
|
+
|
|
310
|
+
<h3 align="center">
|
|
311
|
+
<img src="docs/images/arctic_platform_logo.svg" width=500px><br>
|
|
312
|
+
</h3>
|
|
313
|
+
|
|
314
|
+
# Arctic Platform: Simplifying and Accelerating Post-Training for LLMs
|
|
315
|
+
|
|
316
|
+
Arctic Platform is a framework for addressing challenges in current post-training frameworks, such as limited support for rapid prototyping and the lack of native data generation tools, by offering modularity across training and inference components, simplified code structures, and integrated pipelines for creating and cleaning synthetic data. These features enable users to enhance LLM capabilities, like code generation and complex reasoning, with greater efficiency and flexibility.
|
|
317
|
+
|
|
318
|
+
For example in the RL world if one were to compare various frameworks, one will find that each framework implements its own backends, adds fragmented system optimizations and the overall performance isn't quite portable from framework to framework. Thus the intention is to provide consistent building blocks that can be easily integrated into multiple frameworks while providing:
|
|
319
|
+
|
|
320
|
+
- Unified GPU Backends, e.g. RL on-prem or remote backends
|
|
321
|
+
- Unified System Optimizations leading to High Performance Portability
|
|
322
|
+
|
|
323
|
+
This is a work in progress, starting with the RL components, later integrating more training and inference components.
|
|
324
|
+
|
|
325
|
+
# Project Scope
|
|
326
|
+
|
|
327
|
+
Arctic Platform aims to cover the full post-training stack for LLMs behind a small, composable API. The codebase is being built out incrementally.
|
|
328
|
+
|
|
329
|
+
What is available today:
|
|
330
|
+
|
|
331
|
+
* [**Arctic Reinforcement Learning**](#arctic-reinforcement-learning) — a high-throughput RL training/inference backend that plugs into existing RL frameworks ([docs/rl.md](docs/rl.md)).
|
|
332
|
+
* [**ZoRRo Train**](#zorro-train) — a prompt-deduplication optimization that removes redundant prompt computation during RL training ([docs/rl.md#zorro-train](docs/rl.md#zorro-train)).
|
|
333
|
+
* [**ZoRRo Inference**](#zorro-inference) — forest cascade attention for efficient rollout step that eliminates redundant memory accesses via grouping ([docs/rl.md#zorro-inference-forest-cascade-attention](docs/rl.md#zorro-inference-forest-cascade-attention)).
|
|
334
|
+
|
|
335
|
+
Full documentation index: [docs/index.md](docs/index.md) (RL, shared server infra; SFT docs forthcoming).
|
|
336
|
+
|
|
337
|
+
What's coming next:
|
|
338
|
+
|
|
339
|
+
* additional trainers (SFT/distillation), synthetic data generation and cleaning pipelines, and tighter inference integration.
|
|
340
|
+
|
|
341
|
+
## Arctic Reinforcement Learning
|
|
342
|
+
|
|
343
|
+
Arctic RL is designed to **integrate into existing RL frameworks** rather than replace them. The RL framework keeps ownership of the training loop, rollouts, rewards, and advantage estimation; Arctic Platform provides the heavy compute engines behind a thin client:
|
|
344
|
+
|
|
345
|
+
* **Training engine** — a DeepSpeed engine that runs forward/backward and the optimizer step.
|
|
346
|
+
* **Log-prob / reference engine** — a forward-only DeepSpeed engine for reference / old log-prob computation.
|
|
347
|
+
* **Sampling engine** — a vLLM engine with ArcticInference for fast rollouts.
|
|
348
|
+
|
|
349
|
+
These engines are orchestrated over [Ray](https://www.ray.io/), can be **colocated** on shared GPUs (via fractional Ray resources) or split across separate GPUs, and keep the sampler in sync with the trainer through NCCL or CUDA-IPC weight transfer. The RL framework interacts with the compute engines over Ray. ArcticInference extends the core optimizations with multi-replica scheduling, load-balancing, weight-sync, and router-replay.
|
|
350
|
+
|
|
351
|
+
An RL framework integrates this module by constructing a client and driving the standard operations (generate, forward/backward, optimizer step, sync_weights, and wake/sleep for memory management). For example, on a single 8-GPU node one could do:
|
|
352
|
+
|
|
353
|
+
```python
|
|
354
|
+
from arctic_platform.rl import ArcticRLClientConfig, create_arctic_rl_client
|
|
355
|
+
|
|
356
|
+
config = ArcticRLClientConfig(
|
|
357
|
+
model_name="Qwen/Qwen3-4B",
|
|
358
|
+
training_gpus=8, #
|
|
359
|
+
sampling_gpus=8,
|
|
360
|
+
log_prob_gpus=0,
|
|
361
|
+
colocate=True,
|
|
362
|
+
)
|
|
363
|
+
client = create_arctic_rl_client(config)
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
### Integrated
|
|
367
|
+
|
|
368
|
+
1. [SkyRL](https://github.com/NovaSky-AI/SkyRL/pull/1837). [Documentation](https://github.com/NovaSky-AI/SkyRL/tree/main/integrations/arctic_rl)
|
|
369
|
+
2. TRL on Cortex. GSM8K GRPO from a CPU driver: [recipes/rl/trl/gsm8k_cortex](recipes/rl/trl/gsm8k_cortex).
|
|
370
|
+
|
|
371
|
+
### Integration is done but not yet merged
|
|
372
|
+
|
|
373
|
+
1. Verl. The reference integration is [verl](https://github.com/verl-project/verl) ([https://github.com/verl-project/verl/pull/6422](https://github.com/verl-project/verl/pull/6422)), which drives Arctic RL from its PPO/GRPO trainer. End-to-end recipes live under [recipes/rl/verl](recipes/rl/verl/), including a [simple single-GPU GSM8K](https://github.com/Snowflake-AI-Research/Arctic-Platform/tree/main/recipes/rl/verl/simple) starter, [Txt2SQL](https://github.com/Snowflake-AI-Research/Arctic-Platform/tree/main/recipes/rl/verl/txt2sql) and [long-context QA](https://github.com/Snowflake-AI-Research/Arctic-Platform/tree/main/recipes/rl/verl/long_context_qa).
|
|
374
|
+
2. TRL. The adapter lives under [`arctic_platform/integrations/trl/`](arctic_platform/integrations/trl/) and drives Arctic RL from TRL's `AsyncGRPOTrainer`. End-to-end recipes live under [recipes/rl/trl](recipes/rl/trl/), including a [GSM8K](recipes/rl/trl/gsm8k) starter and [Txt2SQL (BIRD)](recipes/rl/trl/txt2sql).
|
|
375
|
+
|
|
376
|
+
### Upcoming integrations
|
|
377
|
+
|
|
378
|
+
Multiple additional frameworks integrations are in the works and will be added here once available. We are in various stages of planning or working on integration into Axolotl, unsloth, PrimeRL and potentially others.
|
|
379
|
+
|
|
380
|
+
### ZoRRo Train
|
|
381
|
+
|
|
382
|
+
ZoRRo stands for Zero Redundancy Rollouts.
|
|
383
|
+
|
|
384
|
+
In RL training (PPO/GRPO) the same prompt is sampled many times to explore different responses, therefore for long sequence tasks **80–95% of the tokens in a batch are redundant prompt tokens** — and with transformer attention’s O(n²) cost, recomputing those shared prompts dominates the bill for long-context RL.
|
|
385
|
+
|
|
386
|
+
**ZoRRo Train** eliminates that waste with automatic **prompt deduplication** at all levels: it detects sequences that share a prompt, packs each unique prompt once, runs the model a single time over the deduplicated sequence, and transparently reconstructs per-response logprobs/entropy in the original sample order. The result is mathematically equivalent to the naive forward/backward (gradients match the baseline within numerical precision) while substantially cutting memory use and increasing throughput — the longer and more-shared the prompts, the larger the win.
|
|
387
|
+
|
|
388
|
+
It is installed transparently by the DeepSpeed training/log-prob engines and toggled per run via `ds_worker_config.zorro_train_enable` (verl yaml: `remote_backend.train.zorro_train.enable`). See [docs/rl.md § ZoRRo Train](docs/rl.md#zorro-train).
|
|
389
|
+
|
|
390
|
+
The supported model families span dense and MoE models - see the list [here](arctic_platform/rl/zorro_train/README.md#supported-models).
|
|
391
|
+
|
|
392
|
+
See [arctic_platform/rl/zorro_train/README.md](arctic_platform/rl/zorro_train/README.md) for the full design, the deduplication/attention internals, and benchmarks.
|
|
393
|
+
|
|
394
|
+
### ZoRRo Inference
|
|
395
|
+
|
|
396
|
+
During RL rollouts many sequences are generated from the same prompt. In the decode step, standard attention re-reads the KV cache of those shared prefixes **once per request**, so the sampler spends most of its memory bandwidth fetching identical keys and values over and over.
|
|
397
|
+
|
|
398
|
+
**ZoRRo Inference** removes that waste with **Forest Cascade Attention (FCA)**, which deduplicates shared KV reads at the attention layer of the sampling engine. For each decode batch, FCA discovers groups of requests that share a KV-cache prefix. Then it splits each attention call into a single grouped pass over the shared prefix blocks plus a per-request pass over the unique suffix blocks. FCA then reduces the two partial results with rigorous weighting. This reads each shared prefix block once per *group* instead of once per *request*, cutting redundant memory accesses while remaining mathematically equivalent to standard attention — the longer and more-shared the prefixes, the larger the win.
|
|
399
|
+
|
|
400
|
+
It is implemented in the vLLM sampling engine and activates transparently for decode-heavy batches with shared prefixes.
|
|
401
|
+
|
|
402
|
+
See the [Forest Cascade Attention README](https://github.com/snowflakedb/ArcticInference/tree/main/arctic_inference/vllm/attention) in Arctic Inference for the full design, the grouping/attention internals, and the tuning knobs.
|
|
403
|
+
|
|
404
|
+
# Quickstart
|
|
405
|
+
|
|
406
|
+
To get started training a model with Arctic Platform, first [install the package](#installation), then follow [the recipes](recipes/rl/).
|
|
407
|
+
|
|
408
|
+
# Installation
|
|
409
|
+
|
|
410
|
+
## From PyPI
|
|
411
|
+
|
|
412
|
+
The base install carries the config models only. Pick the extra for the backend you are training against, from [PyPI](https://pypi.org/project/arctic-platform/):
|
|
413
|
+
|
|
414
|
+
```shell
|
|
415
|
+
pip install "arctic-platform[cortex]" # drive Cortex training over SnowAPI
|
|
416
|
+
pip install "arctic-platform[sft]" # run a local training-only server
|
|
417
|
+
pip install "arctic-platform[rl]" # ...plus the sampling stack (arctic-inference, vLLM)
|
|
418
|
+
pip install "arctic-platform[verl]" # the verl adapter
|
|
419
|
+
pip install "arctic-platform[trl]" # the TRL adapter
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
`[cortex]` is by far the lightest: it skips DeepSpeed, transformers, Ray and vLLM entirely. Importing on-prem code without the matching extra raises an error naming the one to install.
|
|
423
|
+
|
|
424
|
+
While this project is being very actively developed it's probably better to install directly from [git](#from-source-git).
|
|
425
|
+
|
|
426
|
+
## From source (git)
|
|
427
|
+
|
|
428
|
+
To get the latest development version (or to contribute), clone the repository and install it in editable mode:
|
|
429
|
+
|
|
430
|
+
```shell
|
|
431
|
+
git clone https://github.com/Snowflake-AI-Research/Arctic-Platform.git
|
|
432
|
+
cd Arctic-Platform
|
|
433
|
+
pip install -e ".[rl]"
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
# Citation
|
|
437
|
+
|
|
438
|
+
If you use Arctic Platform in your research, please consider citing:
|
|
439
|
+
|
|
440
|
+
```bibtex
|
|
441
|
+
@misc{arctic_platform_2025,
|
|
442
|
+
title={Arctic Platform: Simplifying and Accelerating Post-Training for LLMs},
|
|
443
|
+
author={Snowflake AI Research},
|
|
444
|
+
year={2025},
|
|
445
|
+
howpublished={\url{https://github.com/Snowflake-AI-Research/Arctic-Platform}}
|
|
446
|
+
}
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
# License
|
|
450
|
+
|
|
451
|
+
Apache License 2.0. See [LICENSE](https://github.com/Snowflake-AI-Research/Arctic-Platform/blob/main/LICENSE).
|