@wenathlan/saddle 1.8.2 → 1.8.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -9
- package/browser/playwright.js +22 -0
- package/docs/.gitkeep +0 -0
- package/docs/gapmatrix.md +3 -3
- package/docs/libraryapi.md +1 -0
- package/docs/logs/.gitkeep +0 -0
- package/docs/packageaudit185.md +23 -0
- package/docs/plans/00.index.md +50 -0
- package/docs/plans/01.architecture.md +86 -0
- package/docs/plans/02.research.computer.use.md +58 -0
- package/docs/plans/03.research.captcha.bypass.md +68 -0
- package/docs/plans/04.research.sandbox.ai.md +52 -0
- package/docs/plans/05.capture.platform.md +57 -0
- package/docs/plans/06.dependencies.md +97 -0
- package/docs/plans/07.captcha.test.page.md +41 -0
- package/docs/plans/08.production.infra.md +70 -0
- package/docs/plans/09.database.schema.md +121 -0
- package/docs/plans/10.cloudinary.storage.md +57 -0
- package/docs/plans/11.movement.logs.json.md +72 -0
- package/docs/plans/12.research.atlas.agent.browser.md +79 -0
- package/docs/plans/13.research.anti.detection.md +898 -0
- package/docs/plans/14.research.proxy.md +1495 -0
- package/docs/plans/15.research.retry.rate.limit.md +1958 -0
- package/docs/plans/16.research.crawling.md +1417 -0
- package/docs/plans/17.research.caching.md +1610 -0
- package/docs/plans/18.research.content.extraction.md +1952 -0
- package/docs/plans/19.research.errors.events.md +1523 -0
- package/docs/plans/20.research.zod.validation.md +1350 -0
- package/docs/plans/21.research.batch.concurrency.md +1888 -0
- package/docs/plans/22.research.universal.runtime.md +944 -0
- package/docs/plans/23.research.ai.integration.md +1465 -0
- package/docs/plans/24.research.memory.persistence.md +1979 -0
- package/docs/plans/25.research.server.api.md +342 -0
- package/docs/plans/26.research.compilation.md +249 -0
- package/docs/plans/27.research.html.parsing.md +251 -0
- package/docs/plans/28.action.plan.md +50 -0
- package/docs/plans/29.api.reference.md +174 -0
- package/docs/plans/30.architecture.plan.md +94 -0
- package/docs/plans/31.auditoria.dados.md +163 -0
- package/docs/plans/32.bots.automacao.computacional.md +214 -0
- package/docs/plans/33.bots.codigo.revisao.md +220 -0
- package/docs/plans/34.bots.seguranca.cicd.md +366 -0
- package/docs/plans/35.comparativo.concorrencia.md +464 -0
- package/docs/plans/36.computational.memory.md +340 -0
- package/docs/plans/37.deploystrategy.md +394 -0
- package/docs/plans/38.flow.md +155 -0
- package/docs/plans/39.multi.platform.bot.md +252 -0
- package/docs/plans/40.npm.publish.md +250 -0
- package/docs/plans/41.o.que.falta.md +407 -0
- package/docs/plans/42.pesquisa.concorrencia.md +721 -0
- package/docs/plans/43.plan.universal.architecture.md +496 -0
- package/docs/plans/44.reference.md +100 -0
- package/docs/plans/45.robotarchitecture.md +237 -0
- package/docs/plans/46.scdnintegration.md +284 -0
- package/docs/plans/47.multiforge.readme.md +129 -0
- package/docs/plans/48.theory.v4.repo.os.md +152 -0
- package/docs/plans/49.third.party.infra.md +12 -0
- package/docs/plans/50.file.as.compute.md +39 -0
- package/docs/plans/51.architecture.virtual.processor.md +80 -0
- package/docs/plans/52.manifesto.v8.md +11 -0
- package/docs/plans/58.cdn.list.md +23 -0
- package/docs/plans/59.sql.frameworks.md +33 -0
- package/docs/plans/60.sql.thirdparty.md +26 -0
- package/docs/plans/61.objective.multiforge.md +63 -0
- package/docs/plans/62.huggingface.upload.md +26 -0
- package/docs/plans/63.kaggle.upload.md +24 -0
- package/docs/plans/64.npm.storage.md +30 -0
- package/docs/plans/65.rclone.terabox.md +32 -0
- package/docs/plans/66.buckets.and.models.todo.md +14 -0
- package/docs/plans/67.database.todo.md +13 -0
- package/docs/plans/68.deploy.packages.todo.md +12 -0
- package/docs/plans/69.report.human.operator.md +133 -0
- package/docs/plans/70.report.brain2qwerty.ems.md +135 -0
- package/docs/plans/71.report.hd.infinito.vram.md +155 -0
- package/docs/plans/72.plan.hd.infinito.node.md +146 -0
- package/docs/plans/73.plan.scifi.repos.md +125 -0
- package/docs/plans/74.000.manifesto.v8.flat.2..md +11 -0
- package/docs/plans/README.md +489 -0
- package/docs/plans/aggregate_platforms.mjs +146 -0
- package/docs/plans/examplesession.json +36 -0
- package/docs/plans/missing-facts.md +192 -0
- package/docs/plans/models.md +64 -0
- package/docs/plans/organize.cjs +270 -0
- package/docs/plans/platforms.md +2887 -0
- package/docs/plans/sites.md +31322 -0
- package/docs/platformpipelineaudit.md +18 -0
- package/docs/platformpipelines.md +13 -0
- package/docs/registryresearch.md +4 -0
- package/docs/release.md +4 -4
- package/docs/release182notes.md +2 -0
- package/docs/release184notes.md +7 -0
- package/docs/release185notes.md +7 -0
- package/docs/sources/farm.py +117 -0
- package/docs/sources/html/saddle1.html +132 -0
- package/docs/sources/html/saddle2.html +157 -0
- package/docs/sources/html/saddle3.html +119 -0
- package/docs/sources/html/saddle4.html +144 -0
- package/docs/sources/html/saddle5.html +72 -0
- package/docs/sources/html/saddle6.html +171 -0
- package/docs/sources/html/saddle7.html +236 -0
- package/docs/sources/saddle.ts +74 -0
- package/docs/sources/schema.prisma +88 -0
- package/docs/sources/script.sh +64 -0
- package/docs/sources/workflows.yml +458 -0
- package/docs/talks1/_body.txt +14 -0
- package/docs/talks1/_index.md +15 -0
- package/docs/talks1/_screenshot.png +0 -0
- package/docs/talks1/assistant-01.md +5 -0
- package/docs/talks1/assistant-02.md +5 -0
- package/docs/talks1/assistant-03.md +531 -0
- package/docs/talks1/assistant-04.md +26 -0
- package/docs/talks1/assistant-05.md +774 -0
- package/docs/talks1/assistant-06.md +1718 -0
- package/docs/talks1/scrape-share.cjs +185 -0
- package/docs/talks1/scrape-share.ts +183 -0
- package/docs/talks1/user-01.md +3 -0
- package/docs/talks1/user-02.md +3 -0
- package/docs/talks1/user-03.md +88 -0
- package/docs/talks1/user-04.md +3 -0
- package/docs/talks1/user-05.md +3 -0
- package/docs/talks1/user-06.md +88 -0
- package/docs/talks1/user-07.md +88 -0
- package/docs/talks2/_body.txt +14 -0
- package/docs/talks2/_index.md +16 -0
- package/docs/talks2/_screenshot.png +0 -0
- package/docs/talks2/assistant-01.md +5 -0
- package/docs/talks2/assistant-02.md +5 -0
- package/docs/talks2/assistant-03.md +424 -0
- package/docs/talks2/assistant-04.md +598 -0
- package/docs/talks2/assistant-05.md +1280 -0
- package/docs/talks2/assistant-06.md +1227 -0
- package/docs/talks2/assistant-07.md +1252 -0
- package/docs/talks2/user-01.md +3 -0
- package/docs/talks2/user-02.md +3 -0
- package/docs/talks2/user-03.md +88 -0
- package/docs/talks2/user-04.md +88 -0
- package/docs/talks2/user-05.md +88 -0
- package/docs/talks2/user-06.md +88 -0
- package/docs/talks2/user-07.md +3 -0
- package/docs/talks3/_body.txt +467 -0
- package/docs/talks3/_index.md +10 -0
- package/docs/talks3/_screenshot.png +0 -0
- package/docs/talks3/assistant-01.md +417 -0
- package/docs/talks3/assistant-02.md +417 -0
- package/docs/talks3/assistant-03.md +29 -0
- package/docs/talks3/assistant-04.md +727 -0
- package/docs/talks3/user-01.md +88 -0
- package/docs/talks3/user-02.md +88 -0
- package/docs/talks3/user-03.md +3 -0
- package/docs/talks3/user-04.md +3 -0
- package/docs/talks4/_body.txt +14 -0
- package/docs/talks4/_index.md +12 -0
- package/docs/talks4/_screenshot.png +0 -0
- package/docs/talks4/assistant-01.md +5 -0
- package/docs/talks4/assistant-02.md +5 -0
- package/docs/talks4/assistant-03.md +35 -0
- package/docs/talks4/assistant-04.md +512 -0
- package/docs/talks4/assistant-05.md +599 -0
- package/docs/talks4/user-01.md +3 -0
- package/docs/talks4/user-02.md +3 -0
- package/docs/talks4/user-03.md +88 -0
- package/docs/talks4/user-04.md +88 -0
- package/docs/talks4/user-05.md +7 -0
- package/docs/talks5/_body.txt +14 -0
- package/docs/talks5/_index.md +13 -0
- package/docs/talks5/_screenshot.png +0 -0
- package/docs/talks5/assistant-01.md +5 -0
- package/docs/talks5/assistant-02.md +5 -0
- package/docs/talks5/assistant-03.md +690 -0
- package/docs/talks5/assistant-04.md +758 -0
- package/docs/talks5/assistant-05.md +974 -0
- package/docs/talks5/user-01.md +3 -0
- package/docs/talks5/user-02.md +3 -0
- package/docs/talks5/user-03.md +105 -0
- package/docs/talks5/user-04.md +105 -0
- package/docs/talks5/user-05.md +63 -0
- package/docs/talks5/user-06.md +105 -0
- package/docs/talks6/_body.txt +14 -0
- package/docs/talks6/_index.md +9 -0
- package/docs/talks6/_screenshot.png +0 -0
- package/docs/talks6/assistant-01.md +5 -0
- package/docs/talks6/assistant-02.md +5 -0
- package/docs/talks6/assistant-03.md +1499 -0
- package/docs/talks6/user-01.md +3 -0
- package/docs/talks6/user-02.md +3 -0
- package/docs/talks6/user-03.md +88 -0
- package/docs/talks6/user-04.md +88 -0
- package/docs/talks7/_body.txt +14 -0
- package/docs/talks7/_index.md +10 -0
- package/docs/talks7/_screenshot.png +0 -0
- package/docs/talks7/assistant-01.md +5 -0
- package/docs/talks7/assistant-02.md +5 -0
- package/docs/talks7/assistant-03.md +523 -0
- package/docs/talks7/assistant-04.md +617 -0
- package/docs/talks7/user-01.md +3 -0
- package/docs/talks7/user-02.md +3 -0
- package/docs/talks7/user-03.md +105 -0
- package/docs/talks7/user-04.md +67 -0
- package/docs/talks8/conversa1.txt +1322 -0
- package/docs/talks8/conversa2.txt +237 -0
- package/docs/talks9/Beyond the Obvious_ 50 Plataformas Auto-Hospedadas de Forja de C/303/263digo para Al/303/251m de Gitea e GitLab.md" +174 -0
- package/docs/talks9/De NPM a Multi-Linguagem_ Uma Arquitetura T/303/251cnica para a Execu/303/247/303/243o Integrada de C/303/263digo no Ecossistema Node.js.md" +59 -0
- package/docs/talks9/De NPM a VMs Virtuais_ Uma An/303/241lise Arquitet/303/264nica para a Realiza/303/247/303/243o do Ciclo de Vida do Projeto SADDLE.md" +91 -0
- package/docs/talks9/Mapeamento da Engrenagem Computacional_ Uma Arquitetura para Execu/303/247/303/243o Isolada e Persist/303/252ncia em Ambientes Distribu/303/255dos.md" +116 -0
- package/docs/talks9/O Cen/303/241rio Pr/303/241tico do SADDLE_ Uma An/303/241lise de Viabilidade e Modelo de Ciclo de Vida Integrado.md" +128 -0
- package/docs/talks9/README (2).md +489 -0
- package/docs/talks9/README.md +198 -0
- package/docs/talks9/Viabilidade do Saddle_ Uma An/303/241lise T/303/251cnica da Transforma/303/247/303/243o de Armazenamento Remoto em Mem/303/263ria Computacional.md" +80 -0
- package/docs/talks9/conversa.txt +544 -0
- package/docs/talks9/other (2).md +39 -0
- package/docs/talks9/other.md +57 -0
- package/docs/talks9/outro.txt +24 -0
- package/extension/README.md +3 -3
- package/extension/serviceworker.js +63 -4
- package/extension/worker.js +3 -1
- package/package.json +12 -2
- package/packager/manifest.js +1 -1
- package/scrape/agent.ts +122 -0
- package/scrape/batch.ts +79 -0
- package/scrape/biome.json +76 -0
- package/scrape/browser.ts +222 -0
- package/scrape/cache.ts +84 -0
- package/scrape/chunking.ts +193 -0
- package/scrape/cli.ts +105 -0
- package/scrape/crawler.ts +115 -0
- package/scrape/dev-server.ts +94 -0
- package/scrape/errors.ts +132 -0
- package/scrape/events.ts +26 -0
- package/scrape/extract.ts +165 -0
- package/scrape/fetch.ts +105 -0
- package/scrape/formats.ts +85 -0
- package/scrape/headers.ts +71 -0
- package/scrape/index.ts +92 -0
- package/scrape/jsdom.d.ts +6 -0
- package/scrape/llms-txt.ts +84 -0
- package/scrape/middleware.ts +90 -0
- package/scrape/package-lock.json +9397 -0
- package/scrape/package.json +1420 -0
- package/scrape/pool.ts +95 -0
- package/scrape/port.ts +18 -0
- package/scrape/proxy.ts +103 -0
- package/scrape/rate-limiter.ts +95 -0
- package/scrape/renderer.ts +194 -0
- package/scrape/retry.ts +64 -0
- package/scrape/robots.ts +137 -0
- package/scrape/scrape.ts +123 -0
- package/scrape/serialize.ts +310 -0
- package/scrape/server.ts +137 -0
- package/scrape/session.ts +109 -0
- package/scrape/sitemap.ts +131 -0
- package/scrape/tokens.ts +45 -0
- package/scrape/tsconfig.json +28 -0
- package/scrape/types.ts +214 -0
- package/scrape/utils.ts +77 -0
- package/scrape/vite.config.ts +55 -0
- package/scrape/vitest.config.ts +17 -0
- package/workflow/templates.js +4 -4
|
@@ -0,0 +1,1718 @@
|
|
|
1
|
+
# assistant
|
|
2
|
+
|
|
3
|
+
[Thinking] The user is asking me to show them a step-by-step implementation plan for building a platform that combines all the technologies they've mentioned. They want to create a sandbox platform that uses:
|
|
4
|
+
|
|
5
|
+
Free infrastructure from git hosting (GitHub, GitLab, Gitea, Forgejo, Codeberg)
|
|
6
|
+
ML/AI platforms (Hugging Face, Kaggle, ModelScope)
|
|
7
|
+
Databases (Drizzle ORM, Prisma, MySQL2)
|
|
8
|
+
Memory virtualization (zram, tmpfs, shared memory)
|
|
9
|
+
GPU virtualization
|
|
10
|
+
Open source sandbox projects
|
|
11
|
+
NPM packages for CDN availability
|
|
12
|
+
|
|
13
|
+
They're in a Windows environment (PowerShell) and want to download and run scripts.I need to create a comprehensive implementation plan that addresses their core question: using free infrastructure from repositories to build a sandbox platform. They want to leverage storage from these repos and buckets, convert storage to RAM via zram and tmpfs, use pipelines for automation, and deploy everything through NPM packages via CDN. This is a substantial project, so I should focus on what's actually feasible with free resources and what they're really trying to accomplish.
|
|
14
|
+
|
|
15
|
+
[Thinking] The user wants me to do more research and show step-by-step implementation. They said the project name is "saddle" and they want me to do 10+ more searches and show detailed implementation. Let me do more research and then present a comprehensive plan.
|
|
16
|
+
They didn't answer the other questions, so I'll make reasonable assumptions and do more research. Let me search for more specific implementation details.
|
|
17
|
+
|
|
18
|
+
[Thinking] Now let me do a few more searches for the remaining topics.
|
|
19
|
+
|
|
20
|
+
[Thinking] Now I have comprehensive research. Let me compile a complete step-by-step implementation plan for the user's project "saddle".
|
|
21
|
+
|
|
22
|
+
Entendi. Você quer um plano completo de implementação que una tudo isso. Deixe-me estruturar um plano detalhado passo a passo.
|
|
23
|
+
Antes de criar o plano final, preciso esclarecer alguns pontos:
|
|
24
|
+
|
|
25
|
+
Aqui está o PLANO COMPLETO DE IMPLEMENTAÇÃO PASSO A PASSO para o projeto "saddle":
|
|
26
|
+
|
|
27
|
+
PLANO DE IMPLEMENTAÇÃO — saddle (Plataforma de Sandbox)
|
|
28
|
+
VISÃO GERAL DA ARQUITETURA
|
|
29
|
+
┌─────────────────────────────────────────────────────────────────┐
|
|
30
|
+
│ saddle PLATFORM │
|
|
31
|
+
│ │
|
|
32
|
+
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
|
|
33
|
+
│ │ API Gateway │ │ Dashboard │ │ CLI / SDK │ │
|
|
34
|
+
│ │ (Node.js) │ │ (Next.js) │ │ (npx saddle) │ │
|
|
35
|
+
│ └──────┬───────┘ └──────┬───────┘ └──────────┬───────────┘ │
|
|
36
|
+
│ │ │ │ │
|
|
37
|
+
│ ┌──────▼─────────────────▼──────────────────────▼───────────┐ │
|
|
38
|
+
│ │ ORCHESTRATOR │ │
|
|
39
|
+
│ │ Drizzle ORM + SQLite/PostgreSQL │ │
|
|
40
|
+
│ └──┬──────────┬──────────┬──────────┬──────────┬───────────┘ │
|
|
41
|
+
│ │ │ │ │ │ │
|
|
42
|
+
│ ┌──▼──┐ ┌───▼───┐ ┌───▼───┐ ┌───▼───┐ ┌───▼──────┐ │
|
|
43
|
+
│ │GitHub│ │GitLab │ │Gitea/ │ │ HF │ │ Kaggle │ │
|
|
44
|
+
│ │Action│ │ CI/CD │ │Forgejo│ │Spaces │ │ Notebooks│ │
|
|
45
|
+
│ └──┬──┘ └───┬───┘ └───┬───┘ └───┬───┘ └───┬──────┘ │
|
|
46
|
+
│ │ │ │ │ │ │
|
|
47
|
+
│ ┌──▼─────────▼──────────▼──────────▼──────────▼───────────┐ │
|
|
48
|
+
│ │ MEMORY BRIDGE (zram + tmpfs) │ │
|
|
49
|
+
│ │ Storage → RAM conversion para cada sandbox │ │
|
|
50
|
+
│ └─────────────────────────────────────────────────────────┘ │
|
|
51
|
+
│ │ │ │ │ │ │
|
|
52
|
+
│ ┌──▼──┐ ┌───▼───┐ ┌───▼───┐ ┌───▼───┐ ┌───▼──────┐ │
|
|
53
|
+
│ │Docker│ │Fire- │ │ LXC │ │gVisor │ │ HiveBox │ │
|
|
54
|
+
│ │ │ │cracker│ │ │ │ │ │ │ │
|
|
55
|
+
│ └──────┘ └───────┘ └───────┘ └───────┘ └──────────┘ │
|
|
56
|
+
└─────────────────────────────────────────────────────────────────┘
|
|
57
|
+
|
|
58
|
+
FASE 1: ESTRUTURA DO PROJETO
|
|
59
|
+
Passo 1.1: Criar monorepo
|
|
60
|
+
saddle/
|
|
61
|
+
├── package.json # Root package (workspace)
|
|
62
|
+
├── turbo.json # Turborepo config
|
|
63
|
+
├── .github/
|
|
64
|
+
│ └── workflows/
|
|
65
|
+
│ ├── ci.yml # CI do saddle
|
|
66
|
+
│ ├── release.yml # Release para npm
|
|
67
|
+
│ └── deploy.yml # Deploy sandbox via API
|
|
68
|
+
├── .gitlab-ci.yml # Mirror GitLab CI
|
|
69
|
+
├── .forgejo/
|
|
70
|
+
│ └── workflows/
|
|
71
|
+
│ └── ci.yml # Forgejo Actions CI
|
|
72
|
+
├── apps/
|
|
73
|
+
│ ├── api/ # API Gateway (Node.js + Hono)
|
|
74
|
+
│ ├── web/ # Dashboard (Next.js)
|
|
75
|
+
│ └── cli/ # CLI (npx saddle)
|
|
76
|
+
├── packages/
|
|
77
|
+
│ ├── core/ # Lógica principal do orchestrator
|
|
78
|
+
│ ├── memory-bridge/ # Conversor storage→RAM
|
|
79
|
+
│ ├── sandbox-docker/ # Provider Docker
|
|
80
|
+
│ ├── sandbox-firecracker/ # Provider Firecracker
|
|
81
|
+
│ ├── sandbox-github/ # Provider GitHub Actions
|
|
82
|
+
│ ├── sandbox-gitlab/ # Provider GitLab CI
|
|
83
|
+
│ ├── sandbox-gitea/ # Provider Gitea/Forgejo
|
|
84
|
+
│ ├── sandbox-hf/ # Provider Hugging Face
|
|
85
|
+
│ ├── sandbox-kaggle/ # Provider Kaggle
|
|
86
|
+
│ ├── db/ # Schema Drizzle ORM
|
|
87
|
+
│ └── shared/ # Tipos, utils
|
|
88
|
+
└── docker/
|
|
89
|
+
├── Dockerfile.api
|
|
90
|
+
├── Dockerfile.sandbox # Imagem base das sandboxes
|
|
91
|
+
└── docker-compose.yml
|
|
92
|
+
Passo 1.2: package.json raiz
|
|
93
|
+
{
|
|
94
|
+
"name": "saddle",
|
|
95
|
+
"version": "0.1.0",
|
|
96
|
+
"private": true,
|
|
97
|
+
"workspaces": ["apps/*", "packages/*"],
|
|
98
|
+
"scripts": {
|
|
99
|
+
"dev": "turbo dev",
|
|
100
|
+
"build": "turbo build",
|
|
101
|
+
"lint": "turbo lint",
|
|
102
|
+
"typecheck": "turbo typecheck",
|
|
103
|
+
"test": "turbo test",
|
|
104
|
+
"db:generate": "turbo db:generate",
|
|
105
|
+
"db:push": "turbo db:push",
|
|
106
|
+
"sandbox:create": "node apps/cli/dist/index.js create",
|
|
107
|
+
"sandbox:destroy": "node apps/cli/dist/index.js destroy"
|
|
108
|
+
},
|
|
109
|
+
"devDependencies": {
|
|
110
|
+
"turbo": "^2.0.0",
|
|
111
|
+
"typescript": "^5.5.0"
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
FASE 2: DATABASE (Drizzle ORM)
|
|
116
|
+
Passo 2.1: Schema do banco
|
|
117
|
+
// packages/db/schema.ts
|
|
118
|
+
import { sqliteTable, text, integer, real } from 'drizzle-orm/sqlite-core';
|
|
119
|
+
|
|
120
|
+
export const users = sqliteTable('users', {
|
|
121
|
+
id: text('id').primaryKey(),
|
|
122
|
+
name: text('name').notNull(),
|
|
123
|
+
email: text('email').notNull().unique(),
|
|
124
|
+
githubToken: text('github_token'),
|
|
125
|
+
gitlabToken: text('gitlab_token'),
|
|
126
|
+
giteaToken: text('gitea_token'),
|
|
127
|
+
hfToken: text('hf_token'),
|
|
128
|
+
kaggleUsername: text('kaggle_username'),
|
|
129
|
+
kaggleKey: text('kaggle_key'),
|
|
130
|
+
createdAt: integer('created_at', { mode: 'timestamp' }).notNull(),
|
|
131
|
+
});
|
|
132
|
+
|
|
133
|
+
export const sandboxes = sqliteTable('sandboxes', {
|
|
134
|
+
id: text('id').primaryKey(),
|
|
135
|
+
userId: text('user_id').references(() => users.id).notNull(),
|
|
136
|
+
name: text('name').notNull(),
|
|
137
|
+
status: text('status').notNull(), // creating, running, sleeping, stopped, destroyed
|
|
138
|
+
provider: text('provider').notNull(), // github, gitlab, gitea, hf, kaggle, docker, firecracker
|
|
139
|
+
containerId: text('container_id'),
|
|
140
|
+
repoUrl: text('repo_url'), // URL do repo criado no provider
|
|
141
|
+
pipelineId: text('pipeline_id'), // ID da pipeline ativa
|
|
142
|
+
|
|
143
|
+
// Recursos alocados
|
|
144
|
+
memoryMb: integer('memory_mb').default(4096),
|
|
145
|
+
cpuCores: integer('cpu_cores').default(2),
|
|
146
|
+
diskGb: integer('disk_gb').default(10),
|
|
147
|
+
|
|
148
|
+
// Memória bridge
|
|
149
|
+
storageToRamEnabled: integer('storage_to_ram_enabled', { mode: 'boolean' }).default(false),
|
|
150
|
+
zramSizeMb: integer('zram_size_mb').default(2048),
|
|
151
|
+
|
|
152
|
+
// Network
|
|
153
|
+
sshHost: text('ssh_host'),
|
|
154
|
+
sshPort: integer('ssh_port'),
|
|
155
|
+
webUrl: text('web_url'),
|
|
156
|
+
|
|
157
|
+
// Metadata
|
|
158
|
+
image: text('image').default('ubuntu:22.04'),
|
|
159
|
+
labels: text('labels'), // JSON
|
|
160
|
+
createdAt: integer('created_at', { mode: 'timestamp' }).notNull(),
|
|
161
|
+
expiresAt: integer('expires_at', { mode: 'timestamp' }),
|
|
162
|
+
lastActivityAt: integer('last_activity_at', { mode: 'timestamp' }),
|
|
163
|
+
});
|
|
164
|
+
|
|
165
|
+
export const pipelines = sqliteTable('pipelines', {
|
|
166
|
+
id: text('id').primaryKey(),
|
|
167
|
+
sandboxId: text('sandbox_id').references(() => sandboxes.id).notNull(),
|
|
168
|
+
provider: text('provider').notNull(),
|
|
169
|
+
externalId: text('external_id'), // ID no provider externo
|
|
170
|
+
status: text('status').notNull(), // pending, running, success, failed
|
|
171
|
+
trigger: text('trigger').notNull(), // manual, cron, webhook, api
|
|
172
|
+
output: text('output'),
|
|
173
|
+
createdAt: integer('created_at', { mode: 'timestamp' }).notNull(),
|
|
174
|
+
finishedAt: integer('finished_at', { mode: 'timestamp' }),
|
|
175
|
+
});
|
|
176
|
+
|
|
177
|
+
export const cronJobs = sqliteTable('cron_jobs', {
|
|
178
|
+
id: text('id').primaryKey(),
|
|
179
|
+
sandboxId: text('sandbox_id').references(() => sandboxes.id),
|
|
180
|
+
schedule: text('schedule').notNull(), // cron expression
|
|
181
|
+
command: text('command').notNull(),
|
|
182
|
+
lastRun: integer('last_run', { mode: 'timestamp' }),
|
|
183
|
+
nextRun: integer('next_run', { mode: 'timestamp' }),
|
|
184
|
+
enabled: integer('enabled', { mode: 'boolean' }).default(true),
|
|
185
|
+
createdAt: integer('created_at', { mode: 'timestamp' }).notNull(),
|
|
186
|
+
});
|
|
187
|
+
Passo 2.2: Config Drizzle
|
|
188
|
+
// packages/db/drizzle.config.ts
|
|
189
|
+
import { defineConfig } from 'drizzle-kit';
|
|
190
|
+
|
|
191
|
+
export default defineConfig({
|
|
192
|
+
schema: './schema.ts',
|
|
193
|
+
out: './migrations',
|
|
194
|
+
dialect: 'sqlite',
|
|
195
|
+
dbCredentials: {
|
|
196
|
+
url: process.env.DATABASE_URL || 'file:../data/saddle.db',
|
|
197
|
+
},
|
|
198
|
+
});
|
|
199
|
+
Passo 2.3: Conexão do banco
|
|
200
|
+
// packages/db/index.ts
|
|
201
|
+
import { drizzle } from 'drizzle-orm/better-sqlite3';
|
|
202
|
+
import Database from 'better-sqlite3';
|
|
203
|
+
import * as schema from './schema';
|
|
204
|
+
|
|
205
|
+
const sqlite = new Database(process.env.DATABASE_URL || './data/saddle.db');
|
|
206
|
+
export const db = drizzle(sqlite, { schema });
|
|
207
|
+
|
|
208
|
+
FASE 3: MEMORY BRIDGE (Storage → RAM)
|
|
209
|
+
Passo 3.1: Módulo memory-bridge
|
|
210
|
+
// packages/memory-bridge/index.ts
|
|
211
|
+
export { MemoryBridge } from './bridge';
|
|
212
|
+
export { ZramManager } from './zram';
|
|
213
|
+
export { TmpfsManager } from './tmpfs';
|
|
214
|
+
export type { MemoryConfig, MemoryStats } from './types';
|
|
215
|
+
Passo 3.2: Implementação do bridge
|
|
216
|
+
// packages/memory-bridge/bridge.ts
|
|
217
|
+
import { execSync } from 'child_process';
|
|
218
|
+
import { ZramManager } from './zram';
|
|
219
|
+
import { TmpfsManager } from './tmpfs';
|
|
220
|
+
|
|
221
|
+
export interface MemoryConfig {
|
|
222
|
+
sandboxId: string;
|
|
223
|
+
storagePath: string; // ex: /data/sandboxes/{id}
|
|
224
|
+
zramSizeMb: number; // Tamanho do zram (ex: 2048)
|
|
225
|
+
tmpfsSizeMb: number; // Tamanho do tmpfs (ex: 4096)
|
|
226
|
+
enableSwap: boolean; // Usar zram como swap
|
|
227
|
+
enableTmpfs: boolean; // Montar diretórios em RAM
|
|
228
|
+
compressionAlgo: string; // lz4, zstd, lzo
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
export interface MemoryStats {
|
|
232
|
+
zramUsed: number;
|
|
233
|
+
zramTotal: number;
|
|
234
|
+
tmpfsUsed: number;
|
|
235
|
+
tmpfsTotal: number;
|
|
236
|
+
swapUsed: number;
|
|
237
|
+
swapTotal: number;
|
|
238
|
+
compressionRatio: number;
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
export class MemoryBridge {
|
|
242
|
+
private zram: ZramManager;
|
|
243
|
+
private tmpfs: TmpfsManager;
|
|
244
|
+
private config: MemoryConfig;
|
|
245
|
+
|
|
246
|
+
constructor(config: MemoryConfig) {
|
|
247
|
+
this.config = config;
|
|
248
|
+
this.zram = new ZramManager(config);
|
|
249
|
+
this.tmpfs = new TmpfsManager(config);
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
async setup(): Promise<void> {
|
|
253
|
+
// 1. Criar dispositivo zram
|
|
254
|
+
if (this.config.enableSwap) {
|
|
255
|
+
await this.zram.create();
|
|
256
|
+
await this.zram.setCompression(this.config.compressionAlgo);
|
|
257
|
+
await this.zram.activateAsSwap();
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
// 2. Montar diretórios em RAM
|
|
261
|
+
if (this.config.enableTmpfs) {
|
|
262
|
+
await this.tmpfs.mount('/tmp', this.config.tmpfsSizeMb);
|
|
263
|
+
await this.tmpfs.mount(`${this.config.storagePath}/cache`, 512);
|
|
264
|
+
}
|
|
265
|
+
|
|
266
|
+
// 3. Configurar swappiness para priorizar zram
|
|
267
|
+
execSync('sysctl vm.swappiness=60');
|
|
268
|
+
}
|
|
269
|
+
|
|
270
|
+
async convertStorageToRam(sourceDir: string, targetMount: string): Promise<void> {
|
|
271
|
+
// Copia dados do storage para RAM via tmpfs
|
|
272
|
+
const size = this.getDirSize(sourceDir);
|
|
273
|
+
await this.tmpfs.mount(targetMount, Math.ceil(size * 1.2)); // 20% overhead
|
|
274
|
+
|
|
275
|
+
// Sincroniza dados
|
|
276
|
+
execSync(`cp -a ${sourceDir}/. ${targetMount}/`);
|
|
277
|
+
|
|
278
|
+
// Cria bind mount para manter sincronizado
|
|
279
|
+
execSync(`mount --bind ${targetMount} ${sourceDir}`);
|
|
280
|
+
}
|
|
281
|
+
|
|
282
|
+
async stats(): Promise<MemoryStats> {
|
|
283
|
+
const zramStats = await this.zram.getStats();
|
|
284
|
+
const tmpfsStats = await this.tmpfs.getStats();
|
|
285
|
+
|
|
286
|
+
return {
|
|
287
|
+
zramUsed: zramStats.used,
|
|
288
|
+
zramTotal: zramStats.total,
|
|
289
|
+
tmpfsUsed: tmpfsStats.used,
|
|
290
|
+
tmpfsTotal: tmpfsStats.total,
|
|
291
|
+
swapUsed: zramStats.swapUsed,
|
|
292
|
+
swapTotal: zramStats.swapTotal,
|
|
293
|
+
compressionRatio: zramStats.compressionRatio,
|
|
294
|
+
};
|
|
295
|
+
}
|
|
296
|
+
|
|
297
|
+
async teardown(): Promise<void> {
|
|
298
|
+
await this.tmpfs.unmountAll();
|
|
299
|
+
await this.zram.deactivate();
|
|
300
|
+
await this.zram.remove();
|
|
301
|
+
}
|
|
302
|
+
|
|
303
|
+
private getDirSize(path: string): number {
|
|
304
|
+
const output = execSync(`du -sm ${path} | cut -f1`).toString().trim();
|
|
305
|
+
return parseInt(output, 10);
|
|
306
|
+
}
|
|
307
|
+
}
|
|
308
|
+
Passo 3.3: Gerenciador zram
|
|
309
|
+
// packages/memory-bridge/zram.ts
|
|
310
|
+
import { execSync } from 'child_process';
|
|
311
|
+
import { MemoryConfig } from './types';
|
|
312
|
+
|
|
313
|
+
export class ZramManager {
|
|
314
|
+
private config: MemoryConfig;
|
|
315
|
+
private devicePath: string;
|
|
316
|
+
|
|
317
|
+
constructor(config: MemoryConfig) {
|
|
318
|
+
this.config = config;
|
|
319
|
+
this.devicePath = '/dev/zram0';
|
|
320
|
+
}
|
|
321
|
+
|
|
322
|
+
async create(): Promise<void> {
|
|
323
|
+
// Carrega módulo zram se necessário
|
|
324
|
+
try {
|
|
325
|
+
execSync('modprobe zram', { stdio: 'ignore' });
|
|
326
|
+
} catch {}
|
|
327
|
+
|
|
328
|
+
// Define tamanho do dispositivo
|
|
329
|
+
execSync(`echo ${this.config.zramSizeMb}M > /sys/block/zram0/disksize`);
|
|
330
|
+
}
|
|
331
|
+
|
|
332
|
+
async setCompression(algorithm: string): Promise<void> {
|
|
333
|
+
execSync(`echo ${algorithm} > /sys/block/zram0/comp_algorithm`);
|
|
334
|
+
}
|
|
335
|
+
|
|
336
|
+
async activateAsSwap(): Promise<void> {
|
|
337
|
+
execSync(`mkswap ${this.devicePath}`);
|
|
338
|
+
execSync(`swapon -p 100 ${this.devicePath}`);
|
|
339
|
+
}
|
|
340
|
+
|
|
341
|
+
async activateAsFilesystem(mountPoint: string): Promise<void> {
|
|
342
|
+
execSync(`mkfs.ext4 ${this.devicePath}`);
|
|
343
|
+
execSync(`mkdir -p ${mountPoint}`);
|
|
344
|
+
execSync(`mount ${this.devicePath} ${mountPoint}`);
|
|
345
|
+
}
|
|
346
|
+
|
|
347
|
+
async getStats(): Promise<{
|
|
348
|
+
used: number;
|
|
349
|
+
total: number;
|
|
350
|
+
swapUsed: number;
|
|
351
|
+
swapTotal: number;
|
|
352
|
+
compressionRatio: number;
|
|
353
|
+
}> {
|
|
354
|
+
const diskSize = this.readFile('/sys/block/zram0/disksize');
|
|
355
|
+
const compData = this.readFile('/sys/block/zram0/mm_stat');
|
|
356
|
+
|
|
357
|
+
const stats = compData.split(/\s+/);
|
|
358
|
+
const origDataSize = parseInt(stats[0]) || 0;
|
|
359
|
+
const comprDataSize = parseInt(stats[1]) || 0;
|
|
360
|
+
|
|
361
|
+
return {
|
|
362
|
+
total: parseInt(diskSize) / 1024 / 1024,
|
|
363
|
+
used: comprDataSize / 1024 / 1024,
|
|
364
|
+
swapTotal: parseInt(diskSize) / 1024 / 1024,
|
|
365
|
+
swapUsed: comprDataSize / 1024 / 1024,
|
|
366
|
+
compressionRatio: origDataSize > 0 ? comprDataSize / origDataSize : 1,
|
|
367
|
+
};
|
|
368
|
+
}
|
|
369
|
+
|
|
370
|
+
async deactivate(): Promise<void> {
|
|
371
|
+
try {
|
|
372
|
+
execSync(`swapoff ${this.devicePath}`);
|
|
373
|
+
} catch {}
|
|
374
|
+
}
|
|
375
|
+
|
|
376
|
+
async remove(): Promise<void> {
|
|
377
|
+
try {
|
|
378
|
+
execSync(`echo 1 > /sys/block/zram0/reset`);
|
|
379
|
+
execSync(`echo 0 > /sys/block/zram0/disksize`);
|
|
380
|
+
} catch {}
|
|
381
|
+
}
|
|
382
|
+
|
|
383
|
+
private readFile(path: string): string {
|
|
384
|
+
return execSync(`cat ${path}`).toString().trim();
|
|
385
|
+
}
|
|
386
|
+
}
|
|
387
|
+
Passo 3.4: Gerenciador tmpfs
|
|
388
|
+
// packages/memory-bridge/tmpfs.ts
|
|
389
|
+
import { execSync } from 'child_process';
|
|
390
|
+
import { MemoryConfig } from './types';
|
|
391
|
+
|
|
392
|
+
export class TmpfsManager {
|
|
393
|
+
private config: MemoryConfig;
|
|
394
|
+
private mountedPaths: Map<string, number> = new Map();
|
|
395
|
+
|
|
396
|
+
constructor(config: MemoryConfig) {
|
|
397
|
+
this.config = config;
|
|
398
|
+
}
|
|
399
|
+
|
|
400
|
+
async mount(mountPoint: string, sizeMb: number): Promise<void> {
|
|
401
|
+
execSync(`mkdir -p ${mountPoint}`);
|
|
402
|
+
execSync(`mount -t tmpfs -o size=${sizeMb}M tmpfs ${mountPoint}`);
|
|
403
|
+
this.mountedPaths.set(mountPoint, sizeMb);
|
|
404
|
+
}
|
|
405
|
+
|
|
406
|
+
async mountWithSync(mountPoint: string, sizeMb: number, sourceDir: string): Promise<void> {
|
|
407
|
+
// Monta tmpfs
|
|
408
|
+
await this.mount(mountPoint, sizeMb);
|
|
409
|
+
|
|
410
|
+
// Copia dados existentes
|
|
411
|
+
execSync(`cp -a ${sourceDir}/. ${mountPoint}/`);
|
|
412
|
+
|
|
413
|
+
// Cria bind mount para persistir
|
|
414
|
+
execSync(`mount --bind ${mountPoint} ${sourceDir}`);
|
|
415
|
+
}
|
|
416
|
+
|
|
417
|
+
async unmount(mountPoint: string): Promise<void> {
|
|
418
|
+
try {
|
|
419
|
+
execSync(`umount ${mountPoint}`);
|
|
420
|
+
this.mountedPaths.delete(mountPoint);
|
|
421
|
+
} catch {}
|
|
422
|
+
}
|
|
423
|
+
|
|
424
|
+
async unmountAll(): Promise<void> {
|
|
425
|
+
for (const [path] of this.mountedPaths) {
|
|
426
|
+
await this.unmount(path);
|
|
427
|
+
}
|
|
428
|
+
}
|
|
429
|
+
|
|
430
|
+
async getStats(): Promise<{ used: number; total: number; mounts: string[] }> {
|
|
431
|
+
const output = execSync('df -m -t tmpfs').toString();
|
|
432
|
+
const lines = output.split('\n').slice(1);
|
|
433
|
+
|
|
434
|
+
let totalUsed = 0;
|
|
435
|
+
let totalSize = 0;
|
|
436
|
+
|
|
437
|
+
for (const line of lines) {
|
|
438
|
+
const parts = line.split(/\s+/);
|
|
439
|
+
if (parts.length >= 5 && parts[0].startsWith('tmpfs')) {
|
|
440
|
+
totalSize += parseInt(parts[1]) || 0;
|
|
441
|
+
totalUsed += parseInt(parts[2]) || 0;
|
|
442
|
+
}
|
|
443
|
+
}
|
|
444
|
+
|
|
445
|
+
return {
|
|
446
|
+
used: totalUsed,
|
|
447
|
+
total: totalSize,
|
|
448
|
+
mounts: Array.from(this.mountedPaths.keys()),
|
|
449
|
+
};
|
|
450
|
+
}
|
|
451
|
+
}
|
|
452
|
+
|
|
453
|
+
FASE 4: PROVIDERS DE SANDBOX
|
|
454
|
+
Passo 4.1: Interface base
|
|
455
|
+
// packages/core/provider.ts
|
|
456
|
+
export interface SandboxConfig {
|
|
457
|
+
name: string;
|
|
458
|
+
image?: string;
|
|
459
|
+
memoryMb: number;
|
|
460
|
+
cpuCores: number;
|
|
461
|
+
diskGb: number;
|
|
462
|
+
networkEnabled: boolean;
|
|
463
|
+
labels?: Record<string, string>;
|
|
464
|
+
env?: Record<string, string>;
|
|
465
|
+
volumes?: string[];
|
|
466
|
+
commands?: string[];
|
|
467
|
+
storageToRam?: boolean;
|
|
468
|
+
zramSizeMb?: number;
|
|
469
|
+
}
|
|
470
|
+
|
|
471
|
+
export interface SandboxInstance {
|
|
472
|
+
id: string;
|
|
473
|
+
status: 'creating' | 'running' | 'sleeping' | 'stopped' | 'destroyed';
|
|
474
|
+
provider: string;
|
|
475
|
+
sshHost?: string;
|
|
476
|
+
sshPort?: number;
|
|
477
|
+
webUrl?: string;
|
|
478
|
+
repoUrl?: string;
|
|
479
|
+
pipelineId?: string;
|
|
480
|
+
createdAt: Date;
|
|
481
|
+
expiresAt?: Date;
|
|
482
|
+
}
|
|
483
|
+
|
|
484
|
+
export interface SandboxProvider {
|
|
485
|
+
name: string;
|
|
486
|
+
|
|
487
|
+
create(config: SandboxConfig): Promise<SandboxInstance>;
|
|
488
|
+
destroy(id: string): Promise<void>;
|
|
489
|
+
start(id: string): Promise<void>;
|
|
490
|
+
stop(id: string): Promise<void>;
|
|
491
|
+
pause(id: string): Promise<void>;
|
|
492
|
+
resume(id: string): Promise<void>;
|
|
493
|
+
|
|
494
|
+
execute(id: string, command: string): Promise<{ stdout: string; stderr: string; exitCode: number }>;
|
|
495
|
+
|
|
496
|
+
getStats(id: string): Promise<{ cpu: number; memory: number; disk: number }>;
|
|
497
|
+
|
|
498
|
+
list(): Promise<SandboxInstance[]>;
|
|
499
|
+
}
|
|
500
|
+
Passo 4.2: Provider GitHub Actions
|
|
501
|
+
// packages/sandbox-github/index.ts
|
|
502
|
+
import { SandboxProvider, SandboxConfig, SandboxInstance } from '@saddle/core';
|
|
503
|
+
import { Octokit } from '@octokit/rest';
|
|
504
|
+
|
|
505
|
+
export class GitHubProvider implements SandboxProvider {
|
|
506
|
+
name = 'github';
|
|
507
|
+
private octokit: Octokit;
|
|
508
|
+
private owner: string;
|
|
509
|
+
|
|
510
|
+
constructor(token: string, owner: string) {
|
|
511
|
+
this.octokit = new Octokit({ auth: token });
|
|
512
|
+
this.owner = owner;
|
|
513
|
+
}
|
|
514
|
+
|
|
515
|
+
async create(config: SandboxConfig): Promise<SandboxInstance> {
|
|
516
|
+
// 1. Criar repositório para a sandbox
|
|
517
|
+
const repoName = `saddle-sandbox-${config.name}-${Date.now()}`;
|
|
518
|
+
const { data: repo } = await this.octokit.repos.createForAuthenticatedUser({
|
|
519
|
+
name: repoName,
|
|
520
|
+
private: true,
|
|
521
|
+
description: `Saddle sandbox: ${config.name}`,
|
|
522
|
+
auto_init: true,
|
|
523
|
+
});
|
|
524
|
+
|
|
525
|
+
// 2. Criar workflow file
|
|
526
|
+
const workflowContent = this.generateWorkflow(config);
|
|
527
|
+
await this.octokit.repos.createOrUpdateFileContents({
|
|
528
|
+
owner: this.owner,
|
|
529
|
+
repo: repoName,
|
|
530
|
+
path: '.github/workflows/sandbox.yml',
|
|
531
|
+
message: 'Create sandbox workflow',
|
|
532
|
+
content: Buffer.from(workflowContent).toString('base64'),
|
|
533
|
+
});
|
|
534
|
+
|
|
535
|
+
// 3. Criar Dockerfile
|
|
536
|
+
const dockerfileContent = this.generateDockerfile(config);
|
|
537
|
+
await this.octokit.repos.createOrUpdateFileContents({
|
|
538
|
+
owner: this.owner,
|
|
539
|
+
repo: repoName,
|
|
540
|
+
path: 'Dockerfile',
|
|
541
|
+
message: 'Create sandbox Dockerfile',
|
|
542
|
+
content: Buffer.from(dockerfileContent).toString('base64'),
|
|
543
|
+
});
|
|
544
|
+
|
|
545
|
+
// 4. Trigger do workflow
|
|
546
|
+
const { data: run } = await this.octokit.actions.createWorkflowDispatch({
|
|
547
|
+
owner: this.owner,
|
|
548
|
+
repo: repoName,
|
|
549
|
+
workflow_id: 'sandbox.yml',
|
|
550
|
+
ref: 'main',
|
|
551
|
+
inputs: {
|
|
552
|
+
memory: `${config.memoryMb}`,
|
|
553
|
+
cpu: `${config.cpuCores}`,
|
|
554
|
+
disk: `${config.diskGb}`,
|
|
555
|
+
},
|
|
556
|
+
});
|
|
557
|
+
|
|
558
|
+
return {
|
|
559
|
+
id: repoName,
|
|
560
|
+
status: 'creating',
|
|
561
|
+
provider: 'github',
|
|
562
|
+
repoUrl: repo.html_url,
|
|
563
|
+
pipelineId: String(run.id),
|
|
564
|
+
createdAt: new Date(),
|
|
565
|
+
};
|
|
566
|
+
}
|
|
567
|
+
|
|
568
|
+
async destroy(id: string): Promise<void> {
|
|
569
|
+
await this.octokit.repos.delete({ owner: this.owner, repo: id });
|
|
570
|
+
}
|
|
571
|
+
|
|
572
|
+
async start(id: string): Promise<void> {
|
|
573
|
+
await this.octokit.actions.createWorkflowDispatch({
|
|
574
|
+
owner: this.owner,
|
|
575
|
+
repo: id,
|
|
576
|
+
workflow_id: 'sandbox.yml',
|
|
577
|
+
ref: 'main',
|
|
578
|
+
inputs: { action: 'start' },
|
|
579
|
+
});
|
|
580
|
+
}
|
|
581
|
+
|
|
582
|
+
async stop(id: string): Promise<void> {
|
|
583
|
+
// Parar workflow run ativo
|
|
584
|
+
const { data: runs } = await this.octokit.actions.listWorkflowRunsForRepo({
|
|
585
|
+
owner: this.owner,
|
|
586
|
+
repo: id,
|
|
587
|
+
status: 'in_progress',
|
|
588
|
+
});
|
|
589
|
+
|
|
590
|
+
for (const run of runs.workflow_runs) {
|
|
591
|
+
await this.octokit.actions.cancelWorkflowRun({
|
|
592
|
+
owner: this.owner,
|
|
593
|
+
repo: id,
|
|
594
|
+
run_id: run.id,
|
|
595
|
+
});
|
|
596
|
+
}
|
|
597
|
+
}
|
|
598
|
+
|
|
599
|
+
async pause(id: string): Promise<void> {
|
|
600
|
+
await this.stop(id);
|
|
601
|
+
}
|
|
602
|
+
|
|
603
|
+
async resume(id: string): Promise<void> {
|
|
604
|
+
await this.start(id);
|
|
605
|
+
}
|
|
606
|
+
|
|
607
|
+
async execute(id: string, command: string): Promise<{ stdout: string; stderr: string; exitCode: number }> {
|
|
608
|
+
// Criar issue com comando para executar via workflow
|
|
609
|
+
const { data: issue } = await this.octokit.issues.create({
|
|
610
|
+
owner: this.owner,
|
|
611
|
+
repo: id,
|
|
612
|
+
title: `Execute: ${command.substring(0, 50)}`,
|
|
613
|
+
body: `## Command\n\`\`\`\n${command}\n\`\`\``,
|
|
614
|
+
labels: ['saddle-execute'],
|
|
615
|
+
});
|
|
616
|
+
|
|
617
|
+
// Aguardar conclusão (polling)
|
|
618
|
+
// ... implementar polling do workflow run
|
|
619
|
+
|
|
620
|
+
return { stdout: '', stderr: '', exitCode: 0 };
|
|
621
|
+
}
|
|
622
|
+
|
|
623
|
+
async getStats(id: string): Promise<{ cpu: number; memory: number; disk: number }> {
|
|
624
|
+
// GitHub Actions não expõe stats diretamente
|
|
625
|
+
// Usar artifacts ou issues para coletar
|
|
626
|
+
return { cpu: 0, memory: 0, disk: 0 };
|
|
627
|
+
}
|
|
628
|
+
|
|
629
|
+
async list(): Promise<SandboxInstance[]> {
|
|
630
|
+
const { data: repos } = await this.octokit.repos.listForAuthenticatedUser({
|
|
631
|
+
per_page: 100,
|
|
632
|
+
sort: 'created',
|
|
633
|
+
direction: 'desc',
|
|
634
|
+
});
|
|
635
|
+
|
|
636
|
+
return repos
|
|
637
|
+
.filter(r => r.name.startsWith('saddle-sandbox-'))
|
|
638
|
+
.map(r => ({
|
|
639
|
+
id: r.name,
|
|
640
|
+
status: 'running' as const,
|
|
641
|
+
provider: 'github',
|
|
642
|
+
repoUrl: r.html_url,
|
|
643
|
+
createdAt: new Date(r.created_at),
|
|
644
|
+
}));
|
|
645
|
+
}
|
|
646
|
+
|
|
647
|
+
private generateWorkflow(config: SandboxConfig): string {
|
|
648
|
+
return `name: Saddle Sandbox
|
|
649
|
+
|
|
650
|
+
on:
|
|
651
|
+
workflow_dispatch:
|
|
652
|
+
inputs:
|
|
653
|
+
action:
|
|
654
|
+
description: 'Action to perform'
|
|
655
|
+
required: false
|
|
656
|
+
default: 'start'
|
|
657
|
+
memory:
|
|
658
|
+
description: 'Memory limit in MB'
|
|
659
|
+
required: false
|
|
660
|
+
default: '${config.memoryMb}'
|
|
661
|
+
cpu:
|
|
662
|
+
description: 'CPU cores'
|
|
663
|
+
required: false
|
|
664
|
+
default: '${config.cpuCores}'
|
|
665
|
+
|
|
666
|
+
jobs:
|
|
667
|
+
sandbox:
|
|
668
|
+
runs-on: ubuntu-latest
|
|
669
|
+
container:
|
|
670
|
+
image: ${config.image || 'ubuntu:22.04'}
|
|
671
|
+
options: --memory=\${{ github.event.inputs.memory || '${config.memoryMb}' }}m --cpus=\${{ github.event.inputs.cpu || '${config.cpuCores}' }}
|
|
672
|
+
steps:
|
|
673
|
+
- name: Setup sandbox
|
|
674
|
+
run: |
|
|
675
|
+
apt-get update && apt-get install -y curl wget git vim
|
|
676
|
+
echo "Sandbox ready with ${{ github.event.inputs.memory || '${config.memoryMb}' }}MB RAM"
|
|
677
|
+
|
|
678
|
+
- name: Keep alive
|
|
679
|
+
run: |
|
|
680
|
+
echo "Sandbox running..."
|
|
681
|
+
while true; do sleep 60; done
|
|
682
|
+
if: github.event.inputs.action != 'stop'
|
|
683
|
+
`;
|
|
684
|
+
}
|
|
685
|
+
|
|
686
|
+
private generateDockerfile(config: SandboxConfig): string {
|
|
687
|
+
return `FROM ${config.image || 'ubuntu:22.04'}
|
|
688
|
+
|
|
689
|
+
RUN apt-get update && apt-get install -y \\
|
|
690
|
+
curl wget git vim nano htop \\
|
|
691
|
+
nodejs npm python3 python3-pip
|
|
692
|
+
|
|
693
|
+
RUN useradd -m -s /bin/bash sandbox
|
|
694
|
+
USER sandbox
|
|
695
|
+
WORKDIR /home/sandbox
|
|
696
|
+
|
|
697
|
+
CMD ["/bin/bash"]
|
|
698
|
+
`;
|
|
699
|
+
}
|
|
700
|
+
}
|
|
701
|
+
Passo 4.3: Provider GitLab CI
|
|
702
|
+
// packages/sandbox-gitlab/index.ts
|
|
703
|
+
import { SandboxProvider, SandboxConfig, SandboxInstance } from '@saddle/core';
|
|
704
|
+
|
|
705
|
+
export class GitLabProvider implements SandboxProvider {
|
|
706
|
+
name = 'gitlab';
|
|
707
|
+
private baseUrl: string;
|
|
708
|
+
private token: string;
|
|
709
|
+
|
|
710
|
+
constructor(token: string, baseUrl = 'https://gitlab.com') {
|
|
711
|
+
this.token = token;
|
|
712
|
+
this.baseUrl = baseUrl;
|
|
713
|
+
}
|
|
714
|
+
|
|
715
|
+
async create(config: SandboxConfig): Promise<SandboxInstance> {
|
|
716
|
+
// 1. Criar projeto
|
|
717
|
+
const projectName = `saddle-sandbox-${config.name}-${Date.now()}`;
|
|
718
|
+
const createRes = await fetch(`${this.baseUrl}/api/v4/projects`, {
|
|
719
|
+
method: 'POST',
|
|
720
|
+
headers: {
|
|
721
|
+
'PRIVATE-TOKEN': this.token,
|
|
722
|
+
'Content-Type': 'application/json',
|
|
723
|
+
},
|
|
724
|
+
body: JSON.stringify({
|
|
725
|
+
name: projectName,
|
|
726
|
+
visibility: 'private',
|
|
727
|
+
description: `Saddle sandbox: ${config.name}`,
|
|
728
|
+
initialize_with_readme: true,
|
|
729
|
+
}),
|
|
730
|
+
});
|
|
731
|
+
const project = await createRes.json();
|
|
732
|
+
|
|
733
|
+
// 2. Criar .gitlab-ci.yml
|
|
734
|
+
const ciContent = this.generateCI(config);
|
|
735
|
+
await fetch(`${this.baseUrl}/api/v4/projects/${project.id}/repository/files/.gitlab-ci.yml`, {
|
|
736
|
+
method: 'POST',
|
|
737
|
+
headers: {
|
|
738
|
+
'PRIVATE-TOKEN': this.token,
|
|
739
|
+
'Content-Type': 'application/json',
|
|
740
|
+
},
|
|
741
|
+
body: JSON.stringify({
|
|
742
|
+
branch: 'main',
|
|
743
|
+
content: ciContent,
|
|
744
|
+
commit_message: 'Create sandbox CI config',
|
|
745
|
+
}),
|
|
746
|
+
});
|
|
747
|
+
|
|
748
|
+
// 3. Criar trigger token
|
|
749
|
+
const triggerRes = await fetch(`${this.baseUrl}/api/v4/projects/${project.id}/triggers`, {
|
|
750
|
+
method: 'POST',
|
|
751
|
+
headers: {
|
|
752
|
+
'PRIVATE-TOKEN': this.token,
|
|
753
|
+
'Content-Type': 'application/json',
|
|
754
|
+
},
|
|
755
|
+
body: JSON.stringify({ description: 'saddle-trigger' }),
|
|
756
|
+
});
|
|
757
|
+
const trigger = await triggerRes.json();
|
|
758
|
+
|
|
759
|
+
// 4. Trigger pipeline
|
|
760
|
+
const pipelineRes = await fetch(
|
|
761
|
+
`${this.baseUrl}/api/v4/projects/${project.id}/trigger/pipeline`,
|
|
762
|
+
{
|
|
763
|
+
method: 'POST',
|
|
764
|
+
headers: { 'Content-Type': 'application/json' },
|
|
765
|
+
body: JSON.stringify({
|
|
766
|
+
token: trigger.token,
|
|
767
|
+
ref: 'main',
|
|
768
|
+
variables: {
|
|
769
|
+
SANDBOX_MEMORY: `${config.memoryMb}`,
|
|
770
|
+
SANDBOX_CPU: `${config.cpuCores}`,
|
|
771
|
+
},
|
|
772
|
+
}),
|
|
773
|
+
}
|
|
774
|
+
);
|
|
775
|
+
const pipeline = await pipelineRes.json();
|
|
776
|
+
|
|
777
|
+
return {
|
|
778
|
+
id: String(project.id),
|
|
779
|
+
status: 'creating',
|
|
780
|
+
provider: 'gitlab',
|
|
781
|
+
repoUrl: project.web_url,
|
|
782
|
+
pipelineId: String(pipeline.id),
|
|
783
|
+
createdAt: new Date(),
|
|
784
|
+
};
|
|
785
|
+
}
|
|
786
|
+
|
|
787
|
+
async destroy(id: string): Promise<void> {
|
|
788
|
+
await fetch(`${this.baseUrl}/api/v4/projects/${id}`, {
|
|
789
|
+
method: 'DELETE',
|
|
790
|
+
headers: { 'PRIVATE-TOKEN': this.token },
|
|
791
|
+
});
|
|
792
|
+
}
|
|
793
|
+
|
|
794
|
+
async start(id: string): Promise<void> {
|
|
795
|
+
await fetch(`${this.baseUrl}/api/v4/projects/${id}/pipeline`, {
|
|
796
|
+
method: 'POST',
|
|
797
|
+
headers: {
|
|
798
|
+
'PRIVATE-TOKEN': this.token,
|
|
799
|
+
'Content-Type': 'application/json',
|
|
800
|
+
},
|
|
801
|
+
body: JSON.stringify({ ref: 'main' }),
|
|
802
|
+
});
|
|
803
|
+
}
|
|
804
|
+
|
|
805
|
+
async stop(id: string): Promise<void> {
|
|
806
|
+
const res = await fetch(`${this.baseUrl}/api/v4/projects/${id}/pipelines?status=running`, {
|
|
807
|
+
headers: { 'PRIVATE-TOKEN': this.token },
|
|
808
|
+
});
|
|
809
|
+
const pipelines = await res.json();
|
|
810
|
+
|
|
811
|
+
for (const p of pipelines) {
|
|
812
|
+
await fetch(`${this.baseUrl}/api/v4/projects/${id}/pipelines/${p.id}/cancel`, {
|
|
813
|
+
method: 'POST',
|
|
814
|
+
headers: { 'PRIVATE-TOKEN': this.token },
|
|
815
|
+
});
|
|
816
|
+
}
|
|
817
|
+
}
|
|
818
|
+
|
|
819
|
+
async pause(id: string): Promise<void> { await this.stop(id); }
|
|
820
|
+
async resume(id: string): Promise<void> { await this.start(id); }
|
|
821
|
+
|
|
822
|
+
async execute(id: string, command: string): Promise<{ stdout: string; stderr: string; exitCode: number }> {
|
|
823
|
+
// Criar pipeline com variável de comando
|
|
824
|
+
const res = await fetch(`${this.baseUrl}/api/v4/projects/${id}/pipeline`, {
|
|
825
|
+
method: 'POST',
|
|
826
|
+
headers: {
|
|
827
|
+
'PRIVATE-TOKEN': this.token,
|
|
828
|
+
'Content-Type': 'application/json',
|
|
829
|
+
},
|
|
830
|
+
body: JSON.stringify({
|
|
831
|
+
ref: 'main',
|
|
832
|
+
variables: [{ key: 'EXECUTE_COMMAND', value: command }],
|
|
833
|
+
}),
|
|
834
|
+
});
|
|
835
|
+
const pipeline = await res.json();
|
|
836
|
+
|
|
837
|
+
// Aguardar conclusão
|
|
838
|
+
// ... polling implementado
|
|
839
|
+
|
|
840
|
+
return { stdout: '', stderr: '', exitCode: 0 };
|
|
841
|
+
}
|
|
842
|
+
|
|
843
|
+
async getStats(id: string): Promise<{ cpu: number; memory: number; disk: number }> {
|
|
844
|
+
return { cpu: 0, memory: 0, disk: 0 };
|
|
845
|
+
}
|
|
846
|
+
|
|
847
|
+
async list(): Promise<SandboxInstance[]> {
|
|
848
|
+
const res = await fetch(
|
|
849
|
+
`${this.baseUrl}/api/v4/projects?search=saddle-sandbox&per_page=100`,
|
|
850
|
+
{ headers: { 'PRIVATE-TOKEN': this.token } }
|
|
851
|
+
);
|
|
852
|
+
const projects = await res.json();
|
|
853
|
+
|
|
854
|
+
return projects.map((p: any) => ({
|
|
855
|
+
id: String(p.id),
|
|
856
|
+
status: 'running' as const,
|
|
857
|
+
provider: 'gitlab',
|
|
858
|
+
repoUrl: p.web_url,
|
|
859
|
+
createdAt: new Date(p.created_at),
|
|
860
|
+
}));
|
|
861
|
+
}
|
|
862
|
+
|
|
863
|
+
private generateCI(config: SandboxConfig): string {
|
|
864
|
+
return `image: ${config.image || 'ubuntu:22.04'}
|
|
865
|
+
|
|
866
|
+
variables:
|
|
867
|
+
SANDBOX_MEMORY: "${config.memoryMb}"
|
|
868
|
+
SANDBOX_CPU: "${config.cpuCores}"
|
|
869
|
+
|
|
870
|
+
sandbox:
|
|
871
|
+
script:
|
|
872
|
+
- apt-get update && apt-get install -y curl wget git
|
|
873
|
+
- echo "Sandbox ready with $SANDBOX_MEMORY MB RAM"
|
|
874
|
+
- |
|
|
875
|
+
if [ -n "$EXECUTE_COMMAND" ]; then
|
|
876
|
+
eval "$EXECUTE_COMMAND"
|
|
877
|
+
else
|
|
878
|
+
while true; do sleep 60; done
|
|
879
|
+
fi
|
|
880
|
+
resource_group: sandbox
|
|
881
|
+
tags:
|
|
882
|
+
- docker
|
|
883
|
+
`;
|
|
884
|
+
}
|
|
885
|
+
}
|
|
886
|
+
Passo 4.4: Provider Hugging Face Spaces
|
|
887
|
+
// packages/sandbox-hf/index.ts
|
|
888
|
+
import { SandboxProvider, SandboxConfig, SandboxInstance } from '@saddle/core';
|
|
889
|
+
|
|
890
|
+
export class HuggingFaceProvider implements SandboxProvider {
|
|
891
|
+
name = 'huggingface';
|
|
892
|
+
private token: string;
|
|
893
|
+
|
|
894
|
+
constructor(token: string) {
|
|
895
|
+
this.token = token;
|
|
896
|
+
}
|
|
897
|
+
|
|
898
|
+
async create(config: SandboxConfig): Promise<SandboxInstance> {
|
|
899
|
+
const spaceName = `saddle-sandbox-${config.name}-${Date.now()}`;
|
|
900
|
+
|
|
901
|
+
// 1. Criar Space via API
|
|
902
|
+
const createRes = await fetch('https://huggingface.co/api/repos/create', {
|
|
903
|
+
method: 'POST',
|
|
904
|
+
headers: {
|
|
905
|
+
'Authorization': `Bearer ${this.token}`,
|
|
906
|
+
'Content-Type': 'application/json',
|
|
907
|
+
},
|
|
908
|
+
body: JSON.stringify({
|
|
909
|
+
name: spaceName,
|
|
910
|
+
repo_type: 'space',
|
|
911
|
+
space_sdk: 'docker',
|
|
912
|
+
license: 'mit',
|
|
913
|
+
}),
|
|
914
|
+
});
|
|
915
|
+
const space = await createRes.json();
|
|
916
|
+
|
|
917
|
+
// 2. Criar Dockerfile via Git
|
|
918
|
+
const dockerfile = this.generateDockerfile(config);
|
|
919
|
+
const readme = this.generateReadme(config);
|
|
920
|
+
|
|
921
|
+
await this.pushFile(space.name, 'Dockerfile', dockerfile);
|
|
922
|
+
await this.pushFile(space.name, 'README.md', readme);
|
|
923
|
+
|
|
924
|
+
return {
|
|
925
|
+
id: space.name,
|
|
926
|
+
status: 'creating',
|
|
927
|
+
provider: 'huggingface',
|
|
928
|
+
webUrl: `https://huggingface.co/spaces/${space.name}`,
|
|
929
|
+
repoUrl: `https://huggingface.co/spaces/${space.name}`,
|
|
930
|
+
createdAt: new Date(),
|
|
931
|
+
};
|
|
932
|
+
}
|
|
933
|
+
|
|
934
|
+
async destroy(id: string): Promise<void> {
|
|
935
|
+
await fetch(`https://huggingface.co/api/repos/delete`, {
|
|
936
|
+
method: 'DELETE',
|
|
937
|
+
headers: {
|
|
938
|
+
'Authorization': `Bearer ${this.token}`,
|
|
939
|
+
'Content-Type': 'application/json',
|
|
940
|
+
},
|
|
941
|
+
body: JSON.stringify({
|
|
942
|
+
name: id,
|
|
943
|
+
type: 'space',
|
|
944
|
+
}),
|
|
945
|
+
});
|
|
946
|
+
}
|
|
947
|
+
|
|
948
|
+
async start(id: string): Promise<void> {
|
|
949
|
+
// HF Spaces auto-start on access
|
|
950
|
+
await fetch(`https://huggingface.co/api/spaces/${id}/restart`, {
|
|
951
|
+
method: 'POST',
|
|
952
|
+
headers: { 'Authorization': `Bearer ${this.token}` },
|
|
953
|
+
});
|
|
954
|
+
}
|
|
955
|
+
|
|
956
|
+
async stop(id: string): Promise<void> {
|
|
957
|
+
await fetch(`https://huggingface.co/api/spaces/${id}/pause`, {
|
|
958
|
+
method: 'POST',
|
|
959
|
+
headers: { 'Authorization': `Bearer ${this.token}` },
|
|
960
|
+
});
|
|
961
|
+
}
|
|
962
|
+
|
|
963
|
+
async pause(id: string): Promise<void> { await this.stop(id); }
|
|
964
|
+
async resume(id: string): Promise<void> { await this.start(id); }
|
|
965
|
+
|
|
966
|
+
async execute(id: string, command: string): Promise<{ stdout: string; stderr: string; exitCode: number }> {
|
|
967
|
+
// HF não tem execução direta via API
|
|
968
|
+
// Usar webhook ou endpoint customizado
|
|
969
|
+
const res = await fetch(`https://${id}.hf.space/execute`, {
|
|
970
|
+
method: 'POST',
|
|
971
|
+
headers: { 'Content-Type': 'application/json' },
|
|
972
|
+
body: JSON.stringify({ command }),
|
|
973
|
+
});
|
|
974
|
+
const data = await res.json();
|
|
975
|
+
return { stdout: data.stdout || '', stderr: data.stderr || '', exitCode: data.exitCode || 0 };
|
|
976
|
+
}
|
|
977
|
+
|
|
978
|
+
async getStats(id: string): Promise<{ cpu: number; memory: number; disk: number }> {
|
|
979
|
+
return { cpu: 0, memory: 0, disk: 0 };
|
|
980
|
+
}
|
|
981
|
+
|
|
982
|
+
async list(): Promise<SandboxInstance[]> {
|
|
983
|
+
const res = await fetch('https://huggingface.co/api/spaces?author=me', {
|
|
984
|
+
headers: { 'Authorization': `Bearer ${this.token}` },
|
|
985
|
+
});
|
|
986
|
+
const spaces = await res.json();
|
|
987
|
+
|
|
988
|
+
return spaces
|
|
989
|
+
.filter((s: any) => s.id.startsWith('saddle-sandbox-'))
|
|
990
|
+
.map((s: any) => ({
|
|
991
|
+
id: s.id,
|
|
992
|
+
status: s.runtime?.stage === 'RUNNING' ? 'running' : 'stopped',
|
|
993
|
+
provider: 'huggingface',
|
|
994
|
+
webUrl: `https://huggingface.co/spaces/${s.id}`,
|
|
995
|
+
createdAt: new Date(s.lastModified),
|
|
996
|
+
}));
|
|
997
|
+
}
|
|
998
|
+
|
|
999
|
+
private async pushFile(spaceName: string, path: string, content: string): Promise<void> {
|
|
1000
|
+
const blob = new Blob([content]);
|
|
1001
|
+
const formData = new FormData();
|
|
1002
|
+
formData.append('file', blob, path);
|
|
1003
|
+
|
|
1004
|
+
await fetch(`https://huggingface.co/api/repos/${spaceName}/upload/main`, {
|
|
1005
|
+
method: 'POST',
|
|
1006
|
+
headers: { 'Authorization': `Bearer ${this.token}` },
|
|
1007
|
+
body: formData,
|
|
1008
|
+
});
|
|
1009
|
+
}
|
|
1010
|
+
|
|
1011
|
+
private generateDockerfile(config: SandboxConfig): string {
|
|
1012
|
+
return `FROM ${config.image || 'python:3.11-slim'}
|
|
1013
|
+
|
|
1014
|
+
RUN useradd -m -u 1000 user
|
|
1015
|
+
WORKDIR /app
|
|
1016
|
+
|
|
1017
|
+
RUN apt-get update && apt-get install -y curl wget git
|
|
1018
|
+
|
|
1019
|
+
USER user
|
|
1020
|
+
ENV HOME=/home/user PATH=/home/user/.local/bin:$PATH
|
|
1021
|
+
|
|
1022
|
+
EXPOSE 7860
|
|
1023
|
+
|
|
1024
|
+
CMD ["python", "-m", "http.server", "7860"]
|
|
1025
|
+
`;
|
|
1026
|
+
}
|
|
1027
|
+
|
|
1028
|
+
private generateReadme(config: SandboxConfig): string {
|
|
1029
|
+
return `---
|
|
1030
|
+
title: Saddle Sandbox
|
|
1031
|
+
emoji: 🐴
|
|
1032
|
+
colorFrom: blue
|
|
1033
|
+
colorTo: green
|
|
1034
|
+
sdk: docker
|
|
1035
|
+
app_port: 7860
|
|
1036
|
+
pinned: false
|
|
1037
|
+
---
|
|
1038
|
+
|
|
1039
|
+
# Saddle Sandbox
|
|
1040
|
+
|
|
1041
|
+
Sandbox environment created by Saddle platform.
|
|
1042
|
+
`;
|
|
1043
|
+
}
|
|
1044
|
+
}
|
|
1045
|
+
|
|
1046
|
+
FASE 5: ORCHESTRATOR (CORE)
|
|
1047
|
+
Passo 5.1: Módulo principal
|
|
1048
|
+
// packages/core/orchestrator.ts
|
|
1049
|
+
import { SandboxProvider, SandboxConfig, SandboxInstance } from './provider';
|
|
1050
|
+
import { MemoryBridge } from '@saddle/memory-bridge';
|
|
1051
|
+
import { db } from '@saddle/db';
|
|
1052
|
+
import { sandboxes, pipelines } from '@saddle/db/schema';
|
|
1053
|
+
import { eq } from 'drizzle-orm';
|
|
1054
|
+
|
|
1055
|
+
export class Orchestrator {
|
|
1056
|
+
private providers: Map<string, SandboxProvider> = new Map();
|
|
1057
|
+
private memoryBridges: Map<string, MemoryBridge> = new Map();
|
|
1058
|
+
|
|
1059
|
+
registerProvider(name: string, provider: SandboxProvider): void {
|
|
1060
|
+
this.providers.set(name, provider);
|
|
1061
|
+
}
|
|
1062
|
+
|
|
1063
|
+
async createSandbox(config: SandboxConfig, providerName: string): Promise<SandboxInstance> {
|
|
1064
|
+
const provider = this.providers.get(providerName);
|
|
1065
|
+
if (!provider) throw new Error(`Provider ${providerName} not found`);
|
|
1066
|
+
|
|
1067
|
+
// 1. Criar sandbox no provider
|
|
1068
|
+
const instance = await provider.create(config);
|
|
1069
|
+
|
|
1070
|
+
// 2. Salvar no banco
|
|
1071
|
+
await db.insert(sandboxes).values({
|
|
1072
|
+
id: instance.id,
|
|
1073
|
+
userId: 'default',
|
|
1074
|
+
name: config.name,
|
|
1075
|
+
status: instance.status,
|
|
1076
|
+
provider: providerName,
|
|
1077
|
+
containerId: instance.id,
|
|
1078
|
+
repoUrl: instance.repoUrl,
|
|
1079
|
+
pipelineId: instance.pipelineId,
|
|
1080
|
+
memoryMb: config.memoryMb,
|
|
1081
|
+
cpuCores: config.cpuCores,
|
|
1082
|
+
diskGb: config.diskGb,
|
|
1083
|
+
storageToRamEnabled: config.storageToRam || false,
|
|
1084
|
+
zramSizeMb: config.zramSizeMb || 2048,
|
|
1085
|
+
webUrl: instance.webUrl,
|
|
1086
|
+
createdAt: new Date(),
|
|
1087
|
+
expiresAt: new Date(Date.now() + 24 * 60 * 60 * 1000), // 24h
|
|
1088
|
+
});
|
|
1089
|
+
|
|
1090
|
+
// 3. Setup memory bridge se habilitado
|
|
1091
|
+
if (config.storageToRam) {
|
|
1092
|
+
const bridge = new MemoryBridge({
|
|
1093
|
+
sandboxId: instance.id,
|
|
1094
|
+
storagePath: `/data/sandboxes/${instance.id}`,
|
|
1095
|
+
zramSizeMb: config.zramSizeMb || 2048,
|
|
1096
|
+
tmpfsSizeMb: config.memoryMb,
|
|
1097
|
+
enableSwap: true,
|
|
1098
|
+
enableTmpfs: true,
|
|
1099
|
+
compressionAlgo: 'lz4',
|
|
1100
|
+
});
|
|
1101
|
+
await bridge.setup();
|
|
1102
|
+
this.memoryBridges.set(instance.id, bridge);
|
|
1103
|
+
}
|
|
1104
|
+
|
|
1105
|
+
return instance;
|
|
1106
|
+
}
|
|
1107
|
+
|
|
1108
|
+
async destroySandbox(id: string): Promise<void> {
|
|
1109
|
+
const provider = this.providers.get('docker'); // Default provider
|
|
1110
|
+
if (provider) {
|
|
1111
|
+
await provider.destroy(id);
|
|
1112
|
+
}
|
|
1113
|
+
|
|
1114
|
+
// Cleanup memory bridge
|
|
1115
|
+
const bridge = this.memoryBridges.get(id);
|
|
1116
|
+
if (bridge) {
|
|
1117
|
+
await bridge.teardown();
|
|
1118
|
+
this.memoryBridges.delete(id);
|
|
1119
|
+
}
|
|
1120
|
+
|
|
1121
|
+
// Atualizar status no banco
|
|
1122
|
+
await db.update(sandboxes)
|
|
1123
|
+
.set({ status: 'destroyed' })
|
|
1124
|
+
.where(eq(sandboxes.id, id));
|
|
1125
|
+
}
|
|
1126
|
+
|
|
1127
|
+
async sleepSandbox(id: string): Promise<void> {
|
|
1128
|
+
const sandbox = await db.select().from(sandboxes).where(eq(sandboxes.id, id)).get();
|
|
1129
|
+
if (!sandbox) throw new Error('Sandbox not found');
|
|
1130
|
+
|
|
1131
|
+
const provider = this.providers.get(sandbox.provider);
|
|
1132
|
+
if (provider) {
|
|
1133
|
+
await provider.pause(id);
|
|
1134
|
+
}
|
|
1135
|
+
|
|
1136
|
+
await db.update(sandboxes)
|
|
1137
|
+
.set({ status: 'sleeping' })
|
|
1138
|
+
.where(eq(sandboxes.id, id));
|
|
1139
|
+
}
|
|
1140
|
+
|
|
1141
|
+
async wakeSandbox(id: string): Promise<void> {
|
|
1142
|
+
const sandbox = await db.select().from(sandboxes).where(eq(sandboxes.id, id)).get();
|
|
1143
|
+
if (!sandbox) throw new Error('Sandbox not found');
|
|
1144
|
+
|
|
1145
|
+
const provider = this.providers.get(sandbox.provider);
|
|
1146
|
+
if (provider) {
|
|
1147
|
+
await provider.resume(id);
|
|
1148
|
+
}
|
|
1149
|
+
|
|
1150
|
+
await db.update(sandboxes)
|
|
1151
|
+
.set({ status: 'running', lastActivityAt: new Date() })
|
|
1152
|
+
.where(eq(sandboxes.id, id));
|
|
1153
|
+
}
|
|
1154
|
+
|
|
1155
|
+
async executeInSandbox(id: string, command: string): Promise<{
|
|
1156
|
+
stdout: string;
|
|
1157
|
+
stderr: string;
|
|
1158
|
+
exitCode: number;
|
|
1159
|
+
}> {
|
|
1160
|
+
const sandbox = await db.select().from(sandboxes).where(eq(sandboxes.id, id)).get();
|
|
1161
|
+
if (!sandbox) throw new Error('Sandbox not found');
|
|
1162
|
+
|
|
1163
|
+
const provider = this.providers.get(sandbox.provider);
|
|
1164
|
+
if (!provider) throw new Error('Provider not found');
|
|
1165
|
+
|
|
1166
|
+
// Wake se sleeping
|
|
1167
|
+
if (sandbox.status === 'sleeping') {
|
|
1168
|
+
await this.wakeSandbox(id);
|
|
1169
|
+
}
|
|
1170
|
+
|
|
1171
|
+
const result = await provider.execute(id, command);
|
|
1172
|
+
|
|
1173
|
+
// Atualizar última atividade
|
|
1174
|
+
await db.update(sandboxes)
|
|
1175
|
+
.set({ lastActivityAt: new Date() })
|
|
1176
|
+
.where(eq(sandboxes.id, id));
|
|
1177
|
+
|
|
1178
|
+
return result;
|
|
1179
|
+
}
|
|
1180
|
+
|
|
1181
|
+
async convertStorageToRam(id: string): Promise<void> {
|
|
1182
|
+
const bridge = this.memoryBridges.get(id);
|
|
1183
|
+
if (!bridge) throw new Error('Memory bridge not initialized');
|
|
1184
|
+
|
|
1185
|
+
await bridge.convertStorageToRam(
|
|
1186
|
+
`/data/sandboxes/${id}`,
|
|
1187
|
+
`/tmp/saddle-ram/${id}`
|
|
1188
|
+
);
|
|
1189
|
+
}
|
|
1190
|
+
|
|
1191
|
+
async getMemoryStats(id: string): Promise<any> {
|
|
1192
|
+
const bridge = this.memoryBridges.get(id);
|
|
1193
|
+
if (!bridge) return null;
|
|
1194
|
+
return bridge.stats();
|
|
1195
|
+
}
|
|
1196
|
+
|
|
1197
|
+
async listSandboxes(userId?: string): Promise<any[]> {
|
|
1198
|
+
let query = db.select().from(sandboxes);
|
|
1199
|
+
if (userId) {
|
|
1200
|
+
query = query.where(eq(sandboxes.userId, userId));
|
|
1201
|
+
}
|
|
1202
|
+
return query.all();
|
|
1203
|
+
}
|
|
1204
|
+
|
|
1205
|
+
// Cron job para cleanup de sandboxes expiradas
|
|
1206
|
+
async cleanupExpired(): Promise<void> {
|
|
1207
|
+
const expired = await db.select().from(sandboxes)
|
|
1208
|
+
.where(eq(sandboxes.status, 'running'))
|
|
1209
|
+
.all();
|
|
1210
|
+
|
|
1211
|
+
for (const sandbox of expired) {
|
|
1212
|
+
if (sandbox.expiresAt && sandbox.expiresAt < new Date()) {
|
|
1213
|
+
console.log(`Destroying expired sandbox: ${sandbox.id}`);
|
|
1214
|
+
await this.destroySandbox(sandbox.id);
|
|
1215
|
+
}
|
|
1216
|
+
}
|
|
1217
|
+
}
|
|
1218
|
+
}
|
|
1219
|
+
|
|
1220
|
+
FASE 6: API GATEWAY (Hono)
|
|
1221
|
+
Passo 6.1: Servidor API
|
|
1222
|
+
// apps/api/server.ts
|
|
1223
|
+
import { Hono } from 'hono';
|
|
1224
|
+
import { cors } from 'hono/cors';
|
|
1225
|
+
import { Orchestrator } from '@saddle/core';
|
|
1226
|
+
import { GitHubProvider } from '@saddle/sandbox-github';
|
|
1227
|
+
import { GitLabProvider } from '@saddle/sandbox-gitlab';
|
|
1228
|
+
import { HuggingFaceProvider } from '@saddle/sandbox-hf';
|
|
1229
|
+
|
|
1230
|
+
const app = new Hono();
|
|
1231
|
+
app.use('*', cors());
|
|
1232
|
+
|
|
1233
|
+
const orchestrator = new Orchestrator();
|
|
1234
|
+
|
|
1235
|
+
// Registrar providers
|
|
1236
|
+
orchestrator.registerProvider('github', new GitHubProvider(
|
|
1237
|
+
process.env.GITHUB_TOKEN!,
|
|
1238
|
+
process.env.GITHUB_OWNER!
|
|
1239
|
+
));
|
|
1240
|
+
|
|
1241
|
+
orchestrator.registerProvider('gitlab', new GitLabProvider(
|
|
1242
|
+
process.env.GITLAB_TOKEN!,
|
|
1243
|
+
process.env.GITLAB_URL
|
|
1244
|
+
));
|
|
1245
|
+
|
|
1246
|
+
orchestrator.registerProvider('huggingface', new HuggingFaceProvider(
|
|
1247
|
+
process.env.HF_TOKEN!
|
|
1248
|
+
));
|
|
1249
|
+
|
|
1250
|
+
// Routes
|
|
1251
|
+
app.post('/api/sandboxes', async (c) => {
|
|
1252
|
+
const config = await c.req.json();
|
|
1253
|
+
const provider = c.req.query('provider') || 'github';
|
|
1254
|
+
|
|
1255
|
+
const instance = await orchestrator.createSandbox(config, provider);
|
|
1256
|
+
return c.json(instance);
|
|
1257
|
+
});
|
|
1258
|
+
|
|
1259
|
+
app.delete('/api/sandboxes/:id', async (c) => {
|
|
1260
|
+
const id = c.req.param('id');
|
|
1261
|
+
await orchestrator.destroySandbox(id);
|
|
1262
|
+
return c.json({ success: true });
|
|
1263
|
+
});
|
|
1264
|
+
|
|
1265
|
+
app.post('/api/sandboxes/:id/execute', async (c) => {
|
|
1266
|
+
const id = c.req.param('id');
|
|
1267
|
+
const { command } = await c.req.json();
|
|
1268
|
+
|
|
1269
|
+
const result = await orchestrator.executeInSandbox(id, command);
|
|
1270
|
+
return c.json(result);
|
|
1271
|
+
});
|
|
1272
|
+
|
|
1273
|
+
app.post('/api/sandboxes/:id/sleep', async (c) => {
|
|
1274
|
+
const id = c.req.param('id');
|
|
1275
|
+
await orchestrator.sleepSandbox(id);
|
|
1276
|
+
return c.json({ success: true });
|
|
1277
|
+
});
|
|
1278
|
+
|
|
1279
|
+
app.post('/api/sandboxes/:id/wake', async (c) => {
|
|
1280
|
+
const id = c.req.param('id');
|
|
1281
|
+
await orchestrator.wakeSandbox(id);
|
|
1282
|
+
return c.json({ success: true });
|
|
1283
|
+
});
|
|
1284
|
+
|
|
1285
|
+
app.post('/api/sandboxes/:id/convert-to-ram', async (c) => {
|
|
1286
|
+
const id = c.req.param('id');
|
|
1287
|
+
await orchestrator.convertStorageToRam(id);
|
|
1288
|
+
return c.json({ success: true });
|
|
1289
|
+
});
|
|
1290
|
+
|
|
1291
|
+
app.get('/api/sandboxes/:id/memory-stats', async (c) => {
|
|
1292
|
+
const id = c.req.param('id');
|
|
1293
|
+
const stats = await orchestrator.getMemoryStats(id);
|
|
1294
|
+
return c.json(stats);
|
|
1295
|
+
});
|
|
1296
|
+
|
|
1297
|
+
app.get('/api/sandboxes', async (c) => {
|
|
1298
|
+
const sandboxes = await orchestrator.listSandboxes();
|
|
1299
|
+
return c.json(sandboxes);
|
|
1300
|
+
});
|
|
1301
|
+
|
|
1302
|
+
// Cron para cleanup
|
|
1303
|
+
setInterval(() => {
|
|
1304
|
+
orchestrator.cleanupExpired();
|
|
1305
|
+
}, 60 * 60 * 1000); // A cada hora
|
|
1306
|
+
|
|
1307
|
+
export default app;
|
|
1308
|
+
|
|
1309
|
+
FASE 7: CLI (npx saddle)
|
|
1310
|
+
Passo 7.1: CLI principal
|
|
1311
|
+
// apps/cli/index.ts
|
|
1312
|
+
#!/usr/bin/env node
|
|
1313
|
+
import { Command } from 'commander';
|
|
1314
|
+
import { Orchestrator } from '@saddle/core';
|
|
1315
|
+
import { GitHubProvider } from '@saddle/sandbox-github';
|
|
1316
|
+
|
|
1317
|
+
const program = new Command();
|
|
1318
|
+
const orchestrator = new Orchestrator();
|
|
1319
|
+
|
|
1320
|
+
// Setup provider baseado em env
|
|
1321
|
+
if (process.env.GITHUB_TOKEN) {
|
|
1322
|
+
orchestrator.registerProvider('github', new GitHubProvider(
|
|
1323
|
+
process.env.GITHUB_TOKEN,
|
|
1324
|
+
process.env.GITHUB_OWNER || 'saddle-sandboxes'
|
|
1325
|
+
));
|
|
1326
|
+
}
|
|
1327
|
+
|
|
1328
|
+
program
|
|
1329
|
+
.name('saddle')
|
|
1330
|
+
.description('Create and manage sandboxes')
|
|
1331
|
+
.version('0.1.0');
|
|
1332
|
+
|
|
1333
|
+
program
|
|
1334
|
+
.command('create')
|
|
1335
|
+
.description('Create a new sandbox')
|
|
1336
|
+
.option('-n, --name <name>', 'Sandbox name')
|
|
1337
|
+
.option('-p, --provider <provider>', 'Provider (github|gitlab|hf)', 'github')
|
|
1338
|
+
.option('-m, --memory <mb>', 'Memory in MB', '4096')
|
|
1339
|
+
.option('-c, --cpu <cores>', 'CPU cores', '2')
|
|
1340
|
+
.option('-d, --disk <gb>', 'Disk in GB', '10')
|
|
1341
|
+
.option('-i, --image <image>', 'Docker image', 'ubuntu:22.04')
|
|
1342
|
+
.option('--ram', 'Enable storage-to-RAM bridge', false)
|
|
1343
|
+
.action(async (opts) => {
|
|
1344
|
+
const instance = await orchestrator.createSandbox({
|
|
1345
|
+
name: opts.name || `sandbox-${Date.now()}`,
|
|
1346
|
+
memoryMb: parseInt(opts.memory),
|
|
1347
|
+
cpuCores: parseInt(opts.cpu),
|
|
1348
|
+
diskGb: parseInt(opts.disk),
|
|
1349
|
+
image: opts.image,
|
|
1350
|
+
networkEnabled: true,
|
|
1351
|
+
storageToRam: opts.ram,
|
|
1352
|
+
}, opts.provider);
|
|
1353
|
+
|
|
1354
|
+
console.log('Sandbox created!');
|
|
1355
|
+
console.log(` ID: ${instance.id}`);
|
|
1356
|
+
console.log(` Status: ${instance.status}`);
|
|
1357
|
+
if (instance.repoUrl) console.log(` Repo: ${instance.repoUrl}`);
|
|
1358
|
+
if (instance.webUrl) console.log(` Web: ${instance.webUrl}`);
|
|
1359
|
+
});
|
|
1360
|
+
|
|
1361
|
+
program
|
|
1362
|
+
.command('destroy <id>')
|
|
1363
|
+
.description('Destroy a sandbox')
|
|
1364
|
+
.action(async (id) => {
|
|
1365
|
+
await orchestrator.destroySandbox(id);
|
|
1366
|
+
console.log(`Sandbox ${id} destroyed`);
|
|
1367
|
+
});
|
|
1368
|
+
|
|
1369
|
+
program
|
|
1370
|
+
.command('exec <id> <command>')
|
|
1371
|
+
.description('Execute command in sandbox')
|
|
1372
|
+
.action(async (id, command) => {
|
|
1373
|
+
const result = await orchestrator.executeInSandbox(id, command);
|
|
1374
|
+
console.log(result.stdout);
|
|
1375
|
+
if (result.stderr) console.error(result.stderr);
|
|
1376
|
+
process.exit(result.exitCode);
|
|
1377
|
+
});
|
|
1378
|
+
|
|
1379
|
+
program
|
|
1380
|
+
.command('sleep <id>')
|
|
1381
|
+
.description('Sleep a sandbox')
|
|
1382
|
+
.action(async (id) => {
|
|
1383
|
+
await orchestrator.sleepSandbox(id);
|
|
1384
|
+
console.log(`Sandbox ${id} sleeping`);
|
|
1385
|
+
});
|
|
1386
|
+
|
|
1387
|
+
program
|
|
1388
|
+
.command('wake <id>')
|
|
1389
|
+
.description('Wake a sandbox')
|
|
1390
|
+
.action(async (id) => {
|
|
1391
|
+
await orchestrator.wakeSandbox(id);
|
|
1392
|
+
console.log(`Sandbox ${id} waking`);
|
|
1393
|
+
});
|
|
1394
|
+
|
|
1395
|
+
program
|
|
1396
|
+
.command('list')
|
|
1397
|
+
.description('List all sandboxes')
|
|
1398
|
+
.action(async () => {
|
|
1399
|
+
const sandboxes = await orchestrator.listSandboxes();
|
|
1400
|
+
console.table(sandboxes.map(s => ({
|
|
1401
|
+
ID: s.id,
|
|
1402
|
+
Name: s.name,
|
|
1403
|
+
Status: s.status,
|
|
1404
|
+
Provider: s.provider,
|
|
1405
|
+
Memory: `${s.memoryMb}MB`,
|
|
1406
|
+
Created: s.createdAt,
|
|
1407
|
+
})));
|
|
1408
|
+
});
|
|
1409
|
+
|
|
1410
|
+
program
|
|
1411
|
+
.command('convert-to-ram <id>')
|
|
1412
|
+
.description('Convert sandbox storage to RAM')
|
|
1413
|
+
.action(async (id) => {
|
|
1414
|
+
await orchestrator.convertStorageToRam(id);
|
|
1415
|
+
console.log(`Storage converted to RAM for ${id}`);
|
|
1416
|
+
});
|
|
1417
|
+
|
|
1418
|
+
program
|
|
1419
|
+
.command('memory-stats <id>')
|
|
1420
|
+
.description('Show memory stats for sandbox')
|
|
1421
|
+
.action(async (id) => {
|
|
1422
|
+
const stats = await orchestrator.getMemoryStats(id);
|
|
1423
|
+
console.log(JSON.stringify(stats, null, 2));
|
|
1424
|
+
});
|
|
1425
|
+
|
|
1426
|
+
program.parse();
|
|
1427
|
+
|
|
1428
|
+
FASE 8: PACKAGES NPM + CDN
|
|
1429
|
+
Passo 8.1: Publicar como pacotes
|
|
1430
|
+
// packages/core/package.json
|
|
1431
|
+
{
|
|
1432
|
+
"name": "@saddle/core",
|
|
1433
|
+
"version": "0.1.0",
|
|
1434
|
+
"main": "dist/index.js",
|
|
1435
|
+
"types": "dist/index.d.ts",
|
|
1436
|
+
"files": ["dist"],
|
|
1437
|
+
"scripts": {
|
|
1438
|
+
"build": "tsup src/index.ts --format cjs,esm --dts",
|
|
1439
|
+
"prepublishOnly": "npm run build"
|
|
1440
|
+
},
|
|
1441
|
+
"exports": {
|
|
1442
|
+
".": {
|
|
1443
|
+
"import": "./dist/index.mjs",
|
|
1444
|
+
"require": "./dist/index.js"
|
|
1445
|
+
}
|
|
1446
|
+
}
|
|
1447
|
+
}
|
|
1448
|
+
Passo 8.2: Uso via CDN (jsDelivr)
|
|
1449
|
+
<!-- No navegador -->
|
|
1450
|
+
<script type="module">
|
|
1451
|
+
import { Orchestrator } from 'https://cdn.jsdelivr.net/npm/@saddle/core/+esm';
|
|
1452
|
+
|
|
1453
|
+
const orch = new Orchestrator();
|
|
1454
|
+
const sandbox = await orch.createSandbox({
|
|
1455
|
+
name: 'test',
|
|
1456
|
+
memoryMb: 4096,
|
|
1457
|
+
cpuCores: 2,
|
|
1458
|
+
diskGb: 10,
|
|
1459
|
+
networkEnabled: true,
|
|
1460
|
+
}, 'github');
|
|
1461
|
+
|
|
1462
|
+
console.log('Created:', sandbox);
|
|
1463
|
+
</script>
|
|
1464
|
+
Passo 8.3: Uso via npx
|
|
1465
|
+
# Criar sandbox
|
|
1466
|
+
npx saddle create --name my-app --provider github --memory 8192
|
|
1467
|
+
|
|
1468
|
+
# Executar comando
|
|
1469
|
+
npx saddle exec <sandbox-id> "apt-get update && apt-get install -y nodejs"
|
|
1470
|
+
|
|
1471
|
+
# Converter storage para RAM
|
|
1472
|
+
npx saddle convert-to-ram <sandbox-id>
|
|
1473
|
+
|
|
1474
|
+
# Ver stats de memória
|
|
1475
|
+
npx saddle memory-stats <sandbox-id>
|
|
1476
|
+
|
|
1477
|
+
# Listar sandboxes
|
|
1478
|
+
npx saddle list
|
|
1479
|
+
|
|
1480
|
+
# Destruir
|
|
1481
|
+
npx saddle destroy <sandbox-id>
|
|
1482
|
+
|
|
1483
|
+
FASE 9: DEPLOY MULTI-PLATAFORMA
|
|
1484
|
+
Passo 9.1: GitHub Actions CI
|
|
1485
|
+
# .github/workflows/ci.yml
|
|
1486
|
+
name: CI
|
|
1487
|
+
|
|
1488
|
+
on:
|
|
1489
|
+
push:
|
|
1490
|
+
branches: [main]
|
|
1491
|
+
pull_request:
|
|
1492
|
+
branches: [main]
|
|
1493
|
+
|
|
1494
|
+
jobs:
|
|
1495
|
+
build:
|
|
1496
|
+
runs-on: ubuntu-latest
|
|
1497
|
+
steps:
|
|
1498
|
+
- uses: actions/checkout@v4
|
|
1499
|
+
- uses: actions/setup-node@v4
|
|
1500
|
+
with:
|
|
1501
|
+
node-version: 20
|
|
1502
|
+
cache: npm
|
|
1503
|
+
- run: npm ci
|
|
1504
|
+
- run: npm run typecheck
|
|
1505
|
+
- run: npm run lint
|
|
1506
|
+
- run: npm test
|
|
1507
|
+
- run: npm run build
|
|
1508
|
+
Passo 9.2: Release npm
|
|
1509
|
+
# .github/workflows/release.yml
|
|
1510
|
+
name: Release
|
|
1511
|
+
|
|
1512
|
+
on:
|
|
1513
|
+
push:
|
|
1514
|
+
tags: ['v*']
|
|
1515
|
+
|
|
1516
|
+
jobs:
|
|
1517
|
+
release:
|
|
1518
|
+
runs-on: ubuntu-latest
|
|
1519
|
+
steps:
|
|
1520
|
+
- uses: actions/checkout@v4
|
|
1521
|
+
- uses: actions/setup-node@v4
|
|
1522
|
+
with:
|
|
1523
|
+
node-version: 20
|
|
1524
|
+
registry-url: https://registry.npmjs.org
|
|
1525
|
+
- run: npm ci
|
|
1526
|
+
- run: npm run build
|
|
1527
|
+
- run: npm publish --access public
|
|
1528
|
+
env:
|
|
1529
|
+
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
|
1530
|
+
Passo 9.3: GitLab CI
|
|
1531
|
+
# .gitlab-ci.yml
|
|
1532
|
+
image: node:20
|
|
1533
|
+
|
|
1534
|
+
stages:
|
|
1535
|
+
- test
|
|
1536
|
+
- build
|
|
1537
|
+
- deploy
|
|
1538
|
+
|
|
1539
|
+
test:
|
|
1540
|
+
stage: test
|
|
1541
|
+
script:
|
|
1542
|
+
- npm ci
|
|
1543
|
+
- npm run typecheck
|
|
1544
|
+
- npm run test
|
|
1545
|
+
rules:
|
|
1546
|
+
- if: $CI_MERGE_REQUEST_ID
|
|
1547
|
+
- if: $CI_COMMIT_BRANCH == "main"
|
|
1548
|
+
|
|
1549
|
+
build:
|
|
1550
|
+
stage: build
|
|
1551
|
+
script:
|
|
1552
|
+
- npm ci
|
|
1553
|
+
- npm run build
|
|
1554
|
+
artifacts:
|
|
1555
|
+
paths:
|
|
1556
|
+
- dist/
|
|
1557
|
+
expire_in: 1 week
|
|
1558
|
+
|
|
1559
|
+
deploy-npm:
|
|
1560
|
+
stage: deploy
|
|
1561
|
+
script:
|
|
1562
|
+
- npm ci
|
|
1563
|
+
- npm run build
|
|
1564
|
+
- npm publish --access public
|
|
1565
|
+
rules:
|
|
1566
|
+
- if: $CI_COMMIT_TAG =~ /^v/
|
|
1567
|
+
variables:
|
|
1568
|
+
NODE_AUTH_TOKEN: $NPM_TOKEN
|
|
1569
|
+
Passo 9.4: Forgejo Actions
|
|
1570
|
+
# .forgejo/workflows/ci.yml
|
|
1571
|
+
name: CI
|
|
1572
|
+
|
|
1573
|
+
on:
|
|
1574
|
+
push:
|
|
1575
|
+
branches: [main]
|
|
1576
|
+
pull_request:
|
|
1577
|
+
|
|
1578
|
+
jobs:
|
|
1579
|
+
build:
|
|
1580
|
+
runs-on: ubuntu-latest
|
|
1581
|
+
container: node:20
|
|
1582
|
+
steps:
|
|
1583
|
+
- uses: actions/checkout@v4
|
|
1584
|
+
- run: npm ci
|
|
1585
|
+
- run: npm run typecheck
|
|
1586
|
+
- run: npm run test
|
|
1587
|
+
- run: npm run build
|
|
1588
|
+
Passo 9.5: Deploy em Hugging Face Spaces
|
|
1589
|
+
# Para o dashboard web
|
|
1590
|
+
---
|
|
1591
|
+
title: Saddle Dashboard
|
|
1592
|
+
emoji: 🐴
|
|
1593
|
+
colorFrom: blue
|
|
1594
|
+
colorTo: green
|
|
1595
|
+
sdk: docker
|
|
1596
|
+
app_port: 3000
|
|
1597
|
+
pinned: false
|
|
1598
|
+
---
|
|
1599
|
+
|
|
1600
|
+
FASE 10: CRON JOBS E TIMELINE
|
|
1601
|
+
Passo 10.1: Cron de cleanup
|
|
1602
|
+
// apps/api/cron.ts
|
|
1603
|
+
import cron from 'node-cron';
|
|
1604
|
+
import { Orchestrator } from '@saddle/core';
|
|
1605
|
+
|
|
1606
|
+
const orchestrator = new Orchestrator();
|
|
1607
|
+
|
|
1608
|
+
// Cleanup a cada hora
|
|
1609
|
+
cron.schedule('0 * * * *', async () => {
|
|
1610
|
+
console.log('Running cleanup...');
|
|
1611
|
+
await orchestrator.cleanupExpired();
|
|
1612
|
+
});
|
|
1613
|
+
|
|
1614
|
+
// Verificar sandboxes dormindo por mais de 48h
|
|
1615
|
+
cron.schedule('0 */6 * * *', async () => {
|
|
1616
|
+
console.log('Checking sleeping sandboxes...');
|
|
1617
|
+
// Destruir sandboxes dormindo por mais de 48h
|
|
1618
|
+
});
|
|
1619
|
+
|
|
1620
|
+
// Stats de memória a cada 5 minutos
|
|
1621
|
+
cron.schedule('*/5 * * * *', async () => {
|
|
1622
|
+
const sandboxes = await orchestrator.listSandboxes();
|
|
1623
|
+
for (const s of sandboxes) {
|
|
1624
|
+
if (s.status === 'running') {
|
|
1625
|
+
const stats = await orchestrator.getMemoryStats(s.id);
|
|
1626
|
+
console.log(`Sandbox ${s.id}:`, stats);
|
|
1627
|
+
}
|
|
1628
|
+
}
|
|
1629
|
+
});
|
|
1630
|
+
Passo 10.2: GitHub Actions cron
|
|
1631
|
+
# .github/workflows/cron-cleanup.yml
|
|
1632
|
+
name: Cron Cleanup
|
|
1633
|
+
|
|
1634
|
+
on:
|
|
1635
|
+
schedule:
|
|
1636
|
+
- cron: '0 * * * *' # A cada hora
|
|
1637
|
+
|
|
1638
|
+
jobs:
|
|
1639
|
+
cleanup:
|
|
1640
|
+
runs-on: ubuntu-latest
|
|
1641
|
+
steps:
|
|
1642
|
+
- uses: actions/checkout@v4
|
|
1643
|
+
- uses: actions/setup-node@v4
|
|
1644
|
+
with:
|
|
1645
|
+
node-version: 20
|
|
1646
|
+
- run: npm ci
|
|
1647
|
+
- run: node apps/api/cron.js
|
|
1648
|
+
env:
|
|
1649
|
+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
1650
|
+
DATABASE_URL: ${{ secrets.DATABASE_URL }}
|
|
1651
|
+
|
|
1652
|
+
RESUMO: O QUE CADA COMPONENTE FAZ
|
|
1653
|
+
|
|
1654
|
+
|
|
1655
|
+
|
|
1656
|
+
Componente
|
|
1657
|
+
Função
|
|
1658
|
+
Infra Utilizada
|
|
1659
|
+
|
|
1660
|
+
|
|
1661
|
+
|
|
1662
|
+
@saddle/core
|
|
1663
|
+
Orquestrador principal
|
|
1664
|
+
Node.js
|
|
1665
|
+
|
|
1666
|
+
|
|
1667
|
+
@saddle/memory-bridge
|
|
1668
|
+
Converte storage→RAM
|
|
1669
|
+
zram + tmpfs (Linux)
|
|
1670
|
+
|
|
1671
|
+
|
|
1672
|
+
@saddle/sandbox-github
|
|
1673
|
+
Cria sandboxes via GitHub Actions
|
|
1674
|
+
GitHub Free (2000 min/mês)
|
|
1675
|
+
|
|
1676
|
+
|
|
1677
|
+
@saddle/sandbox-gitlab
|
|
1678
|
+
Cria sandboxes via GitLab CI
|
|
1679
|
+
GitLab Free (400 min/mês)
|
|
1680
|
+
|
|
1681
|
+
|
|
1682
|
+
@saddle/sandbox-hf
|
|
1683
|
+
Cria sandboxes via HF Spaces
|
|
1684
|
+
HF Free (16GB RAM)
|
|
1685
|
+
|
|
1686
|
+
|
|
1687
|
+
@saddle/db
|
|
1688
|
+
Schema Drizzle ORM
|
|
1689
|
+
SQLite/PostgreSQL
|
|
1690
|
+
|
|
1691
|
+
|
|
1692
|
+
saddle (CLI)
|
|
1693
|
+
Interface CLI
|
|
1694
|
+
npx saddle
|
|
1695
|
+
|
|
1696
|
+
|
|
1697
|
+
saddle-api
|
|
1698
|
+
API REST
|
|
1699
|
+
Hono + Cloudflare Workers
|
|
1700
|
+
|
|
1701
|
+
|
|
1702
|
+
saddle-web
|
|
1703
|
+
Dashboard
|
|
1704
|
+
Next.js
|
|
1705
|
+
|
|
1706
|
+
|
|
1707
|
+
|
|
1708
|
+
MACETES E POSSIBILIDADES
|
|
1709
|
+
|
|
1710
|
+
GitHub Actions = 2000 min/mês grátis → Cada sandbox usa ~6h = 360 min → ~5 sandboxes/mês
|
|
1711
|
+
GitLab CI = 400 min/mês grátis → Cada sandbox usa ~50min → ~8 sandboxes/mês
|
|
1712
|
+
HF Spaces = 16GB RAM grátis → Sandbox permanente, dorme após 48h inatividade
|
|
1713
|
+
Kaggle = 30GB RAM + GPU grátis → 12h/session, 30h/semana GPU
|
|
1714
|
+
zram = 2-3x compressão → 8GB storage vira ~3GB RAM efetiva
|
|
1715
|
+
NPM package via jsDelivr → Executa no browser a qualquer momento
|
|
1716
|
+
Multi-provider → Fallback automático se um provider estiver lotado
|
|
1717
|
+
|
|
1718
|
+
Quer que eu implemente alguma parte específica? Por exemplo, o memory-bridge completo, ou o provider GitHub?
|