@thierrynakoa/fire-flow 12.2.1 → 13.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CREDITS.md +25 -0
- package/DOMINION-FLOW-OVERVIEW.md +182 -38
- package/README.md +399 -455
- package/TROUBLESHOOTING.md +264 -264
- package/agents/fire-debugger.md +54 -0
- package/agents/fire-executor.md +1610 -1033
- package/agents/fire-fact-checker.md +1 -1
- package/agents/fire-planner.md +85 -17
- package/agents/fire-project-researcher.md +1 -1
- package/agents/fire-researcher.md +4 -22
- package/agents/{fire-phoenix-analyst.md → fire-resurrection-analyst.md} +394 -394
- package/agents/fire-reviewer.md +552 -499
- package/agents/fire-verifier.md +114 -19
- package/bin/cli.js +18 -101
- package/commands/fire-0-orient.md +2 -2
- package/commands/fire-1a-new.md +50 -15
- package/commands/fire-1c-setup.md +33 -5
- package/commands/fire-1d-discuss.md +87 -1
- package/commands/fire-2-plan.md +556 -527
- package/commands/fire-3-execute.md +2046 -1356
- package/commands/fire-4-verify.md +975 -906
- package/commands/fire-5-handoff.md +46 -5
- package/commands/fire-6-resume.md +2 -31
- package/commands/fire-add-new-skill.md +138 -19
- package/commands/fire-autonomous.md +14 -2
- package/commands/fire-complete-milestone.md +1 -1
- package/commands/fire-cost.md +179 -183
- package/commands/fire-debug.md +1 -6
- package/commands/fire-loop-resume.md +2 -2
- package/commands/fire-loop-stop.md +1 -1
- package/commands/fire-loop.md +2 -15
- package/commands/fire-map-codebase.md +1 -1
- package/commands/fire-migrate-database.md +548 -0
- package/commands/fire-new-milestone.md +1 -1
- package/commands/fire-reflect.md +1 -2
- package/commands/fire-research.md +142 -21
- package/commands/{fire-phoenix.md → fire-resurrect.md} +859 -603
- package/commands/fire-scaffold.md +297 -0
- package/commands/fire-search.md +1 -2
- package/commands/fire-security-scan.md +483 -484
- package/commands/fire-setup.md +359 -0
- package/commands/fire-skill.md +770 -0
- package/commands/fire-skills-diff.md +506 -506
- package/commands/fire-skills-history.md +388 -388
- package/commands/fire-skills-rollback.md +7 -7
- package/commands/fire-skills-sync.md +470 -470
- package/commands/fire-test.md +5 -5
- package/commands/fire-todos.md +1 -1
- package/commands/fire-update.md +5 -5
- package/commands/fire-validate-skills.md +282 -0
- package/commands/fire-vuln-scan.md +492 -493
- package/hooks/run-hook.sh +8 -8
- package/hooks/run-session-end.sh +7 -7
- package/hooks/session-end.sh +90 -90
- package/hooks/session-start.sh +1 -1
- package/package.json +4 -24
- package/plugin.json +7 -7
- package/references/autonomy-levels.md +235 -0
- package/references/behavioral-directives.md +95 -3
- package/references/blocker-tracking.md +1 -1
- package/references/circuit-breaker.md +93 -2
- package/references/context-engineering.md +227 -9
- package/references/honesty-protocols.md +70 -1
- package/references/issue-to-pr-pipeline.md +149 -150
- package/references/metrics-and-trends.md +1 -2
- package/references/research-improvements.md +4 -108
- package/references/sdlc-mapping.md +73 -0
- package/references/state-machine.md +151 -0
- package/skills-library/AVAILABLE_TOOLS_REFERENCE.md +333 -0
- package/skills-library/SKILLS-INDEX.md +57 -558
- package/skills-library/SKILLS_LIBRARY_INDEX.md +532 -0
- package/skills-library/_general/api-patterns/api-field-name-mismatch.md +107 -0
- package/skills-library/_general/api-patterns/streaming-command-timeout.md +122 -0
- package/skills-library/_general/api-patterns/streaming-proxy-cors-bypass.md +102 -0
- package/skills-library/_general/automation/settings-gui-generator.md +172 -0
- package/skills-library/_general/database-solutions/data-type-mapping-reference.md +181 -0
- package/skills-library/_general/database-solutions/mysql-limit-offset-string-coercion.md +102 -0
- package/skills-library/_general/database-solutions/mysql-to-pg-migration.md +195 -0
- package/skills-library/_general/database-solutions/orm-schema-portability.md +193 -0
- package/skills-library/_general/database-solutions/persistent-analysis-storage.md +207 -0
- package/skills-library/_general/database-solutions/pg-to-mysql-schema-migration-methodology.md +190 -0
- package/skills-library/_general/database-solutions/sql-dialect-compatibility-matrix.md +306 -0
- package/skills-library/_general/database-solutions/sqlite-to-pg-migration.md +219 -0
- package/skills-library/_general/frontend/canvas-bubble-animation-grouping.md +270 -0
- package/skills-library/_general/frontend/color-token-migration.md +112 -0
- package/skills-library/_general/frontend/framer-motion-layoutid-grouping.md +150 -0
- package/skills-library/_general/frontend/pyqt6-settings-dialog.md +191 -0
- package/skills-library/_general/frontend/react-flow-animated-layout-switching.md +101 -0
- package/skills-library/_general/frontend/react-hooks-order-debugging.md +141 -0
- package/skills-library/_general/frontend/redux-localstorage-auth-desync.md +126 -0
- package/skills-library/_general/frontend/safari-csp-theme-color-debugging.md +124 -0
- package/skills-library/_general/frontend/safari-sw-cache-poisoning.md +138 -0
- package/skills-library/_general/frontend/svg-sparkline-no-charting-library.md +131 -0
- package/skills-library/_general/growth-marketing/oss-daily-growth-intelligence.md +224 -0
- package/skills-library/_general/integrations/claude-code-local-mcp-integration.md +250 -0
- package/skills-library/_general/integrations/mcp-composite-tool-orchestration.md +200 -0
- package/skills-library/_general/methodology/AGENT_SDK_STANDALONE_TOOLING.md +181 -0
- package/skills-library/_general/methodology/AGENT_TEAMS_GUIDE.md +169 -0
- package/skills-library/_general/methodology/ALAS_STATEFUL_EXECUTION.md +207 -0
- package/skills-library/_general/methodology/AUTO_REVIEWER_SUBAGENT.md +211 -0
- package/skills-library/_general/methodology/CONSISTENCY_CHECK_AMBIGUITY_GATE.md +96 -0
- package/skills-library/_general/methodology/DEAD_ENDS_SHELF.md +4 -4
- package/skills-library/_general/methodology/DISTILL_NOT_DUMP.md +108 -0
- package/skills-library/_general/methodology/EXECUTION_PROGRESS_MONITOR.md +157 -0
- package/skills-library/_general/methodology/HIERARCHICAL_REVIEW_MARS.md +122 -0
- package/skills-library/_general/methodology/MCP_INTER_AGENT_BRIDGE.md +207 -0
- package/skills-library/_general/methodology/MERMAID_WIZARD_DIAGRAMS.md +77 -0
- package/skills-library/_general/methodology/MISSING_DIMENSION_DETECTOR.md +89 -0
- package/skills-library/_general/methodology/MULTI_AGENT_COORDINATION.md +397 -0
- package/skills-library/_general/methodology/OBSERVATION_MASKING.md +100 -0
- package/skills-library/_general/methodology/PHOENIX_REBUILD_METHODOLOGY.md +82 -11
- package/skills-library/_general/methodology/REVIEW_BACKTRACK_PANEL.md +140 -0
- package/skills-library/_general/methodology/REVIEW_FIX_LOOP.md +117 -0
- package/skills-library/_general/methodology/VOTING_VERDICT_ARBITRATION.md +155 -0
- package/skills-library/_general/methodology/ZERO_FRICTION_CLI_SETUP.md +2 -2
- package/skills-library/_general/methodology/dead-code-activation.md +123 -0
- package/skills-library/_general/methodology/debug-swarm-researcher-escape-hatch.md +240 -240
- package/skills-library/_general/methodology/shell-autonomous-loop-fixplan.md +1 -1
- package/skills-library/_general/patterns-standards/GOF_DESIGN_PATTERNS_FOR_AI_AGENTS.md +5 -5
- package/skills-library/_general/patterns-standards/cascading-failure-diagnosis.md +119 -0
- package/skills-library/_general/patterns-standards/domain-specific-layout-algorithms.md +209 -0
- package/skills-library/_general/patterns-standards/python-desktop-app-architecture.md +399 -0
- package/skills-library/_general/patterns-standards/realtime-monitoring-dashboard.md +457 -0
- package/skills-library/_general/patterns-standards/togglable-processing-pipeline.md +169 -0
- package/skills-library/_general/performance/liveclock-extraction.md +112 -0
- package/skills-library/_general/performance/ref-based-canvas-animation.md +117 -0
- package/skills-library/_general/performance/use-visible-interval.md +131 -0
- package/skills-library/_general/testing/playwright-firefox-withcredentials-auth-issue.md +104 -0
- package/skills-library/_quarantine/README.md +30 -0
- package/skills-library/api-patterns/BROADCAST_SCHEDULER_SHARED_EXECUTE_FUNCTION.md +150 -0
- package/skills-library/api-patterns/ERROR_RESPONSE_STANDARDS.md +145 -0
- package/skills-library/api-patterns/EXPRESS_ROUTE_ORDERING_MIDDLEWARE_INTERCEPTION.md +326 -0
- package/skills-library/api-patterns/PAGINATION_PATTERNS.md +137 -0
- package/skills-library/api-patterns/PODCAST_PROGRESS_TRACKING_THREE_ROOT_CAUSES.md +277 -0
- package/skills-library/api-patterns/RATE_LIMITING_TOGGLE.md +155 -0
- package/skills-library/api-patterns/graphql-content-queries.md +708 -0
- package/skills-library/appointment-scheduler-design.md +423 -0
- package/skills-library/automation/AUTO_POPULATE_COMPLETE_GUIDE.md +631 -0
- package/skills-library/automation/CC_WORKFLOW_STUDIO.md +83 -0
- package/skills-library/automation/CLAUDE_CODE_SWARM_MODE.md +95 -0
- package/skills-library/automation/DAEMON_TRIGGER_FILE_IPC.md +195 -0
- package/skills-library/automation/scheduled-content-publishing.md +608 -0
- package/skills-library/awesome-workflows/Blogging-Platform-Instructions/view_commands.md +25 -0
- package/skills-library/awesome-workflows/CREDENTIAL-SECURITY-WORKFLOW.md +109 -0
- package/skills-library/awesome-workflows/DEBUGGING-WORKFLOW.md +124 -0
- package/skills-library/awesome-workflows/Design-Review-Workflow/README.md +31 -0
- package/skills-library/awesome-workflows/Design-Review-Workflow/design-principles-example.md +129 -0
- package/skills-library/awesome-workflows/Design-Review-Workflow/design-review-agent.md +107 -0
- package/skills-library/awesome-workflows/Design-Review-Workflow/design-review-claude-md-snippet.md +24 -0
- package/skills-library/awesome-workflows/Design-Review-Workflow/design-review-slash-command.md +38 -0
- package/skills-library/awesome-workflows/PARALLEL-RESEARCH-WORKFLOW.md +89 -0
- package/skills-library/awesome-workflows/PHASE-EXECUTION-WORKFLOW.md +97 -0
- package/skills-library/awesome-workflows/SESSION-HANDOFF-WORKFLOW.md +116 -0
- package/skills-library/cms-patterns/content-branch-preview.md +515 -0
- package/skills-library/cms-patterns/inline-visual-editing.md +666 -0
- package/skills-library/cms-patterns/mdx-component-content.md +649 -0
- package/skills-library/cms-patterns/media-manager-abstraction.md +827 -0
- package/skills-library/cms-patterns/schema-driven-form-generator.md +838 -0
- package/skills-library/complexity-metrics/complexity-divider.md +707 -0
- package/skills-library/complexity-metrics/work-with-complexity.md +193 -0
- package/skills-library/creative-multimedia/animation-stack-guide.md +577 -0
- package/skills-library/creative-multimedia/audio-enhancement-pipeline.md +625 -0
- package/skills-library/creative-multimedia/content-repurposing-pipeline.md +1146 -0
- package/skills-library/creative-multimedia/data-visualization-generator.md +862 -0
- package/skills-library/creative-multimedia/doc-to-podcast-pipeline.md +2184 -0
- package/skills-library/creative-multimedia/ffmpeg-command-generator.md +405 -0
- package/skills-library/creative-multimedia/image-optimization-pipeline.md +605 -0
- package/skills-library/creative-multimedia/multi-format-content-generator.md +1759 -0
- package/skills-library/creative-multimedia/og-image-generator.md +635 -0
- package/skills-library/creative-multimedia/podcast-audio-composition.md +1355 -0
- package/skills-library/creative-multimedia/podcast-quality-evaluation.md +1452 -0
- package/skills-library/creative-multimedia/podcast-script-generation.md +1841 -0
- package/skills-library/creative-multimedia/svg-generation.md +750 -0
- package/skills-library/creative-multimedia/text-to-speech-provider-selector.md +1414 -0
- package/skills-library/creative-multimedia/transcription-pipeline-selector.md +677 -0
- package/skills-library/creative-multimedia/video-streaming-setup.md +559 -0
- package/skills-library/database-solutions/AI_RESPONSE_DATABASE_CACHING.md +520 -0
- package/skills-library/database-solutions/CONDITIONAL_SQL_MIGRATION_PATTERN.md +119 -0
- package/skills-library/database-solutions/DATABASE_COLUMN_NAME_MISMATCH.md +393 -0
- package/skills-library/database-solutions/DATABASE_SCHEMA.md +394 -0
- package/skills-library/database-solutions/DATABASE_SCHEMA_VERIFICATION_GUIDE.md +348 -0
- package/skills-library/database-solutions/DATABASE_STRATEGY.md +71 -0
- package/skills-library/database-solutions/ES_MODULE_SEED_SCRIPT_PATTERN.md +52 -0
- package/skills-library/database-solutions/MIGRATION_GUIDE.md +3 -0
- package/skills-library/database-solutions/PLPGSQL_VARIABLE_CONFLICT_FIX.md +208 -0
- package/skills-library/database-solutions/POSTGRESQL_JSONB_DOUBLE_STRINGIFY_FIX.md +245 -0
- package/skills-library/database-solutions/POSTGRESQL_LICENSE_TABLE_DESIGN.md +393 -0
- package/skills-library/database-solutions/POSTGRESQL_UUID_DOCUMENT_RAG_DUAL_SCOPE.md +732 -0
- package/skills-library/database-solutions/POSTGRES_SQL_TEMPLATE_BINDING_ERROR.md +240 -0
- package/skills-library/database-solutions/PRISMA_DB_PUSH_DATA_LOSS_PREVENTION.md +141 -0
- package/skills-library/database-solutions/PRODUCTION_QUERY_OPTIMIZATION_RESTART_FIX.md +389 -0
- package/skills-library/database-solutions/RLS_SECURITY_GUIDE.md +107 -0
- package/skills-library/database-solutions/SCHEMA_ENHANCEMENTS_GUIDE.md +373 -0
- package/skills-library/database-solutions/SCHEMA_MIGRATION_GUIDE.md +368 -0
- package/skills-library/database-solutions/SCHEMA_VERIFICATION_QUICK_REFERENCE.md +104 -0
- package/skills-library/database-solutions/ai-erd-generator.md +1213 -0
- package/skills-library/database-solutions/content-publishing-states.md +631 -0
- package/skills-library/database-solutions/database-schema-designer.md +522 -0
- package/skills-library/database-solutions/er-diagram-components.md +569 -0
- package/skills-library/database-solutions/er-to-ddl-mapping.md +1405 -0
- package/skills-library/database-solutions/erd-creator-textbook-research.md +433 -0
- package/skills-library/database-solutions/erd-react-flow-architecture.md +1965 -0
- package/skills-library/database-solutions/mariadb-aggregate-function-replacement.md +145 -0
- package/skills-library/database-solutions/normalization-validator.md +778 -0
- package/skills-library/database-solutions/postgres-full-text-search-content.md +494 -0
- package/skills-library/database-solutions/postgresql-to-mysql-runtime-translation.md +286 -0
- package/skills-library/database-solutions/regex-alternation-ordering-sql-types.md +92 -0
- package/skills-library/database-solutions/reserved-word-context-aware-quoting.md +142 -0
- package/skills-library/database-solutions/sql-ddl-generator.md +756 -0
- package/skills-library/database-solutions/supabase-connection-pooler-fix.md +102 -0
- package/skills-library/deployment-security/CPANEL_NODE_DEPLOYMENT.md +166 -0
- package/skills-library/deployment-security/DEPLOYMENT.md +275 -0
- package/skills-library/deployment-security/DEPLOYMENT_CHECKLIST.md +363 -0
- package/skills-library/deployment-security/DEPLOYMENT_PLAN.md +669 -0
- package/skills-library/deployment-security/KNEX_DATABASE_ABSTRACTION.md +444 -0
- package/skills-library/deployment-security/LICENSE_KEY_SYSTEM.md +206 -0
- package/skills-library/deployment-security/NODE18_DEPENDENCY_COMPATIBILITY.md +284 -0
- package/skills-library/deployment-security/PHP_INSTALLER_WIZARD_GUIDE.md +315 -0
- package/skills-library/deployment-security/PM2_ENVIRONMENT_VARIABLE_CACHING.md +256 -0
- package/skills-library/deployment-security/PM2_MEMORY_EXHAUSTION_FIX.md +370 -0
- package/skills-library/deployment-security/PRODUCTION_DEPLOYMENT_GUIDE.md +592 -0
- package/skills-library/deployment-security/PRODUCTION_HARDENING_DOCUMENTATION.md +307 -0
- package/skills-library/deployment-security/PRODUCTION_RECOVERY_CHERRY_PICK_PATTERN.md +202 -0
- package/skills-library/deployment-security/PYINSTALLER_CUDA_WHISPER_BUNDLING.md +236 -0
- package/skills-library/deployment-security/SECURITY.md +41 -0
- package/skills-library/deployment-security/SMTP_SSL_HOSTNAME_MISMATCH_SHARED_HOSTING.md +220 -0
- package/skills-library/deployment-security/SPA_SEO_OPTIMIZATION_CPANEL.md +200 -0
- package/skills-library/deployment-security/SUPABASE_EDGE_FUNCTIONS.md +338 -0
- package/skills-library/deployment-security/VERCEL_GITHUB_DEPLOYMENT_GUIDE.md +858 -0
- package/skills-library/deployment-security/VPS_DEPLOYMENT_READINESS.md +356 -0
- package/skills-library/deployment-security/deployment-changes-not-applying.md +241 -0
- package/skills-library/deployment-security/env-file-management-production-local.md +203 -0
- package/skills-library/deployment-security/express-secure-file-downloads.md +413 -0
- package/skills-library/deployment-security/react-production-deployment-desktop-guide.md +2011 -0
- package/skills-library/deployment-security/self-hosted-supabase-coolify-guide.md +1684 -0
- package/skills-library/deployment-security/unique-features-ai-strategy-plaid-security.md +1613 -0
- package/skills-library/deployment-security/vps-deployment.md +135 -0
- package/skills-library/document-processing/WORD_EXPORT_MARKDOWN_FORMATTING.md +482 -0
- package/skills-library/document-processing/document-ai-landingai-integration.md +677 -0
- package/skills-library/document-processing/express-secure-file-downloads-mern.md +413 -0
- package/skills-library/document-processing/express-secure-file-downloads.md +413 -0
- package/skills-library/document-processing/md-to-word-converter.md +318 -0
- package/skills-library/document-processing/pdf-forms-integration/README.md +101 -0
- package/skills-library/document-processing/pdf-forms-integration/SKILL.md +662 -0
- package/skills-library/ecommerce/ADMIN_PRODUCTS_GUIDE.md +428 -0
- package/skills-library/ecommerce/ECOMMERCE_API_REFERENCE.md +776 -0
- package/skills-library/ecommerce/ECOMMERCE_COMPLETION_SUMMARY.md +673 -0
- package/skills-library/ecommerce/ECOMMERCE_IMPLEMENTATION_GUIDE.md +729 -0
- package/skills-library/ecommerce/ECOMMERCE_QUICK_REFERENCE.md +521 -0
- package/skills-library/ecommerce/ECOMMERCE_TESTING_CHECKLIST.md +565 -0
- package/skills-library/ecommerce/ECOMMERCE_WORKFLOW_GUIDE.md +1059 -0
- package/skills-library/ecommerce/PRODUCT_CREATION_EXPANDED.md +522 -0
- package/skills-library/ecommerce/agentic-commerce-protocol.md +203 -0
- package/skills-library/ecommerce/cart-abandonment-recovery.md +236 -0
- package/skills-library/ecommerce/cart-architecture-patterns.md +300 -0
- package/skills-library/ecommerce/cart-item-count-indicator.md +264 -0
- package/skills-library/ecommerce/checkout-ux-conversion.md +227 -0
- package/skills-library/ecommerce/composable-commerce-selection.md +166 -0
- package/skills-library/ecommerce/ecommerce-analytics-patterns.md +167 -0
- package/skills-library/ecommerce/fraud-detection-patterns.md +179 -0
- package/skills-library/ecommerce/inventory-stock-management.md +270 -0
- package/skills-library/ecommerce/order-saga-state-machine.md +336 -0
- package/skills-library/ecommerce/payment-provider-abstraction.md +245 -0
- package/skills-library/ecommerce/pci-compliance-checklist.md +192 -0
- package/skills-library/ecommerce/refund-chargeback-handling.md +177 -0
- package/skills-library/ecommerce/shipping-carrier-integration.md +218 -0
- package/skills-library/ecommerce/webhook-idempotency-patterns.md +253 -0
- package/skills-library/excalidraw-diagrams/.github/workflows/ci.yml +558 -0
- package/skills-library/excalidraw-diagrams/.github/workflows/prompt-gallery.yml +448 -0
- package/skills-library/excalidraw-diagrams/.github/workflows/release.yml +42 -0
- package/skills-library/excalidraw-diagrams/.github/workflows/test-reusable-ci.yml +25 -0
- package/skills-library/excalidraw-diagrams/CLAUDE.md +57 -0
- package/skills-library/excalidraw-diagrams/LICENSE +21 -0
- package/skills-library/excalidraw-diagrams/README.md +178 -0
- package/skills-library/excalidraw-diagrams/SKILL.md +715 -0
- package/skills-library/form-solutions/BUTTON_TYPE_FORM_SUBMISSION.md +336 -0
- package/skills-library/form-solutions/FILLABLE_PDF_IMPLEMENTATION.md +226 -0
- package/skills-library/form-solutions/SURVEYJS_QUESTIONNAIRE_SYSTEM.md +367 -0
- package/skills-library/form-solutions/tiptap-minimal-setup.md +690 -0
- package/skills-library/frontend/scholarly-classification-bubble-map.md +149 -0
- package/skills-library/infrastructure/ci-cd-pipeline-builder.md +517 -0
- package/skills-library/infrastructure/observability-designer.md +264 -0
- package/skills-library/infrastructure/performance-profiler.md +621 -0
- package/skills-library/installer-wizard-patterns.md +249 -0
- package/skills-library/integrations/CLAUDE_CODE_TOKEN_ANALYTICS.md +160 -0
- package/skills-library/integrations/CONFIGURABLE_AI_PROVIDER_SELECTION.md +728 -0
- package/skills-library/integrations/SOCKET_IO_BROADCAST_ALL_VS_ROOM.md +141 -0
- package/skills-library/integrations/VIRTUAL_MEETINGS_IMPLEMENTATION.md +374 -0
- package/skills-library/integrations/WORDPRESS_LEARNDASH_DATA_RECOVERY.md +53 -0
- package/skills-library/integrations/YOUTUBE_API_SETUP.md +141 -0
- package/skills-library/integrations/YOUTUBE_BOOKMARKING_EXPLANATION.md +252 -0
- package/skills-library/integrations/YOUTUBE_BOOKMARKING_SOLUTION.md +268 -0
- package/skills-library/integrations/YOUTUBE_OAUTH_SETUP_GUIDE.md +200 -0
- package/skills-library/integrations/YOUTUBE_VIDEO_FIX_COMPLETE.md +192 -0
- package/skills-library/integrations/ai-ml/GEMINI_AI_RAG_PIPELINE_COMPLETE_GUIDE.md +195 -0
- package/skills-library/integrations/ai-ml/GEMINI_IMAGE_GENERATION_SETUP.md +64 -0
- package/skills-library/integrations/cloudflare/cloudflare-turnstile-debugging.md +202 -0
- package/skills-library/integrations/cloudflare/cloudflare-turnstile-implementation.md +476 -0
- package/skills-library/integrations/cloudflare-turnstile-debugging.md +202 -0
- package/skills-library/integrations/cloudflare-turnstile-implementation.md +476 -0
- package/skills-library/integrations/ghost-creator-monetization-pattern.md +454 -0
- package/skills-library/integrations/headless-cms-architecture.md +484 -0
- package/skills-library/integrations/headless-cms-stack-selection.md +183 -0
- package/skills-library/integrations/payload-cms-patterns.md +674 -0
- package/skills-library/integrations/realtimestt-openwakeword-cuda-windows.md +229 -0
- package/skills-library/integrations/rss-podcast-integration.md +300 -0
- package/skills-library/integrations/wordpress/WORDPRESS_LEARNDASH_DATA_RECOVERY.md +53 -0
- package/skills-library/integrations/youtube/YOUTUBE_API_SETUP.md +141 -0
- package/skills-library/integrations/youtube/YOUTUBE_BOOKMARKING_EXPLANATION.md +252 -0
- package/skills-library/integrations/youtube/YOUTUBE_BOOKMARKING_SOLUTION.md +268 -0
- package/skills-library/integrations/youtube/YOUTUBE_OAUTH_SETUP_GUIDE.md +200 -0
- package/skills-library/integrations/youtube/YOUTUBE_VIDEO_FIX_COMPLETE.md +192 -0
- package/skills-library/marketing/campaign-analytics.md +97 -0
- package/skills-library/marketing/content-creator.md +105 -0
- package/skills-library/marketing/marketing-strategy-pmm.md +94 -0
- package/skills-library/marketing/social-media-analyzer.md +81 -0
- package/skills-library/methodology/ADVANCED_ORCHESTRATION_PATTERNS.md +401 -0
- package/skills-library/methodology/AGENT_SELF_IMPROVEMENT_LOOP.md +179 -0
- package/skills-library/methodology/BREATH_BASED_PARALLEL_EXECUTION.md +1 -1
- package/skills-library/methodology/CLEANSING_CYCLE.md +358 -0
- package/skills-library/methodology/CONFIDENCE_ANNOTATION_PATTERN.md +143 -0
- package/skills-library/methodology/CRITICAL_PATTERNS_DOCUMENTATION_COMPLETE.md +204 -0
- package/skills-library/methodology/DELIVERABLES_SUMMARY.md +341 -0
- package/skills-library/methodology/DIFFICULTY_AWARE_AGENT_ROUTING.md +252 -0
- package/skills-library/methodology/EVOLUTIONARY_SKILL_SYNTHESIS.md +219 -0
- package/skills-library/methodology/GLOMERULUS_DECISION_GATE.md +223 -0
- package/skills-library/methodology/HIBERNATION_SYSTEM.md +231 -0
- package/skills-library/methodology/INSTRUMENTATION_OVER_RESTRICTION.md +192 -0
- package/skills-library/methodology/MASTER_COMPLETION_SUMMARY.md +444 -0
- package/skills-library/methodology/MASTER_SESSION_COMPLETION.md +743 -0
- package/skills-library/methodology/MERN_QUICK_REFERENCE.md +358 -0
- package/skills-library/methodology/ORGAN_AGENT_MAPPING.md +177 -0
- package/skills-library/methodology/PARALLEL_WAVE_BASED_REFACTORING.md +440 -0
- package/skills-library/methodology/QUICK_REFERENCE.md +358 -0
- package/skills-library/methodology/SDFT_ONPOLICY_SELF_DISTILLATION.md +186 -0
- package/skills-library/methodology/SELF_QUESTIONING_TASK_GENERATION.md +270 -0
- package/skills-library/methodology/SESSION_COMPLETION_SUMMARY.md +304 -0
- package/skills-library/methodology/SESSION_SUMMARY.md +432 -0
- package/skills-library/methodology/WARRIOR_WORKFLOW_DEBUGGING_PROTOCOL.md +252 -0
- package/skills-library/methodology/tech-debt-tracker.md +570 -0
- package/skills-library/parallel-debug/SKILL.md +60 -0
- package/skills-library/patterns-standards/API_PATTERN_FIX_SUMMARY.md +236 -0
- package/skills-library/patterns-standards/BATCH_OPERATIONS_WITH_PROGRESS_MODAL.md +362 -0
- package/skills-library/patterns-standards/CRITICAL_CODING_PATTERNS.md +639 -0
- package/skills-library/patterns-standards/DARK_MODE_MODAL_VISIBILITY.md +258 -0
- package/skills-library/patterns-standards/ERROR_RESILIENCE_IMPLEMENTATION.md +375 -0
- package/skills-library/patterns-standards/ES_MODULE_IMPORT_HOISTING_DOTENV.md +298 -0
- package/skills-library/patterns-standards/NESTED_BACKDROP_FILTER_CSS_ARTIFACT_FIX.md +76 -0
- package/skills-library/patterns-standards/ORDERED_DETECTOR_PIPELINE_GRACEFUL_FALLBACK.md +333 -0
- package/skills-library/patterns-standards/PHASE_IMPORT_ERROR_DEBUGGING.md +271 -0
- package/skills-library/patterns-standards/PYNPUT_GLOBAL_HOTKEY_VK_MATCHING.md +252 -0
- package/skills-library/patterns-standards/REACT_USEEFFECT_CASCADE_RESET_FIX.md +132 -0
- package/skills-library/patterns-standards/SUBMENU_HOVER_DROPDOWN_PATTERN.md +225 -0
- package/skills-library/patterns-standards/TAILWIND_TEXT_VISIBILITY_OVERRIDE.md +322 -0
- package/skills-library/patterns-standards/THEME_AWARE_CSS_VARIABLES_PATTERN.md +209 -0
- package/skills-library/patterns-standards/THEME_USER_OBJECT_PROPERTY_NAMING.md +194 -0
- package/skills-library/patterns-standards/TOOLTIP_BLOCKING_CLICKS_FIX.md +267 -0
- package/skills-library/patterns-standards/claude-code-plugin-structure.md +235 -0
- package/skills-library/patterns-standards/react-i18next-setup.md +429 -0
- package/skills-library/patterns-standards/thesys-c1-generative-ui-integration.md +967 -0
- package/skills-library/plugin-development/CLAUDE_CODE_COMMAND_REGISTRATION_SILENT_FAILURE.md +315 -0
- package/skills-library/plugin-development/plugin-command-namespace-vs-global.md +390 -0
- package/skills-library/plugin-development/plugin-doc-auto-generation.md +172 -0
- package/skills-library/security/GITHUB_REPO_SECURITY_AUDIT.md +115 -0
- package/skills-library/security/admin-deletion-safety.md +396 -0
- package/skills-library/security/application-vuln-patterns.md +477 -0
- package/skills-library/security/env-secrets-manager.md +686 -0
- package/skills-library/security/secure-ai-application-templates.md +347 -0
- package/skills-library/security/sql-injection-prevention-postgresjs.md +151 -0
- package/skills-library/supabase-connection-pooler-fix.md +102 -0
- package/skills-library/system-context/POWERSHELL_BASH_INTEROP.md +82 -0
- package/skills-library/system-context/SERVICE_LIFECYCLE_MANAGEMENT.md +119 -0
- package/skills-library/system-context/SKILL.md +40 -0
- package/skills-library/system-context/WINDOWS_DEV_ENVIRONMENT.md +73 -0
- package/skills-library/testing/E2E_PLAYWRIGHT_PATTERNS.md +99 -0
- package/skills-library/testing/INTEGRATION_TEST_STRATEGY.md +82 -0
- package/skills-library/testing/RED_GREEN_BUGFIX_GATE.md +203 -0
- package/skills-library/testing/TEST_DATA_MANAGEMENT.md +69 -0
- package/skills-library/testing/VITEST_UNIT_TEST_PATTERNS.md +75 -0
- package/skills-library/testing/playwright-api-security-tests.md +202 -0
- package/skills-library/toolbox/SKILL.md +84 -0
- package/skills-library/toolbox/code-graph-and-web-scraping-mcps.md +237 -0
- package/skills-library/ui-ux-pro-max/ACCESSIBILITY_ESSENTIALS.md +115 -0
- package/skills-library/ui-ux-pro-max/DESIGN_SYSTEM_SCAFFOLDING.md +133 -0
- package/skills-library/ui-ux-pro-max/RESPONSIVE_LAYOUT_PATTERNS.md +119 -0
- package/skills-library/ui-ux-pro-max/SKILL.md +386 -0
- package/skills-library/ui-ux-pro-max/data/charts.csv +26 -0
- package/skills-library/ui-ux-pro-max/data/colors.csv +97 -0
- package/skills-library/ui-ux-pro-max/data/icons.csv +101 -0
- package/skills-library/ui-ux-pro-max/data/landing.csv +31 -0
- package/skills-library/ui-ux-pro-max/data/products.csv +97 -0
- package/skills-library/ui-ux-pro-max/data/react-performance.csv +45 -0
- package/skills-library/ui-ux-pro-max/data/stacks/astro.csv +54 -0
- package/skills-library/ui-ux-pro-max/data/stacks/flutter.csv +53 -0
- package/skills-library/ui-ux-pro-max/data/stacks/html-tailwind.csv +56 -0
- package/skills-library/ui-ux-pro-max/data/stacks/jetpack-compose.csv +53 -0
- package/skills-library/ui-ux-pro-max/data/stacks/nextjs.csv +53 -0
- package/skills-library/ui-ux-pro-max/data/stacks/nuxt-ui.csv +51 -0
- package/skills-library/ui-ux-pro-max/data/stacks/nuxtjs.csv +59 -0
- package/skills-library/ui-ux-pro-max/data/stacks/react-native.csv +52 -0
- package/skills-library/ui-ux-pro-max/data/stacks/react.csv +54 -0
- package/skills-library/ui-ux-pro-max/data/stacks/shadcn.csv +61 -0
- package/skills-library/ui-ux-pro-max/data/stacks/svelte.csv +54 -0
- package/skills-library/ui-ux-pro-max/data/stacks/swiftui.csv +51 -0
- package/skills-library/ui-ux-pro-max/data/stacks/vue.csv +50 -0
- package/skills-library/ui-ux-pro-max/data/styles.csv +68 -0
- package/skills-library/ui-ux-pro-max/data/typography.csv +58 -0
- package/skills-library/ui-ux-pro-max/data/ui-reasoning.csv +101 -0
- package/skills-library/ui-ux-pro-max/data/ux-guidelines.csv +100 -0
- package/skills-library/ui-ux-pro-max/data/web-interface.csv +31 -0
- package/skills-library/wordpress-style-theme-components.md +1526 -0
- package/templates/ASSUMPTIONS.md +1 -1
- package/templates/DECISION_LOG.md +0 -1
- package/templates/phase-prompt.md +1 -1
- package/templates/phoenix-comparison.md +6 -6
- package/templates/skill-api-integration.md +106 -0
- package/templates/skill-architecture-pattern.md +92 -0
- package/templates/skill-debug-pattern.md +98 -0
- package/templates/skill-devops-recipe.md +107 -0
- package/templates/skill-general.md +65 -0
- package/templates/skill-ui-component.md +113 -0
- package/tools/uat-runner.py +179 -0
- package/version.json +7 -3
- package/workflows/handoff-session.md +2 -2
- package/workflows/new-project.md +2 -2
- package/workflows/plan-phase.md +1 -1
- package/.claude-plugin/plugin.json +0 -64
- package/skills-library/_general/methodology/LIVE_BREADCRUMB_PROTOCOL.md +0 -242
- package/skills-library/_general/methodology/llm-judge-memory-crud.md +0 -241
- package/skills-library/methodology/REFLEXION_MEMORY_PATTERN.md +0 -183
- package/skills-library/methodology/RESEARCH_BACKED_WORKFLOW_UPGRADE.md +0 -263
- package/skills-library/methodology/SABBATH_REST_PATTERN.md +0 -267
- package/skills-library/methodology/STONE_AND_SCAFFOLD.md +0 -220
- package/skills-library/specialists/api-architecture/api-designer.md +0 -49
- package/skills-library/specialists/api-architecture/graphql-architect.md +0 -49
- package/skills-library/specialists/api-architecture/mcp-developer.md +0 -51
- package/skills-library/specialists/api-architecture/microservices-architect.md +0 -50
- package/skills-library/specialists/api-architecture/websocket-engineer.md +0 -48
- package/skills-library/specialists/backend/django-expert.md +0 -52
- package/skills-library/specialists/backend/fastapi-expert.md +0 -52
- package/skills-library/specialists/backend/laravel-specialist.md +0 -52
- package/skills-library/specialists/backend/nestjs-expert.md +0 -51
- package/skills-library/specialists/backend/rails-expert.md +0 -53
- package/skills-library/specialists/backend/spring-boot-engineer.md +0 -56
- package/skills-library/specialists/data-ml/fine-tuning-expert.md +0 -48
- package/skills-library/specialists/data-ml/ml-pipeline.md +0 -47
- package/skills-library/specialists/data-ml/pandas-pro.md +0 -47
- package/skills-library/specialists/data-ml/rag-architect.md +0 -51
- package/skills-library/specialists/data-ml/spark-engineer.md +0 -47
- package/skills-library/specialists/frontend/angular-architect.md +0 -52
- package/skills-library/specialists/frontend/flutter-expert.md +0 -51
- package/skills-library/specialists/frontend/nextjs-developer.md +0 -54
- package/skills-library/specialists/frontend/react-native-expert.md +0 -50
- package/skills-library/specialists/frontend/vue-expert.md +0 -51
- package/skills-library/specialists/infrastructure/chaos-engineer.md +0 -74
- package/skills-library/specialists/infrastructure/cloud-architect.md +0 -70
- package/skills-library/specialists/infrastructure/database-optimizer.md +0 -64
- package/skills-library/specialists/infrastructure/devops-engineer.md +0 -70
- package/skills-library/specialists/infrastructure/kubernetes-specialist.md +0 -52
- package/skills-library/specialists/infrastructure/monitoring-expert.md +0 -70
- package/skills-library/specialists/infrastructure/sre-engineer.md +0 -70
- package/skills-library/specialists/infrastructure/terraform-engineer.md +0 -51
- package/skills-library/specialists/languages/cpp-pro.md +0 -74
- package/skills-library/specialists/languages/csharp-developer.md +0 -69
- package/skills-library/specialists/languages/dotnet-core-expert.md +0 -54
- package/skills-library/specialists/languages/golang-pro.md +0 -51
- package/skills-library/specialists/languages/java-architect.md +0 -49
- package/skills-library/specialists/languages/javascript-pro.md +0 -68
- package/skills-library/specialists/languages/kotlin-specialist.md +0 -68
- package/skills-library/specialists/languages/php-pro.md +0 -49
- package/skills-library/specialists/languages/python-pro.md +0 -52
- package/skills-library/specialists/languages/react-expert.md +0 -51
- package/skills-library/specialists/languages/rust-engineer.md +0 -50
- package/skills-library/specialists/languages/sql-pro.md +0 -56
- package/skills-library/specialists/languages/swift-expert.md +0 -69
- package/skills-library/specialists/languages/typescript-pro.md +0 -51
- package/skills-library/specialists/platform/atlassian-mcp.md +0 -52
- package/skills-library/specialists/platform/embedded-systems.md +0 -53
- package/skills-library/specialists/platform/game-developer.md +0 -53
- package/skills-library/specialists/platform/salesforce-developer.md +0 -53
- package/skills-library/specialists/platform/shopify-expert.md +0 -49
- package/skills-library/specialists/platform/wordpress-pro.md +0 -49
- package/skills-library/specialists/quality/code-documenter.md +0 -51
- package/skills-library/specialists/quality/code-reviewer.md +0 -67
- package/skills-library/specialists/quality/debugging-wizard.md +0 -51
- package/skills-library/specialists/quality/fullstack-guardian.md +0 -51
- package/skills-library/specialists/quality/legacy-modernizer.md +0 -50
- package/skills-library/specialists/quality/playwright-expert.md +0 -65
- package/skills-library/specialists/quality/spec-miner.md +0 -56
- package/skills-library/specialists/quality/test-master.md +0 -65
- package/skills-library/specialists/security/secure-code-guardian.md +0 -55
- package/skills-library/specialists/security/security-reviewer.md +0 -53
- package/skills-library/specialists/workflow/architecture-designer.md +0 -53
- package/skills-library/specialists/workflow/cli-developer.md +0 -70
- package/skills-library/specialists/workflow/feature-forge.md +0 -65
- package/skills-library/specialists/workflow/prompt-engineer.md +0 -54
- package/skills-library/specialists/workflow/the-fool.md +0 -62
- /package/skills-library/{performance → _general/performance}/cache-augmented-generation.md +0 -0
- /package/skills-library/{debugging → parallel-debug}/FAILURE_TAXONOMY_CLASSIFICATION.md +0 -0
- /package/skills-library/{debugging → parallel-debug}/THREE_AGENT_HYPOTHESIS_DEBUGGING.md +0 -0
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: streaming-command-timeout
|
|
3
|
+
category: api-patterns
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
contributed: 2026-03-12
|
|
6
|
+
contributor: c3-server
|
|
7
|
+
last_updated: 2026-03-12
|
|
8
|
+
contributors:
|
|
9
|
+
- c3-server
|
|
10
|
+
tags: [bun, node, process, timeout, streaming, pm2, ssh, spawn]
|
|
11
|
+
difficulty: easy
|
|
12
|
+
usage_count: 0
|
|
13
|
+
success_rate: 100
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Streaming Command Timeout — Preventing Hanging Process Execution
|
|
17
|
+
|
|
18
|
+
## Problem
|
|
19
|
+
|
|
20
|
+
When executing shell commands via `Bun.spawn` or `child_process.spawn`, some commands are streaming by default and never exit (e.g., `pm2 logs`, `tail -f`, `docker logs -f`). If you `await` their stdout, the process hangs forever — the API endpoint never responds, the UI spinner runs indefinitely, and server resources are leaked.
|
|
21
|
+
|
|
22
|
+
**Symptoms:**
|
|
23
|
+
- UI button spins forever after clicking "View Logs"
|
|
24
|
+
- API endpoint never returns a response
|
|
25
|
+
- Server eventually runs out of file descriptors or memory
|
|
26
|
+
- `pm2 logs` (without `--nostream`) never exits
|
|
27
|
+
- SSH commands to remote servers hang if the remote command streams
|
|
28
|
+
|
|
29
|
+
## Solution Pattern
|
|
30
|
+
|
|
31
|
+
Two-layer defense:
|
|
32
|
+
|
|
33
|
+
1. **Fix the command itself:** Add flags that make commands exit after producing output (`--nostream`, `--lines N`, `-n N`)
|
|
34
|
+
2. **Add a timeout wrapper:** Use `Promise.race` between process completion and a timeout timer. On timeout, kill the process and return partial output.
|
|
35
|
+
|
|
36
|
+
## Code Example
|
|
37
|
+
|
|
38
|
+
```typescript
|
|
39
|
+
// Before (hangs forever on streaming commands)
|
|
40
|
+
const proc = Bun.spawn(['bash', '-c', 'pm2 logs --lines 30'], {
|
|
41
|
+
stdout: 'pipe', stderr: 'pipe'
|
|
42
|
+
})
|
|
43
|
+
const stdout = await new Response(proc.stdout).text() // HANGS FOREVER
|
|
44
|
+
|
|
45
|
+
// After (exits cleanly with timeout protection)
|
|
46
|
+
const ACTION_TIMEOUT_MS = 30_000
|
|
47
|
+
|
|
48
|
+
const proc = Bun.spawn(['bash', '-c', 'pm2 logs --lines 30 --nostream'], {
|
|
49
|
+
stdout: 'pipe', stderr: 'pipe'
|
|
50
|
+
})
|
|
51
|
+
|
|
52
|
+
const timeout = new Promise<'timeout'>((resolve) =>
|
|
53
|
+
setTimeout(() => resolve('timeout'), ACTION_TIMEOUT_MS)
|
|
54
|
+
)
|
|
55
|
+
|
|
56
|
+
const result = await Promise.race([
|
|
57
|
+
(async () => {
|
|
58
|
+
const stdout = await new Response(proc.stdout).text()
|
|
59
|
+
const stderr = await new Response(proc.stderr).text()
|
|
60
|
+
const exitCode = await proc.exited
|
|
61
|
+
return { stdout, stderr, exitCode } as const
|
|
62
|
+
})(),
|
|
63
|
+
timeout,
|
|
64
|
+
])
|
|
65
|
+
|
|
66
|
+
if (result === 'timeout') {
|
|
67
|
+
proc.kill()
|
|
68
|
+
const partial = await Promise.race([
|
|
69
|
+
new Response(proc.stdout).text(),
|
|
70
|
+
new Promise<string>((r) => setTimeout(() => r(''), 1000)),
|
|
71
|
+
])
|
|
72
|
+
return {
|
|
73
|
+
output: (partial || '') + '\n[Timed out after 30s]',
|
|
74
|
+
success: true, // Partial output is still useful
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Implementation Steps
|
|
80
|
+
|
|
81
|
+
1. Audit all commands that can stream: `pm2 logs`, `docker logs`, `tail`, `ssh` with streaming remotes
|
|
82
|
+
2. Add exit flags: `--nostream`, `--lines N`, `-n N`
|
|
83
|
+
3. Wrap execution in `Promise.race` with a timeout (30s is reasonable for most commands)
|
|
84
|
+
4. On timeout: kill process, collect partial output, return with timeout notice
|
|
85
|
+
5. Return partial output as success (user still gets useful data)
|
|
86
|
+
|
|
87
|
+
## Common Streaming Commands & Fixes
|
|
88
|
+
|
|
89
|
+
| Command | Fix |
|
|
90
|
+
|---------|-----|
|
|
91
|
+
| `pm2 logs` | Add `--nostream` |
|
|
92
|
+
| `docker logs` | Remove `-f`, add `--tail N` |
|
|
93
|
+
| `tail -f` | Use `tail -n N` instead |
|
|
94
|
+
| `ssh ... 'pm2 logs'` | Add `--nostream` to remote command |
|
|
95
|
+
| `kubectl logs -f` | Remove `-f`, add `--tail N` |
|
|
96
|
+
|
|
97
|
+
## When to Use
|
|
98
|
+
|
|
99
|
+
- Any API endpoint that executes shell commands
|
|
100
|
+
- Quick action / one-click command systems
|
|
101
|
+
- Remote command execution via SSH
|
|
102
|
+
- CI/CD pipeline step execution
|
|
103
|
+
- Any `Bun.spawn` / `child_process.spawn` where the command set is user-configurable
|
|
104
|
+
|
|
105
|
+
## When NOT to Use
|
|
106
|
+
|
|
107
|
+
- SSE/WebSocket streaming endpoints (you WANT streaming there)
|
|
108
|
+
- Background jobs with their own lifecycle management
|
|
109
|
+
- Commands you've verified always exit quickly
|
|
110
|
+
|
|
111
|
+
## Common Mistakes
|
|
112
|
+
|
|
113
|
+
- Setting timeout too short (5s) — SSH commands to remote servers can take 10-15s to connect
|
|
114
|
+
- Killing the process but not collecting partial output — user gets "timeout" with no useful data
|
|
115
|
+
- Forgetting to clean up the timeout timer on successful completion (minor memory leak)
|
|
116
|
+
- Not adding `--nostream` AND adding timeout — the timeout alone works but wastes 30s every time
|
|
117
|
+
|
|
118
|
+
## References
|
|
119
|
+
|
|
120
|
+
- Bun docs: `Bun.spawn` API
|
|
121
|
+
- PM2 docs: `--nostream` flag (outputs then exits)
|
|
122
|
+
- Contributed from: c3-server action executor (PM2 logs hanging forever)
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: streaming-proxy-cors-bypass
|
|
3
|
+
category: api-patterns
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
contributed: 2026-03-12
|
|
6
|
+
contributor: mern-community-lms
|
|
7
|
+
last_updated: 2026-03-12
|
|
8
|
+
contributors:
|
|
9
|
+
- mern-community-lms
|
|
10
|
+
tags: [cors, proxy, streaming, pipeline, node, express, video, cdn]
|
|
11
|
+
difficulty: medium
|
|
12
|
+
usage_count: 0
|
|
13
|
+
success_rate: 100
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Streaming Proxy CORS Bypass
|
|
17
|
+
|
|
18
|
+
## Problem
|
|
19
|
+
|
|
20
|
+
Third-party CDNs (Zoom, Vimeo, cloud storage) serve media files without CORS headers. When your frontend tries to play or fetch these URLs directly, the browser blocks the request with a CORS error. A common workaround is to redirect the client (302) to the CDN URL, but this still fails because the browser's media element follows the redirect and hits the same CORS wall.
|
|
21
|
+
|
|
22
|
+
**Symptoms:**
|
|
23
|
+
- Video/audio playback fails silently or shows "blocked by CORS policy"
|
|
24
|
+
- `fetch()` to CDN URLs returns opaque responses or errors
|
|
25
|
+
- 302 redirects work in Postman but fail in browser
|
|
26
|
+
- `Access-Control-Allow-Origin` header missing from CDN response
|
|
27
|
+
|
|
28
|
+
## Solution Pattern
|
|
29
|
+
|
|
30
|
+
Instead of redirecting the client to the CDN, your server acts as a **streaming proxy** — it fetches the CDN resource server-side (no CORS restriction) and pipes the response body directly to the client using Node.js `stream.pipeline()`. The client only talks to your origin server, so CORS is never an issue.
|
|
31
|
+
|
|
32
|
+
**Key insight:** Server-to-server requests are not subject to CORS. By interposing your server as a transparent pipe, you eliminate the browser's CORS check entirely while adding minimal latency (streaming, not buffering).
|
|
33
|
+
|
|
34
|
+
## Code Example
|
|
35
|
+
|
|
36
|
+
```javascript
|
|
37
|
+
// Before (broken — 302 redirect hits CORS wall)
|
|
38
|
+
app.get('/api/video/replay/:meetingId', async (req, res) => {
|
|
39
|
+
const downloadUrl = await getZoomDownloadUrl(req.params.meetingId);
|
|
40
|
+
res.redirect(downloadUrl); // Browser follows redirect, CDN blocks CORS
|
|
41
|
+
});
|
|
42
|
+
|
|
43
|
+
// After (streaming proxy — no CORS issue)
|
|
44
|
+
import { pipeline } from 'stream/promises';
|
|
45
|
+
|
|
46
|
+
app.get('/api/video/replay/:meetingId', async (req, res) => {
|
|
47
|
+
const downloadUrl = await getZoomDownloadUrl(req.params.meetingId);
|
|
48
|
+
|
|
49
|
+
const cdnResponse = await fetch(downloadUrl, { redirect: 'follow' });
|
|
50
|
+
|
|
51
|
+
if (!cdnResponse.ok) {
|
|
52
|
+
return res.status(cdnResponse.status).json({ error: 'CDN fetch failed' });
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
// Forward content headers
|
|
56
|
+
res.set('Content-Type', cdnResponse.headers.get('content-type') || 'video/mp4');
|
|
57
|
+
const contentLength = cdnResponse.headers.get('content-length');
|
|
58
|
+
if (contentLength) res.set('Content-Length', contentLength);
|
|
59
|
+
|
|
60
|
+
// Stream body directly — no buffering in memory
|
|
61
|
+
await pipeline(cdnResponse.body, res);
|
|
62
|
+
});
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Implementation Steps
|
|
66
|
+
|
|
67
|
+
1. Replace `res.redirect(cdnUrl)` with a server-side `fetch(cdnUrl)`
|
|
68
|
+
2. Forward relevant headers (`Content-Type`, `Content-Length`, `Content-Disposition`)
|
|
69
|
+
3. Use `stream.pipeline()` (or `pipe()`) to stream the response body to the client
|
|
70
|
+
4. Handle errors — if CDN returns 403/404, return a clean error to client
|
|
71
|
+
5. For authenticated CDNs (Zoom, AWS S3), add auth headers/tokens to the server-side fetch
|
|
72
|
+
|
|
73
|
+
## When to Use
|
|
74
|
+
|
|
75
|
+
- CDN serves media without CORS headers and you can't control the CDN config
|
|
76
|
+
- Third-party API returns download URLs that work server-side but not in browsers
|
|
77
|
+
- You need to proxy large files without buffering them entirely in memory
|
|
78
|
+
- OAuth-protected downloads (Zoom, Google Drive) where tokens can't be exposed to client
|
|
79
|
+
|
|
80
|
+
## When NOT to Use
|
|
81
|
+
|
|
82
|
+
- You control the CDN and can add CORS headers directly (simpler)
|
|
83
|
+
- The CDN already includes proper `Access-Control-Allow-Origin` headers
|
|
84
|
+
- Files are small enough to buffer — just use `res.send(await fetch(url).then(r => r.buffer()))`
|
|
85
|
+
- Static assets that should be served from a CDN for performance (proxy adds latency)
|
|
86
|
+
|
|
87
|
+
## Common Mistakes
|
|
88
|
+
|
|
89
|
+
- **Buffering the entire file in memory** — use `pipeline()` for streaming, not `await response.buffer()`
|
|
90
|
+
- **Not following redirects** — CDN URLs often redirect; use `{ redirect: 'follow' }` in fetch
|
|
91
|
+
- **Forgetting Content-Type** — browser won't play video if header is `application/octet-stream`
|
|
92
|
+
- **Not handling CDN auth expiry** — Zoom/S3 signed URLs expire; refresh token before proxying
|
|
93
|
+
|
|
94
|
+
## Related Skills
|
|
95
|
+
|
|
96
|
+
- [streaming-command-timeout](../api-patterns/streaming-command-timeout.md) — Timeout handling for streaming operations
|
|
97
|
+
|
|
98
|
+
## References
|
|
99
|
+
|
|
100
|
+
- Node.js `stream.pipeline()` documentation
|
|
101
|
+
- Zoom Server-to-Server OAuth API
|
|
102
|
+
- Contributed from: mern-community-lms (Zoom replay fix, 2026-03-12)
|
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: settings-gui-generator
|
|
3
|
+
category: automation
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
contributed: 2026-02-24
|
|
6
|
+
contributor: voice-bridge-v4
|
|
7
|
+
last_updated: 2026-02-24
|
|
8
|
+
contributors:
|
|
9
|
+
- voice-bridge-v4
|
|
10
|
+
tags: [pyqt6, python, settings, config, code-generation, yaml, dark-theme, gui]
|
|
11
|
+
difficulty: medium
|
|
12
|
+
usage_count: 0
|
|
13
|
+
success_rate: 100
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Settings GUI Generator (Config-to-Dialog Pattern)
|
|
17
|
+
|
|
18
|
+
## Problem
|
|
19
|
+
|
|
20
|
+
Every Python desktop app with a YAML/JSON/TOML config file eventually needs a settings dialog. The typical anti-pattern:
|
|
21
|
+
- `os.startfile(config_path)` opens raw YAML in Notepad
|
|
22
|
+
- Users break YAML indentation
|
|
23
|
+
- No discoverability — users can't see what options exist
|
|
24
|
+
- No validation — invalid values crash at runtime
|
|
25
|
+
- No live reload — changes require full restart
|
|
26
|
+
|
|
27
|
+
Building a settings dialog from scratch takes 30-60 minutes and commonly hits the **cross-thread bug** when called from pystray callbacks.
|
|
28
|
+
|
|
29
|
+
## Solution Pattern
|
|
30
|
+
|
|
31
|
+
Map config structure to PyQt6 widgets automatically:
|
|
32
|
+
|
|
33
|
+
| Config Type | Widget | Example |
|
|
34
|
+
|-------------|--------|---------|
|
|
35
|
+
| `boolean` | QCheckBox | `enabled: true` → ☑ Enabled |
|
|
36
|
+
| `string` (enum) | QComboBox | `backend: "piper"` → dropdown [piper, edge] |
|
|
37
|
+
| `number` | QSpinBox / QSlider | `speed: 1.0` → slider 0.5-2.0 |
|
|
38
|
+
| `string` (free) | QLineEdit | `voice: "en_US-lessac"` → text input |
|
|
39
|
+
| Nested dict with `enabled` | QCheckBox in QGroupBox | Feature group toggle |
|
|
40
|
+
|
|
41
|
+
### Thread-Safety Pattern (CRITICAL)
|
|
42
|
+
|
|
43
|
+
System tray callbacks (pystray) run on a background thread. Qt widgets MUST be created on the Qt GUI thread. The dialog will RENDER but be UNCLICKABLE if created from the wrong thread.
|
|
44
|
+
|
|
45
|
+
**Solution:** Signal marshaling:
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
# In overlay/main window:
|
|
49
|
+
sig_open_settings = pyqtSignal() # Thread-safe bridge
|
|
50
|
+
|
|
51
|
+
# Tray callback (WRONG thread):
|
|
52
|
+
def _open_settings(self):
|
|
53
|
+
self.overlay.sig_open_settings.emit() # Just emit signal
|
|
54
|
+
|
|
55
|
+
# Qt slot (RIGHT thread, auto-marshaled):
|
|
56
|
+
def _on_open_settings(self):
|
|
57
|
+
from .settings_dialog import SettingsDialog
|
|
58
|
+
dialog = SettingsDialog(self)
|
|
59
|
+
dialog.settings_changed.connect(self._on_settings_changed)
|
|
60
|
+
dialog.exec()
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
## Code Example
|
|
64
|
+
|
|
65
|
+
```python
|
|
66
|
+
# Before (problematic) — opens raw YAML in Notepad
|
|
67
|
+
def _open_settings(self) -> None:
|
|
68
|
+
os.startfile(str(CONFIG_FILE))
|
|
69
|
+
|
|
70
|
+
# After (solution) — proper settings GUI with live reload
|
|
71
|
+
class SettingsDialog(QDialog):
|
|
72
|
+
settings_changed = pyqtSignal(dict)
|
|
73
|
+
|
|
74
|
+
def __init__(self, parent=None):
|
|
75
|
+
super().__init__(parent)
|
|
76
|
+
self.config = load_config()
|
|
77
|
+
self._widgets = {} # config_path → widget mapping
|
|
78
|
+
self._build_ui()
|
|
79
|
+
|
|
80
|
+
def _build_ui(self):
|
|
81
|
+
features = [
|
|
82
|
+
("post_processing.hallucination_filter", "Hallucination Filter",
|
|
83
|
+
"Reject garbage like 'thank you for watching'"),
|
|
84
|
+
("post_processing.filler_removal", "Filler Removal",
|
|
85
|
+
"Strip um, uh, like, basically"),
|
|
86
|
+
]
|
|
87
|
+
for path, label, desc in features:
|
|
88
|
+
cb = QCheckBox(f"{label}\n{desc}")
|
|
89
|
+
value = self._get_nested(self.config, path)
|
|
90
|
+
cb.setChecked(value.get("enabled", False))
|
|
91
|
+
self._widgets[path] = cb
|
|
92
|
+
|
|
93
|
+
def _save(self):
|
|
94
|
+
config = load_config() # Fresh read!
|
|
95
|
+
for path, widget in self._widgets.items():
|
|
96
|
+
self._set_nested(config, path, "enabled", widget.isChecked())
|
|
97
|
+
save_config(config)
|
|
98
|
+
self.settings_changed.emit(config)
|
|
99
|
+
self.accept()
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
### Dark Theme Stylesheet
|
|
103
|
+
|
|
104
|
+
```python
|
|
105
|
+
STYLESHEET = """
|
|
106
|
+
QDialog { background-color: #0f0f14; color: #f0f0f5; }
|
|
107
|
+
QGroupBox {
|
|
108
|
+
background-color: #1a1a24; border: 1px solid #2a2a3a;
|
|
109
|
+
border-radius: 8px; margin-top: 12px; padding: 16px;
|
|
110
|
+
}
|
|
111
|
+
QGroupBox::title { color: #6366f1; }
|
|
112
|
+
QCheckBox::indicator:checked {
|
|
113
|
+
background-color: #6366f1; border-color: #6366f1;
|
|
114
|
+
}
|
|
115
|
+
QPushButton {
|
|
116
|
+
background-color: #6366f1; color: white;
|
|
117
|
+
border-radius: 6px; padding: 8px 24px;
|
|
118
|
+
}
|
|
119
|
+
"""
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
### Live Reload Pattern
|
|
123
|
+
|
|
124
|
+
```python
|
|
125
|
+
def _on_settings_changed(self, new_config: dict):
|
|
126
|
+
self.text_processor = TextProcessor(new_config) # Recreate from config
|
|
127
|
+
# No restart needed — next call uses new settings
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
## Implementation Steps
|
|
131
|
+
|
|
132
|
+
1. Create `settings_dialog.py` alongside your main module
|
|
133
|
+
2. Define feature list as `(config_path, label, description)` tuples
|
|
134
|
+
3. Map each config type to appropriate Qt widget
|
|
135
|
+
4. Build UI with QGroupBox sections for logical grouping
|
|
136
|
+
5. Wire save button: load_config() fresh → apply widget states → save_config()
|
|
137
|
+
6. Emit `settings_changed` signal with full config dict
|
|
138
|
+
7. In main app: connect signal via Qt signal (NOT direct call from tray)
|
|
139
|
+
8. Add fallback to `os.startfile()` if dialog import fails
|
|
140
|
+
|
|
141
|
+
## When to Use
|
|
142
|
+
|
|
143
|
+
- Any Python desktop app with a YAML/JSON/TOML config file
|
|
144
|
+
- When "Settings" currently opens the raw config in a text editor
|
|
145
|
+
- When features are togglable and users shouldn't understand config syntax
|
|
146
|
+
- When you want live reload without full app restart
|
|
147
|
+
|
|
148
|
+
## When NOT to Use
|
|
149
|
+
|
|
150
|
+
- Web apps (use a proper web settings page)
|
|
151
|
+
- CLI-only tools (use command-line flags)
|
|
152
|
+
- When config has only 1-2 simple values (overkill)
|
|
153
|
+
- When config changes ALWAYS require restart (signal adds complexity)
|
|
154
|
+
|
|
155
|
+
## Common Mistakes
|
|
156
|
+
|
|
157
|
+
- **Cross-thread dialog creation** — The #1 bug. ALWAYS use signal marshaling from tray callbacks. The dialog renders but is unclickable without it.
|
|
158
|
+
- **Reading stale config in `_save()`** — Always `load_config()` fresh before applying widget states
|
|
159
|
+
- **Not handling nested dicts** — Config like `{post_processing: {filter: {enabled: true}}}` needs recursive navigation
|
|
160
|
+
- **Missing deep_merge for defaults** — If config file doesn't have new feature sections, `deep_merge(DEFAULT_CONFIG, loaded)` ensures they appear
|
|
161
|
+
- **Not writing defaults to disk** — After adding new features, call `load_config()` then `save_config()` to persist the merged defaults
|
|
162
|
+
|
|
163
|
+
## Related Skills
|
|
164
|
+
|
|
165
|
+
- [pyqt6-settings-dialog](../frontend/pyqt6-settings-dialog.md) — The pattern this generates
|
|
166
|
+
- [togglable-processing-pipeline](../patterns-standards/togglable-processing-pipeline.md) — Config-driven feature ordering
|
|
167
|
+
|
|
168
|
+
## References
|
|
169
|
+
|
|
170
|
+
- Contributed from: voice-bridge-v4
|
|
171
|
+
- Real implementation: `voice-bridge-v3/src/voice_bridge_v3/settings_dialog.py`
|
|
172
|
+
- PyQt6 QDialog docs: https://doc.qt.io/qtforpython-6/PySide6/QtWidgets/QDialog.html
|
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: data-type-mapping-reference
|
|
3
|
+
category: database-solutions
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
contributed: 2026-03-10
|
|
6
|
+
contributor: research
|
|
7
|
+
last_updated: 2026-03-10
|
|
8
|
+
tags: [postgresql, mysql, sqlite, sql-server, data-types, migration, type-mapping]
|
|
9
|
+
difficulty: medium
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# Data Type Mapping Reference
|
|
13
|
+
|
|
14
|
+
## Problem
|
|
15
|
+
|
|
16
|
+
Database migrations fail silently when type mappings are wrong. Unsigned integers overflow, booleans store unexpected values, dates corrupt, and JSON loses indexability. Developers need a precise, copy-pasteable mapping table for every common type across PostgreSQL, MySQL, SQLite, and SQL Server.
|
|
17
|
+
## Complete Type Mapping Table
|
|
18
|
+
|
|
19
|
+
### Numeric Types
|
|
20
|
+
|
|
21
|
+
| PostgreSQL | MySQL | SQLite | SQL Server | Migration Notes |
|
|
22
|
+
|---|---|---|---|---|
|
|
23
|
+
| `SMALLINT` | `SMALLINT` | `INTEGER` | `SMALLINT` | Direct mapping |
|
|
24
|
+
| `INTEGER` | `INT` | `INTEGER` | `INT` | Direct mapping |
|
|
25
|
+
| `BIGINT` | `BIGINT` | `INTEGER` | `BIGINT` | SQLite uses 8-byte INTEGER for all |
|
|
26
|
+
| N/A | `TINYINT` (signed, -128 to 127) | `INTEGER` | `TINYINT` (unsigned, 0-255) | PG has no TINYINT — use SMALLINT |
|
|
27
|
+
| N/A | `INT UNSIGNED` | `INTEGER` | N/A | PG: upsize to `BIGINT` for range |
|
|
28
|
+
| `NUMERIC(p,s)` | `DECIMAL(p,s)` | `REAL` or `TEXT` | `DECIMAL(p,s)` | Validate precision alignment for financial data |
|
|
29
|
+
| `REAL` (4 bytes) | `FLOAT` | `REAL` | `REAL` | Direct mapping |
|
|
30
|
+
| `DOUBLE PRECISION` | `DOUBLE` | `REAL` | `FLOAT(53)` | Direct mapping |
|
|
31
|
+
| `MONEY` | `DECIMAL(19,2)` | N/A | `MONEY` | Avoid PG MONEY — use NUMERIC(19,2) |
|
|
32
|
+
|
|
33
|
+
**Unsigned integer rule:** MySQL `UNSIGNED INT` → PG `BIGINT` (upsize to accommodate 0-4.29B range). pgloader does this automatically.
|
|
34
|
+
|
|
35
|
+
### String Types
|
|
36
|
+
|
|
37
|
+
| PostgreSQL | MySQL | SQLite | SQL Server | Migration Notes |
|
|
38
|
+
|---|---|---|---|---|
|
|
39
|
+
| `CHAR(n)` | `CHAR(n)` (max 255) | `TEXT` | `CHAR(n)` (max 8000) | Check length limits |
|
|
40
|
+
| `VARCHAR(n)` | `VARCHAR(n)` (max 65535) | `TEXT` | `VARCHAR(n)` / `VARCHAR(MAX)` | PG max ~10M; MySQL max 65K |
|
|
41
|
+
| `TEXT` | `TEXT` (64KB) | `TEXT` | `TEXT` | MySQL TEXT is 64KB; use LONGTEXT for larger |
|
|
42
|
+
| `TEXT` | `MEDIUMTEXT` (16MB) | `TEXT` | `VARCHAR(MAX)` | pgloader maps all → PG TEXT |
|
|
43
|
+
| `TEXT` | `LONGTEXT` (4GB) | `TEXT` | `VARCHAR(MAX)` | pgloader maps all → PG TEXT |
|
|
44
|
+
|
|
45
|
+
**Charset gotcha:** MySQL defaults to `latin1` in older versions. Always specify `utf8mb4` for full Unicode. PG is always UTF-8.
|
|
46
|
+
|
|
47
|
+
### Boolean
|
|
48
|
+
|
|
49
|
+
| PostgreSQL | MySQL | SQLite | SQL Server |
|
|
50
|
+
|---|---|---|---|
|
|
51
|
+
| `BOOLEAN` | `TINYINT(1)` / `BOOL` | `INTEGER` (0/1) | `BIT` |
|
|
52
|
+
|
|
53
|
+
**pgloader rule:** MySQL `TINYINT(1)` → PG `BOOLEAN` (only when display width = 1).
|
|
54
|
+
**Validation:** MySQL TINYINT(1) can store 0-127. Values > 1 have no boolean mapping — validate before migration.
|
|
55
|
+
|
|
56
|
+
### Date / Time
|
|
57
|
+
|
|
58
|
+
| PostgreSQL | MySQL | SQLite | SQL Server | Migration Notes |
|
|
59
|
+
|---|---|---|---|---|
|
|
60
|
+
| `DATE` | `DATE` | `TEXT` | `DATE` | Direct |
|
|
61
|
+
| `TIME` | `TIME` | `TEXT` | `TIME(p)` | Direct |
|
|
62
|
+
| `TIMESTAMP` | `DATETIME` | `TEXT` | `DATETIME2(p)` | MySQL DATETIME has no timezone |
|
|
63
|
+
| `TIMESTAMPTZ` | `TIMESTAMP` | `TEXT` | `DATETIMEOFFSET(p)` | MySQL TIMESTAMP auto-converts UTC |
|
|
64
|
+
| `INTERVAL` | N/A | N/A | N/A | PG-only — convert to seconds or use app logic |
|
|
65
|
+
|
|
66
|
+
**Critical gotchas:**
|
|
67
|
+
- MySQL `0000-00-00` and `0000-00-00 00:00:00` → PG: **rejected**. Convert to NULL.
|
|
68
|
+
- MySQL `TIMESTAMP` range: 1970-2038. PG `TIMESTAMPTZ` range: 4713 BC - 294276 AD.
|
|
69
|
+
- SQL Server `DATETIME2` precision 0-7; PG max precision 6 (microseconds truncated).
|
|
70
|
+
- **pgloader rule:** MySQL `DATETIME`/`TIMESTAMP` → PG `TIMESTAMPTZ`.
|
|
71
|
+
|
|
72
|
+
### Binary
|
|
73
|
+
|
|
74
|
+
| PostgreSQL | MySQL | SQLite | SQL Server |
|
|
75
|
+
|---|---|---|---|
|
|
76
|
+
| `BYTEA` | `BINARY(n)` / `VARBINARY(n)` | `BLOB` | `BINARY(n)` / `VARBINARY(n)` |
|
|
77
|
+
| `BYTEA` | `TINYBLOB/BLOB/MEDIUMBLOB/LONGBLOB` | `BLOB` | `IMAGE` (deprecated) |
|
|
78
|
+
|
|
79
|
+
**pgloader rule:** All MySQL binary/BLOB variants → PG `BYTEA`.
|
|
80
|
+
|
|
81
|
+
### UUID
|
|
82
|
+
|
|
83
|
+
| PostgreSQL | MySQL | SQLite | SQL Server |
|
|
84
|
+
|---|---|---|---|
|
|
85
|
+
| `UUID` (native, 16 bytes) | `CHAR(36)` | `TEXT` | `UNIQUEIDENTIFIER` |
|
|
86
|
+
| `gen_random_uuid()` | `UUID()` | N/A (app-side) | `NEWID()` |
|
|
87
|
+
|
|
88
|
+
**MINISTRY-LMS lesson:** MySQL `LAST_INSERT_ID()` returns 0 for `CHAR(36)` PKs. Generate UUID in JavaScript with `uuidv4()` and include `id` in INSERT column list.
|
|
89
|
+
|
|
90
|
+
### JSON
|
|
91
|
+
|
|
92
|
+
| PostgreSQL | MySQL | SQLite | SQL Server | MariaDB |
|
|
93
|
+
|---|---|---|---|---|
|
|
94
|
+
| `JSON` (text, validated) | `JSON` (binary since 8.0) | `TEXT` or `JSON` (3.45+) | `NVARCHAR(MAX)` | `LONGTEXT` alias |
|
|
95
|
+
| `JSONB` (binary, indexable) | N/A | `JSONB` (3.45+) | N/A | N/A |
|
|
96
|
+
|
|
97
|
+
**Key difference:** PG JSONB supports containment (`@>`), existence (`?`), and GIN indexing. MySQL JSON requires generated columns + B-tree for indexing. MariaDB JSON is just LONGTEXT with no binary optimization.
|
|
98
|
+
|
|
99
|
+
### Enum
|
|
100
|
+
|
|
101
|
+
| PostgreSQL | MySQL | SQLite | SQL Server |
|
|
102
|
+
|---|---|---|---|
|
|
103
|
+
| `CREATE TYPE x AS ENUM(...)` | `ENUM(...)` inline on column | N/A | N/A |
|
|
104
|
+
|
|
105
|
+
**pgloader rule:** MySQL inline ENUM → PG `CREATE TYPE enum_table_column AS ENUM(...)` + column reference.
|
|
106
|
+
**Reverse (PG→MySQL):** Flatten CREATE TYPE into inline ENUM on each column that uses it.
|
|
107
|
+
|
|
108
|
+
### PostgreSQL-Only Types (No Direct Equivalent)
|
|
109
|
+
|
|
110
|
+
| PG Type | MySQL Workaround | SQLite Workaround | SQL Server Workaround |
|
|
111
|
+
|---|---|---|---|
|
|
112
|
+
| `TEXT[]` / `INT[]` | `JSON` (array) | `TEXT` (JSON) | `NVARCHAR(MAX)` (JSON) |
|
|
113
|
+
| `JSONB` | `JSON` (lose GIN indexing) | `TEXT` | `NVARCHAR(MAX)` |
|
|
114
|
+
| `CIDR` / `INET` | `VARCHAR(43)` | `TEXT` | `VARCHAR(43)` |
|
|
115
|
+
| `MACADDR` | `VARCHAR(17)` | `TEXT` | `VARCHAR(17)` |
|
|
116
|
+
| `TSVECTOR` / `TSQUERY` | FULLTEXT index (different API) | FTS5 extension | Full-text catalog |
|
|
117
|
+
| `HSTORE` | `JSON` | `TEXT` | `NVARCHAR(MAX)` |
|
|
118
|
+
| Geometric types | Spatial extensions | N/A | `GEOMETRY` |
|
|
119
|
+
|
|
120
|
+
## pgloader Default Casting Rules (MySQL → PostgreSQL)
|
|
121
|
+
|
|
122
|
+
```
|
|
123
|
+
TINYINT(1) → BOOLEAN
|
|
124
|
+
TINYINT (other) → SMALLINT
|
|
125
|
+
INT UNSIGNED → BIGINT
|
|
126
|
+
INT AUTO_INCREMENT → SERIAL
|
|
127
|
+
BIGINT AUTO_INCREMENT → BIGSERIAL
|
|
128
|
+
BIT(1) → BOOLEAN
|
|
129
|
+
DATETIME/TIMESTAMP → TIMESTAMPTZ
|
|
130
|
+
YEAR → INTEGER
|
|
131
|
+
All TEXT variants → TEXT
|
|
132
|
+
All BLOB variants → BYTEA
|
|
133
|
+
ENUM(...) → CREATE TYPE + column
|
|
134
|
+
SET(...) → TEXT
|
|
135
|
+
0000-00-00 dates → NULL
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
## Safest Portable Type Subset
|
|
139
|
+
|
|
140
|
+
These types work identically across PG, MySQL, SQLite, and SQL Server with zero translation:
|
|
141
|
+
|
|
142
|
+
```
|
|
143
|
+
STRING: VARCHAR(n) where n ≤ 255
|
|
144
|
+
INTEGER: INT / INTEGER
|
|
145
|
+
FLOAT: REAL / DOUBLE PRECISION
|
|
146
|
+
BOOLEAN: Use INTEGER 0/1 (universally supported)
|
|
147
|
+
TEXT: TEXT (for unlimited strings)
|
|
148
|
+
TIMESTAMP: Store as ISO 8601 TEXT for max portability
|
|
149
|
+
UUID: CHAR(36) (string representation)
|
|
150
|
+
JSON: TEXT containing valid JSON (parse in app layer)
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
**Trade-off:** This subset sacrifices database-specific optimizations (JSONB indexing, native UUID, array types) for portability.
|
|
154
|
+
|
|
155
|
+
## When to Use
|
|
156
|
+
|
|
157
|
+
- Planning any database migration
|
|
158
|
+
- Choosing column types for cross-database applications
|
|
159
|
+
- Configuring pgloader / SQLines custom type mappings
|
|
160
|
+
- Reviewing ORM schema definitions for portability issues
|
|
161
|
+
- Building the DATABASE phase of `/fire-resurrect`
|
|
162
|
+
|
|
163
|
+
## When NOT to Use
|
|
164
|
+
|
|
165
|
+
- Single-database projects committed to one vendor
|
|
166
|
+
- NoSQL databases
|
|
167
|
+
- Data warehouse columns (different optimization concerns)
|
|
168
|
+
|
|
169
|
+
## Related Skills
|
|
170
|
+
|
|
171
|
+
- [sql-dialect-compatibility-matrix](sql-dialect-compatibility-matrix.md)
|
|
172
|
+
- [pg-to-mysql-schema-migration-methodology](pg-to-mysql-schema-migration-methodology.md)
|
|
173
|
+
- [mysql-to-pg-migration](mysql-to-pg-migration.md)
|
|
174
|
+
|
|
175
|
+
## References
|
|
176
|
+
|
|
177
|
+
- pgloader MySQL documentation — default casting rules
|
|
178
|
+
- MySQL Workbench PostgreSQL type mapping reference
|
|
179
|
+
- SQLines data type mapping configuration
|
|
180
|
+
- MINISTRY-LMS sql-compat.js — production-proven type handling
|
|
181
|
+
- "I Migrated 400M Rows" (Medium, 2026) — type mapping breakage points
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: mysql-limit-offset-string-coercion
|
|
3
|
+
category: database-solutions
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
contributed: 2026-03-07
|
|
6
|
+
contributor: ministry-lms
|
|
7
|
+
last_updated: 2026-03-07
|
|
8
|
+
contributors:
|
|
9
|
+
- ministry-lms
|
|
10
|
+
tags: [mysql, mariadb, postgresql, limit, offset, pagination, express, req-query, type-coercion]
|
|
11
|
+
difficulty: easy
|
|
12
|
+
usage_count: 0
|
|
13
|
+
success_rate: 100
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# MySQL LIMIT/OFFSET String Coercion
|
|
17
|
+
|
|
18
|
+
## Problem
|
|
19
|
+
|
|
20
|
+
When porting a PostgreSQL application to MySQL, paginated endpoints break with SQL syntax errors. The root cause: Express `req.query` parameters are always strings (`'20'`, not `20`). PostgreSQL silently auto-casts `LIMIT '20'` to `LIMIT 20`. MySQL strictly rejects string values in LIMIT/OFFSET clauses.
|
|
21
|
+
|
|
22
|
+
**Symptoms:**
|
|
23
|
+
- `You have an error in your SQL syntax` on any paginated endpoint
|
|
24
|
+
- Works fine in PostgreSQL, breaks in MySQL
|
|
25
|
+
- Only affects endpoints that pass `req.query.limit` or `req.query.page` into SQL
|
|
26
|
+
- Error appears as `LIMIT '20' OFFSET 0` in the generated SQL
|
|
27
|
+
|
|
28
|
+
## Solution Pattern
|
|
29
|
+
|
|
30
|
+
Fix this **systemically** in the SQL compatibility/translation layer, not per-controller. Any runtime SQL translation layer that converts PG queries to MySQL should coerce LIMIT/OFFSET values from strings to integers before query execution.
|
|
31
|
+
|
|
32
|
+
**Why systemic:** Fixing in every controller/model is fragile — each new paginated endpoint would need the same fix. One fix in the SQL layer covers all endpoints, past and future.
|
|
33
|
+
|
|
34
|
+
## Code Example
|
|
35
|
+
|
|
36
|
+
```javascript
|
|
37
|
+
// Before (broken on MySQL)
|
|
38
|
+
// req.query.limit = '20' (string from Express)
|
|
39
|
+
const result = await sql`
|
|
40
|
+
SELECT * FROM meetings
|
|
41
|
+
ORDER BY created_at DESC
|
|
42
|
+
LIMIT ${limit} OFFSET ${offset}
|
|
43
|
+
`;
|
|
44
|
+
// Generated SQL: SELECT * FROM meetings ORDER BY created_at DESC LIMIT '20' OFFSET 0
|
|
45
|
+
// MySQL error: You have an error in your SQL syntax
|
|
46
|
+
|
|
47
|
+
// After (works on both PG and MySQL)
|
|
48
|
+
// Option A: Fix in SQL compatibility layer (RECOMMENDED — one fix for all endpoints)
|
|
49
|
+
// In the template literal parser, detect LIMIT/OFFSET context and coerce to int:
|
|
50
|
+
const precedingSQL = sql.trimEnd().toUpperCase();
|
|
51
|
+
if (typeof value === 'string' && /\b(LIMIT|OFFSET)\s*\??\s*$/.test(precedingSQL)) {
|
|
52
|
+
const parsed = parseInt(value, 10);
|
|
53
|
+
if (!isNaN(parsed)) value = parsed;
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
// Option B: Fix per-controller (NOT recommended — must repeat everywhere)
|
|
57
|
+
const limit = parseInt(req.query.limit, 10) || 50;
|
|
58
|
+
const offset = parseInt(req.query.page, 10) * limit || 0;
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Implementation Steps
|
|
62
|
+
|
|
63
|
+
1. Locate the SQL template literal parser in your compatibility layer (the function that processes tagged template literals and generates parameterized queries)
|
|
64
|
+
2. In the section that handles parameter values, check if the preceding SQL text ends with `LIMIT` or `OFFSET`
|
|
65
|
+
3. If the value is a string and the context is LIMIT/OFFSET, coerce it to an integer with `parseInt(value, 10)`
|
|
66
|
+
4. Only apply the coercion if `parseInt` produces a valid number (not `NaN`)
|
|
67
|
+
5. Restart the server and test any paginated endpoint
|
|
68
|
+
|
|
69
|
+
## When to Use
|
|
70
|
+
|
|
71
|
+
- Porting a PostgreSQL Express app to MySQL/MariaDB
|
|
72
|
+
- Building a SQL compatibility layer between PG and MySQL
|
|
73
|
+
- Any scenario where `req.query` values pass through to LIMIT/OFFSET in MySQL
|
|
74
|
+
- When you see `LIMIT '20'` style errors in MySQL logs
|
|
75
|
+
|
|
76
|
+
## When NOT to Use
|
|
77
|
+
|
|
78
|
+
- If your app already uses an ORM (Prisma, Sequelize, Knex) that handles type coercion
|
|
79
|
+
- If you're only targeting PostgreSQL (PG handles this automatically)
|
|
80
|
+
- If LIMIT/OFFSET values come from validated/parsed sources (not raw req.query)
|
|
81
|
+
- If you're using prepared statements where the driver handles type binding
|
|
82
|
+
|
|
83
|
+
## Common Mistakes
|
|
84
|
+
|
|
85
|
+
- Fixing in individual controllers instead of the SQL layer (fragile, doesn't scale)
|
|
86
|
+
- Using `Number()` instead of `parseInt()` (Number('') returns 0, parseInt('') returns NaN — different edge cases)
|
|
87
|
+
- Forgetting OFFSET (same issue as LIMIT — both need coercion)
|
|
88
|
+
- Not checking for NaN before coercing (could turn invalid input into NaN parameter)
|
|
89
|
+
- Assuming `req.query` values are numbers (they are ALWAYS strings in Express)
|
|
90
|
+
|
|
91
|
+
## Related Skills
|
|
92
|
+
|
|
93
|
+
- [pg-to-mysql-schema-migration-methodology](../database-solutions/pg-to-mysql-schema-migration-methodology.md) - Schema-level PG to MySQL migration
|
|
94
|
+
- [sql-injection-prevention-postgresjs](../../security/sql-injection-prevention-postgresjs.md) - Safe SQL parameter handling
|
|
95
|
+
|
|
96
|
+
## References
|
|
97
|
+
|
|
98
|
+
- MySQL LIMIT syntax: requires integer expressions, not string literals
|
|
99
|
+
- Express req.query: all values are strings (parsed from URL query string)
|
|
100
|
+
- Discovered during: MINISTRY-LMS PG-to-MySQL migration — meetings endpoint broke with LIMIT '20'
|
|
101
|
+
- Fix applied in: `server/database/sql-compat.js` _parseTemplate method
|
|
102
|
+
- Contributed from: ministry-lms
|