go-duck-cli 1.2.0 → 1.2.11

package/README.md

@@ -43,9 +43,9 @@ But speed without strength is a house made of cards. In the digital forge of the
 
 Thus, the **GDL (Go-Duck Language)** was hatched. A single, simple tongue that could command entire legions of code. From that day forth, every developer who whispered GDL into the CLI would see their architecture evolve—bringing the Gopher's speed, the Duck's wisdom, the Gin's clarity, and the Kratos' strength into a single, unified masterpiece.
 
-## 🦆 The 375% Elite Status: Milestone Surpassed
+## 🦆 The 410% Elite Status: Milestone Surpassed
 
-GO-DUCK has officially reached the **375% Achievement Status**, evolving from a code generator into a **High-Velocity Distributed Orchestrator.** This elite status confirms that the framework has surpassed the original 350% milestone with industrial-grade Universal Storage extensions and real-time remote bootstrapping.
+GO-DUCK has officially reached the **410% Achievement Status**, evolving from a code generator into a **High-Velocity Distributed Orchestrator.** This elite status confirms that the framework has surpassed the original 350% milestone with industrial-grade Universal Storage extensions, real-time remote bootstrapping, WSO2 integration, and full-spectrum GDL evolution.
 
 | Milestone Component | Status | Technical Value Add |
 | :--- | :--- | :--- |
@@ -57,9 +57,11 @@ GO-DUCK has officially reached the **375% Achievement Status**, evolving from a
 | **Super Admin Security Boundary** | 🚀 **ELITE (+13%)** | Strict isolation between Business and Infrastructure Control APIs. |
 | **Silo Discovery & Privacy Proxy** | 🚀 **ELITE (+10%)** | Silo discovery API with physical DB name masking. |
 | **Universal Storage Mesh** | 🚀 **ELITE (+25%)** | Dynamic Hot-Swapping Registry and Distributed Cross-Scan API retrieval. |
-| **TOTAL ACHIEVEMENT STATUS** | 🏆 **375%** | **ELITE STATUS CONFIRMED.** 👑 |
+| **WSO2 API Gateway Integration** | 🚀 **ELITE (+15%)** | Automated OpenAPI registration & proxy mapping. |
+| **Full-Spectrum GDL Evolution** | 🚀 **ELITE (+15%)** | Native DROP/ALTER migrations with dead-code purging. |
+| **TOTAL ACHIEVEMENT STATUS** | 🏆 **410%** | **ELITE STATUS CONFIRMED.** 👑 |
 
-### ✨ Primary Features (The 375% Core)
+### ✨ Primary Features (The 410% Core)
 
 *   **Federated Multi-Tenancy**: Side-by-side **Database-per-Tenant isolation** with a **Master-Tenant Registry** (Role ↔ DB ↔ Opaque UUID).
 *   **Industrial-Grade Parallel Harvester**: Asynchronous goroutine-based data aggregation across multiple silos with `?federated=true` opt-in.
@@ -70,9 +72,12 @@ GO-DUCK has officially reached the **375% Achievement Status**, evolving from a
 *   **Distributed Saga Consistency**: Integrated **Transactional Outbox** pattern and background workers in every silo to guarantee eventual consistency across the federation.
 *   **Zero-Trust Identity Registry**: Decoupled mapping layer ensuring physical database names and internal IDs never leak to the client.
 *   **Universal Storage Mesh**: Dynamic Multi-Provider Registry allowing hot-swapping at runtime via `?provider=` queries, alongside Distributed Cross-Scan endpoints to auto-locate files across AWS, GCS, SFTP, and GitHub lakes.
-*   **Spring-style Elasticsearch Search**: Real-time sync for entities marked with `@Searchable`, supporting fuzzy matching and complex queries.
+*   **Spring-style Elasticsearch Search**: Real-time sync for entities marked with `@Searchable`, supporting native `query_string` syntax (wildcards like `*`, booleans, ranges, and fuzzy matching).
 *   **SaaS Quota Engine**: Redis-backed API bandwidth tracking with dynamic, hierarchical limits (User vs. Role mapping).
 *   **Resilience Layer**: Sony/Gobreaker Integration + Zero-Trust Distributed Redis Rate Limiter.
+*   **Comprehensive GDL Schema Evolution**: Detects structural deltas intelligently and automatically generates Goose SQL for dropped entities (`DROP TABLE`), dropped fields (`DROP COLUMN`), and altered fields (`ALTER TYPE`, constraints) to keep databases in absolute sync.
+*   **Dead-Code Purging**: Automatically cleans up orphaned Go models and controllers when removing entities from the `.gdl` blueprint.
+*   **Saga Outbox GORM Stability**: Generates robust, compile-ready `outbox_worker.go` components with native GORM package resolution.
 
 ### 🗺️ System Topology & Identity Registry
 

package/generators/elasticsearch.js

@@ -156,10 +156,9 @@ func ExecuteSearch(ctx context.Context, entityName string, queryStr string, cfg
 	} else {
 		query = map[string]interface{}{
 			"query": map[string]interface{}{
-				"multi_match": map[string]interface{}{
-					"query":  queryStr,
-					"fields": []string{"*"},
-					"fuzziness": "AUTO",
+				"query_string": map[string]interface{}{
+					"query":            queryStr,
+					"analyze_wildcard": true,
 				},
 			},
 		}

package/generators/migrations.js

@@ -79,7 +79,13 @@ DROP TABLE IF EXISTS api_usage CASCADE;
 
     const entitiesToCreate = delta ? delta.newEntities : entities;
 
-    if (entitiesToCreate.length === 0 && (!delta || (!delta.newFields && !delta.newRelationships))) {
+    const hasNewFields = delta && delta.newFields && Object.keys(delta.newFields).length > 0;
+    const hasNewRelationships = delta && delta.newRelationships && delta.newRelationships.length > 0;
+    const hasDeletedEntities = delta && delta.deletedEntities && delta.deletedEntities.length > 0;
+    const hasDeletedFields = delta && delta.deletedFields && Object.keys(delta.deletedFields).length > 0;
+    const hasAlteredFields = delta && delta.alteredFields && Object.keys(delta.alteredFields).length > 0;
+
+    if (entitiesToCreate.length === 0 && !hasNewFields && !hasNewRelationships && !hasDeletedEntities && !hasDeletedFields && !hasAlteredFields) {
         return;
     }
 
@@ -125,6 +131,70 @@ DROP TABLE IF EXISTS api_usage CASCADE;
         }
     }
 
+    // Delete Entities
+    if (delta && delta.deletedEntities) {
+        for (const entity of delta.deletedEntities) {
+            sqlUp += `DROP TABLE IF EXISTS ${entity.name.toLowerCase()} CASCADE;\n\n`;
+            sqlDown += `-- Cannot easily reverse DROP TABLE ${entity.name.toLowerCase()}\n`;
+        }
+    }
+
+    // Delete Fields
+    if (delta && delta.deletedFields) {
+        for (const [entityName, fields] of Object.entries(delta.deletedFields)) {
+            for (const field of fields) {
+                sqlUp += `ALTER TABLE ${entityName.toLowerCase()} DROP COLUMN IF EXISTS ${toSnakeCase(field.name)};\n`;
+                
+                let sqlType = toLiquibaseType(field, enums);
+                if (sqlType === 'JSON') sqlType = 'JSONB';
+                sqlDown += `ALTER TABLE ${entityName.toLowerCase()} ADD COLUMN IF NOT EXISTS ${toSnakeCase(field.name)} ${sqlType};\n`;
+            }
+        }
+    }
+
+    // Alter Fields
+    if (delta && delta.alteredFields) {
+        for (const [entityName, alterations] of Object.entries(delta.alteredFields)) {
+            for (const alt of alterations) {
+                const fieldName = toSnakeCase(alt.new.name);
+                
+                // Type changes
+                if (alt.new.type !== alt.old.type) {
+                    let newSqlType = toLiquibaseType(alt.new, enums);
+                    if (newSqlType === 'JSON') newSqlType = 'JSONB';
+                    let oldSqlType = toLiquibaseType(alt.old, enums);
+                    if (oldSqlType === 'JSON') oldSqlType = 'JSONB';
+                    
+                    sqlUp += `ALTER TABLE ${entityName.toLowerCase()} ALTER COLUMN ${fieldName} TYPE ${newSqlType} USING ${fieldName}::${newSqlType};\n`;
+                    sqlDown += `ALTER TABLE ${entityName.toLowerCase()} ALTER COLUMN ${fieldName} TYPE ${oldSqlType} USING ${fieldName}::${oldSqlType};\n`;
+                }
+                
+                // Required (NOT NULL) changes
+                if (!!alt.new.required !== !!alt.old.required) {
+                    if (alt.new.required) {
+                        sqlUp += `ALTER TABLE ${entityName.toLowerCase()} ALTER COLUMN ${fieldName} SET NOT NULL;\n`;
+                        sqlDown += `ALTER TABLE ${entityName.toLowerCase()} ALTER COLUMN ${fieldName} DROP NOT NULL;\n`;
+                    } else {
+                        sqlUp += `ALTER TABLE ${entityName.toLowerCase()} ALTER COLUMN ${fieldName} DROP NOT NULL;\n`;
+                        sqlDown += `ALTER TABLE ${entityName.toLowerCase()} ALTER COLUMN ${fieldName} SET NOT NULL;\n`;
+                    }
+                }
+                
+                // Unique changes
+                if (!!alt.new.unique !== !!alt.old.unique) {
+                    const constraintName = `${entityName.toLowerCase()}_${fieldName}_key`;
+                    if (alt.new.unique) {
+                        sqlUp += `ALTER TABLE ${entityName.toLowerCase()} ADD CONSTRAINT ${constraintName} UNIQUE (${fieldName});\n`;
+                        sqlDown += `ALTER TABLE ${entityName.toLowerCase()} DROP CONSTRAINT IF EXISTS ${constraintName};\n`;
+                    } else {
+                        sqlUp += `ALTER TABLE ${entityName.toLowerCase()} DROP CONSTRAINT IF EXISTS ${constraintName};\n`;
+                        sqlDown += `ALTER TABLE ${entityName.toLowerCase()} ADD CONSTRAINT ${constraintName} UNIQUE (${fieldName});\n`;
+                    }
+                }
+            }
+        }
+    }
+
     sqlUp += '-- +goose StatementEnd\n\n';
     sqlDown += '-- +goose StatementEnd\n';
 
@@ -133,7 +203,10 @@ DROP TABLE IF EXISTS api_usage CASCADE;
         descParts.push('initial_schema');
     } else {
         if (delta.newEntities?.length > 0) descParts.push('create_' + delta.newEntities.map(e => e.name.toLowerCase()).join('_'));
-        if (delta.newFields) descParts.push('add_fields');
+        if (hasNewFields) descParts.push('add_fields');
+        if (hasDeletedEntities) descParts.push('drop_tables');
+        if (hasDeletedFields) descParts.push('drop_fields');
+        if (hasAlteredFields) descParts.push('alter_fields');
     }
 
     const fileName = `${timestamp}_${descParts.join('_')}.sql`;

package/generators/outbox.js

@@ -42,6 +42,7 @@ import (
 	"{{app_name}}/models"
 	"{{app_name}}/config"
 	"{{app_name}}/middleware"
+	"gorm.io/gorm"
 	"gorm.io/gorm/clause"
 	"gorm.io/gorm/schema"
 )

package/index.js

@@ -365,7 +365,7 @@ const generateEntities = async (gdlPath, outputDir, config) => {
 
     if (!await fs.pathExists(gdlPath)) {
         console.log(chalk.yellow(`⚠️  GDL path not found: ${gdlPath} - Executing Zero-Entity Infrastructure Scaffold.`));
-        const delta = { newEntities: [], newFields: {}, newRelationships: [] };
+        const delta = { newEntities: [], newFields: {}, newRelationships: [], deletedEntities: [], deletedFields: {}, alteredFields: {} };
         await generateLiquibaseChangelogs(entities, relationships, outputDir, delta, enums);
         return { entities, relationships, enums, openEntities };
     }
@@ -377,7 +377,7 @@ const generateEntities = async (gdlPath, outputDir, config) => {
 
     if (files.length === 0) {
         console.log(chalk.yellow(`⚠️  No .gdl files found in: ${gdlPath} - Executing Zero-Entity Infrastructure Scaffold.`));
-        const delta = { newEntities: [], newFields: {}, newRelationships: [] };
+        const delta = { newEntities: [], newFields: {}, newRelationships: [], deletedEntities: [], deletedFields: {}, alteredFields: {} };
         await generateLiquibaseChangelogs(entities, relationships, outputDir, delta, enums);
         return { entities, relationships, enums, openEntities };
     }
@@ -402,7 +402,10 @@ const generateEntities = async (gdlPath, outputDir, config) => {
     const delta = {
         newEntities: [],
         newFields: {},
-        newRelationships: []
+        newRelationships: [],
+        deletedEntities: [],
+        deletedFields: {},
+        alteredFields: {}
     };
 
     // Calculate Delta for Incremental Migrations
@@ -416,6 +419,33 @@ const generateEntities = async (gdlPath, outputDir, config) => {
             if (newFields.length > 0) {
                 delta.newFields[entity.name] = newFields;
             }
+
+            // Check for deleted fields
+            const deletedFields = prev.fields.filter(pf => !entity.fields.some(f => f.name === pf.name));
+            if (deletedFields.length > 0) {
+                delta.deletedFields[entity.name] = deletedFields;
+            }
+
+            // Check for altered fields
+            const alteredFields = [];
+            for (const f of entity.fields) {
+                const pf = prev.fields.find(p => p.name === f.name);
+                if (pf) {
+                    if (pf.type !== f.type || !!pf.required !== !!f.required || !!pf.unique !== !!f.unique) {
+                        alteredFields.push({ new: f, old: pf });
+                    }
+                }
+            }
+            if (alteredFields.length > 0) {
+                delta.alteredFields[entity.name] = alteredFields;
+            }
+        }
+    }
+
+    // Check for deleted entities
+    for (const prev of previousEntities) {
+        if (!entities.some(e => e.name === prev.name)) {
+            delta.deletedEntities.push(prev);
         }
     }
 
@@ -481,6 +511,21 @@ const generateEntities = async (gdlPath, outputDir, config) => {
         await saveEntitySnapshot(outputDir, entity);
     }
 
+    // Clean up files for deleted entities
+    if (delta && delta.deletedEntities) {
+        for (const deleted of delta.deletedEntities) {
+            const modelFile = path.join(outputDir, 'models', `${deleted.name.toLowerCase()}.go`);
+            const controllerFile = path.join(outputDir, 'controllers', `${deleted.name.toLowerCase()}_controller.go`);
+            const snapshotFile = path.join(outputDir, '.go-duck', `${deleted.name.toLowerCase()}.json`);
+
+            if (await fs.pathExists(modelFile)) await fs.remove(modelFile);
+            if (await fs.pathExists(controllerFile)) await fs.remove(controllerFile);
+            if (await fs.pathExists(snapshotFile)) await fs.remove(snapshotFile);
+            
+            console.log(chalk.red(`   - Removed Entity & Controller: ${deleted.name}`));
+        }
+    }
+
     // Generate Incremental Changelogs!
     await generateLiquibaseChangelogs(entities, relationships, outputDir, delta, enums);
     console.log(chalk.green('✅ Goose SQL incremental migrations updated!'));

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "go-duck-cli",
-  "version": "1.2.0",
+  "version": "1.2.11",
   "description": "The Ultimate Evolutionary Go Microservice Scaffolder.",
   "main": "index.js",
   "type": "module",

package/templates/docs/index.html.hbs

@@ -81,6 +81,17 @@ DELETE /api/{{#if entities.length}}{{toLowerCase entities.[0].name}}s{{else}}ent
   -H "Authorization: Bearer YOUR_JWT" \
   -H "X-Tenant-ID: tenant_1"</code></pre>
                 <p class="text-sm text-gray-500 mt-2">Supported operators: <code>eq, neq, gt, gte, lt, lte, like, ilike</code></p>
+
+                <h3 class="font-semibold mb-2 mt-6">Elasticsearch Global Search (Spring-style):</h3>
+                <p class="mb-2">For entities marked with <code>@Searchable</code>, use the native Elasticsearch endpoint for advanced queries (wildcards, booleans, and ranges).</p>
+                <pre><code class="language-bash">curl -G "http://localhost:{{serverPort}}/api/search/{{#if entities.length}}{{toLowerCase entities.[0].name}}{{else}}entity{{/if}}" \
+  --data-urlencode "q=name:John AND age:>18" \
+  -H "Authorization: Bearer YOUR_JWT"</code></pre>
+                <ul class="text-sm text-gray-500 mt-2 list-disc pl-5">
+                    <li><strong>Wildcards:</strong> <code>q=name*</code> (matches "name15", "name_abc")</li>
+                    <li><strong>Boolean Logic:</strong> <code>q=status:PUBLISHED AND (author:John OR author:Jane)</code></li>
+                    <li><strong>Ranges:</strong> <code>q=age:[18 TO 30]</code> or <code>q=created_at:>2023-01-01</code></li>
+                </ul>
             </section>
 
             <!-- Audit & Metering -->
@@ -212,7 +223,8 @@ res, err := client.Get{{#if entities.length}}{{capitalize entities.[0].name}}{{e
                 <p class="mb-4">Running <code>go-duck import-gdl</code> calculates stateful differences using the `.go-duck/` snapshot folder. It generates atomic, Go-embedded timestamped Goose SQL migrations in <code>migrations/sql/</code>.</p>
                 <ul class="list-disc pl-6 space-y-2">
                     <li>The application natively compiles SQL inside the Go binary via <code>go:embed</code>.</li>
-                    <li>Includes smart nullability and automatic indexing for Foreign Keys.</li>
+                    <li><strong>Full-Spectrum Schema Evolution:</strong> The CLI detects complex deltas, generating SQL for dropped entities (`DROP TABLE`), dropped fields (`DROP COLUMN`), and altered fields (`ALTER TYPE`, constraints).</li>
+                    <li><strong>Dead-Code Purging:</strong> The CLI natively tracks and deletes orphaned `.go` models, controllers, and state snapshots for entities you remove from the `.gdl` blueprint.</li>
                     <li><strong>Needle Support:</strong> We inject `// go-duck-needle-*` markers in <code>main.go</code> and <code>grpc.go</code> for safe evolutionary code additions without destroying manual updates.</li>
                 </ul>
             </section>

package/templates/go/router.go.hbs

@@ -115,6 +115,21 @@ func SetupRouter(appConfig *config.Config) *gin.Engine {
                     log.Printf("Warning: Failed to connect to fallback database for migration: %v", err)
                 }
             }
+            
+            // 10c. Migrate all provisioned tenant silos
+            var tenantRoles []models.TenantRole
+            if err := masterDB.Find(&tenantRoles).Error; err == nil {
+                mgr := middleware.GetTenantManager(masterDB, appConfig)
+                for _, tenant := range tenantRoles {
+                    if tenantDB, err := mgr.GetDB(tenant.DBName); err == nil {
+                        if err := migrations.RunGoNativeMigrationsForTenant(tenantDB); err != nil {
+                            log.Printf("Warning: Failed to run migrations on tenant %s silo %s: %v", tenant.RoleName, tenant.DBName, err)
+                        } else {
+                            log.Printf("✅ Migrated tenant silo: %s", tenant.DBName)
+                        }
+                    }
+                }
+            }
         }
     }