quickbase-js 0.1.0-beta.7 → 0.1.0-beta.8

package/tsconfig.json

@@ -17,13 +17,16 @@
     "baseUrl": ".",
     "paths": {
       "@/*": ["src/*"],
-      "@tests/*": ["tests/*"]
+      "@tests/*": ["tests/*"],
+      "undici-types": ["node_modules/undici-types/index.d.ts"],
+      "openapi-types": ["node_modules/openapi-types/index.d.ts"],
+      "jsondiffpatch": ["node_modules/jsondiffpatch/lib/index.d.ts"]
     }
   },
   "include": ["src/**/*", "tests/**/*", "open-api/**/*", "tools/**/*"],
   "exclude": ["node_modules", "dist/**/*", "generated/**/*", "specs"],
   "ts-node": {
     "esm": true,
-    "transpileOnly": true
+    "experimentalResolver": true
   }
 }

package/vitest.config.ts

@@ -1,7 +1,6 @@
-// vitest.config.ts
 import { defineConfig } from "vitest/config";
 import path from "path";
-import dotenv from "dotenv"; // Add this import
+import dotenv from "dotenv";
 
 // Load .env variables before tests run
 dotenv.config({ path: path.resolve(__dirname, ".env") });
@@ -17,6 +16,7 @@ export default defineConfig({
         singleFork: true, // Run tests sequentially
       },
     },
+    testTimeout: 10000, // Global timeout of 10 seconds for all tests
   },
   resolve: {
     alias: {

package/notes.txt

@@ -1,158 +0,0 @@
-At the core of my challenge is to update the QuickbaseClient.ts file with the correct types signatures, removing "any" types.Right now the way I have been doing that is by adding spec overrides to the raw specs/QuickBase_RESTful_API_*.json file by creating open-api/paths/* and open-api/definitions/* files based on the quickbase documentation that override the raw specs file. I think this has been working okay, its fairly manual, and I feel like I am duplicating a lot of work that exists in the raw specs file. The reason I'm doing this is because the raw specs file has been difficult to work with in the past.Please give me a detailed summary of how the QuickbaseClient.ts is generated,
-
-Yes, I understand your concern, and you're correct to point out that enhancing raw spec processing represents a shift from your current strategy of using path/* and definitions/* folder overrides. Let's clarify the relationship between these approaches and why enhancing raw spec processing could be a complementary or alternative strategy.
-Current Strategy (Path/* and Definitions/* Overrides):
-Manual overrides in open-api/paths/* and open-api/definitions/*
-
-Directly modifies specific endpoints and types
-
-Pros:
-Precise control over problematic endpoints
-
-Immediate fixes for known issues
-
-Cons:
-Manual and time-consuming
-
-Duplicates information from raw spec
-
-Doesn't scale well for large APIs
-
-Maintenance burden as API evolves
-
-Proposed Strategy (Enhanced Raw Spec Processing):
-Focuses on improving the initial processing of QuickBase_RESTful_API_*.json
-
-Aims to extract better type information from the raw spec
-
-Pros:
-Addresses root causes of "any" types
-
-Scales better for large APIs
-
-Reduces need for manual overrides
-
-More maintainable long-term
-
-Cons:
-Requires more upfront development
-
-May need validation to ensure accuracy
-
-Still might require some manual overrides for edge cases
-
-Relationship Between Strategies:
-Complementary Nature:
-Enhanced raw spec processing can reduce the need for manual overrides
-
-Overrides can still be used for edge cases or QuickBase-specific quirks
-
-Better raw spec processing makes overrides more targeted and efficient
-
-Evolution, Not Replacement:
-We're not abandoning the override system entirely
-
-Instead, we're adding a new layer of processing to make overrides less necessary
-
-The goal is to shift from heavy reliance on manual overrides to automated type extraction
-
-Strategic Shift:
-Moving from reactive (fixing issues after they're found) to proactive (preventing issues at the source)
-
-Investing in better raw spec processing now will reduce maintenance costs later
-
-Allows for more systematic type improvements across the entire API
-
-Implementation Plan for Enhanced Raw Spec Processing:
-Spec Analysis Tool:
-Develop a tool to analyze QuickBase_RESTful_API_*.json
-
-Identify:
-Endpoints with incomplete type information
-
-Patterns leading to "any" types
-
-Common schema structures
-
-Output a type inventory report
-
-Enhanced fix-spec-main.ts:
-Implement partial spec loading for efficiency
-
-Add type validation and inference:
-Detect missing schema information
-
-Infer types from examples or patterns
-
-Flag areas needing manual review
-
-Enhance array and object schema processing
-
-Type Mapping Improvements:
-Extend mapOpenApiTypeToTs in generate-unified-interface.ts:
-Handle more OpenAPI type variations
-
-Add fallback type inference
-
-Improve mapRefToType:
-Handle complex schema compositions
-
-Better reference resolution
-
-Validation and Reporting:
-Add type quality validation in generate-unified-interface.ts
-
-Generate reports on:
-"any" type occurrences
-
-Missing type information
-
-Suggested manual overrides
-
-Integration with Existing Overrides:
-Modify fix-spec-main.ts to:
-Apply automated fixes first
-
-Then merge path/* and definitions/* overrides
-
-Create a report comparing automated vs. manual type definitions
-
-Transition Strategy:
-Parallel Development:
-Continue using current overrides while developing new processing
-
-Run both systems in parallel for validation
-
-Gradual Migration:
-Start with automated processing for new endpoints
-
-Gradually reduce manual overrides as confidence grows
-
-Maintain overrides for edge cases
-
-Validation and Feedback:
-Compare generated types from both approaches
-
-Identify areas where overrides are still necessary
-
-Refine automated processing based on findings
-
-Next Steps:
-Given that this is a strategic shift, I recommend:
-Starting with the spec analysis tool to understand raw spec patterns
-
-Implementing basic enhancements to fix-spec-main.ts
-
-Running parallel generation (old vs. new) for comparison
-
-Gradually reducing reliance on manual overrides
-
-Would you like to:
-See a detailed implementation plan for the spec analysis tool?
-
-Start with specific enhancements to fix-spec-main.ts?
-
-Develop a validation process to compare old vs. new approaches?
-
-Discuss how to maintain both strategies during transition?
-

package/tools/generate-types.ts

@@ -1,29 +0,0 @@
-// tools/generate-types.ts
-import { promises as fs } from "fs";
-import path from "path";
-import { fileURLToPath } from "url";
-import * as glob from "glob";
-import { execSync } from "child_process";
-
-async function generateTypes(): Promise<void> {
-  const CODEGEN_DIR = path.dirname(fileURLToPath(import.meta.url));
-  const SPLIT_DIR = path.join(CODEGEN_DIR, "..", "specs", "split");
-  const OUTPUT_DIR = path.join(CODEGEN_DIR, "..", "src", "generated-types");
-
-  await fs.mkdir(OUTPUT_DIR, { recursive: true });
-
-  const specFiles = glob.sync(path.join(SPLIT_DIR, "*.json"));
-  for (const specFile of specFiles) {
-    const category = path.basename(specFile, ".json");
-    const outputFile = path.join(OUTPUT_DIR, `${category}.ts`);
-    console.log(`Generating types for ${category}...`);
-    execSync(`npx openapi-typescript ${specFile} --output ${outputFile}`, {
-      stdio: "inherit",
-    });
-  }
-}
-
-generateTypes().catch((error) => {
-  console.error("Failed to generate types:", error);
-  process.exit(1);
-});

package/tools/openapi-schema-validator.ts

@@ -1,38 +0,0 @@
-// tools/validate-spec.ts
-import { promises as fs } from "fs";
-import path from "path";
-import { fileURLToPath } from "url";
-import * as glob from "glob";
-import OpenAPISchemaValidator from "openapi-schema-validator";
-
-async function validateSpecs(): Promise<void> {
-  const CODEGEN_DIR = path.dirname(fileURLToPath(import.meta.url));
-  const SPLIT_DIR = path.join(CODEGEN_DIR, "..", "specs", "split");
-
-  const specFiles = glob.sync(path.join(SPLIT_DIR, "*.json"));
-  if (specFiles.length === 0) {
-    console.error("No split spec files found in specs/split/ folder.");
-    process.exit(1);
-  }
-
-  const validator = new OpenAPISchemaValidator({ version: 2 }); // Adjust based on spec version
-
-  for (const specFile of specFiles) {
-    console.log(`Validating ${path.basename(specFile)}...`);
-    const specContent = await fs.readFile(specFile, "utf8");
-    const spec = JSON.parse(specContent);
-
-    const result = validator.validate(spec);
-    if (result.errors.length > 0) {
-      console.error(`Validation errors in ${specFile}:`, result.errors);
-      process.exit(1);
-    } else {
-      console.log(`${specFile} is valid.`);
-    }
-  }
-}
-
-validateSpecs().catch((error) => {
-  console.error("Failed to validate specs:", error);
-  process.exit(1);
-});

package/tools/spec-analyzer.ts

@@ -1,163 +0,0 @@
-import { promises as fs } from "fs";
-import path from "path";
-import { fileURLToPath } from "url";
-import * as glob from "glob";
-
-interface Parameter {
-  name: string;
-  in: string;
-  required?: boolean;
-  type?: string;
-  schema?: { type?: string; items?: any; $ref?: string; properties?: any };
-  example?: any;
-}
-
-interface Operation {
-  parameters?: Parameter[];
-  responses?: Record<string, { description: string; schema?: any }>;
-  operationId?: string;
-  summary?: string;
-  tags?: string[];
-}
-
-interface Spec {
-  paths: Record<string, Record<string, Operation>>;
-  definitions?: Record<string, any>;
-}
-
-interface TypeInference {
-  fieldId: string;
-  inferredType: string;
-  source: "schema" | "example";
-  confidence: number;
-  context: string;
-}
-
-interface AnalysisReport {
-  requestTypes: TypeInference[];
-  responseTypes: { [status: string]: TypeInference[] };
-  dynamicFields: TypeInference[];
-  errors: string[];
-}
-
-function inferTypeFromValue(value: any): string {
-  if (typeof value === "string") return "string";
-  if (typeof value === "number")
-    return Number.isInteger(value) ? "integer" : "number";
-  if (typeof value === "boolean") return "boolean";
-  if (Array.isArray(value)) {
-    if (value.length === 0) return "any[]";
-    const itemTypes = value.map(inferTypeFromValue);
-    const uniqueTypes = [...new Set(itemTypes)];
-    return uniqueTypes.length === 1 ? `${uniqueTypes[0]}[]` : "any[]";
-  }
-  if (typeof value === "object" && value !== null) {
-    return "object";
-  }
-  return "any";
-}
-
-function analyzeDynamicFields(
-  schema: any,
-  analysis: AnalysisReport,
-  context: string
-) {
-  const examples = schema["x-amf-examples"];
-  if (examples) {
-    for (const [exampleName, example] of Object.entries(examples)) {
-      const data = example.value?.data;
-      if (Array.isArray(data)) {
-        data.forEach((record: any, index: number) => {
-          for (const [fieldId, fieldValue] of Object.entries(record)) {
-            const inferredType = inferTypeFromValue(fieldValue.value);
-            analysis.dynamicFields.push({
-              fieldId,
-              inferredType,
-              source: "example",
-              confidence: 0.8,
-              context: `${context}-example-${exampleName}-${index}`,
-            });
-          }
-        });
-      }
-    }
-  }
-
-  // Add schema-based type extraction logic
-  if (schema.properties?.data?.items) {
-    // Add logic to extract types from schema
-  }
-}
-
-function analyzeUpsertSpec(spec: Spec): AnalysisReport {
-  const upsertPath = spec.paths["/records"]?.post;
-  if (!upsertPath)
-    return {
-      errors: ["Upsert endpoint not found"],
-      requestTypes: [],
-      responseTypes: {},
-      dynamicFields: [],
-    };
-
-  const analysis: AnalysisReport = {
-    requestTypes: [],
-    responseTypes: { "200": [], "207": [] },
-    dynamicFields: [],
-    errors: [],
-  };
-
-  const requestSchema = upsertPath.parameters?.find(
-    (p) => p.in === "body"
-  )?.schema;
-  if (requestSchema) {
-    analyzeDynamicFields(requestSchema, analysis, "request");
-  }
-
-  for (const status of ["200", "207"]) {
-    const responseSchema = upsertPath.responses?.[status]?.schema;
-    if (responseSchema) {
-      analyzeDynamicFields(responseSchema, analysis, `response-${status}`);
-    }
-  }
-
-  return analysis;
-}
-
-async function runAnalysis() {
-  try {
-    const CODEGEN_DIR = path.dirname(fileURLToPath(import.meta.url));
-    const SPECS_DIR = path.join(CODEGEN_DIR, "..", "specs");
-    const OUTPUT_DIR = path.join(CODEGEN_DIR, "..", "tools", "analysis");
-    console.log("Finding latest QuickBase RESTful API spec...");
-    const specFiles = glob.sync(
-      path.join(SPECS_DIR, "QuickBase_RESTful_*.json")
-    );
-    if (specFiles.length === 0) {
-      console.error(
-        "No QuickBase_RESTful_*.json files found in specs/ folder."
-      );
-      process.exit(1);
-    }
-    const inputFile = specFiles.sort().pop() as string;
-    console.log(`Reading ${path.basename(inputFile)} from specs/...`);
-    const specContent = await fs.readFile(inputFile, "utf8");
-    const spec: Spec = JSON.parse(specContent);
-
-    const analysis = analyzeUpsertSpec(spec);
-    await fs.mkdir(OUTPUT_DIR, { recursive: true });
-    await fs.writeFile(
-      path.join(OUTPUT_DIR, "upsert-analysis.json"),
-      JSON.stringify(analysis, null, 2),
-      "utf8"
-    );
-    console.log(
-      "Analysis complete. Output written to:",
-      path.join(OUTPUT_DIR, "upsert-analysis.json")
-    );
-  } catch (error) {
-    console.error("Failed to analyze spec:", error);
-    process.exit(1);
-  }
-}
-
-runAnalysis();

package/tools/split-spec.ts

@@ -1,148 +0,0 @@
-// tools/split-spec.ts
-import { promises as fs } from "fs";
-import path from "path";
-import { fileURLToPath } from "url";
-import * as glob from "glob";
-
-interface Parameter {
-  name: string;
-  in: string;
-  required?: boolean;
-  type?: string;
-  schema?: any;
-  example?: any;
-  description?: string;
-}
-
-interface Operation {
-  parameters?: Parameter[];
-}
-
-interface Spec {
-  swagger?: string;
-  openapi?: string;
-  info: any;
-  paths: Record<string, Record<string, Operation>>;
-  definitions?: Record<string, any>;
-}
-
-function normalizeParameters(parameters: Parameter[] = []): Parameter[] {
-  return parameters.map((param) => {
-    const normalized: Parameter = {
-      name: param.name,
-      in: param.in,
-      required: param.required,
-      description: param.description,
-    };
-
-    // OpenAPI v2: 'body' parameters use 'schema', others use 'type'
-    if (param.in === "body") {
-      if (param.schema) {
-        normalized.schema = param.schema;
-      }
-    } else {
-      normalized.type = param.type || "string";
-      if (param.example !== undefined) {
-        delete param.example; // Remove 'example' as it’s not allowed in v2
-      }
-    }
-
-    // Ensure 'in' is valid
-    if (
-      !["query", "header", "path", "formData", "body"].includes(normalized.in)
-    ) {
-      console.warn(
-        `Invalid 'in' value '${normalized.in}' for parameter '${normalized.name}', defaulting to 'query'`
-      );
-      normalized.in = "query";
-    }
-
-    return normalized;
-  });
-}
-
-async function splitSpec(): Promise<void> {
-  const CODEGEN_DIR = path.dirname(fileURLToPath(import.meta.url));
-  const SPECS_DIR = path.join(CODEGEN_DIR, "..", "specs");
-  const OUTPUT_DIR = path.join(SPECS_DIR, "split");
-
-  // Find the latest spec file
-  const specFiles = glob.sync(path.join(SPECS_DIR, "QuickBase_RESTful_*.json"));
-  if (specFiles.length === 0) {
-    console.error("No QuickBase_RESTful_*.json files found in specs/ folder.");
-    process.exit(1);
-  }
-  const inputFile = specFiles.sort().pop() as string;
-  const specContent = await fs.readFile(inputFile, "utf8");
-  const spec: Spec = JSON.parse(specContent);
-
-  // Define categories based on endpoint paths
-  const categories: Record<string, string[]> = {
-    apps: Object.keys(spec.paths).filter((p) => p.startsWith("/apps")),
-    auth: Object.keys(spec.paths).filter((p) => p.startsWith("/auth")),
-    fields: Object.keys(spec.paths).filter((p) => p.startsWith("/fields")),
-    records: Object.keys(spec.paths).filter((p) => p.startsWith("/records")),
-    tables: Object.keys(spec.paths).filter((p) => p.startsWith("/tables")),
-    reports: Object.keys(spec.paths).filter((p) => p.startsWith("/reports")),
-    relationships: Object.keys(spec.paths).filter((p) =>
-      p.startsWith("/relationships")
-    ),
-    events: Object.keys(spec.paths).filter((p) => p.startsWith("/events")),
-  };
-
-  await fs.mkdir(OUTPUT_DIR, { recursive: true });
-
-  // Split spec by category
-  for (const [category, paths] of Object.entries(categories)) {
-    if (paths.length === 0) continue;
-
-    const categorySpec: Spec = {
-      swagger: "2.0", // OpenAPI v2
-      info: spec.info || { title: `${category} API`, version: "1.0.0" },
-      paths: {},
-      definitions: {},
-    };
-
-    // Add relevant paths and normalize parameters
-    for (const p of paths) {
-      const pathObj = { ...spec.paths[p] };
-      for (const method in pathObj) {
-        if (pathObj[method].parameters) {
-          pathObj[method].parameters = normalizeParameters(
-            pathObj[method].parameters
-          );
-        }
-      }
-      categorySpec.paths[p] = pathObj;
-    }
-
-    // Add relevant definitions
-    const refs = new Set<string>();
-    const refRegex = /#\/definitions\/([^"]+)/g;
-    const categorySpecString = JSON.stringify(categorySpec);
-    let match;
-    while ((match = refRegex.exec(categorySpecString))) {
-      refs.add(match[1]);
-    }
-    if (spec.definitions) {
-      for (const ref of refs) {
-        if (spec.definitions[ref]) {
-          categorySpec.definitions![ref] = spec.definitions[ref];
-        }
-      }
-    }
-
-    const outputFile = path.join(OUTPUT_DIR, `${category}.json`);
-    await fs.writeFile(
-      outputFile,
-      JSON.stringify(categorySpec, null, 2),
-      "utf8"
-    );
-    console.log(`Wrote ${outputFile}`);
-  }
-}
-
-splitSpec().catch((error) => {
-  console.error("Failed to split spec:", error);
-  process.exit(1);
-});

package/tools/test-import.ts

@@ -1,3 +0,0 @@
-// tools/test-import.ts
-import OpenAPISchemaValidator from "openapi-schema-validator";
-console.log(OpenAPISchemaValidator);

package/tools/validate-spec.ts

@@ -1,40 +0,0 @@
-// tools/validate-spec.ts
-import { promises as fs } from "fs";
-import path from "path";
-import { fileURLToPath } from "url";
-import * as glob from "glob";
-import OpenAPISchemaValidatorModule from "openapi-schema-validator";
-
-const OpenAPISchemaValidator = OpenAPISchemaValidatorModule.default;
-
-async function validateSpecs(): Promise<void> {
-  const CODEGEN_DIR = path.dirname(fileURLToPath(import.meta.url));
-  const SPLIT_DIR = path.join(CODEGEN_DIR, "..", "specs", "split");
-
-  const specFiles = glob.sync(path.join(SPLIT_DIR, "*.json"));
-  if (specFiles.length === 0) {
-    console.error("No split spec files found in specs/split/ folder.");
-    process.exit(1);
-  }
-
-  const validator = new OpenAPISchemaValidator({ version: 2 });
-
-  for (const specFile of specFiles) {
-    console.log(`Validating ${path.basename(specFile)}...`);
-    const specContent = await fs.readFile(specFile, "utf8");
-    const spec = JSON.parse(specContent);
-
-    const result = validator.validate(spec);
-    if (result.errors.length > 0) {
-      console.error(`Validation errors in ${specFile}:`, result.errors);
-      process.exit(1);
-    } else {
-      console.log(`${specFile} is valid.`);
-    }
-  }
-}
-
-validateSpecs().catch((error) => {
-  console.error("Failed to validate specs:", error);
-  process.exit(1);
-});