@codeyam/codeyam-cli 0.1.0-staging.1 → 0.1.0-staging.28f73cf

package/analyzer-template/packages/ai/src/lib/generateEntityKeyAttributes.ts

@@ -57,8 +57,8 @@ export default async function generateEntityKeyAttributes({
     staticAnalysisContext,
   });
 
-  console.info('Generated prompt for generateEntityKeyAttributes:');
-  console.info(prompt);
+  console.log('Generated prompt for generateEntityKeyAttributes:');
+  console.log(prompt);
 
   const response = await completionCall({
     type: `generateEntityKeyAttributes`,
@@ -151,224 +151,101 @@ export default async function generateEntityKeyAttributes({
   };
 }
 
-const FRONTEND_SYSTEM_MESSAGE = `We have a frontend function and need to determine how best to vary what is returned from this function and displayed to the user.
+const FRONTEND_SYSTEM_MESSAGE = `You analyze frontend components to identify key data attributes that significantly impact rendering.
 
-We're not interested in causing errors or in high-level guard clauses that often display nothing on the screen. We're interested in always displaying a robust return value that shows what this frontend component does. This does not mean that values can't be undefined, but only allow a "validValueOption" to be "undefined" if the component will still display something meaningful to the user. 
+## What Makes a Key Attribute
 
-In order to do so we want to identify the key data attributes in the code. We've included a list of attributes we've identified as being used in conditionals in the code. Can you start with those, order them by importance and remove any that will result in nothing being displayed?
+A key attribute is a data path from "Attributes and Value Types" that dramatically changes how the component renders:
 
-Then can you idfentify any other key attributes from the "Attributes and Value Types" list that will dramatically change how the component is displayed to the user?
+- Controls a conditional that shows/hides significant UI sections
+- Changes the component's primary visual state (not just minor text differences)
+- Determines which branch of rendering logic executes
 
-Each key attribute represents a path from a top level variable and that path must be noted exactly as it is in the "Attributes and Value Types" list.
+NOT key attributes:
+- Values that only affect non-visible behavior (hrefs, image srcs)
+- Fallback values that won't be used if primary values exist
+- Values that would cause errors if changed
+- Guard clauses that render literally nothing
 
-Key attributes must come either from the function signature or from a function call that is made from within the function that return data that is used in the function. Please capture the attrribute path exactly from the list.
+## Conditional Key Attributes
 
-If an attribute has the keyword "functionCallReturnValue" in it, it is a function call that returns data that is used in the function. That keyword simply designated the return value of the function call. If an attribute path has "[]" in it (e.g. "items[]") that represents one item in that array.
+We've provided a list of variables that are used in conditional statements in the code. These are very likely key attributes.
 
-## Key Attribute Rules
+Each conditional lists it's source and how it is used. This should help you identify which attributes to include and what values to suggest.
 
-- All key attributes must come from the "Attributes and Value Types" provided. All data traces through these attributes and these are the only attributes we can use to provide data to the function. Only use these attributes. Do not reference internally computed variables. Trace their origin back to an attribute from the "Attributes and Value Types" list.
+## Path Format
 
-### Should be a key attribute
+Only return attributes from the "Attributes and Value Types" list — these are the only values we can control. Use the exact \`fullPath\` provided. Use \`functionCallReturnValue\` for function return values. Use \`[]\` for array items (e.g., \`items[]\`).
 
-- Changing the presence, value, length (e.g. for strings or arrays), etc of this variable will dramatically change how the component displays (the return value of the function).
+## Valid Value Options
 
-- Is used in a conditional (especially conditionals in the JSX such as \`{varName1 && varName2 && ([JSX])}\` where the absence or presence of both \`varName1\` and \`varName2\` directly impact what the user sees) that alters how the component displays. Capture how the conditional uses the variable. Does it require the variable to be true/false, defined/not defined (e.g. undefined), greater than/less than a certain value, compared with another variable, etc. Really any attribute that is used in a conditional should be a key attribute.
+- For \`validValueOptions\`, include boundary/extreme values that trigger different behavior 
+- Be very specific with values. Do not make them ambiguous in any way
+— Do NOT include values that would cause unexpected/unhandled errors.
+- Ensure all values or value ranges that will trigger different rendering behavior are included
 
-### Should NOT be a key attribute
+## Response Format
 
-- Changing the value of this variable will cause an error or exception to be thrown.
-
-- Changing the value of this variable will cause a guard clause around the whole method or function to be triggered that results in nothing at all being displayed.
-
-- Changing the value of this variable will cause a minor change in display that is not significant (e.g. slightly different text).
-
-- The value is used as a fallback or alternative for another value and won't even be used if the other value is present.
-
-- The value changes non-visible behavior (e.g. the href in a link or src in an image - all images sources are mocked out).
-
-## Instructions
-
-- Read each line of code. Pay particular attention to the return value of the function and how this frontend component will be displayed. What will cause it display distinctly different but functional and robust results to the user?
-- Find the most impactful key attributes that will change how the frontend component is displayed. This should be a limited subset of the "Attributes and Value Types" provided.
-- Return each key attribute with the following information:
-  - \`fullPath\`: The fullPath to the data attribute in the code
-  - \`description\`: A short description of the data attribute and how it is used in the code.
-  - \`validValueOptions\`: A list of possible values for the data attribute that should be tested. This list should be minimal and only include values that the code will handle properly and will clearly make a difference in how the component is displayed. Do not make up arbitrary values without clear evidence that each different value will cause a different display behavior.
-- List the key attributes in the order of importance (most important first, with importance determined by how much changing the attribute will impact how the component is displayed).
-- Each attribute should reference exactly the full fullPath of a data attribute provided in the "Attributes and Value Types" section.
-
-## Example
-
-Given the following "Attributes and Value Types":
-\`\`\`
-  'notification.read': { fullPath: 'signature[0].notification.read', type: 'boolean' }
-  'notification.message': { fullPath: 'signature[0].notification.message', type: 'string' }
-\`\`\`
-
-and code:
-
-\`\`\`typescript
-const NotificationText = (notification) => {
-  if (notification.read) {
-    return (
-      <span onClick={markAsRead(notification.id)}>
-        <img src={notification.icon} alt="Notification Icon" />
-        {notification.message}
-      </span>
-    );
-  } else {
-    return <strong>{notification.message}</strong>;
-  }
-};
-\`\`\`
-
-The response might be:
+List all key attributes in order from most impactful to least impactful regarding how dramatically they will change the rendering of the component using the following format:
 
 \`\`\`json
 {
   "keyAttributes": [
     {
-      "fullPath": "signature[0].notification.read",
-      "description": "The read status of the notification determines how the message is displayed.",
-      "validValueOptions": [
-        "true",
-        "false"
-      ]
-    },
-    {
-      "fullPath": "signature[0].notification.message",
-      "description": "The message text is displayed and should be tested for various lengths.",
-      "validValueOptions": [
-        "An empty message (no characters)",
-        "A medium message (around 100 characters)",
-        "A long message (longer than 200 characters)"
-      ]
+      "fullPath": "signature[0].user",
+      "description": "Controls whether the main form or redirect screen displays",
+      "validValueOptions": ["an empty object {}", "null/undefined"]
     }
   ]
 }
 \`\`\`
-
-## Important
-
-- A key attribute must be an exact attribute fullPath from the "Attributes and Value Types" provided. Do not return any other attribute fullPaths.  The only data we can provide to the function comes from these attributes so any other attribute is useless. For internally computed variables trace them back to one of the higher level variables that we can actually impact. Be sure to match the text from the attribute list exactly including spaces, commas, etc.
 `;
 
-const BACKEND_SYSTEM_MESSAGE = `We have a function that we want to test in various ways by passing in appropriate data.
-
-Can you help us identify the key data attributes in the code that will cause the function to return distinctly different but functional and robust results?
-
-We're not interested in causing errors or in high-level guard clauses that result in nothing being returned. We're interested in always returning a robust result that demonstrates the core intent of the function.
+const BACKEND_SYSTEM_MESSAGE = `You analyze backend functions to identify key data attributes that significantly impact behavior.
 
-In order to do so we want to identify the key data attributes in the code. We've included a list of attributes we've identified as being used in conditionals in the code. Can you start with those, order them by importance and remove any that will result in nothing being displayed?
+## What Makes a Key Attribute
 
-Then can you idfentify any other key attributes from the "Attributes and Value Types" list that will dramatically change how the component is displayed to the user?
+A key attribute is a data path from "Attributes and Value Types" that dramatically changes the function's behavior:
 
-Each key attribute represents a path from a top level variable and that fullPath must be noted exactly as it is in the "Attributes and Value Types" list.
+- Controls a conditional that significantly alters execution flow or return value
+- Determines which branch of logic executes
+- For side-effect functions: changes how external systems are called (database, email, etc.)
 
-Key attributes must come either from the function signature or from a function call that is made from within the function that return data that is used in the function. Please capture the attrribute fullPath exactly from the list.
+NOT key attributes:
+- Fallback values that won't be used if primary values exist
+- Values that would cause errors if changed
+- Guard clauses that return nothing/minimal data
 
-If an attribute has the keyword "functionCallReturnValue" in it, it is a function call that returns data that is used in the function. That keyword simply designated the return value of the function call. If an attribute path has "[]" in it (e.g. "items[]") that represents one item in that array.
+## Conditional Key Attributes
 
-## Key Attribute Rules
+We've provided a list of variables that are used in conditional statements in the code. These are very likely key attributes.
 
-- All key attributes must come from the "Attributes and Value Types" provided. All data traces through these attributes and these are the only attributes we can use to provide data to the function. Only use these attributes. Do not reference internally computed variables. Trace their origin back to an attribute from the "Attributes and Value Types" list.
+Each conditional lists it's source and how it is used. This should help you identify which attributes to include and what values to suggest.
 
-### Should be a key attribute
+## Path Format
 
-- Changing the value, length (e.g. for strings or arrays), etc of this variable will dramatically impact the return value of the function.
+Only return attributes from the "Attributes and Value Types" list — these are the only values we can control. Use the exact \`fullPath\` provided. Use \`functionCallReturnValue\` for function return values. Use \`[]\` for array items (e.g., \`items[]\`).
 
-- Is used in a conditional that dramatically the execution flow of the function and as a result the return value of the function (note: if the conditional compares two or more variables, note which variables are involved and how they must be set to cause different behavior).
+## Valid Value Options
 
-- Note: some functions do not return anything but instead call out to other systems (e.g. send an email, write to a database, etc). In those cases, the key attributes should be those that dramatically change how the function interacts with those other systems.
+- For \`validValueOptions\`, include boundary/extreme values that trigger different behavior 
+- Be very specific with values. Do not make them ambiguous in any way
+— Do NOT include values that would cause unexpected/unhandled errors.
+- Ensure all values or value ranges that will trigger different rendering behavior are included
 
-### Should NOT be a key attribute
+## Response Format
 
-- Changing the value of this variable will cause an error or exception to be thrown.
-
-- Changing the value of this variable will cause a guard clause to be triggered that results in nothing or very little being returned.
-
-- The value is used as a fallback or alternative for another value and won't even be used if the other value is present.
-
-## Instructions
-
-- Read each line of code. Pay particular attention to the return value of the function. What data attributes will cause it to return distinctly different but functional and robust results?
-- Find the most impactful key attributes that will impact the return value of the function. This should be a limited subset of the "Attributes and Value Types" provided.
-- Return each key attribute with the following information:
-  - \`fullPath\`: The fullPath to the data attribute in the code
-  - \`description\`: A short description of the data attribute and how it is used in the code.
-  - \`validValueOptions\`: A list of possible values for the data attribute that should be tested. This list should be minimal and only include values that the code will handle properly and will clearly make a difference in the functions's behavior. Do not make up arbitrary values without clear evidence that each different value will clearly result in a different function behavior.
-- List the key attributes in the order of importance (most important first, with importance determined by how much changing the attribute will change the return value of the function).
-- Each attribute should reference exactly the fullPath of a data attribute provided in the "Attributes and Value Types" section.
-
-## Example
-
-Given the following "Attributes and Value Types":
-\`\`\`
-[
-  items[].quantity: { fullPath: 'signature[0][].quantity', type: 'number' },
-  items[].price: { fullPath: 'signature[0][].price', type: 'number' },
-  discount: { fullPath: 'signature[1]', type: 'number' },
-  taxRate: { fullPath: 'signature[2]', type: 'number' },
-]
-\`\`\`
-
-and code:
-
-\`\`\`typescript
-const calculateTotal = (items, discount, taxRate) => {
-  const subtotal = items.reduce((sum, item) => sum + item.quantity * item.price, 0);
-  const discountedTotal = subtotal - discount;
-  const tax = discountedTotal * taxRate;
-  return discountedTotal + tax;
-}
-\`\`\`
-
-The response might be:
+List all key attributes in order from most impactful to least impactful using the following format:
 
 \`\`\`json
 {
   "keyAttributes": [
     {
-      "fullPath": "signature[0][].quantity",
-      "description": "The quantity of each item affects the subtotal calculation.",
-      "validValueOptions": [
-        "0 (no items)",
-        "1 (single item)",
-        "5 (multiple items)",
-        "100 (large quantity)"
-      ]
-    },
-    {
-      "fullPath": "signature[0][].price",
-      "description": "The price of each item affects the subtotal calculation.",
-      "validValueOptions": [
-        "0.00 (free item)",
-        "19.99 (average price)",
-        "999.99 (high price)"
-      ]
-    },
-    {
-      "fullPath": "signature[1]",
-      "description": "The discount amount directly reduces the subtotal.",
-      "validValueOptions": [
-        "0 (no discount)",
-        "20.00 (discount)"
-      ]
-    },
-    {
-      "fullPath": "signature[2]",
-      "description": "The tax rate affects the final total calculation.",
-      "validValueOptions": [
-        "0 (no tax)",
-        "0.20 (20% tax)"
-      ]
+      "fullPath": "signature[0].config.enabled",
+      "description": "Controls whether the main processing logic runs",
+      "validValueOptions": ["true", "false"]
     }
   ]
 }
 \`\`\`
-
-## Important
-
-- A key attribute must be an exact attribute fullPath from the "Attributes and Value Types" provided. Do not return any other attribute fullPaths.  The only data we can provide to the function comes from these attributes so any other attribute is useless. For internally computed variables trace them back to one of the higher level variables that we can actually impact. Be sure to match the text from the attribute list exactly including spaces, commas, etc.
 `;

package/analyzer-template/packages/ai/src/lib/generateEntityScenarioData.ts

@@ -280,176 +280,75 @@ export default async function generateEntityScenarioData({
 export const generateSystemMessage = (
   scenarioName: string,
   defaultScenario: boolean,
-) => `## Overview
-- You will be provided with a structure for scenario data, the "mockData structure", and for the arguments of the function, the "argumentsData structure", and a scenario describing a new state of data for this function.
-
-- Your goal is to generate data to fulfill both the data structures and the scenario, to be returned in a JSON object.
-
-## Provided Information
-
-- The "mockData structure" and "argumentsData structure" describe all of the data required for the simulation to work. Copy the data structures exactly (if an attribute is a function it will look like "myFunction()" and that exact key "myFunction()" should be in the return value structure) and replace any primitives (e.g. "string", "number", "boolean", "date") with realistic data. 
-
-${defaultScenario ? '' : '- The "Default Scenario Data" provides the data that has already been generated for the default scenario. This data will be merged into the data for each scenario. Your goal is to modify this data as needed to satisfy the requirements of this scenario. As such you only need to specify data that is different or should be removed from the default scenario data. To remove data set some portion of the data structure to null.'}
-
-- Follow the "keyAttributeInstructions" for the scenario carefully.${defaultScenario ? ' As the Default Scenario ensure that the entire data structure is filled in with appropriate values even if they are not explicitly described in the "keyAttributeInstructions".' : ''}
-
-## Instructions
-${
-  defaultScenario
-    ? `
-- We need data for the default scenario.
-
-- The default scenario requires robust and complete data that fulfills all the "mockData" and "argumentsData" data structures if they exist, except for error attributes.
-
-- The one exception to this is error data. Sometimes the data structure will contain structures for both success and error data. In this case you should only fill in the success data and leave out the error data completely.
-
-- This data should cover for every object, attribute, and nested attribute described in the data structures (other than error data). If an item in a data structure is an array then provide multiple objects in the array. Fill in every nested attribute with realistic data. This should provide a robust and complete "database" for the simulation. If the type is described as "any" then create a realistic example of that type based on its name. If the type is "date" or "timestamp" then use the instructions below to generate a dynamic date or timestamp. 
-
-- Remember to fill in each and every attribute and nested attribute in the data structure for the default scenario (other than error data). This will make the simulation more realistic and valuable. Do not add any attributes that are not in the data structure. If an attribute is not in the data structure then it should not be included in the data.
-`
-    : `
-- This scenario is a modification of the default scenario. Figure out what needs to change about the Default Scenario in order to satisfy this scenario. If data must be removed then mark is as null at the appropriate level of the structure that the deletion should occur. Otherwise you can change any data that needs to be modified. Do not include any aspect of the structure that should remain the same as the Default Scenario.
-
-- The data generated will be merged into the Default Scenario Data in order to create the full data for this scenario.
-`
-}
-
-## Example Response
-
-{
-  "scenarioData": {
-    "scenarioName": ${scenarioName},
-    "data': {
-      "mockData": {
-        "useUser()": {
-          "user": {
-            "name": "John Doe",
-            "metadata": {
-              "hobbies": ["fishing", "hiking", "camping"]
-              "favoriteColor": "blue",
-            },
-            "joinedAt": {
-              "~~codeyam-code~~": "new Date(Date.now() - 3 * 24 * 60 * 60 * 1000)"         
-            }
-          },
-        },
-        "loadProject()": {
-          "project": {
-            "name": "New Project",
-            "slug": "new-project",
-            "files": [
-              {
-                "name": "File 1",
-                "slug": "file-1",
-                "content": "This is the content of file 1"
-              },
-              {
-                "name": "File 2",
-                "slug": "file-2",
-                "content": "This is the content of file 2"
-              },
-              {
-                "name": "File 3",
-                "slug": "file-3",
-                "content": "This is the content of file 3"
-              }
-            ]
-          }
-        }
-      },
-      "argumentsData": [
-        {
-          "open": "true",
-          "children: {
-            "~~codeyam-jsx~~": "<div>Hi!</div>"         
-          }      
-        }    
-      ]  
-    }
-  }
-}
-
-## scenarioData
-
-You will return a JSON object called scenarioData containing the name and data for this scenario.
-
-If you encounter a "date" then you should use "~~codeyam-code~~" along with code in string format. That string will be evaled to create the date or timestamp needed.
-
-For example the "timestamp" above should probably not be a hard-coded timestamp but should be a function that returns a timestamp. If you want to return a timestamp that is 24 hours in the future, you would return this object:
+) => {
+  const scenarioType = defaultScenario
+    ? `## Default Scenario
+Generate COMPLETE, robust data for the entire data structure.
+- Fill ALL fields with realistic values (except error attributes)
+- Arrays should have 2-3 items
+- Don't skip nested attributes
+- This provides the baseline data for all other scenarios`
+    : `## Non-Default Scenario
+Generate ONLY the differences from the default scenario.
+- Include only fields that need to change
+- Set to \`null\` to remove data
+- Omit unchanged fields—they merge from default`;
+
+  return `You are a test data generator. Create mock data matching a data structure and scenario requirements.
+
+${scenarioType}
+
+## CRITICAL: NO ERROR DATA
+NEVER include "error" fields in responses. Skip them entirely.
+- If structure has \`{ data: {...}, error: {...} }\`, only fill \`data\`
+- Leave out any attribute named "error"—do not set to null, omit entirely
+
+## Special Markers
+
+### Dynamic Dates (\`~~codeyam-code~~\`)
+\`\`\`json
+{ "createdAt": { "~~codeyam-code~~": "new Date(Date.now() - 24*60*60*1000)" } }
+\`\`\`
+Use for relative dates. Code runs in Node (no browser APIs, no external libraries).
 
+### JSX Children (\`~~codeyam-jsx~~\`)
+\`\`\`json
+{ "children": { "~~codeyam-jsx~~": "<div>Hello</div>" } }
 \`\`\`
+Use simple elements only (\`<div>\`, \`<span>\`). No custom components.
+
+## Function Keys
+Preserve function call syntax exactly as written in the structure:
+\`\`\`json
 {
-  "scenarioData": {
-    "scenarioName": ${scenarioName},
-    "data": {
-      "mockData": {
-        "dataLoaders": {
-          "fetchUser()": {
-            "user": {
-              "name": "John Doe",
-              "metadata": {
-                "hobbies": ["fishing", "hiking", "camping"],
-                "favoriteColor": "blue"
-              },
-              "joinedAt": {
-                "~~codeyam-code~~": "new Date(Date.now() - 3 * 24 * 60 * 60 * 1000).toISOString()"
-              }  
-            }
-          }
-        }
-      }
-    }
+  "mockData": {
+    "useUser()": { "user": { "name": "John" } },
+    "from().select()": [{ "id": "1" }]
   }
 }
 \`\`\`
 
-Keep in mind the above pattern is also valid for argumentsData.
-
-All code should be written in JavaScript. It will run in a node environment (not the browser) with no reference to external libraries. It will essentially be run in a sandboxed environment. Do not reference any external libraries in the code.
-
-With argumentsData you may also need to return a "children" argument. "children" can either be a string or a jsx element.
-
-If you want to return a jsx element you would return something like this:
-
-\`\`\`
+## Response Format
+\`\`\`json
 {
   "scenarioData": {
-    "scenarioName": ${scenarioName},
+    "scenarioName": "${scenarioName}",
     "data": {
-      "argumentsData": [
-        {
-          "children": {
-            "~~codeyam-jsx~~": "<div>Click Me!</div>"
-          }
-        }
-      ]
+      "mockData": { ... },
+      "argumentsData": [ ... ]
     }
   }
 }
 \`\`\`
 
-The "~~codeyam-jsx~~" key in a "children" prop will be rendered directly to the code as a jsx element, not a string. Only return basic elements like "<div>".
-
-Do not import or reference any libraries in this code. No external libraries will be imported.
-
-If either mockData or argumentsData is empty you can return an empty object for mockData ("{}") or empty array for argumentsData ("[]"). 
-
-If an object inside of mockData or argumentsData is intended to not exist you can set it to null.
-
-Never use "~~codeyam-code~~" or "~~codeyam-jsx~~" as a partial key in the response. Even if the key looks like it should have a relative date or code. "~~codeyam-code~~" and "~~codeyam-jsx~~" are only used as independent keys used to mark values that are code.
-
-## Common Mistakes
-
-- Do not use the term undefined in the response. If something is missing leave it out or use null.
-- Leave out error data (e.g. a data attribute labeled as "error"), especially when it is in a data structure that has success data as well. Only include the success data and simply leave out the error data. Error data is usually labeled as "error" in the structure. Leave it out entirely.
-  - Please double check this. Ensure no error data is included in the response unless specifically requested. ${defaultScenario ? `This includes the ${`DEFAULT_SCENARIO_NAME`}.` : ''} Leave out any error attributes from the response completely!
-- Do not reference data in the response (e.g. if there was a "posts" field you can not say "posts[0]" elsewhere in the response - you need to duplicate the data). Unfortunately all data must be written out directly in the response.
-- Ensure each scenario in the response matches the full scenario name, commas and all, in the scenarios provided.
-- Do not throw errors from this data in any manner. Simple leave out any error data.
-- If the data structure contains a type that is a "date" then use ~~codeyam-code~~ to generate a dynamic timestamp or date. Do not hard-code dates unless it needs to be a very specific date. Otherwise it should be a relative, dynamic date.
-- The response must be valid JSON. Do not include raw code in the response.
-${defaultScenario ? `- As the ${DEFAULT_SCENARIO_NAME} be very robust in generating data that reflects common real-world usage. All arrays and attributes, except error attributes, should be filled in with high quality data so that default simulation provides a good default of how the function behaves` : ''}
+## Rules
+- Valid JSON only—no raw code outside markers
+- No \`undefined\`—use \`null\` or omit
+- No data references (can't use \`posts[0]\` elsewhere—duplicate the value)
+- Scenario name must match exactly: "${scenarioName}"
+- Empty mockData: \`{}\`, empty argumentsData: \`[]\`
 `;
+};
 
 export const generateIncompleteSystemMessage = (
   scenarioName: string,