ad-agent_architecture 0.0.4 → 0.0.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ddf07e6fca1337e7800b3c85b1c48f44524aa14660827eb81ffe541bce122c49
-  data.tar.gz: be8299cec2e18b52eebeb1af83fc243b172685b490cd581241d1d471b7de68ee
+  metadata.gz: daa2d7eacfdb1c2650cf23106d3f001040d765d7aa3ccf44081a989361e9ad48
+  data.tar.gz: b3fe1ef5e9d26a809d13ddb6200d125a17549b6af88b5995ba05e35788fcb608
 SHA512:
-  metadata.gz: 92afeb1cb3bc9315da02dc33dbe280fa6ab1c43404886852c5e4b542345e00f8f0b48b504a5f45dce205dde146e6dc3f7cebc35b8d2cee777aee6f81ed7a6c9f
-  data.tar.gz: 0167d391cb38acb70bd39dd0e454c21570531b86bb411b9b473869437f70a33374992f266c7cd10daa8f9ffd17697ffbd5784bb46056ffb4c71d3e34966926a2
+  metadata.gz: b09d423f903b09b704e958627b4a2cc4d1dbae248aee623e0750623e366f13247ab943043742702f8e0353ce4fe8a22ea7303f0f267a2ccce90c365239970947
+  data.tar.gz: be77db10d9e1fc55d9d18464714b57a8039b194a5c53ec963d7becdc743533ef60b06af6d18cf9ba13048446d09031d679fe239c3c5c3352870a8b3cbeac706f

data/.rubocop.yml

@@ -14,6 +14,7 @@ AllCops:
   Exclude:
     - ".builders/**/*"
     - "spec/samples/**/*"
+    - "lib/ad/agent_architecture/db/*"
 
 Metrics/BlockLength:
   Exclude:

data/CHANGELOG.md

@@ -1,3 +1,18 @@
+## [0.0.5](https://github.com/appydave/ad-agent_architecture/compare/v0.0.4...v0.0.5) (2024-06-28)
+
+
+### Bug Fixes
+
+* add sqlite3, sequal and setup a schema ([a8fa9c9](https://github.com/appydave/ad-agent_architecture/commit/a8fa9c9ff29ed223644d3622bd46ca5d626af9a9))
+* add sqlite3, sequal and setup a schema ([87ded1e](https://github.com/appydave/ad-agent_architecture/commit/87ded1e6886d3190fc83ecaa4e215bacf044378f))
+
+## [0.0.4](https://github.com/appydave/ad-agent_architecture/compare/v0.0.3...v0.0.4) (2024-06-26)
+
+
+### Bug Fixes
+
+* update schema ([dd2d6fb](https://github.com/appydave/ad-agent_architecture/commit/dd2d6fbd1eb477da5e44f3d5417aa05ec291fb25))
+
 ## [0.0.3](https://github.com/appydave/ad-agent_architecture/compare/v0.0.2...v0.0.3) (2024-06-26)
 
 

data/docs/requirements.md

@@ -1,3 +1,65 @@
+# AppyDave - Agent Architecture Schema
+
+## Purpose
+
+The application is designed to handle structured workflows for building documents or text-based assets using LLMs from OpenAi, Anthropic and others. It should be adaptable to any workflow that follows a structured concept or a series of steps.
+
+## Sample Workflows
+- YouTube titles
+- YouTube scripts
+- YouTube transcription to Medium, LinkedIn, etc.
+
+## Key Components
+
+### Static Workflow Definition
+- **Workflows**: Define a series of tasks to achieve a specific goal.
+- **Sections**: Subdivisions within workflows to organize steps logically.
+- **Steps**: Individual tasks within a section, each associated with an AI prompt and various input/output attributes.
+- **Attributes**: Parameters used in steps, which can be simple values or arrays.
+- **Input/Output Attributes**: Map attributes to steps as either inputs or outputs.
+
+### Dynamic Workflow Execution
+- **Workflow Runs**: Instances of workflow executions.
+- **Section Runs**: Instances of section executions within a workflow run.
+- **Step Runs**: Instances of step executions within a section run, supporting multiple branches.
+- **Attribute Values**: Store the values of attributes during step executions.
+
+## Features
+
+### Schema Creation and Models
+- Use Sequel and SQLite to define and manage the database schema.
+- Models are defined for workflows, sections, steps, attributes, and their relationships.
+
+### Testing and Validation
+- Unit tests to verify the schema and associations using RSpec.
+- Guard setup to automatically run tests when certain files are modified.
+
+### Data Management
+- Methods to export the database schema and data to JSON files for backup.
+- Methods to restore the database from JSON files for easy editing and re-importing.
+
+## Initial Workflows
+- These include YouTube video scripts, YouTube title creation, and transcription to articles.
+- These workflows serve as examples and validation for the broader concept.
+
+## Implementation Details
+- **Database**: Use SQLite with Sequel for ORM.
+- **Code Structure**: Organize code into modules and classes for database schema creation and model definitions.
+- **Testing**: Ensure models and associations are correctly implemented and tested.
+
+## Next Steps
+
+### Refine DSL for Static Workflows
+- Enhance the DSL to create and update static workflows.
+- Ensure it updates the database and exports workflows in JSON/YAML format.
+- Implement rendering of the workflow structure as HTML.
+
+### Plan Dynamic Execution
+- Determine the approach for executing workflows dynamically.
+- Decide on using a DSL or a web application for this purpose.
+
+This summary should help guide the development and focus on the core objectives and future directions.
+
 
 ## Schema for AI Agents
 
@@ -7,7 +69,16 @@
 
 ### Static Workflow Definition
 
+Static workflow definition defines the structure of a workflow, including its sections, steps, attributes, and their relationships.
+
+![Static Workflow Definition](./images/sample-workflow.png)
+
+### Static Workflow Entities
+
 #### Table: workflows
+
+This table stores the basic information about each workflow, including its unique identifier, name, and description.
+
 | Field       | Type   | Description                         |
 |-------------|--------|-------------------------------------|
 | id          | string | Unique identifier for the workflow  |
@@ -15,40 +86,60 @@
 | description | string | A brief description of the workflow |
 
 #### Table: sections
+This table stores the sections that belong to a workflow. Each section has an order within its workflow.
+
 | Field       | Type    | Description                            |
 |-------------|---------|----------------------------------------|
 | id          | string  | Unique identifier for the section      |
+| workflow_id | string  | Foreign key referencing workflows      |
 | name        | string  | The name of the section                |
 | description | string  | A brief description of the section     |
 | order       | integer | The order of the section in the workflow|
-| workflow_id | string  | Foreign key referencing workflows      |
 
 #### Table: steps
+
+This table stores the steps that belong to a section. Each step has an order within its section and an associated prompt.
+
+Example workflows include "YouTube Video Script", "YouTube Title Creator" and "YouTube Transcription to Medium Article".
+
 | Field       | Type    | Description                         |
 |-------------|---------|-------------------------------------|
 | id          | string  | Unique identifier for the step      |
+| section_id  | string  | Foreign key referencing sections    |
 | name        | string  | The name of the step                |
+| action      | string  | The action to be performed by the step, default ('gpt') |
 | description | string  | A brief description of the step     |
 | order       | integer | The order of the step in the section|
-| section_id  | string  | Foreign key referencing sections    |
 | prompt      | string  | The template string for the prompt  |
 
 #### Table: attributes
+
+This table stores the attributes associated with a workflow. Each attribute can be a simple value or an array.
+
+Example attributes include "simple_title", "basic_factsheet" and "working_title".
+
 | Field       | Type    | Description                                 |
 |-------------|---------|---------------------------------------------|
 | id          | string  | Unique identifier for the attribute         |
+| workflow_id | string  | Foreign key referencing workflows           |
 | name        | string  | The name of the attribute                   |
 | type        | string  | The type of the attribute (e.g., string)    |
 | is_array    | boolean | Indicates whether the attribute is an array |
-| workflow_id | string  | Foreign key referencing workflows           |
 
 #### Table: input_attributes
+
+This table maps input attributes to steps. This becomes the input parameters for the AI agent.
+
 | Field       | Type    | Description                        |
 |-------------|---------|------------------------------------|
 | step_id     | string  | Foreign key referencing steps      |
 | attribute_id| string  | Foreign key referencing attributes |
+| required    | boolean | Indicates whether the attribute is required |
 
 #### Table: output_attributes
+
+This table maps output attributes to steps. This becomes a result of the AI agent.
+
 | Field       | Type    | Description                        |
 |-------------|---------|------------------------------------|
 | step_id     | string  | Foreign key referencing steps      |
@@ -56,20 +147,31 @@
 
 ### Dynamic Workflow Execution
 
+Dynamic workflow execution captures the actual execution of a workflow, including the workflow runs, section runs, step runs, and attribute values.
+
+### Dynamic Workflow Entities
+
 #### Table: workflow_runs
+
+This table stores instances of workflow executions.
+
 | Field       | Type   | Description                           |
 |-------------|--------|---------------------------------------|
 | id          | string | Unique identifier for the workflow run|
 | workflow_id | string | Foreign key referencing workflows     |
 
 #### Table: section_runs
+This table stores instances of section executions within a workflow run.
+
 | Field           | Type   | Description                            |
 |-----------------|--------|----------------------------------------|
 | id              | string | Unique identifier for the section run  |
-| workflow_run_id | string | Foreign key referencing workflow_runs  |
 | section_id      | string | Foreign key referencing sections       |
+| workflow_run_id | string | Foreign key referencing workflow_runs  |
 
 #### Table: step_runs
+This table stores instances of step executions within a section run. Each step run can have multiple branches.
+
 | Field            | Type    | Description                                                         |
 |------------------|---------|---------------------------------------------------------------------|
 | id               | string  | Unique identifier for the step run                                  |
@@ -78,9 +180,526 @@
 | branch_number    | integer | Branch number to distinguish different instances (branches) of the same step |
 
 #### Table: attribute_values
+This table stores the values of attributes during step executions.
+
 | Field         | Type   | Description                             |
 |---------------|--------|-----------------------------------------|
 | id            | string | Unique identifier for the attribute value|
 | attribute_id  | string | Foreign key referencing attributes      |
 | step_run_id   | string | Foreign key referencing step_runs       |
 | value         | text   | The actual value of the attribute during the step execution |
+
+## Gemfile
+
+```ruby
+
+# Gemfile
+
+source 'https://rubygems.org'
+
+gem 'sequel'
+gem 'sqlite3'
+```
+
+## Database Setup
+
+
+```ruby
+# db_setup.rb
+
+require 'sequel'
+
+DB = Sequel.sqlite('workflow.db') # Creates an SQLite database called workflow.db
+
+# Create the tables for static workflow definition
+DB.create_table :workflows do
+  primary_key :id
+  String :name, null: false
+  String :description
+end
+
+DB.create_table :sections do
+  primary_key :id
+  String :name, null: false
+  String :description
+  Integer :order
+  foreign_key :workflow_id, :workflows
+end
+
+DB.create_table :steps do
+  primary_key :id
+  String :name, null: false
+  String :description
+  Integer :order
+  foreign_key :section_id, :sections
+  String :prompt
+end
+
+DB.create_table :attributes do
+  primary_key :id
+  String :name, null: false
+  String :type
+  Boolean :is_array
+  foreign_key :workflow_id, :workflows
+end
+
+DB.create_table :input_attributes do
+  foreign_key :step_id, :steps
+  foreign_key :attribute_id, :attributes
+end
+
+DB.create_table :output_attributes do
+  foreign_key :step_id, :steps
+  foreign_key :attribute_id, :attributes
+end
+
+# Create the tables for dynamic workflow execution
+DB.create_table :workflow_runs do
+  primary_key :id
+  foreign_key :workflow_id, :workflows
+end
+
+DB.create_table :section_runs do
+  primary_key :id
+  foreign_key :workflow_run_id, :workflow_runs
+  foreign_key :section_id, :sections
+end
+
+DB.create_table :step_runs do
+  primary_key :id
+  foreign_key :section_run_id, :section_runs
+  foreign_key :step_id, :steps
+  Integer :branch_number
+end
+
+DB.create_table :attribute_values do
+  primary_key :id
+  foreign_key :attribute_id, :attributes
+  foreign_key :step_run_id, :step_runs
+  String :value
+end
+```
+
+## Models
+
+```ruby
+# models.rb
+
+require 'sequel'
+
+DB = Sequel.sqlite('workflow.db') # Connect to the SQLite database
+
+# Static Workflow Definition
+
+class Workflow < Sequel::Model
+  one_to_many :sections
+  one_to_many :attributes
+  one_to_many :workflow_runs
+end
+
+class Section < Sequel::Model
+  many_to_one :workflow
+  one_to_many :steps
+  one_to_many :section_runs
+end
+
+class Step < Sequel::Model
+  many_to_one :section
+  one_to_many :input_attributes, class: :StepInputAttribute
+  one_to_many :output_attributes, class: :StepOutputAttribute
+  one_to_many :step_runs
+end
+
+class Attribute < Sequel::Model
+  many_to_one :workflow
+  one_to_many :input_attributes, class: :StepInputAttribute
+  one_to_many :output_attributes, class: :StepOutputAttribute
+end
+
+class StepInputAttribute < Sequel::Model
+  many_to_one :step
+  many_to_one :attribute
+end
+
+class StepOutputAttribute < Sequel::Model
+  many_to_one :step
+  many_to_one :attribute
+end
+
+# Dynamic Workflow Execution
+
+class WorkflowRun < Sequel::Model
+  many_to_one :workflow
+  one_to_many :section_runs
+end
+
+class SectionRun < Sequel::Model
+  many_to_one :workflow_run
+  many_to_one :section
+  one_to_many :step_runs
+end
+
+class StepRun < Sequel::Model
+  many_to_one :section_run
+  many_to_one :step
+  one_to_many :attribute_values
+end
+
+class AttributeValue < Sequel::Model
+  many_to_one :attribute
+  many_to_one :step_run
+end
+
+```
+
+## Usage via Models
+
+```ruby
+# example_usage.rb
+
+require_relative 'models'
+
+# Create a new workflow
+workflow = Workflow.create(name: "YouTube Video Script", description: "Workflow for creating a YouTube video script")
+
+# Create sections for the workflow
+section1 = Section.create(name: "Research", description: "Research phase", order: 1, workflow: workflow)
+section2 = Section.create(name: "Script Writing", description: "Script writing phase", order: 2, workflow: workflow)
+
+# Create steps for the sections
+step1 = Step.create(name: "01-1-basic-meta", description: "Basic metadata for Fotor AI tool", order: 1, section: section1, prompt: "Generate 5 titles to get started")
+step2 = Step.create(name: "01-2-basic-factsheet", description: "Detailed factsheet about Fotor AI Tool", order: 2, section: section1, prompt: "Generate detailed factsheet")
+
+# Create attributes
+attribute1 = Attribute.create(name: "simple_title", type: "string", is_array: false, workflow: workflow)
+attribute2 = Attribute.create(name: "basic_factsheet", type: "string", is_array: false, workflow: workflow)
+
+# Associate input and output attributes with steps
+StepInputAttribute.create(step: step1, attribute: attribute1)
+StepOutputAttribute.create(step: step1, attribute: attribute2)
+
+# Create a workflow run
+workflow_run = WorkflowRun.create(workflow: workflow)
+
+# Create section runs
+section_run1 = SectionRun.create(workflow_run: workflow_run, section: section1)
+section_run2 = SectionRun.create(workflow_run: workflow_run, section: section2)
+
+# Create step runs
+step_run1 = StepRun.create(section_run: section_run1, step: step1, branch_number: 1)
+step_run2 = StepRun.create(section_run: section_run1, step: step2, branch_number: 1)
+
+# Add attribute values to step runs
+AttributeValue.create(attribute: attribute1, step_run: step_run1, value: "Fotor AI tool")
+AttributeValue.create(attribute: attribute2, step_run: step_run2, value: "Detailed factsheet about Fotor AI Tool")
+```
+
+## Exporting to JSON
+
+```ruby
+# export_to_json.rb
+
+require 'json'
+require_relative 'models'
+
+# Export all workflows to JSON files
+Workflow.all.each do |workflow|
+  workflow_data = {
+    id: workflow.id,
+    name: workflow.name,
+    description: workflow.description,
+    sections: workflow.sections.map do |section|
+      {
+        id: section.id,
+        name: section.name,
+        description: section.description,
+        order: section.order,
+        steps: section.steps.map do |step|
+          {
+            id: step.id,
+            name: step.name,
+            description: step.description,
+            order: step.order,
+            prompt: step.prompt,
+            input_attributes: step.input_attributes.map { |ia| { id: ia.attribute.id, name: ia.attribute.name } },
+            output_attributes: step.output_attributes.map { |oa| { id: oa.attribute.id, name: oa.attribute.name } }
+          }
+        end
+      }
+    end,
+    attributes: workflow.attributes.map { |attr| { id: attr.id, name: attr.name, type: attr.type, is_array: attr.is_array } }
+  }
+
+  File.write("workflow_#{workflow.id}.json", JSON.pretty_generate(workflow_data))
+end
+
+# Export all workflow runs to JSON files
+WorkflowRun.all.each do |workflow_run|
+  workflow_run_data = {
+    id: workflow_run.id,
+    workflow_id: workflow_run.workflow_id,
+    section_runs: workflow_run.section_runs.map do |section_run|
+      {
+        id: section_run.id,
+        section_id: section_run.section_id,
+        step_runs: section_run.step_runs.map do |step_run|
+          {
+            id: step_run.id,
+            step_id: step_run.step_id,
+            branch_number: step_run.branch_number,
+            attribute_values: step_run.attribute_values.map { |av| { id: av.attribute.id, name: av.attribute.name, value: av.value } }
+          }
+        end
+      }
+    end
+  }
+
+  File.write("workflow_run_#{workflow_run.id}.json", JSON.pretty_generate(workflow_run_data))
+end
+```
+
+## Importing from JSON
+
+```ruby
+# import_from_json.rb
+
+require 'json'
+require_relative 'models'
+
+# Helper function to find or create an attribute
+def find_or_create_attribute(attr_data, workflow)
+  Attribute.find_or_create(name: attr_data['name'], workflow: workflow) do |attribute|
+    attribute.type = attr_data['type']
+    attribute.is_array = attr_data['is_array']
+  end
+end
+
+# Restore workflows from JSON files
+Dir.glob('workflow_*.json').each do |file|
+  data = JSON.parse(File.read(file))
+  
+  workflow = Workflow.find_or_create(id: data['id']) do |w|
+    w.name = data['name']
+    w.description = data['description']
+  end
+
+  data['sections'].each do |section_data|
+    section = Section.find_or_create(id: section_data['id'], workflow: workflow) do |s|
+      s.name = section_data['name']
+      s.description = section_data['description']
+      s.order = section_data['order']
+    end
+
+    section_data['steps'].each do |step_data|
+      step = Step.find_or_create(id: step_data['id'], section: section) do |s|
+        s.name = step_data['name']
+        s.description = step_data['description']
+        s.order = step_data['order']
+        s.prompt = step_data['prompt']
+      end
+
+      step_data['input_attributes'].each do |attr_data|
+        attribute = find_or_create_attribute(attr_data, workflow)
+        StepInputAttribute.find_or_create(step: step, attribute: attribute)
+      end
+
+      step_data['output_attributes'].each do |attr_data|
+        attribute = find_or_create_attribute(attr_data, workflow)
+        StepOutputAttribute.find_or_create(step: step, attribute: attribute)
+      end
+    end
+  end
+
+  data['attributes'].each do |attr_data|
+    find_or_create_attribute(attr_data, workflow)
+  end
+end
+
+# Restore workflow runs from JSON files
+Dir.glob('workflow_run_*.json').each do |file|
+  data = JSON.parse(File.read(file))
+
+  workflow_run = WorkflowRun.find_or_create(id: data['id'], workflow_id: data['workflow_id'])
+
+  data['section_runs'].each do |section_run_data|
+    section_run = SectionRun.find_or_create(id: section_run_data['id'], workflow_run: workflow_run, section_id: section_run_data['section_id'])
+
+    section_run_data['step_runs'].each do |step_run_data|
+      step_run = StepRun.find_or_create(id: step_run_data['id'], section_run: section_run, step_id: step_run_data['step_id'], branch_number: step_run_data['branch_number'])
+
+      step_run_data['attribute_values'].each do |attr_value_data|
+        attribute = Attribute.find(id: attr_value_data['id'])
+        AttributeValue.find_or_create(attribute: attribute, step_run: step_run) do |av|
+          av.value = attr_value_data['value']
+        end
+      end
+    end
+  end
+end
+```
+
+## Sample Implementation of Workflow Builder (DSL)
+
+```ruby
+require 'yaml'
+require 'sequel'
+
+# Assuming the database and models are already set up
+DB = Sequel.sqlite('workflow.db')
+
+# Models
+class Workflow < Sequel::Model
+  one_to_many :sections
+  one_to_many :attributes
+  one_to_many :workflow_runs
+end
+
+class Section < Sequel::Model
+  many_to_one :workflow
+  one_to_many :steps
+  one_to_many :section_runs
+end
+
+class Step < Sequel::Model
+  many_to_one :section
+  one_to_many :input_attributes, class: :StepInputAttribute
+  one_to_many :output_attributes, class: :StepOutputAttribute
+  one_to_many :step_runs
+end
+
+class Attribute < Sequel::Model
+  many_to_one :workflow
+  one_to_many :input_attributes, class: :StepInputAttribute
+  one_to_many :output_attributes, class: :StepOutputAttribute
+end
+
+class StepInputAttribute < Sequel::Model
+  many_to_one :step
+  many_to_one :attribute
+end
+
+class StepOutputAttribute < Sequel::Model
+  many_to_one :step
+  many_to_one :attribute
+end
+
+# WorkflowBuilder
+class WorkflowBuilder
+  def initialize(name:, description: nil)
+    @workflow = Workflow.new(name: name, description: description)
+    @current_section_order = 1
+  end
+
+  def description(desc)
+    @workflow.description = desc
+  end
+
+  def section(name:, &block)
+    @current_step_order = 1
+    @current_section = Section.new(name: name, order: @current_section_order)
+    @current_section_order += 1
+    instance_eval(&block) if block_given?
+    @workflow.add_section(@current_section)
+  end
+
+  def step(name:, &block)
+    step = Step.new(name: name, order: @current_step_order)
+    @current_step_order += 1
+    instance_eval(&block) if block_given?
+    @current_section.add_step(step)
+  end
+
+  def prompt(prompt)
+    @current_section.steps.last.prompt = prompt
+  end
+
+  def save
+    @workflow.save
+  end
+
+  def to_yaml
+    workflow_data = {
+      id: @workflow.id,
+      name: @workflow.name,
+      description: @workflow.description,
+      sections: @workflow.sections.map do |section|
+        {
+          id: section.id,
+          name: section.name,
+          description: section.description,
+          order: section.order,
+          steps: section.steps.map do |step|
+            {
+              id: step.id,
+              name: step.name,
+              description: step.description,
+              order: step.order,
+              prompt: step.prompt
+            }
+          end
+        }
+      end
+    }
+    workflow_data.to_yaml
+  end
+end
+
+```
+
+## Usage via DSL
+
+```ruby
+builder = WorkflowBuilder.new(name: 'YouTube Video Script')
+
+builder.section(name: 'Research') do
+  step(name: '01-1-basic-meta') do
+    prompt 'Generate 5 titles to get started'
+  end
+
+  step(name: '01-2-basic-factsheet') do
+    prompt 'Generate detailed factsheet'
+  end
+end
+
+builder.section(name: 'Script Writing') do
+  step(name: '02-1-create-script') do
+    prompt 'Generate basic script'
+  end
+end
+
+puts builder.to_yaml
+
+# Save to the database
+builder.save
+
+
+builder = WorkflowBuilder.new(name: 'YouTube Title Creator')
+
+builder.section(name: 'Research') do
+  step(name: '01-1-working-title') do
+    prompt 'Come up with a simple working title for the YouTube video.'
+  end
+
+  step(name: '01-2-keyword-research') do
+    prompt 'Perform basic keyword research to identify relevant keywords.'
+  end
+
+  step(name: '01-3-topic-research') do
+    prompt 'Conduct basic topic research to gather information on the subject.'
+  end
+
+  step(name: '01-4-powerful-titles') do
+    prompt 'Ask GPT for 10 powerful titles based on the research.'
+  end
+
+  step(name: '01-5-title-rules') do
+    prompt 'Follow specific rules for title creation to ensure effectiveness.'
+  end
+end
+
+puts builder.to_yaml
+
+# Save to the database
+builder.save
+
+```

data/docs/structure-youtube-script.yaml

@@ -0,0 +1,177 @@
+workflows:
+  - id: 1
+    name: "YouTube Video Script"
+    description: "Workflow for creating a YouTube video script"
+
+sections:
+  - id: 1
+    name: "Research"
+    description: "Research phase of the workflow"
+    order: 1
+    workflow_id: 1
+  - id: 2
+    name: "Script Writing"
+    description: "Script writing phase of the workflow"
+    order: 2
+    workflow_id: 1
+
+steps:
+  - id: 1
+    name: "01-1-basic-meta"
+    description: "Basic metadata for Fotor AI tool"
+    order: 1
+    section_id: 1
+    prompt: "Generate 5 titles to get started"
+  - id: 2
+    name: "01-2-basic-factsheet"
+    description: "Detailed factsheet about Fotor AI Tool"
+    order: 2
+    section_id: 1
+    prompt: "Generate detailed factsheet"
+  - id: 3
+    name: "01-3-video-type"
+    description: "Identify video types"
+    order: 3
+    section_id: 1
+    prompt: "Identify video types based on basic factsheet"
+  - id: 4
+    name: "01-4-expanded-factsheet"
+    description: "Expanded factsheet about Fotor AI Tool"
+    order: 4
+    section_id: 1
+    prompt: "Generate expanded factsheet"
+  - id: 5
+    name: "01-5-expanded-meta"
+    description: "Expanded metadata for Fotor AI Tool"
+    order: 5
+    section_id: 1
+    prompt: "Generate engaging titles, keywords, and topics"
+  - id: 6
+    name: "02-1-create-script"
+    description: "Create basic script"
+    order: 1
+    section_id: 2
+    prompt: "Generate basic script"
+  - id: 7
+    name: "02-2-clean-transcript"
+    description: "Clean and revise transcript"
+    order: 2
+    section_id: 2
+    prompt: "Generate basic transcript"
+  - id: 8
+    name: "02-3-transcript-factchecked"
+    description: "Fact-check and revise transcript"
+    order: 3
+    section_id: 2
+    prompt: "Generate fact-checked transcript"
+
+attributes:
+  - id: 1
+    name: "simple_title"
+    type: "string"
+    is_array: false
+    workflow_id: 1
+  - id: 2
+    name: "basic_factsheet"
+    type: "string"
+    is_array: false
+    workflow_id: 1
+  - id: 3
+    name: "focus_video_type"
+    type: "string"
+    is_array: false
+    workflow_id: 1
+  - id: 4
+    name: "expanded_factsheet"
+    type: "string"
+    is_array: false
+    workflow_id: 1
+  - id: 5
+    name: "video_types"
+    type: "string"
+    is_array: true
+    workflow_id: 1
+  - id: 6
+    name: "engaging_titles"
+    type: "string"
+    is_array: true
+    workflow_id: 1
+  - id: 7
+    name: "keywords"
+    type: "string"
+    is_array: true
+    workflow_id: 1
+  - id: 8
+    name: "topics"
+    type: "string"
+    is_array: true
+    workflow_id: 1
+  - id: 9
+    name: "basic_script"
+    type: "string"
+    is_array: false
+    workflow_id: 1
+  - id: 10
+    name: "basic_transcript"
+    type: "string"
+    is_array: false
+    workflow_id: 1
+  - id: 11
+    name: "transcript_qa"
+    type: "string"
+    is_array: false
+    workflow_id: 1
+
+input_attributes:
+  - step_id: 1
+    attribute_id: 1
+  - step_id: 2
+    attribute_id: 1
+  - step_id: 3
+    attribute_id: 1
+  - step_id: 3
+    attribute_id: 2
+  - step_id: 4
+    attribute_id: 1
+  - step_id: 4
+    attribute_id: 2
+  - step_id: 4
+    attribute_id: 3
+  - step_id: 5
+    attribute_id: 1
+  - step_id: 5
+    attribute_id: 3
+  - step_id: 5
+    attribute_id: 4
+  - step_id: 6
+    attribute_id: 1
+  - step_id: 6
+    attribute_id: 3
+  - step_id: 6
+    attribute_id: 4
+  - step_id: 7
+    attribute_id: 9
+  - step_id: 8
+    attribute_id: 10
+
+output_attributes:
+  - step_id: 1
+    attribute_id: 6
+  - step_id: 2
+    attribute_id: 2
+  - step_id: 3
+    attribute_id: 5
+  - step_id: 4
+    attribute_id: 4
+  - step_id: 5
+    attribute_id: 6
+  - step_id: 5
+    attribute_id: 7
+  - step_id: 5
+    attribute_id: 8
+  - step_id: 6
+    attribute_id: 9
+  - step_id: 7
+    attribute_id: 10
+  - step_id: 8
+    attribute_id: 11

data/lib/ad/agent_architecture/db/create_schema.rb

@@ -0,0 +1,159 @@
+# frozen_string_literal: true
+
+# Usage
+# DB = Sequel.sqlite # In-memory database
+
+# schema_creator = Ad::AgentArchitecture::Database::CreateSchema.new(DB)
+# schema_creator.execute
+module Ad
+  module AgentArchitecture
+    module Database
+      # Create the database schema
+      class CreateSchema
+        attr_reader :db
+
+        def initialize(db)
+          @db = db
+        end
+
+        def execute
+          create_tables
+          define_models
+        end
+
+        private
+
+        def create_tables
+          db.create_table :workflows do
+            primary_key :id
+            String :name, null: false
+            String :description
+          end
+
+          db.create_table :sections do
+            primary_key :id
+            foreign_key :workflow_id, :workflows
+            String :name, null: false
+            String :description
+            Integer :order
+          end
+
+          db.create_table :steps do
+            primary_key :id
+            foreign_key :section_id, :sections
+            String :name, null: false
+            String :action, default: 'gpt'
+            String :description
+            Integer :order
+            String :prompt
+          end
+
+          db.create_table :attributes do
+            primary_key :id
+            foreign_key :workflow_id, :workflows
+            String :name, null: false
+            String :type
+            Boolean :is_array
+          end
+
+          db.create_table :input_attributes do
+            primary_key :id
+            foreign_key :step_id, :steps
+            foreign_key :attribute_id, :attributes
+            Boolean :required
+          end
+
+          db.create_table :output_attributes do
+            primary_key :id
+            foreign_key :step_id, :steps
+            foreign_key :attribute_id, :attributes
+          end
+
+          db.create_table :workflow_runs do
+            primary_key :id
+            foreign_key :workflow_id, :workflows
+          end
+
+          db.create_table :section_runs do
+            primary_key :id
+            foreign_key :workflow_run_id, :workflow_runs
+            foreign_key :section_id, :sections
+          end
+
+          db.create_table :step_runs do
+            primary_key :id
+            foreign_key :section_run_id, :section_runs
+            foreign_key :step_id, :steps
+            Integer :branch_number
+          end
+
+          db.create_table :attribute_values do
+            primary_key :id
+            foreign_key :attribute_id, :attributes
+            foreign_key :step_run_id, :step_runs
+            String :value
+          end
+        end
+
+        def define_models
+          Class.new(Sequel::Model(:workflows)) do
+            one_to_many :sections
+            one_to_many :attributes
+            one_to_many :workflow_runs
+          end
+
+          Class.new(Sequel::Model(:sections)) do
+            many_to_one :workflow
+            one_to_many :steps
+            one_to_many :section_runs
+          end
+
+          Class.new(Sequel::Model(:steps)) do
+            many_to_one :section
+            one_to_many :input_attributes, class: :InputAttribute
+            one_to_many :output_attributes, class: :OutputAttribute
+            one_to_many :step_runs
+          end
+
+          Class.new(Sequel::Model(:attributes)) do
+            many_to_one :workflow
+            one_to_many :input_attributes, class: :InputAttribute
+            one_to_many :output_attributes, class: :OutputAttribute
+          end
+
+          Class.new(Sequel::Model(:input_attributes)) do
+            many_to_one :step
+            many_to_one :attribute
+          end
+
+          Class.new(Sequel::Model(:output_attributes)) do
+            many_to_one :step
+            many_to_one :attribute
+          end
+
+          Class.new(Sequel::Model(:workflow_runs)) do
+            many_to_one :workflow
+            one_to_many :section_runs
+          end
+
+          Class.new(Sequel::Model(:section_runs)) do
+            many_to_one :workflow_run
+            many_to_one :section
+            one_to_many :step_runs
+          end
+
+          Class.new(Sequel::Model(:step_runs)) do
+            many_to_one :section_run
+            many_to_one :step
+            one_to_many :attribute_values
+          end
+
+          Class.new(Sequel::Model(:attribute_values)) do
+            many_to_one :attribute
+            many_to_one :step_run
+          end
+        end
+      end
+    end
+  end
+end

data/lib/ad/agent_architecture/db/models.rb

@@ -0,0 +1,76 @@
+# frozen_string_literal: true
+
+require 'sequel'
+
+module Ad
+  module AgentArchitecture
+    module Database
+      # Workflow model represents a workflow entity in the database.
+      class Workflow < Sequel::Model
+        one_to_many :sections, class: 'Ad::AgentArchitecture::Database::Section'
+        one_to_many :attributes, class: 'Ad::AgentArchitecture::Database::Attribute'
+        one_to_many :workflow_runs, class: 'Ad::AgentArchitecture::Database::WorkflowRun'
+      end
+
+      # Section model represents a section entity in the database.
+      class Section < Sequel::Model
+        many_to_one :workflow, class: 'Ad::AgentArchitecture::Database::Workflow'
+        one_to_many :steps, class: 'Ad::AgentArchitecture::Database::Step'
+        one_to_many :section_runs, class: 'Ad::AgentArchitecture::Database::SectionRun'
+      end
+
+      # Step model represents a step entity in the database.
+      class Step < Sequel::Model
+        many_to_one :section, class: 'Ad::AgentArchitecture::Database::Section'
+        one_to_many :input_attributes, class: 'Ad::AgentArchitecture::Database::InputAttribute'
+        one_to_many :output_attributes, class: 'Ad::AgentArchitecture::Database::OutputAttribute'
+        one_to_many :step_runs, class: 'Ad::AgentArchitecture::Database::StepRun'
+      end
+
+      # Attribute model represents an attribute entity in the database.
+      class Attribute < Sequel::Model
+        many_to_one :workflow, class: 'Ad::AgentArchitecture::Database::Workflow'
+        one_to_many :input_attributes, class: 'Ad::AgentArchitecture::Database::InputAttribute'
+        one_to_many :output_attributes, class: 'Ad::AgentArchitecture::Database::OutputAttribute'
+      end
+
+      # InputAttribute model represents an input attribute entity in the database.
+      class InputAttribute < Sequel::Model
+        many_to_one :step, class: 'Ad::AgentArchitecture::Database::Step'
+        many_to_one :attribute, class: 'Ad::AgentArchitecture::Database::Attribute'
+      end
+
+      # OutputAttribute model represents an output attribute entity in the database.
+      class OutputAttribute < Sequel::Model
+        many_to_one :step, class: 'Ad::AgentArchitecture::Database::Step'
+        many_to_one :attribute, class: 'Ad::AgentArchitecture::Database::Attribute'
+      end
+
+      # WorkflowRun model represents a workflow run entity in the database.
+      class WorkflowRun < Sequel::Model
+        many_to_one :workflow, class: 'Ad::AgentArchitecture::Database::Workflow'
+        one_to_many :section_runs, class: 'Ad::AgentArchitecture::Database::SectionRun'
+      end
+
+      # SectionRun model represents a section run entity in the database.
+      class SectionRun < Sequel::Model
+        many_to_one :workflow_run, class: 'Ad::AgentArchitecture::Database::WorkflowRun'
+        many_to_one :section, class: 'Ad::AgentArchitecture::Database::Section'
+        one_to_many :step_runs, class: 'Ad::AgentArchitecture::Database::StepRun'
+      end
+
+      # StepRun model represents a step run entity in the database.
+      class StepRun < Sequel::Model
+        many_to_one :section_run, class: 'Ad::AgentArchitecture::Database::SectionRun'
+        many_to_one :step, class: 'Ad::AgentArchitecture::Database::Step'
+        one_to_many :attribute_values, class: 'Ad::AgentArchitecture::Database::AttributeValue'
+      end
+
+      # AttributeValue model represents an attribute value entity in the database.
+      class AttributeValue < Sequel::Model
+        many_to_one :attribute, class: 'Ad::AgentArchitecture::Database::Attribute'
+        many_to_one :step_run, class: 'Ad::AgentArchitecture::Database::StepRun'
+      end
+    end
+  end
+end

data/lib/ad/agent_architecture/version.rb

@@ -2,6 +2,6 @@
 
 module Ad
   module AgentArchitecture
-    VERSION = '0.0.4'
+    VERSION = '0.0.6'
   end
 end

data/lib/ad/agent_architecture.rb

@@ -1,6 +1,16 @@
 # frozen_string_literal: true
 
+require 'sequel'
+require 'sqlite3'
+
 require 'ad/agent_architecture/version'
+require 'ad/agent_architecture/db/create_schema'
+
+DB = Sequel.sqlite
+
+Ad::AgentArchitecture::Database::CreateSchema.new(DB).execute
+
+require 'ad/agent_architecture/db/models'
 
 module Ad
   module AgentArchitecture

data/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "ad-agent_architecture",
-  "version": "0.0.4",
+  "version": "0.0.6",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "ad-agent_architecture",
-      "version": "0.0.4",
+      "version": "0.0.6",
       "devDependencies": {
         "@klueless-js/semantic-release-rubygem": "github:klueless-js/semantic-release-rubygem",
         "@semantic-release/changelog": "^6.0.3",

data/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ad-agent_architecture",
-  "version": "0.0.4",
+  "version": "0.0.6",
   "description": "Architecture/Schema for AI Agents",
   "scripts": {
     "release": "semantic-release"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ad-agent_architecture
 version: !ruby/object:Gem::Version
-  version: 0.0.4
+  version: 0.0.6
 platform: ruby
 authors:
 - David Cruwys
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2024-06-26 00:00:00.000000000 Z
+date: 2024-06-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: k_log
@@ -24,6 +24,34 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: sequel
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2'
 description: "    Architecture/Schema for AI Agents\n"
 email:
 - david@ideasmen.com.au
@@ -48,8 +76,12 @@ files:
 - bin/console
 - bin/setup
 - docs/erd.svg
+- docs/images/sample-workflow.png
 - docs/requirements.md
+- docs/structure-youtube-script.yaml
 - lib/ad/agent_architecture.rb
+- lib/ad/agent_architecture/db/create_schema.rb
+- lib/ad/agent_architecture/db/models.rb
 - lib/ad/agent_architecture/version.rb
 - package-lock.json
 - package.json