janito 0.5.0__py3-none-any.whl → 0.7.0__py3-none-any.whl

janito/clear_statement_parser/clear_statement_format.txt

@@ -0,0 +1,328 @@
+# CLEAR STATEMENT FORMAT
+
+Version: 1.9.8
+
+## OVERVIEW
+
+This document provides a format for clear, unambiguous statements that can be reliably interpreted by both humans and computers. The format emphasizes literal interpretation and straightforward parsing rules to ensure consistent understanding across different implementations and use cases.
+
+## FUNDAMENTAL PRINCIPLES
+
+1. Whitespace Significance
+   - Whitespace only has meaning when it appears between two symbols in the same line
+   - Leading and trailing whitespace on lines has no meaning
+   - Empty lines (containing only whitespace) are ignored
+   - Multiple consecutive whitespace characters are treated the same as a single whitespace
+   - Indentation is for human readability only and has no semantic meaning
+
+2. Keep It Literal
+   - Content is interpreted exactly as written without escape sequences
+   - Special characters are marked by their position (e.g., leading dot for literal blocks) rather than by escaping
+   - No character escaping mechanism is provided or needed
+   - Line endings are normalized but not escaped
+   - What you see is what you get - the format avoids hidden or transformed characters
+
+## FORMAT ELEMENTS
+
+### 1. Basic Line Types
+
+The following symbols are special: # . : - /, particularly when wrapping lines.
+
+Line type is determined by these rules in order, once there is a match the type is set:
+
+1. All lines are stripped of leading whitespace and trailing whitespace/end-of-line symbols (\n, \r, \r\n), parsing is based on line content, not indentation
+2. Empty Lines: Lines that are completely empty or consist only of whitespace are skipped
+3. Comment: Lines starting with # are treated as comments and ignored
+4. List Item: Lines beginning with one or more - characters
+5. Statement: Lines containing only alphanumeric characters and spaces
+6. Literal Block: Lines beginning with ., terminating when a non-dot-prefixed line is encountered (excluding comments) or when the document ends
+7. Key/Value: Lines containing : are treated as key/value pairs
+8. Block Marker: Lines matching one of these patterns:
+   - Block Start: Line begins with / followed immediately by an alphanumeric block name (e.g., /BlockName)
+   - Block End: Line consists of an alphanumeric block name followed immediately by / (e.g., BlockName/)
+
+Note: The order of these rules is significant. A line's type is determined by the first matching rule.
+
+### 1.1 Block Marker Syntax Rules
+
+Block markers follow these specific rules:
+1. Start marker must be:
+   - A forward slash (/)
+   - Immediately followed by an alphanumeric block name
+   - No whitespace between slash and name
+   - Examples: /Environment, /Database, /Service
+
+2. End marker must be:
+   - An alphanumeric block name
+   - Immediately followed by a forward slash (/)
+   - Must match exactly the name used in the corresponding start marker
+   - No whitespace between name and slash
+   - Examples: Environment/, Database/, Service/
+
+3. Block Naming Conventions:
+   - Use nouns or noun phrases to name blocks
+   - Names should describe the scope or category of contained statements
+   - Use PascalCase for multi-word names
+   - Examples:
+     - Good: /Configuration, /Network, /SecurityPolicy
+     - Avoid: /Configure, /Running, /ProcessItems
+
+4. Invalid block markers:
+   - / (slash alone)
+   - /block name (contains space)
+   - / BlockName (contains space)
+   - BlockName / (contains space)
+   - block_name/ (contains underscore)
+   - /block-name (contains hyphen)
+
+### 1.2 Statement Naming Conventions
+
+Statements follow these naming conventions:
+1. Statement Structure:
+   - Use verb phrases that clearly describe actions
+   - Start with an imperative verb
+   - Follow with appropriate objects or context
+   - Examples: Create File, Deploy Application, Update Configuration
+
+2. Verb Usage:
+   - Use precise, specific verbs
+   - Choose verbs that clearly indicate the action's purpose
+   - Examples:
+     - Good: Deploy Service (specific action)
+     - Avoid: Do Service (vague action)
+     - Good: Create Database (clear intent)
+     - Avoid: Make Database (imprecise)
+
+3. Naming Format:
+   - Use Title Case for statement names
+   - Separate words with single spaces
+   - Keep names concise but descriptive
+   - Examples:
+     - Good: Install Dependencies
+     - Good: Configure Network Settings
+     - Good: Create Backup Schedule
+     - Avoid: installDependencies (wrong format)
+     - Avoid: INSTALL_DEPENDENCIES (wrong format)
+     - Avoid: Install the System Dependencies for the Application (too verbose)
+
+4. Common Statement Verbs:
+   - Creation: Create, Generate, Initialize
+   - Modification: Update, Modify, Configure
+   - Removal: Delete, Remove, Clear
+   - Deployment: Deploy, Install, Provision
+   - Status: Check, Validate, Verify
+
+### 1.3 Statement and Block Name Relationships
+
+When using statements within blocks, maintain clear relationships:
+1. Blocks use nouns describing scope/category (see section 1.1)
+2. Statements use verb phrases describing actions
+3. Together they form clear action-scope pairs:
+
+```
+/Database
+    Create Table
+        name: users
+    Update Schema
+        version: 2
+Database/
+
+/SecurityPolicy
+    Configure Rules
+        type: firewall
+    Enable Monitoring
+        level: high
+SecurityPolicy/
+```
+
+### Line Type Priority Examples
+
+The following examples demonstrate how the order of rules determines line type:
+
+```
+# key: value          -> Rule 3: Comment (despite containing a colon)
+#.literal content     -> Rule 3: Comment (despite starting with dot after #)
+#- item              -> Rule 3: Comment (despite starting with dash after #)
+# /BlockName          -> Rule 3: Comment (despite looking like block start)
+#BlockName/           -> Rule 3: Comment (despite looking like block end)
+- key: value         -> Rule 4: List Item (despite containing a colon)
+-.literal content    -> Rule 4: List Item (despite containing a dot)
+- /BlockName          -> Rule 4: List Item (despite containing block start syntax)
+- BlockName/          -> Rule 4: List Item (despite containing block end syntax)
+.key: value          -> Rule 6: Literal Block (despite containing a colon)
+.# not a comment     -> Rule 6: Literal Block (despite containing #)
+.- not a list        -> Rule 6: Literal Block (despite containing -)
+./BlockName           -> Rule 6: Literal Block (despite containing block start syntax)
+.BlockName/           -> Rule 6: Literal Block (despite containing block end syntax)
+key: /BlockName       -> Rule 7: Key/Value (despite containing block start syntax)
+key: BlockName/       -> Rule 7: Key/Value (despite containing block end syntax)
+/BlockName            -> Rule 8: Block Start Marker
+BlockName/            -> Rule 8: Block End Marker
+```
+
+### 2. Statements Format
+
+A statement is a single line that represents an action or instruction, followed by parameters that provide additional context or data for the statement. Parameters can be key-value pairs, list items, literal block lines, or statement blocks. Statement blocks can be freely mixed with other parameter types, while key-value pairs, list items, and literal blocks cannot be mixed with each other.
+
+#### 2.1 Statement Parameters
+
+Parameters must follow these rules:
+- Base parameter types are mutually exclusive:
+  - A series of key/value lines (with enhanced support for literal blocks or lists as values)
+  - A sequence of list items, where each item is a single line of text
+  - A literal block, where lines are preserved exactly as written including whitespace and formatting
+- Statement blocks can be freely mixed with any other parameter type
+- Statement blocks can contain their own parameters following these same rules
+
+#### 2.2 Enhanced Key/Value Lines
+
+- Keys within a scope must be unique:
+  - All keys within a single statement's parameters must be unique
+  - All keys within the same nesting level of a list must be unique
+  - Keys can be reused in different scopes (different statements or different list items)
+- Values:
+  - If the value is inline (non-empty), it must remain on a single line
+  - If the value is empty (e.g., key:), either a literal block or a list must follow:
+    - Literal Block: Lines starting with . must immediately follow the key and will be treated as the value for the key. The literal block terminates when a non-dot-prefixed line is encountered (except comments) or when the document finishes. All lines are joined using \n after stripping the leading dot.
+    - List: Lines starting with - must immediately follow the key and will be treated as a list for the key. Nesting rules for lists apply.
+  - Mixing Error: Mixing literal blocks and lists under the same key is not allowed and must result in a syntax error.
+
+##### 2.2.1 Literal Blocks
+
+- A leading dot is required for each line of a literal block
+- The content after the leading dot is preserved exactly as written
+- The literal block terminates when:
+  - A line without a leading dot (excluding comments) is encountered, or
+  - The document finishes
+- Join all lines with \n to form a single string value
+- Literal blocks are stored as strings
+- All whitespace and characters after the leading dot are preserved exactly as written
+- The dot prefix allows literal blocks to contain content that would otherwise be interpreted as comments or other special line types
+
+##### 2.2.2 Lists
+
+- List items are marked by one or more leading dash characters (-) to indicate nesting level
+- Nesting Levels: 
+  - Determined by the number of consecutive dash characters (-, --, ---)
+  - Maximum depth of 5 levels
+  - Each level must use exactly one more dash than its parent
+- List Item Content:
+  - Each list item must contain exactly one line of text
+  - The text starts immediately after the dashes and any whitespace
+  - No continuation lines are allowed
+  - No empty items are allowed
+- List Storage: Lists are stored as a list type within the dictionary under the corresponding key they follow
+
+##### 2.2.3 Mixing Rules
+
+- A key cannot have both a literal block and a list as its value
+
+#### 2.3 Statement Blocks
+
+Statement blocks are a powerful mechanism for organizing complex, nested instructions within a single statement. They provide a structured way to group related sub-operations, transformations, or configurations.
+
+Key Characteristics:
+- Blocks create a hierarchical, nested structure for organizing instructions
+- Block markers must follow the syntax rules defined in section 1.1
+- Block names can be duplicated within the same level to represent parallel or repeated operations
+- Allow nested blocks by providing new blocks within inner statement parameters
+- Maximum nesting depth of 10 levels
+- Enables complex, multi-step processes within a single statement
+- A block name cannot be the same as its direct parent block name
+
+Block Rules:
+- Statement block names must be alphanumeric
+- Multiple blocks with the same name are allowed within the same level to represent parallel or repeated operations
+- A block name cannot be the same as its direct parent block name
+- Block names can be reused at any level except as a direct child of a block with the same name
+- Content: Must consist of statements, where each statement can have parameters in the form of key/value pairs, literal blocks, lists, or nested blocks
+- Nesting: Maximum depth of 10 levels (not counting the initial statement level)
+- Empty Blocks: Allowed
+- Scope: Each block creates a new scope for key uniqueness
+
+### 3. Error Handling
+
+- Errors must include the following details:
+  - Statement Context: The statement that caused the error
+  - Line Number: The location of the error
+  - Error Message: Descriptive message explaining the issue
+- Errors should be reported in plain text format for simplicity, with one error per line
+- Uniqueness violations must be reported with clear scope context
+
+### 4. Examples
+
+#### Statement Without Parameters:
+Clear All Cache
+
+#### Statement with Key/Value and Literal Block:
+Create New File
+    name: test.py
+    content:
+    .def greet():
+    .    print("Hello")
+
+#### Statement with Lists:
+Update Menu Items
+    categories:
+    - Main Course
+    -- Pasta
+    -- Pizza
+    - Desserts
+    -- Ice Cream
+    -- Cake
+
+#### Statement with Block Name Constraints:
+Deploy Application
+    /Environment
+        Configure Settings
+            region: us-west-2
+            
+            /Service  # Valid: Different name than parent
+                Deploy Container
+                    name: auth-service
+                    
+                    /Environment  # Valid: Same name as grandparent
+                        Set Parameters
+                            memory: 512Mi
+                    Environment/
+            Service/
+    Environment/
+
+#### Statement with Mixed Parameter Types:
+Create Database
+    name: users
+    /Configuration
+        Set Parameters
+            engine: postgresql
+            version: 14
+    Configuration/
+    schema:
+    .CREATE TABLE users (
+    .    id SERIAL PRIMARY KEY,
+    .    name VARCHAR(255)
+    .);
+
+#### Statement with Duplicate Blocks:
+Deploy Microservices
+    /Service
+        Deploy Container
+            name: auth-service
+            port: 8000
+            image: auth:latest
+    Service/
+    
+    /Service
+        Deploy Container
+            name: payment-service
+            port: 8001
+            image: payments:latest
+    Service/
+    
+    /Service
+        Deploy Container
+            name: user-service
+            port: 8002
+            image: users:latest
+    Service/
+
+

janito/clear_statement_parser/examples.txt

@@ -0,0 +1,326 @@
+# CLEAR STATEMENT FORMAT EXAMPLES
+
+## Configuration Management
+
+#### Web Server Configuration
+Configure Web Server
+    hostname: web-01
+    port: 80
+    worker_processes: auto
+    client_max_body_size: 100M
+    access_log: /var/log/access.log
+    error_log: /var/log/error.log
+    ssl_enabled: true
+    ssl_certificate: /etc/ssl/cert.pem
+    ssl_key: /etc/ssl/key.pem
+
+#### Environment Configuration
+Set Environment Variables
+    /Production
+        Set Variables
+            database_url: postgres://prod-db:5432
+            redis_host: redis-master
+            log_level: info
+    Production/
+    
+    /Development
+        Set Variables
+            database_url: postgres://dev-db:5432
+            redis_host: localhost
+            log_level: debug
+    Development/
+
+## Data Processing
+
+#### Customer Data Processing
+Transform Customer Data
+    input_file: customers.csv
+    output_format: json
+    transformations:
+    - Capitalize all name fields
+    - Convert phone numbers to E.164 format
+    - Validate all email addresses
+    - Remove special characters from addresses
+    filters:
+    - Exclude accounts inactive for 90+ days
+    - Remove records marked as test data
+    - Skip entries with missing required fields
+    field_mapping:
+    .customer_id=id
+    .first_name=given_name
+    .last_name=family_name
+    .phone=contact_number
+    .email=contact_email
+
+#### Report Generation
+Generate Monthly Report
+    title: Monthly Sales Analysis
+    period: 2024-03
+    sections:
+    - Revenue Overview and Trends
+    -- Total Monthly Revenue
+    -- Year-over-Year Growth
+    -- Revenue by Product Line
+    - Customer Insights
+    -- New Customer Acquisition
+    -- Customer Retention Rates
+    -- Average Order Value
+    - Regional Performance
+    -- North America Sales
+    -- European Market Analysis
+    -- Asia Pacific Growth
+    output_format: pdf
+    include_charts: true
+    add_executive_summary: true
+
+## System Administration
+
+#### Database Backup Process
+Backup Database
+    /Preparation
+        Verify Space
+            minimum_free: 100GB
+            backup_path: /mnt/backups
+            temp_space: /tmp/backup
+        Check Connections
+            max_active: 50
+            timeout: 30s
+            verify_replica: true
+    Preparation/
+
+    /Execution
+        Create Snapshot
+            type: hot
+            compression: true
+            verify_checksums: true
+        Copy Files
+            destination: s3://backups/
+            retention_days: 30
+            encryption: AES256
+    Execution/
+
+#### Service Mesh Deployment
+Deploy Service Mesh
+    /Infrastructure
+        Setup Network
+            primary_subnet: 10.0.1.0/24
+            secondary_subnet: 10.0.2.0/24
+            encryption_enabled: true
+            firewall_enabled: true
+    Infrastructure/
+
+    /Services
+        Deploy Gateway
+            replicas: 3
+            ingress_port: 80
+            egress_port: 443
+            health_check: /health
+        Deploy Monitoring
+            monitoring_stack:
+            - Enable Prometheus metrics collection
+            - Install Grafana dashboards
+            - Configure Alert Manager rules
+            - Set up logging pipeline
+    Services/
+
+## Content Management
+
+#### Blog Post Creation
+Create Blog Post
+    title: Getting Started with Clear Statement Format
+    author: John Smith
+    status: draft
+    content:
+    .# Getting Started with Clear Statement Format
+    .
+    .The Clear Statement Format provides a simple yet powerful way to structure configuration and commands.
+    .
+    .## Key Benefits
+    .
+    .* Easy to read and write
+    .* No escape characters needed
+    .* Supports nested structures
+    .
+    .## Examples
+    .
+    .Let's look at some practical examples...
+    categories:
+    - Technical Documentation
+    - Best Practices
+    - Getting Started Guides
+
+#### Media Library Management
+Process Media Library
+    /Scan
+        Process Images
+            source_dir: /uploads/images
+            allowed_formats:
+            - Process all JPEG files
+            - Handle PNG images
+            - Convert WEBP files
+            processing_steps:
+            - Create thumbnails for all images
+            - Extract EXIF metadata
+            - Optimize file sizes
+            - Generate alt text
+    Scan/
+
+    /Organize
+        Create Structure
+            root_path: /media/library
+            categories:
+            - Create Events folder
+            -- Add Conferences subfolder
+            -- Add Meetups subfolder
+            - Create Products folder
+            -- Add Product Launches subfolder
+            -- Add Demo Videos subfolder
+    Organize/
+
+## Security Operations
+
+#### Firewall Configuration
+Update Firewall Rules
+    /Inbound
+        Configure HTTP
+            port: 80
+            source_cidr: 0.0.0.0/0
+            description: Allow incoming HTTP
+        Configure HTTPS
+            port: 443
+            source_cidr: 0.0.0.0/0
+            description: Allow incoming HTTPS
+        Configure SSH
+            port: 22
+            source_cidr: 10.0.0.0/8
+            description: Internal SSH access
+    Inbound/
+
+    /Outbound
+        Set Default Policy
+            default_action: deny
+            log_blocked: true
+        Configure External Access
+            allowed_destinations:
+            - Allow DNS to 8.8.8.8/32
+            - Allow HTTPS to all destinations
+            - Allow NTP to pool.ntp.org
+    Outbound/
+
+#### Access Key Management
+Rotate Access Keys
+    /Audit
+        Check Usage
+            maximum_age: 90
+            inactive_period: 30
+            audit_enabled: true
+    Audit/
+
+    /Update
+        Configure Keys
+            algorithm: rsa-4096
+            validity_days: 365
+            require_rotation: true
+        Update Service Keys
+            services:
+            - Rotate API Gateway credentials
+            - Update Auth Service keys
+            - Refresh Monitoring tokens
+            - Renew Backup service keys
+    Update/
+
+## CI/CD Pipeline
+
+#### Build Process
+Execute Build Pipeline
+    /Prepare
+        Setup Environment
+            cache_enabled: true
+            package_file: package.json
+            lockfile: yarn.lock
+    Prepare/
+
+    /Build
+        Run Build Steps
+            environment: production
+            optimizations:
+            - Minimize all JavaScript bundles
+            - Optimize and compress images
+            - Generate source maps
+            - Tree shake unused code
+            
+        Execute Tests
+            test_types:
+            - Run all unit tests
+            - Execute integration test suite
+            - Perform end-to-end testing
+            coverage_threshold: 80
+            report_path: /reports/coverage
+    Build/
+
+    /Deploy
+        Push Artifacts
+            registry: gcr.io/myproject
+            versions:
+            - Tag as latest version
+            - Add semantic version tag
+            - Include build metadata
+        Update Cluster
+            namespace: production
+            replica_count: 3
+            rolling_update: true
+    Deploy/
+
+#### Deployment Monitoring
+Monitor Deployment Status
+    application_name: user-service
+    watch_metrics:
+    - Track error rate percentage
+    - Monitor average response times
+    - Watch CPU utilization
+    - Check memory consumption
+    alert_conditions:
+    - Alert on error rate above 1%
+    - Notify when response time exceeds 500ms
+    - Warn on CPU above 80%
+    remediation_steps:
+    - Send team notification
+    - Increase replica count
+    - Create incident ticket
+    - Start investigation
+
+## Data Migration
+
+#### Database Migration
+Migrate Database
+    /Planning
+        Analyze Size
+            table_stats:
+            - Users table contains 1M records
+            - Orders table has 5M entries
+            - Products catalog has 100K items
+            - Comments section with 10M rows
+        Plan Schedule
+            maintenance_window: 2h
+            transfer_speed: 100MB/s
+            backup_required: true
+    Planning/
+
+    /Execution
+        Extract Data
+            output_format: binary
+            enable_compression: true
+            verify_integrity: true
+        Update Database
+            schema_changes:
+            - Update users table structure
+            - Add order history indexes
+            - Modify product categories
+            - Optimize query patterns
+        Verify Migration
+            validation_steps:
+            - Compare record counts
+            - Verify data integrity
+            - Test all indexes
+            - Validate constraints
+    Execution/