@defai.digital/automatosx 12.8.7 → 13.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +1 -1
- package/README.md +48 -754
- package/dist/bin.d.ts +8 -0
- package/dist/bin.d.ts.map +1 -0
- package/dist/bin.js +16 -0
- package/dist/bin.js.map +1 -0
- package/dist/index.d.ts +8 -2
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +7 -74239
- package/dist/index.js.map +1 -0
- package/package.json +35 -160
- package/.github/assets/ax-cli.png +0 -0
- package/.github/assets/axlogo.png +0 -0
- package/CHANGELOG.md +0 -81
- package/SECURITY.md +0 -173
- package/dist/mcp/index.d.ts +0 -2
- package/dist/mcp/index.js +0 -43627
- package/examples/AGENTS_INFO.md +0 -187
- package/examples/README.md +0 -434
- package/examples/abilities/accessibility.md +0 -115
- package/examples/abilities/api-design.md +0 -168
- package/examples/abilities/best-practices.md +0 -102
- package/examples/abilities/caching-strategy.md +0 -165
- package/examples/abilities/ci-cd.md +0 -61
- package/examples/abilities/clean-code.md +0 -398
- package/examples/abilities/code-generation.md +0 -333
- package/examples/abilities/code-review.md +0 -51
- package/examples/abilities/component-architecture.md +0 -112
- package/examples/abilities/content-creation.md +0 -97
- package/examples/abilities/data-modeling.md +0 -171
- package/examples/abilities/data-validation.md +0 -50
- package/examples/abilities/db-modeling.md +0 -167
- package/examples/abilities/debugging.md +0 -52
- package/examples/abilities/design-patterns.md +0 -437
- package/examples/abilities/design-system-implementation.md +0 -126
- package/examples/abilities/documentation.md +0 -54
- package/examples/abilities/etl-pipelines.md +0 -44
- package/examples/abilities/feasibility-study.md +0 -20
- package/examples/abilities/general-assistance.md +0 -26
- package/examples/abilities/idea-evaluation.md +0 -21
- package/examples/abilities/infra-as-code.md +0 -57
- package/examples/abilities/job-orchestration.md +0 -44
- package/examples/abilities/literature-review.md +0 -19
- package/examples/abilities/longform-report.md +0 -25
- package/examples/abilities/mathematical-reasoning.md +0 -170
- package/examples/abilities/observability.md +0 -61
- package/examples/abilities/orbital-mechanics.md +0 -50
- package/examples/abilities/our-architecture-decisions.md +0 -180
- package/examples/abilities/our-code-review-checklist.md +0 -149
- package/examples/abilities/our-coding-standards.md +0 -369
- package/examples/abilities/our-project-structure.md +0 -183
- package/examples/abilities/performance.md +0 -89
- package/examples/abilities/problem-solving.md +0 -50
- package/examples/abilities/propulsion-systems.md +0 -50
- package/examples/abilities/quantum-algorithm-design.md +0 -54
- package/examples/abilities/quantum-error-correction.md +0 -56
- package/examples/abilities/quantum-frameworks-transpilation.md +0 -53
- package/examples/abilities/quantum-noise-modeling.md +0 -58
- package/examples/abilities/refactoring.md +0 -223
- package/examples/abilities/release-strategy.md +0 -58
- package/examples/abilities/secrets-policy.md +0 -61
- package/examples/abilities/secure-coding-review.md +0 -60
- package/examples/abilities/software-architecture.md +0 -394
- package/examples/abilities/solid-principles.md +0 -341
- package/examples/abilities/sql-optimization.md +0 -84
- package/examples/abilities/state-management.md +0 -96
- package/examples/abilities/task-planning.md +0 -65
- package/examples/abilities/technical-writing.md +0 -77
- package/examples/abilities/telemetry-diagnostics.md +0 -51
- package/examples/abilities/testing.md +0 -56
- package/examples/abilities/threat-modeling.md +0 -58
- package/examples/abilities/troubleshooting.md +0 -80
- package/examples/abilities/typescript-zod-validation.md +0 -830
- package/examples/agents/AGENTS_INTEGRATION.md +0 -99
- package/examples/agents/aerospace-scientist.yaml +0 -159
- package/examples/agents/architecture.yaml +0 -244
- package/examples/agents/automatosx.config.json +0 -286
- package/examples/agents/backend.yaml +0 -141
- package/examples/agents/ceo.yaml +0 -105
- package/examples/agents/creative-marketer.yaml +0 -173
- package/examples/agents/cto.yaml +0 -118
- package/examples/agents/data-scientist.yaml +0 -200
- package/examples/agents/data.yaml +0 -106
- package/examples/agents/design.yaml +0 -115
- package/examples/agents/devops.yaml +0 -124
- package/examples/agents/frontend.yaml +0 -171
- package/examples/agents/fullstack.yaml +0 -172
- package/examples/agents/mobile.yaml +0 -185
- package/examples/agents/product.yaml +0 -103
- package/examples/agents/quality.yaml +0 -117
- package/examples/agents/quantum-engineer.yaml +0 -166
- package/examples/agents/researcher.yaml +0 -122
- package/examples/agents/security.yaml +0 -115
- package/examples/agents/standard.yaml +0 -214
- package/examples/agents/writer.yaml +0 -122
- package/examples/providers/README.md +0 -117
- package/examples/providers/claude/CLAUDE_INTEGRATION.md +0 -302
- package/examples/providers/claude/mcp/automatosx.json +0 -244
- package/examples/providers/codex/CODEX_INTEGRATION.md +0 -593
- package/examples/providers/codex/README.md +0 -349
- package/examples/providers/codex/usage-examples.ts +0 -421
- package/examples/providers/gemini/GEMINI_INTEGRATION.md +0 -236
- package/examples/providers/gemini/README.md +0 -76
- package/examples/providers/openai-codex-example.ts +0 -421
- package/examples/pytorch_resnet50_training.py +0 -289
- package/examples/specs/automatosx-release.ax.yaml +0 -380
- package/examples/specs/enterprise.ax.yaml +0 -121
- package/examples/specs/enterprise.yaml.mustache +0 -121
- package/examples/specs/government.ax.yaml +0 -148
- package/examples/specs/government.yaml.mustache +0 -148
- package/examples/specs/minimal.ax.yaml +0 -21
- package/examples/specs/minimal.yaml.mustache +0 -21
- package/examples/teams/business.yaml +0 -56
- package/examples/teams/core.yaml +0 -60
- package/examples/teams/design.yaml +0 -58
- package/examples/teams/engineering.yaml +0 -69
- package/examples/teams/research.yaml +0 -56
- package/examples/use-cases/01-web-app-development.md +0 -374
- package/examples/workflows/analyst.yaml +0 -60
- package/examples/workflows/assistant.yaml +0 -48
- package/examples/workflows/basic-agent.yaml +0 -28
- package/examples/workflows/code-reviewer.yaml +0 -52
- package/examples/workflows/debugger.yaml +0 -63
- package/examples/workflows/designer.yaml +0 -69
- package/examples/workflows/developer.yaml +0 -60
- package/examples/workflows/fullstack-developer.yaml +0 -395
- package/examples/workflows/qa-specialist.yaml +0 -71
- package/schema/ability-metadata.json +0 -21
- package/schema/config.json +0 -703
- package/schema/spec-schema.json +0 -608
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
# Code Review Ability
|
|
2
|
-
|
|
3
|
-
Perform thorough code reviews focusing on quality, security, and maintainability.
|
|
4
|
-
|
|
5
|
-
## Review Areas
|
|
6
|
-
|
|
7
|
-
1. **Correctness**
|
|
8
|
-
- Logic errors and bugs
|
|
9
|
-
- Edge case handling
|
|
10
|
-
- Error handling and validation
|
|
11
|
-
|
|
12
|
-
2. **Security**
|
|
13
|
-
- Input validation
|
|
14
|
-
- SQL injection, XSS vulnerabilities
|
|
15
|
-
- Secrets and credentials exposure
|
|
16
|
-
- Authentication and authorization
|
|
17
|
-
|
|
18
|
-
3. **Performance**
|
|
19
|
-
- Algorithm complexity (Big O)
|
|
20
|
-
- Resource usage (memory, CPU)
|
|
21
|
-
- Database query optimization
|
|
22
|
-
- Caching opportunities
|
|
23
|
-
|
|
24
|
-
4. **Maintainability**
|
|
25
|
-
- Code clarity and readability
|
|
26
|
-
- Documentation quality
|
|
27
|
-
- Test coverage
|
|
28
|
-
- Modularity and coupling
|
|
29
|
-
|
|
30
|
-
5. **Style**
|
|
31
|
-
- Coding standards compliance
|
|
32
|
-
- Naming conventions
|
|
33
|
-
- Code organization
|
|
34
|
-
- Consistent formatting
|
|
35
|
-
|
|
36
|
-
## Review Process
|
|
37
|
-
|
|
38
|
-
1. Understand the context and requirements
|
|
39
|
-
2. Review code systematically (file by file)
|
|
40
|
-
3. Identify issues and improvements
|
|
41
|
-
4. Prioritize findings (critical/major/minor)
|
|
42
|
-
5. Provide actionable, constructive feedback
|
|
43
|
-
|
|
44
|
-
## Application Hints
|
|
45
|
-
|
|
46
|
-
When reviewing code:
|
|
47
|
-
- Focus on correctness and security first; style issues are secondary
|
|
48
|
-
- Verify edge cases and error paths are handled, not just the happy path
|
|
49
|
-
- Check that tests cover the changed behavior, not just line coverage
|
|
50
|
-
- Prioritize feedback by severity (critical/major/minor) to respect author's time
|
|
51
|
-
- Provide specific improvements with code examples rather than vague criticism
|
|
@@ -1,112 +0,0 @@
|
|
|
1
|
-
# Component Architecture
|
|
2
|
-
|
|
3
|
-
## Overview
|
|
4
|
-
Design and implement reusable, maintainable component structures following modern frontend architecture patterns. Focus on composition, separation of concerns, and clear component hierarchies.
|
|
5
|
-
|
|
6
|
-
## Core Principles
|
|
7
|
-
|
|
8
|
-
### 1. Single Responsibility
|
|
9
|
-
Each component should have one clear purpose. Avoid components that try to do too much.
|
|
10
|
-
|
|
11
|
-
### 2. Composition Over Inheritance
|
|
12
|
-
Build complex UIs by composing smaller, focused components rather than creating deep inheritance chains.
|
|
13
|
-
|
|
14
|
-
### 3. Container/Presentational Pattern
|
|
15
|
-
- **Container components**: Handle logic, state, and side effects
|
|
16
|
-
- **Presentational components**: Focus purely on rendering UI based on props
|
|
17
|
-
|
|
18
|
-
### 4. Props Design
|
|
19
|
-
- Keep props interfaces simple and explicit
|
|
20
|
-
- Use TypeScript for type safety
|
|
21
|
-
- Avoid prop drilling—use context or state management for deep data
|
|
22
|
-
|
|
23
|
-
## Component Structure
|
|
24
|
-
|
|
25
|
-
```
|
|
26
|
-
/components
|
|
27
|
-
/Button
|
|
28
|
-
Button.tsx # Component implementation
|
|
29
|
-
Button.test.tsx # Tests
|
|
30
|
-
Button.stories.tsx # Storybook stories
|
|
31
|
-
index.ts # Public exports
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
## Atomic Design
|
|
35
|
-
|
|
36
|
-
Organize components by complexity:
|
|
37
|
-
- **Atoms**: Button, Input, Label
|
|
38
|
-
- **Molecules**: FormField (Input + Label), SearchBar
|
|
39
|
-
- **Organisms**: LoginForm, Header, Card
|
|
40
|
-
- **Templates**: Page layouts
|
|
41
|
-
- **Pages**: Complete pages
|
|
42
|
-
|
|
43
|
-
## Best Practices
|
|
44
|
-
|
|
45
|
-
### Component Design
|
|
46
|
-
- Single responsibility principle
|
|
47
|
-
- TypeScript interfaces for props
|
|
48
|
-
- Proper error boundaries
|
|
49
|
-
- Clear naming conventions
|
|
50
|
-
|
|
51
|
-
### Performance
|
|
52
|
-
- Memoization where appropriate (React.memo, useMemo)
|
|
53
|
-
- Callback memoization (useCallback)
|
|
54
|
-
- Code splitting for heavy components
|
|
55
|
-
- Lazy loading with Suspense
|
|
56
|
-
|
|
57
|
-
### Testing
|
|
58
|
-
- Unit tests for component logic
|
|
59
|
-
- Integration tests for user interactions
|
|
60
|
-
- Accessibility testing
|
|
61
|
-
- Visual regression tests
|
|
62
|
-
|
|
63
|
-
## Anti-Patterns to Avoid
|
|
64
|
-
|
|
65
|
-
### 1. God Components
|
|
66
|
-
Components that handle too many responsibilities
|
|
67
|
-
|
|
68
|
-
### 2. Prop Drilling
|
|
69
|
-
Passing props through many levels—use Context API or state management
|
|
70
|
-
|
|
71
|
-
### 3. Tight Coupling
|
|
72
|
-
Components depending on specific parent or child implementations
|
|
73
|
-
|
|
74
|
-
### 4. Mixed Concerns
|
|
75
|
-
Mixing business logic, API calls, and rendering in one component
|
|
76
|
-
|
|
77
|
-
## Component Checklist
|
|
78
|
-
|
|
79
|
-
- [ ] Single responsibility
|
|
80
|
-
- [ ] TypeScript interfaces for props
|
|
81
|
-
- [ ] Proper error boundaries
|
|
82
|
-
- [ ] Memoization where appropriate
|
|
83
|
-
- [ ] Accessibility attributes (ARIA)
|
|
84
|
-
- [ ] Unit tests for logic
|
|
85
|
-
- [ ] Integration tests for interactions
|
|
86
|
-
- [ ] Storybook documentation
|
|
87
|
-
- [ ] Performance optimized
|
|
88
|
-
- [ ] Code split if heavy
|
|
89
|
-
|
|
90
|
-
## Performance Considerations
|
|
91
|
-
|
|
92
|
-
### Memoization
|
|
93
|
-
- Memoize expensive computations (useMemo)
|
|
94
|
-
- Memoize callback functions (useCallback)
|
|
95
|
-
- Memoize components that render often (React.memo)
|
|
96
|
-
|
|
97
|
-
### Code Splitting
|
|
98
|
-
- Lazy load heavy components
|
|
99
|
-
- Use Suspense for loading states
|
|
100
|
-
- Route-based code splitting
|
|
101
|
-
|
|
102
|
-
### Optimization Techniques
|
|
103
|
-
- Virtual scrolling for long lists
|
|
104
|
-
- Debounce/throttle for frequent events
|
|
105
|
-
- Avoid unnecessary re-renders
|
|
106
|
-
- Use production builds
|
|
107
|
-
|
|
108
|
-
## Resources
|
|
109
|
-
|
|
110
|
-
- [React Component Patterns](https://reactpatterns.com/)
|
|
111
|
-
- [Atomic Design Methodology](https://atomicdesign.bradfrost.com/)
|
|
112
|
-
- [Testing Library Best Practices](https://testing-library.com/docs/react-testing-library/intro)
|
|
@@ -1,97 +0,0 @@
|
|
|
1
|
-
# Content Creation Ability
|
|
2
|
-
|
|
3
|
-
Create engaging, informative content for various audiences.
|
|
4
|
-
|
|
5
|
-
## Content Types
|
|
6
|
-
|
|
7
|
-
1. **Blog Posts**
|
|
8
|
-
- Catchy title
|
|
9
|
-
- Hook in first paragraph
|
|
10
|
-
- Clear structure (intro, body, conclusion)
|
|
11
|
-
- Scannable (headings, bullets, short paragraphs)
|
|
12
|
-
- Call to action
|
|
13
|
-
|
|
14
|
-
2. **Tutorials**
|
|
15
|
-
- Clear learning objectives
|
|
16
|
-
- Prerequisites listed
|
|
17
|
-
- Step-by-step instructions
|
|
18
|
-
- Code examples
|
|
19
|
-
- Screenshots/diagrams
|
|
20
|
-
- What you learned / next steps
|
|
21
|
-
|
|
22
|
-
3. **Documentation**
|
|
23
|
-
- Purpose and overview
|
|
24
|
-
- Installation/setup
|
|
25
|
-
- Usage examples
|
|
26
|
-
- API reference
|
|
27
|
-
- Troubleshooting
|
|
28
|
-
- FAQ
|
|
29
|
-
|
|
30
|
-
4. **Presentations**
|
|
31
|
-
- One idea per slide
|
|
32
|
-
- Visual > text
|
|
33
|
-
- Tell a story
|
|
34
|
-
- Practice delivery
|
|
35
|
-
- End with key takeaways
|
|
36
|
-
|
|
37
|
-
## Writing Process
|
|
38
|
-
|
|
39
|
-
1. **Research**
|
|
40
|
-
- Understand the topic
|
|
41
|
-
- Know your audience
|
|
42
|
-
- Check existing content
|
|
43
|
-
- Gather examples
|
|
44
|
-
|
|
45
|
-
2. **Outline**
|
|
46
|
-
- Main points
|
|
47
|
-
- Logical flow
|
|
48
|
-
- Supporting details
|
|
49
|
-
- Conclusion
|
|
50
|
-
|
|
51
|
-
3. **Draft**
|
|
52
|
-
- Get ideas down
|
|
53
|
-
- Don't self-edit yet
|
|
54
|
-
- Write more than needed
|
|
55
|
-
- Focus on clarity
|
|
56
|
-
|
|
57
|
-
4. **Edit**
|
|
58
|
-
- Cut unnecessary content
|
|
59
|
-
- Improve clarity
|
|
60
|
-
- Fix grammar/spelling
|
|
61
|
-
- Check flow and structure
|
|
62
|
-
|
|
63
|
-
5. **Review**
|
|
64
|
-
- Read aloud
|
|
65
|
-
- Get feedback
|
|
66
|
-
- Check accuracy
|
|
67
|
-
- Final polish
|
|
68
|
-
|
|
69
|
-
## Writing Tips
|
|
70
|
-
|
|
71
|
-
**Clarity:**
|
|
72
|
-
|
|
73
|
-
- Use simple words
|
|
74
|
-
- Short sentences
|
|
75
|
-
- Active voice
|
|
76
|
-
- Concrete examples
|
|
77
|
-
|
|
78
|
-
**Engagement:**
|
|
79
|
-
|
|
80
|
-
- Hook readers early
|
|
81
|
-
- Use stories and examples
|
|
82
|
-
- Ask questions
|
|
83
|
-
- Vary sentence length
|
|
84
|
-
|
|
85
|
-
**Structure:**
|
|
86
|
-
|
|
87
|
-
- Clear headings
|
|
88
|
-
- Logical progression
|
|
89
|
-
- Bullet points for lists
|
|
90
|
-
- White space for readability
|
|
91
|
-
|
|
92
|
-
**Credibility:**
|
|
93
|
-
|
|
94
|
-
- Cite sources
|
|
95
|
-
- Use data and statistics
|
|
96
|
-
- Show expertise
|
|
97
|
-
- Acknowledge limitations
|
|
@@ -1,171 +0,0 @@
|
|
|
1
|
-
# Data Modeling
|
|
2
|
-
|
|
3
|
-
## Overview
|
|
4
|
-
Design efficient, scalable data models for relational and NoSQL databases. Focus on normalization, relationships, and schema design that supports business requirements while maintaining data integrity.
|
|
5
|
-
|
|
6
|
-
## Core Principles
|
|
7
|
-
|
|
8
|
-
### 1. Normalization (Relational)
|
|
9
|
-
Eliminate data redundancy through normal forms (1NF, 2NF, 3NF, BCNF).
|
|
10
|
-
|
|
11
|
-
### 2. Denormalization (When Needed)
|
|
12
|
-
Strategic denormalization for read-heavy workloads and performance optimization.
|
|
13
|
-
|
|
14
|
-
### 3. Schema Design Patterns
|
|
15
|
-
- **Star Schema**: Dimensional modeling for analytics (fact + dimension tables)
|
|
16
|
-
- **Snowflake Schema**: Normalized dimensional model
|
|
17
|
-
- **Data Vault**: Flexible, auditable data warehouse modeling
|
|
18
|
-
|
|
19
|
-
## Do's ✅
|
|
20
|
-
|
|
21
|
-
### Relational Data Model
|
|
22
|
-
```sql
|
|
23
|
-
-- ✅ Good: Normalized design
|
|
24
|
-
CREATE TABLE users (
|
|
25
|
-
id SERIAL PRIMARY KEY,
|
|
26
|
-
email VARCHAR(255) UNIQUE NOT NULL,
|
|
27
|
-
created_at TIMESTAMP DEFAULT NOW()
|
|
28
|
-
);
|
|
29
|
-
|
|
30
|
-
CREATE TABLE orders (
|
|
31
|
-
id SERIAL PRIMARY KEY,
|
|
32
|
-
user_id INTEGER REFERENCES users(id),
|
|
33
|
-
total_amount DECIMAL(10,2),
|
|
34
|
-
status VARCHAR(50),
|
|
35
|
-
created_at TIMESTAMP DEFAULT NOW()
|
|
36
|
-
);
|
|
37
|
-
|
|
38
|
-
CREATE TABLE order_items (
|
|
39
|
-
id SERIAL PRIMARY KEY,
|
|
40
|
-
order_id INTEGER REFERENCES orders(id),
|
|
41
|
-
product_id INTEGER REFERENCES products(id),
|
|
42
|
-
quantity INTEGER,
|
|
43
|
-
price DECIMAL(10,2)
|
|
44
|
-
);
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
### Indexes
|
|
48
|
-
```sql
|
|
49
|
-
-- ✅ Good: Index frequently queried columns
|
|
50
|
-
CREATE INDEX idx_orders_user_id ON orders(user_id);
|
|
51
|
-
CREATE INDEX idx_orders_created_at ON orders(created_at);
|
|
52
|
-
CREATE INDEX idx_order_items_order_id ON order_items(order_id);
|
|
53
|
-
|
|
54
|
-
-- ✅ Good: Composite index for multi-column queries
|
|
55
|
-
CREATE INDEX idx_orders_user_status ON orders(user_id, status);
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
### NoSQL Data Model (MongoDB)
|
|
59
|
-
```javascript
|
|
60
|
-
// ✅ Good: Embedded documents for one-to-few
|
|
61
|
-
{
|
|
62
|
-
_id: ObjectId("..."),
|
|
63
|
-
user_email: "user@example.com",
|
|
64
|
-
addresses: [
|
|
65
|
-
{ type: "home", street: "123 Main St", city: "NYC" },
|
|
66
|
-
{ type: "work", street: "456 Office Ave", city: "LA" }
|
|
67
|
-
]
|
|
68
|
-
}
|
|
69
|
-
|
|
70
|
-
// ✅ Good: References for one-to-many
|
|
71
|
-
// User document
|
|
72
|
-
{
|
|
73
|
-
_id: ObjectId("user123"),
|
|
74
|
-
email: "user@example.com"
|
|
75
|
-
}
|
|
76
|
-
|
|
77
|
-
// Order documents (separate collection)
|
|
78
|
-
{
|
|
79
|
-
_id: ObjectId("order456"),
|
|
80
|
-
user_id: ObjectId("user123"),
|
|
81
|
-
items: [...],
|
|
82
|
-
total: 99.99
|
|
83
|
-
}
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
## Don'ts ❌
|
|
87
|
-
|
|
88
|
-
### Over-Normalization
|
|
89
|
-
```sql
|
|
90
|
-
-- ❌ Bad: Excessive normalization
|
|
91
|
-
CREATE TABLE user_first_names (
|
|
92
|
-
user_id INTEGER REFERENCES users(id),
|
|
93
|
-
first_name VARCHAR(100)
|
|
94
|
-
);
|
|
95
|
-
|
|
96
|
-
CREATE TABLE user_last_names (
|
|
97
|
-
user_id INTEGER REFERENCES users(id),
|
|
98
|
-
last_name VARCHAR(100)
|
|
99
|
-
);
|
|
100
|
-
|
|
101
|
-
-- ✅ Good: Appropriate normalization
|
|
102
|
-
CREATE TABLE users (
|
|
103
|
-
id SERIAL PRIMARY KEY,
|
|
104
|
-
first_name VARCHAR(100),
|
|
105
|
-
last_name VARCHAR(100),
|
|
106
|
-
email VARCHAR(255) UNIQUE
|
|
107
|
-
);
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
### Missing Constraints
|
|
111
|
-
```sql
|
|
112
|
-
-- ❌ Bad: No constraints
|
|
113
|
-
CREATE TABLE orders (
|
|
114
|
-
id INTEGER,
|
|
115
|
-
user_id INTEGER,
|
|
116
|
-
total DECIMAL
|
|
117
|
-
);
|
|
118
|
-
|
|
119
|
-
-- ✅ Good: Proper constraints
|
|
120
|
-
CREATE TABLE orders (
|
|
121
|
-
id SERIAL PRIMARY KEY,
|
|
122
|
-
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
|
123
|
-
total DECIMAL(10,2) CHECK (total >= 0),
|
|
124
|
-
status VARCHAR(50) DEFAULT 'pending',
|
|
125
|
-
created_at TIMESTAMP NOT NULL DEFAULT NOW()
|
|
126
|
-
);
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
### Embedding Large Arrays (NoSQL)
|
|
130
|
-
```javascript
|
|
131
|
-
// ❌ Bad: Unbounded array growth
|
|
132
|
-
{
|
|
133
|
-
user_id: "user123",
|
|
134
|
-
orders: [ /* Could grow to thousands */ ]
|
|
135
|
-
}
|
|
136
|
-
|
|
137
|
-
// ✅ Good: Reference pattern
|
|
138
|
-
{
|
|
139
|
-
user_id: "user123",
|
|
140
|
-
recent_orders: [ /* Last 5 orders */ ],
|
|
141
|
-
total_order_count: 1247
|
|
142
|
-
}
|
|
143
|
-
// Store full orders in separate collection
|
|
144
|
-
```
|
|
145
|
-
|
|
146
|
-
## Best Practices
|
|
147
|
-
|
|
148
|
-
### 1. Use Appropriate Data Types
|
|
149
|
-
- `TIMESTAMP` for dates (not VARCHAR)
|
|
150
|
-
- `DECIMAL` for money (not FLOAT)
|
|
151
|
-
- `ENUM` or `CHECK` constraints for fixed options
|
|
152
|
-
- `UUID` for distributed primary keys
|
|
153
|
-
|
|
154
|
-
### 2. Plan for Growth
|
|
155
|
-
- Partition large tables by date/region
|
|
156
|
-
- Use sharding for horizontal scalability
|
|
157
|
-
- Implement archival strategy for old data
|
|
158
|
-
|
|
159
|
-
### 3. Document Relationships
|
|
160
|
-
```
|
|
161
|
-
Users (1) ← (M) Orders (1) ← (M) OrderItems (M) → (1) Products
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
### 4. Version Schema Changes
|
|
165
|
-
Use migration tools (Flyway, Liquibase, Alembic) for schema versioning.
|
|
166
|
-
|
|
167
|
-
## Resources
|
|
168
|
-
|
|
169
|
-
- [Database Normalization](https://en.wikipedia.org/wiki/Database_normalization)
|
|
170
|
-
- [MongoDB Schema Design Patterns](https://www.mongodb.com/blog/post/building-with-patterns-a-summary)
|
|
171
|
-
- [PostgreSQL Documentation](https://www.postgresql.org/docs/)
|
|
@@ -1,50 +0,0 @@
|
|
|
1
|
-
# Data Validation
|
|
2
|
-
|
|
3
|
-
Implement comprehensive data quality checks to ensure accuracy, completeness, and consistency. Catch data issues early before they impact downstream systems.
|
|
4
|
-
|
|
5
|
-
## Do's ✅
|
|
6
|
-
|
|
7
|
-
```python
|
|
8
|
-
# ✅ Good: Schema validation with Pydantic
|
|
9
|
-
from pydantic import BaseModel, validator
|
|
10
|
-
|
|
11
|
-
class UserRecord(BaseModel):
|
|
12
|
-
id: int
|
|
13
|
-
email: str
|
|
14
|
-
age: int
|
|
15
|
-
|
|
16
|
-
@validator('email')
|
|
17
|
-
def email_must_be_valid(cls, v):
|
|
18
|
-
if '@' not in v:
|
|
19
|
-
raise ValueError('Invalid email')
|
|
20
|
-
return v
|
|
21
|
-
|
|
22
|
-
# ✅ Good: Data quality checks
|
|
23
|
-
def validate_data(df):
|
|
24
|
-
checks = [
|
|
25
|
-
('No nulls', df[['id', 'email']].notnull().all().all()),
|
|
26
|
-
('Unique IDs', df['id'].is_unique),
|
|
27
|
-
('Valid ages', df['age'].between(0, 150).all()),
|
|
28
|
-
]
|
|
29
|
-
failed = [check for check, passed in checks if not passed]
|
|
30
|
-
if failed:
|
|
31
|
-
raise ValueError(f"Validation failed: {failed}")
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
## Don'ts ❌
|
|
35
|
-
|
|
36
|
-
```python
|
|
37
|
-
# ❌ Bad: No validation
|
|
38
|
-
df = pd.read_csv('data.csv')
|
|
39
|
-
df.to_sql('users', engine) # Hope for the best!
|
|
40
|
-
|
|
41
|
-
# ❌ Bad: Silent failures
|
|
42
|
-
df = df.dropna() # What was dropped? Why?
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
## Best Practices
|
|
46
|
-
- Define data quality SLAs
|
|
47
|
-
- Implement continuous monitoring
|
|
48
|
-
- Document validation rules
|
|
49
|
-
- Use validation frameworks (Great Expectations, Pandera)
|
|
50
|
-
- Create data quality dashboards
|
|
@@ -1,167 +0,0 @@
|
|
|
1
|
-
# Database Modeling
|
|
2
|
-
|
|
3
|
-
## Overview
|
|
4
|
-
Design efficient, scalable, and maintainable database schemas for relational and NoSQL databases, focusing on data integrity, performance, and evolution.
|
|
5
|
-
|
|
6
|
-
## Core Principles
|
|
7
|
-
|
|
8
|
-
### 1. Normalization & Denormalization
|
|
9
|
-
- 3NF for transactional systems
|
|
10
|
-
- Strategic denormalization for read-heavy workloads
|
|
11
|
-
- Balance between data integrity and query performance
|
|
12
|
-
|
|
13
|
-
### 2. Data Integrity
|
|
14
|
-
- Primary keys on all tables
|
|
15
|
-
- Foreign key constraints
|
|
16
|
-
- Check constraints for data validation
|
|
17
|
-
- Unique constraints for business rules
|
|
18
|
-
|
|
19
|
-
### 3. Indexing Strategy
|
|
20
|
-
- Primary key indexes (clustered)
|
|
21
|
-
- Secondary indexes for frequent queries
|
|
22
|
-
- Composite indexes for multi-column filters
|
|
23
|
-
- Index maintenance and statistics
|
|
24
|
-
|
|
25
|
-
### 4. Schema Evolution
|
|
26
|
-
- Migration strategy (forward-only)
|
|
27
|
-
- Backward compatibility during deploys
|
|
28
|
-
- Column additions vs. breaking changes
|
|
29
|
-
- Versioning for schema changes
|
|
30
|
-
|
|
31
|
-
## Relational Database Design
|
|
32
|
-
|
|
33
|
-
### Table Design Essentials
|
|
34
|
-
- Primary key (BIGSERIAL, UUID)
|
|
35
|
-
- Timestamps (created_at, updated_at)
|
|
36
|
-
- Soft delete (deleted_at)
|
|
37
|
-
- Proper data types
|
|
38
|
-
|
|
39
|
-
### Relationship Patterns
|
|
40
|
-
- One-to-Many: Foreign key in child table
|
|
41
|
-
- Many-to-Many: Junction table with composite primary key
|
|
42
|
-
- One-to-One: Rare; consider column addition instead
|
|
43
|
-
|
|
44
|
-
### Partitioning Strategies
|
|
45
|
-
- Range partitioning (dates, IDs)
|
|
46
|
-
- List partitioning (regions, categories)
|
|
47
|
-
- Hash partitioning (even distribution)
|
|
48
|
-
- When to partition: >10M rows, predictable access patterns
|
|
49
|
-
|
|
50
|
-
## NoSQL Design Patterns
|
|
51
|
-
|
|
52
|
-
### Document Stores (MongoDB, DynamoDB)
|
|
53
|
-
- Embed vs. reference decision tree
|
|
54
|
-
- Avoid unbounded arrays
|
|
55
|
-
- Duplicate data strategically
|
|
56
|
-
- Query pattern drives schema
|
|
57
|
-
|
|
58
|
-
### Key-Value Stores (Redis, DynamoDB)
|
|
59
|
-
- Compound key design
|
|
60
|
-
- TTL for expiring data
|
|
61
|
-
- Secondary indexes trade-offs
|
|
62
|
-
|
|
63
|
-
### Graph Databases (Neo4j)
|
|
64
|
-
- Node and relationship properties
|
|
65
|
-
- Index on frequently queried properties
|
|
66
|
-
- Cypher query optimization
|
|
67
|
-
|
|
68
|
-
## Performance Optimization
|
|
69
|
-
|
|
70
|
-
### Query Performance
|
|
71
|
-
- EXPLAIN ANALYZE for execution plans
|
|
72
|
-
- Index usage verification
|
|
73
|
-
- Avoid N+1 queries (eager loading)
|
|
74
|
-
- Connection pooling configuration
|
|
75
|
-
|
|
76
|
-
### Write Optimization
|
|
77
|
-
- Batch inserts for bulk data
|
|
78
|
-
- Async writes for non-critical data
|
|
79
|
-
- Write-ahead logging (WAL) tuning
|
|
80
|
-
|
|
81
|
-
### Read Optimization
|
|
82
|
-
- Read replicas for scaling
|
|
83
|
-
- Query result caching
|
|
84
|
-
- Materialized views for complex aggregations
|
|
85
|
-
|
|
86
|
-
## Data Types Best Practices
|
|
87
|
-
|
|
88
|
-
### Choosing Types
|
|
89
|
-
- VARCHAR vs. TEXT vs. CHAR
|
|
90
|
-
- INTEGER vs. BIGINT (consider growth)
|
|
91
|
-
- DECIMAL for money (never FLOAT)
|
|
92
|
-
- TIMESTAMPTZ for dates (always use timezone)
|
|
93
|
-
- JSON/JSONB for semi-structured data
|
|
94
|
-
|
|
95
|
-
### Type Performance
|
|
96
|
-
- Smaller types = better performance
|
|
97
|
-
- Fixed-length types for indexes
|
|
98
|
-
- Avoid TEXT in WHERE clauses without indexes
|
|
99
|
-
|
|
100
|
-
## Migration Strategies
|
|
101
|
-
|
|
102
|
-
### Safe Migration Patterns
|
|
103
|
-
1. Add nullable column → Backfill → Add NOT NULL
|
|
104
|
-
2. Create new table → Dual write → Swap → Drop old
|
|
105
|
-
3. Add index CONCURRENTLY (PostgreSQL)
|
|
106
|
-
4. Rename via view (zero-downtime)
|
|
107
|
-
|
|
108
|
-
### Rollback Strategies
|
|
109
|
-
- Idempotent migrations
|
|
110
|
-
- Reversible changes
|
|
111
|
-
- Feature flags for data migrations
|
|
112
|
-
- Separate schema and data migrations
|
|
113
|
-
|
|
114
|
-
## Security & Compliance
|
|
115
|
-
|
|
116
|
-
### Access Control
|
|
117
|
-
- Principle of least privilege
|
|
118
|
-
- Row-level security (RLS)
|
|
119
|
-
- Column-level permissions
|
|
120
|
-
- Audit logging for sensitive data
|
|
121
|
-
|
|
122
|
-
### Data Encryption
|
|
123
|
-
- Encryption at rest
|
|
124
|
-
- Encryption in transit (SSL/TLS)
|
|
125
|
-
- Column-level encryption for PII
|
|
126
|
-
- Key rotation policies
|
|
127
|
-
|
|
128
|
-
### GDPR/Privacy
|
|
129
|
-
- Soft delete vs. hard delete
|
|
130
|
-
- Data retention policies
|
|
131
|
-
- PII identification and protection
|
|
132
|
-
- Right to be forgotten implementation
|
|
133
|
-
|
|
134
|
-
## Common Patterns
|
|
135
|
-
|
|
136
|
-
### Soft Delete
|
|
137
|
-
Use `deleted_at` timestamp with partial index
|
|
138
|
-
|
|
139
|
-
### Audit Trails
|
|
140
|
-
Track created_at, created_by, updated_at, updated_by
|
|
141
|
-
|
|
142
|
-
### Status/State Machine
|
|
143
|
-
Use CHECK constraints for valid states
|
|
144
|
-
|
|
145
|
-
## Design Checklist
|
|
146
|
-
|
|
147
|
-
- [ ] All tables have primary keys
|
|
148
|
-
- [ ] Foreign key constraints defined
|
|
149
|
-
- [ ] Indexes cover frequent query patterns
|
|
150
|
-
- [ ] Data types appropriately sized
|
|
151
|
-
- [ ] Nullable columns justified
|
|
152
|
-
- [ ] Constraints enforce business rules
|
|
153
|
-
- [ ] Migration scripts tested
|
|
154
|
-
- [ ] Rollback plan documented
|
|
155
|
-
- [ ] Performance testing completed
|
|
156
|
-
- [ ] Schema documented
|
|
157
|
-
- [ ] Backup/restore verified
|
|
158
|
-
- [ ] Security audit completed
|
|
159
|
-
|
|
160
|
-
## Application Hints
|
|
161
|
-
|
|
162
|
-
When modeling databases:
|
|
163
|
-
- Choose data types for expected growth; BIGINT for IDs that may exceed 2 billion
|
|
164
|
-
- Use TIMESTAMPTZ (not TIMESTAMP) for all date/time columns to avoid timezone bugs
|
|
165
|
-
- Never use FLOAT/DOUBLE for money; use DECIMAL or store as integer cents
|
|
166
|
-
- Design migrations to be reversible and test rollback before deploying
|
|
167
|
-
- Add indexes based on actual query patterns, not theoretical access needs
|
|
@@ -1,52 +0,0 @@
|
|
|
1
|
-
# Debugging Ability
|
|
2
|
-
|
|
3
|
-
Systematically identify and fix bugs in code.
|
|
4
|
-
|
|
5
|
-
## Debugging Process
|
|
6
|
-
|
|
7
|
-
1. **Reproduce** the issue
|
|
8
|
-
- Understand error messages
|
|
9
|
-
- Identify conditions that trigger the bug
|
|
10
|
-
- Create minimal reproduction case
|
|
11
|
-
|
|
12
|
-
2. **Investigate** the problem
|
|
13
|
-
- Read stack traces carefully
|
|
14
|
-
- Add logging and breakpoints
|
|
15
|
-
- Check recent changes (git diff)
|
|
16
|
-
- Verify assumptions
|
|
17
|
-
|
|
18
|
-
3. **Isolate** the root cause
|
|
19
|
-
- Use binary search (comment out code blocks)
|
|
20
|
-
- Test components individually
|
|
21
|
-
- Check inputs and outputs
|
|
22
|
-
- Trace execution flow
|
|
23
|
-
|
|
24
|
-
4. **Fix** the bug
|
|
25
|
-
- Implement targeted fix
|
|
26
|
-
- Avoid introducing new bugs
|
|
27
|
-
- Update related code if needed
|
|
28
|
-
- Add regression test
|
|
29
|
-
|
|
30
|
-
5. **Verify** the solution
|
|
31
|
-
- Test the fix thoroughly
|
|
32
|
-
- Check for side effects
|
|
33
|
-
- Run full test suite
|
|
34
|
-
- Document the fix
|
|
35
|
-
|
|
36
|
-
## Common Bug Types
|
|
37
|
-
|
|
38
|
-
- Off-by-one errors
|
|
39
|
-
- Null/undefined reference errors
|
|
40
|
-
- Race conditions and timing issues
|
|
41
|
-
- Type mismatches
|
|
42
|
-
- Resource leaks (memory, file handles)
|
|
43
|
-
- Logic errors in conditionals
|
|
44
|
-
|
|
45
|
-
## Application Hints
|
|
46
|
-
|
|
47
|
-
When debugging:
|
|
48
|
-
- Reproduce the issue first; never attempt to fix what you cannot reliably reproduce
|
|
49
|
-
- Isolate to the smallest failing case before investigating root cause
|
|
50
|
-
- Verify the fix doesn't introduce regressions by running the full test suite
|
|
51
|
-
- Document the root cause in the commit message, not just the symptom
|
|
52
|
-
- Check for similar patterns elsewhere in the codebase that may have the same bug
|