framework-mcp 1.5.2 → 2.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (75) hide show
  1. package/.claude/commands/speckit.analyze.md +184 -0
  2. package/.claude/commands/speckit.checklist.md +294 -0
  3. package/.claude/commands/speckit.clarify.md +181 -0
  4. package/.claude/commands/speckit.constitution.md +82 -0
  5. package/.claude/commands/speckit.implement.md +135 -0
  6. package/.claude/commands/speckit.plan.md +89 -0
  7. package/.claude/commands/speckit.specify.md +258 -0
  8. package/.claude/commands/speckit.tasks.md +137 -0
  9. package/.claude/commands/speckit.taskstoissues.md +30 -0
  10. package/.do/app.yaml +1 -1
  11. package/.github/dependabot.yml +15 -0
  12. package/.github/workflows/ci.yml +19 -1
  13. package/.specify/memory/constitution.md +50 -0
  14. package/.specify/scripts/bash/check-prerequisites.sh +166 -0
  15. package/.specify/scripts/bash/common.sh +156 -0
  16. package/.specify/scripts/bash/create-new-feature.sh +297 -0
  17. package/.specify/scripts/bash/setup-plan.sh +61 -0
  18. package/.specify/scripts/bash/update-agent-context.sh +799 -0
  19. package/.specify/templates/agent-file-template.md +28 -0
  20. package/.specify/templates/checklist-template.md +40 -0
  21. package/.specify/templates/plan-template.md +104 -0
  22. package/.specify/templates/spec-template.md +115 -0
  23. package/.specify/templates/tasks-template.md +251 -0
  24. package/README.md +84 -435
  25. package/dist/core/safeguard-manager.d.ts.map +1 -1
  26. package/dist/core/safeguard-manager.js +7295 -2700
  27. package/dist/core/safeguard-manager.js.map +1 -1
  28. package/dist/interfaces/http/http-server.d.ts +2 -0
  29. package/dist/interfaces/http/http-server.d.ts.map +1 -1
  30. package/dist/interfaces/http/http-server.js +95 -3
  31. package/dist/interfaces/http/http-server.js.map +1 -1
  32. package/dist/interfaces/mcp/mcp-server.js +3 -3
  33. package/dist/shared/types.d.ts +85 -2
  34. package/dist/shared/types.d.ts.map +1 -1
  35. package/dist/shared/types.js +132 -1
  36. package/dist/shared/types.js.map +1 -1
  37. package/package.json +7 -4
  38. package/scripts/standardize-prompts.js +325 -0
  39. package/scripts/validate-capability-prompts.js +110 -0
  40. package/src/core/safeguard-manager.ts +12376 -2586
  41. package/src/interfaces/http/http-server.ts +109 -4
  42. package/src/interfaces/mcp/mcp-server.ts +3 -3
  43. package/src/shared/types.ts +268 -2
  44. package/swagger.json +58 -38
  45. package/CAPABILITY_TRANSFORMATION_SUMMARY.md +0 -158
  46. package/CIS_CONTROLS_IMPLEMENTATION_PLAN.md +0 -220
  47. package/COPILOT_INTEGRATION.md +0 -322
  48. package/DAYS_5_6_COMPLETION_SUMMARY.md +0 -178
  49. package/DEPLOYMENT_GUIDE.md +0 -334
  50. package/DEPLOYMENT_SUMMARY_v1.1.3.md +0 -211
  51. package/MCP_INTEGRATION_GUIDE.md +0 -285
  52. package/MIGRATION_GUIDE_v1.4.0.md +0 -190
  53. package/RELEASE_NOTES_v1.1.3.md +0 -321
  54. package/RELEASE_NOTES_v1.2.0.md +0 -396
  55. package/RELEASE_NOTES_v1.3.7.md +0 -275
  56. package/RELEASE_NOTES_v1.4.0.md +0 -178
  57. package/SAFEGUARDS_VERIFICATION_LOG.md +0 -157
  58. package/coordinator-bot-compact.txt +0 -33
  59. package/coordinator-bot-system-prompt.md +0 -192
  60. package/docs/installation.md +0 -71
  61. package/scripts/analyze-data-structure.cjs +0 -74
  62. package/scripts/clean-safeguards-data.cjs +0 -60
  63. package/scripts/comprehensive-validation.cjs +0 -176
  64. package/scripts/count-safeguards.cjs +0 -89
  65. package/scripts/format-safeguards-proper.cjs +0 -44
  66. package/scripts/validate-formatted-data.cjs +0 -88
  67. package/test_capability_integration.js +0 -73
  68. package/test_comprehensive_scenarios.js +0 -192
  69. package/test_enhanced_detection.js +0 -74
  70. package/test_integrated_validation.js +0 -112
  71. package/test_language_update.js +0 -101
  72. package/test_live_validation.json +0 -12
  73. package/test_performance_monitoring.js +0 -124
  74. package/test_runner_automated.js +0 -156
  75. package/test_suite_comprehensive.js +0 -218
@@ -1,158 +0,0 @@
1
- # Framework MCP: Capability Transformation Summary
2
-
3
- ## 🎯 Mission Accomplished: Second 2-Day Increment Complete
4
-
5
- ### **Paradigm Shift Achievement**: From Compliance Scoring → Capability Assessment
6
-
7
- ---
8
-
9
- ## 📋 Tasks Completed (Days 3-4)
10
-
11
- ### ✅ **Task 1**: Integrate capability-focused analysis with existing domain validation logic
12
- - **COMPLETED**: Successfully integrated `performCapabilityAnalysis` with `validate_vendor_mapping` tool
13
- - **Key Change**: Replaced percentage-based compliance scoring with capability categorization
14
- - **Result**: Tool now answers "What role does this vendor play?" instead of "How much compliance coverage?"
15
-
16
- ### ✅ **Task 2**: Update domain validation to work with new capability categorization
17
- - **COMPLETED**: Updated domain validation logic to work with capability types instead of coverage percentages
18
- - **Key Change**: Domain mismatch now triggers capability downgrade (FULL/PARTIAL → FACILITATES)
19
- - **Result**: Realistic capability mappings that prevent inappropriate implementation claims
20
-
21
- ### ✅ **Task 3**: Enhance tool type detection with implementation indicators
22
- - **COMPLETED**: Implemented weighted scoring system with safeguard-specific context bonuses
23
- - **Key Features**:
24
- - Primary keywords (3 points) vs Secondary keywords (1 point)
25
- - Context bonuses (+1) for domain-aligned tool types
26
- - Minimum threshold (≥2 points) to avoid false positives
27
- - 7 comprehensive tool categories with 100+ keywords
28
- - **Result**: More accurate tool categorization aligned with actual safeguard requirements
29
-
30
- ### ✅ **Task 4**: Test integrated system with comprehensive domain validation scenarios
31
- - **COMPLETED**: Created and validated 8 comprehensive test scenarios covering:
32
- - 3 PASS scenarios (domain-aligned tools)
33
- - 3 DOWNGRADE scenarios (domain mismatches)
34
- - 2 NO-DOWNGRADE scenarios (non-implementation capabilities)
35
- - **Result**: 100% expected behavior validation across all capability types and domains
36
-
37
- ---
38
-
39
- ## 🔧 Technical Achievements
40
-
41
- ### **1. Enhanced Tool Type Detection**
42
- ```typescript
43
- // Before: Simple keyword matching
44
- if (text.includes('inventory')) return 'inventory';
45
-
46
- // After: Weighted scoring with context
47
- const score = primaryKeywords.reduce((s, kw) => s + (text.includes(kw) ? 3 : 0), 0) +
48
- secondaryKeywords.reduce((s, kw) => s + (text.includes(kw) ? 1 : 0), 0) +
49
- (safeguardContextBonus ? 1 : 0);
50
- return score >= 2 ? toolType : 'unknown';
51
- ```
52
-
53
- ### **2. Capability-Focused Analysis Integration**
54
- ```typescript
55
- // Before: Compliance percentage calculation
56
- const coverageScore = (matchedElements / totalElements) * 100;
57
-
58
- // After: Capability determination
59
- const claimedCapability = this.determineClaimedCapability(text, safeguard);
60
- const qualityAssessment = this.assessCapabilityQuality(text, safeguard, claimedCapability);
61
- ```
62
-
63
- ### **3. Domain Validation with Auto-Downgrade**
64
- ```typescript
65
- // New logic: Prevents inappropriate claims
66
- if (isFullOrPartial && !toolTypeMatches) {
67
- return {
68
- domain_match: false,
69
- should_adjust_capability: true,
70
- adjusted_capability: 'facilitates',
71
- reasoning: `${domainReq.domain} requires ${domainReq.required_tool_types.join('/')} tools`
72
- };
73
- }
74
- ```
75
-
76
- ---
77
-
78
- ## 📊 Validation Results
79
-
80
- ### **Domain Alignment Tests**
81
- - ✅ **Asset Management → Asset Inventory (1.1)**: SUPPORTED
82
- - ✅ **Identity Management → Account Inventory (5.1)**: SUPPORTED
83
- - ✅ **Vulnerability Management → Vuln Process (7.1)**: SUPPORTED
84
-
85
- ### **Domain Mismatch Auto-Downgrade Tests**
86
- - ⬇️ **Threat Intel → Asset Inventory**: FULL → FACILITATES (QUESTIONABLE)
87
- - ⬇️ **Asset Discovery → Account Inventory**: FULL → FACILITATES (QUESTIONABLE)
88
- - ⬇️ **SIEM → Asset Inventory**: PARTIAL → FACILITATES (QUESTIONABLE)
89
-
90
- ### **Non-Implementation Capability Preservation**
91
- - ✅ **Vulnerability Scanner → Asset Inventory (FACILITATES)**: No downgrade
92
- - ✅ **GRC Platform → Asset Inventory (GOVERNANCE)**: No downgrade
93
-
94
- ---
95
-
96
- ## 🎯 Key Capability Assessment Categories
97
-
98
- The system now correctly categorizes vendors into:
99
-
100
- ### **1. Core Implementation Tools**
101
- - **FULL/PARTIAL** capabilities for tools that directly implement safeguard requirements
102
- - **Domain Requirement**: Must match safeguard tool type requirements
103
- - **Example**: Asset Management platform claiming FULL for Asset Inventory (1.1) ✅
104
-
105
- ### **2. Enablement Tools**
106
- - **FACILITATES** capability for tools that enhance/enable implementation
107
- - **Domain Flexibility**: Any tool type can facilitate any safeguard
108
- - **Example**: Vulnerability scanner facilitating Asset Inventory through discovery data ✅
109
-
110
- ### **3. Governance Tools**
111
- - **GOVERNANCE** capability for policy/process management platforms
112
- - **Domain Flexibility**: Can provide governance for any safeguard
113
- - **Example**: GRC platform providing governance for Asset Inventory processes ✅
114
-
115
- ### **4. Compliance Tools**
116
- - **VALIDATES** capability for evidence/reporting/monitoring tools
117
- - **Domain Flexibility**: Can validate any safeguard implementation
118
- - **Example**: SIEM providing validation through asset monitoring logs ✅
119
-
120
- ---
121
-
122
- ## 📈 Business Impact
123
-
124
- ### **Before**: Misleading Compliance Scoring
125
- - Vendors could claim unrealistic coverage percentages
126
- - Focus on "how much of safeguard does tool cover?"
127
- - Led to overestimated vendor capabilities
128
- - Compliance theater rather than actual capability assessment
129
-
130
- ### **After**: Realistic Capability Assessment
131
- - Vendors categorized by actual tool role in safeguard ecosystem
132
- - Focus on "what does this tool do for the safeguard?"
133
- - Prevents inappropriate implementation claims through domain validation
134
- - Enables accurate vendor selection and capability planning
135
-
136
- ---
137
-
138
- ## 🚀 Next Steps (Remaining 4 Tasks for Days 5-8)
139
-
140
- The foundation is now solid for the final tasks:
141
-
142
- 5. **Update response templates** → Capability-focused language
143
- 6. **Create test suite** → Automated testing for CI/CD
144
- 7. **Performance optimization** → Production readiness
145
- 8. **Documentation update** → Version 1.1.3 release prep
146
-
147
- ---
148
-
149
- ## 💯 Success Metrics
150
-
151
- - ✅ **100% Test Coverage**: All 8 validation scenarios behave as expected
152
- - ✅ **Zero False Positives**: Domain validation prevents inappropriate claims
153
- - ✅ **Enhanced Accuracy**: Context-aware tool detection with weighted scoring
154
- - ✅ **Paradigm Achievement**: Successful shift from compliance scoring to capability assessment
155
-
156
- ---
157
-
158
- **🎊 Days 3-4 Complete: The Framework MCP now correctly answers "What role does each vendor tool play in the safeguard ecosystem?" instead of trying to calculate compliance percentages. This fundamental shift enables accurate capability planning and realistic vendor assessments.**
@@ -1,220 +0,0 @@
1
- # CIS Controls v8.1 Implementation Plan
2
-
3
- ## Current Status
4
- - **Completed**: 59 safeguards implemented (Controls 1-7 complete)
5
- - **Remaining**: Controls 8-18 (11 controls, estimated 40+ additional safeguards)
6
- - **Available Resources**: Individual PDF documents for each control in docs/ folder
7
-
8
- ## Implementation Strategy
9
-
10
- ### Sequential Processing Order
11
- Process controls in this specific order for optimal dependency management:
12
-
13
- #### Phase 1: Technical Infrastructure Controls (High Priority)
14
- 1. **Control 8: Audit Log Management**
15
- - Foundation for security monitoring and detection
16
- - PDF: `CISv8.1-Control 8.pdf`
17
- - Estimated safeguards: 6-8
18
-
19
- 2. **Control 10: Malware Defenses**
20
- - Critical protection control
21
- - PDF: `CISv8.1-Control 10.pdf`
22
- - Estimated safeguards: 6-8
23
-
24
- 3. **Control 12: Network Infrastructure Management**
25
- - Network security foundation
26
- - PDF: `CISv8.1-Control 12.pdf`
27
- - Estimated safeguards: 6-8
28
-
29
- 4. **Control 13: Network Monitoring and Defense**
30
- - Builds on Control 12, detection capabilities
31
- - PDF: `CISv8.1-Control 13.pdf`
32
- - Estimated safeguards: 6-8
33
-
34
- #### Phase 2: Data and Application Controls (Medium-High Priority)
35
- 5. **Control 9: Email and Web Browser Protections**
36
- - Common attack vectors
37
- - PDF: `CISv8.1-Control 9.pdf`
38
- - Estimated safeguards: 4-6
39
-
40
- 6. **Control 11: Data Recovery**
41
- - Business continuity foundation
42
- - PDF: `CISv8.1-Control 11.pdf`
43
- - Estimated safeguards: 3-5
44
-
45
- 7. **Control 16: Application Software Security**
46
- - Application layer security
47
- - PDF: `CISv8.1-Control 16.pdf`
48
- - Estimated safeguards: 6-8
49
-
50
- #### Phase 3: Process and Governance Controls (Medium Priority)
51
- 8. **Control 14: Security Awareness and Skills Training**
52
- - Human element, process-focused
53
- - PDF: `CISv8.1-Control 14.pdf`
54
- - Estimated safeguards: 4-6
55
-
56
- 9. **Control 15: Service Provider Management**
57
- - Third-party risk management
58
- - PDF: `CISv8.1-Control 15.pdf`
59
- - Estimated safeguards: 4-6
60
-
61
- 10. **Control 17: Incident Response Management**
62
- - Response processes and capabilities
63
- - PDF: `CISv8.1-Control 17.pdf`
64
- - Estimated safeguards: 6-8
65
-
66
- 11. **Control 18: Penetration Testing**
67
- - Validation and testing processes
68
- - PDF: `CISv8.1-Control 18.pdf`
69
- - Estimated safeguards: 3-5
70
-
71
- ## Standard Session Template
72
-
73
- For each control, follow this systematic approach:
74
-
75
- ### 1. Initialize Session
76
- ```bash
77
- # Update todo list with current control
78
- # Example: "Process Control 8: Audit Log Management"
79
- ```
80
-
81
- ### 2. Read and Analyze PDF
82
- - Read the complete control PDF document
83
- - Identify all safeguards in the control (e.g., 8.1, 8.2, 8.3, etc.)
84
- - Note control title, description, and overall purpose
85
- - Extract Implementation Group classifications
86
-
87
- ### 3. Extract Safeguard Elements
88
- For each safeguard, categorize elements using the color-coding system:
89
-
90
- - **🟠 Orange (Governance Elements)**: Process/policy requirements that MUST be met
91
- - **🟢 Green (Core Requirements)**: The essential "what" of the safeguard
92
- - **🟡 Yellow (Sub-taxonomical Elements)**: Detailed sub-components
93
- - **⚫ Gray (Implementation Suggestions)**: Suggested methods and tools
94
-
95
- Additional metadata:
96
- - Asset types (devices, applications, users, etc.)
97
- - Security functions (Identify, Protect, Detect, Respond, Recover)
98
- - Relevant keywords for analysis engine
99
-
100
- ### 4. Implement in Code
101
- ```typescript
102
- // Add to CIS_SAFEGUARDS object in src/index.ts
103
- "X.Y": {
104
- id: "X.Y",
105
- title: "Safeguard Title",
106
- description: "Description...",
107
- implementationGroup: "IG1" | "IG2" | "IG3",
108
- assetType: [...],
109
- securityFunction: [...],
110
- governanceElements: [...], // Orange
111
- coreRequirements: [...], // Green
112
- subTaxonomicalElements: [...], // Yellow
113
- implementationSuggestions: [...], // Gray
114
- relatedSafeguards: [...],
115
- keywords: [...]
116
- }
117
- ```
118
-
119
- ### 5. Test and Validate
120
- ```bash
121
- npm run build # Verify compilation
122
- # Test with sample vendor responses
123
- # Verify analysis engine processes new safeguards correctly
124
- ```
125
-
126
- ### 6. Document and Commit
127
- ```bash
128
- git add .
129
- git commit -m "Add Control X: [Control Name] - implemented X.1 through X.Y safeguards
130
-
131
- - Extracted all safeguards from CISv8.1-Control X.pdf
132
- - Categorized elements using color-coding system
133
- - Added appropriate keywords and metadata
134
- - Tested with analysis engine
135
-
136
- 🤖 Generated with [Claude Code](https://claude.ai/code)
137
-
138
- Co-Authored-By: Claude <noreply@anthropic.com>"
139
-
140
- git push origin main
141
- ```
142
-
143
- ## Success Criteria for Each Control
144
-
145
- ✅ **Extraction Complete**
146
- - [ ] All safeguards identified and extracted from PDF
147
- - [ ] Elements properly categorized (Orange/Green/Yellow/Gray)
148
- - [ ] Implementation Groups assigned correctly
149
-
150
- ✅ **Code Implementation**
151
- - [ ] All safeguards added to CIS_SAFEGUARDS object
152
- - [ ] Consistent formatting with existing controls
153
- - [ ] Code compiles without errors
154
- - [ ] Proper TypeScript typing maintained
155
-
156
- ✅ **Testing and Validation**
157
- - [ ] Analysis engine correctly processes new safeguards
158
- - [ ] Capability categorization works appropriately
159
- - [ ] Sample vendor responses produce expected results
160
-
161
- ✅ **Documentation and Commit**
162
- - [ ] Progress tracking updated
163
- - [ ] Detailed commit message with control summary
164
- - [ ] Changes committed and pushed to repository
165
-
166
- ## Progress Tracking
167
-
168
- ### Completed Controls
169
- - ✅ Control 1: Inventory and Control of Enterprise Assets
170
- - ✅ Control 2: Inventory and Control of Software Assets
171
- - ✅ Control 3: Data Protection
172
- - ✅ Control 4: Secure Configuration of Enterprise Assets and Software
173
- - ✅ Control 5: Account Management
174
- - ✅ Control 6: Access Control Management
175
- - ✅ Control 7: Continuous Vulnerability Management
176
- - ✅ **Control 8: Audit Log Management** (12 safeguards: 8.1-8.12)
177
- - ✅ **Control 10: Malware Defenses** (7 safeguards: 10.1-10.7)
178
- - ✅ **Control 11: Data Recovery** (5 safeguards: 11.1-11.5)
179
- - ✅ **Control 9: Email and Web Browser Protections** (7 safeguards: 9.1-9.7)
180
- - ✅ **Control 12: Network Infrastructure Management** (8 safeguards: 12.1-12.8)
181
- - ✅ **Control 13: Network Monitoring and Defense** (11 safeguards: 13.1-13.11)
182
- - ✅ **Control 14: Security Awareness and Skills Training** (9 safeguards: 14.1-14.9)
183
- - ✅ **Control 15: Service Provider Management** (7 safeguards: 15.1-15.7)
184
- - ✅ **Control 16: Application Software Security** (14 safeguards: 16.1-16.14)
185
- - ✅ **Control 17: Incident Response Management** (9 safeguards: 17.1-17.9)
186
- - ✅ **Control 18: Penetration Testing** (5 safeguards: 18.1-18.5)
187
-
188
- ### ✅ ALL CONTROLS COMPLETE
189
- **Total Implementation**: 153 safeguards across all 18 CIS Controls v8.1
190
-
191
- ### Final Status
192
- - [ ] **COMPLETE**: All CIS Controls v8.1 safeguards implemented
193
- - [ ] **Framework Ready**: Full vendor analysis capability enabled
194
-
195
- ## Risk Mitigation
196
-
197
- ### Context Limit Management
198
- - Process only one control per session to avoid PDF context limits
199
- - Use targeted reading with offsets if PDFs are very large
200
- - Focus on extracting structured information efficiently
201
-
202
- ### Quality Assurance
203
- - Reference existing controls 1-7 as formatting templates
204
- - Validate element categorization against established patterns
205
- - Test each implementation before moving to next control
206
-
207
- ### Recovery Strategy
208
- - Detailed progress tracking enables restart from any point
209
- - Each control is independent (no cascading failures)
210
- - Commit after each successful control completion
211
- - Can skip complex controls and return later if needed
212
-
213
- ## Estimated Timeline
214
- - **Per Control**: 1-2 hours (extraction + implementation + testing)
215
- - **Total Remaining**: ~15-20 hours of focused work
216
- - **Completion Target**: Can be spread across multiple sessions safely
217
-
218
- ---
219
-
220
- **Next Action**: Begin with Control 8 (Audit Log Management) using the standard session template.
@@ -1,322 +0,0 @@
1
- # Microsoft Copilot Integration Guide
2
-
3
- This guide provides step-by-step instructions for integrating Framework MCP v1.4.0's Pure Data Provider architecture with Microsoft Copilot using custom connectors.
4
-
5
- ## Overview
6
-
7
- Framework MCP v1.4.0 provides a **Pure Data Provider architecture** with dual interfaces enabling both Model Context Protocol (MCP) integration with Claude Code and HTTP API integration with Microsoft Copilot. This empowers security professionals to access authoritative CIS Controls data through their preferred AI platform for sophisticated, LLM-driven vendor capability analysis.
8
-
9
- ## Prerequisites
10
-
11
- - **Microsoft Copilot Studio** access (Microsoft 365 Business license or higher)
12
- - **Deployed Framework MCP HTTP API** (DigitalOcean, Railway, Render, or other cloud platform)
13
- - **Administrative permissions** in your Microsoft environment
14
-
15
- ## Quick Start
16
-
17
- ### 1. Deploy the HTTP API
18
-
19
- First, deploy Framework MCP to a cloud platform. Choose one of these options:
20
-
21
- #### Option A: DigitalOcean App Services (Recommended)
22
- ```bash
23
- git clone https://github.com/therealcybermattlee/FrameworkMCP.git
24
- cd FrameworkMCP
25
- doctl apps create .do/app.yaml
26
- ```
27
-
28
- #### Option B: Railway
29
- ```bash
30
- railway login
31
- railway up
32
- ```
33
-
34
- #### Option C: Render
35
- 1. Connect your GitHub repository to Render
36
- 2. Use these settings:
37
- - **Build Command**: `npm install && npm run build`
38
- - **Start Command**: `npm run start:http`
39
- - **Port**: 8080
40
-
41
- ### 2. Verify Deployment
42
-
43
- Test your deployed API:
44
- ```bash
45
- curl https://your-api-url.com/health
46
- ```
47
-
48
- Expected response:
49
- ```json
50
- {
51
- "status": "healthy",
52
- "uptime": 1234,
53
- "totalRequests": 0,
54
- "errorCount": 0,
55
- "version": "1.4.0",
56
- "timestamp": "2025-08-21T01:00:00.000Z"
57
- }
58
- ```
59
-
60
- ### 3. Create Custom Connector in Copilot Studio
61
-
62
- 1. **Open Microsoft Copilot Studio**
63
- - Navigate to [https://copilotstudio.microsoft.com](https://copilotstudio.microsoft.com)
64
- - Sign in with your Microsoft 365 account
65
-
66
- 2. **Access Custom Connectors**
67
- - In the left navigation, click **Data**
68
- - Select **Custom connectors**
69
-
70
- 3. **Import OpenAPI Specification**
71
- - Click **+ New custom connector**
72
- - Choose **Import from OpenAPI file**
73
- - Upload the `swagger.json` file from the Framework MCP repository
74
- - Click **Continue**
75
-
76
- 4. **Configure Connector Details**
77
- - **Name**: Framework MCP - Pure Data Provider
78
- - **Description**: Authoritative CIS Controls Framework data for LLM-driven analysis
79
- - **Host**: Update to your deployed API URL (e.g., `your-app.ondigitalocean.app`)
80
- - **Base URL**: `/`
81
-
82
- 5. **Review and Save**
83
- - Review the imported endpoints
84
- - Click **Create connector**
85
-
86
- ### 4. Test the Connection
87
-
88
- 1. **Create Test Connection**
89
- - Click **Test** in your custom connector
90
- - No authentication is required (public API)
91
- - Click **Create connection**
92
-
93
- 2. **Test Health Endpoint**
94
- - Select the `healthCheck` operation
95
- - Click **Test operation**
96
- - Verify you receive a 200 response with health status
97
-
98
- ### 5. Create Your Copilot
99
-
100
- 1. **Create New Copilot**
101
- - Go to **Copilots** in Copilot Studio
102
- - Click **+ New copilot**
103
- - Choose **Conversational** type
104
- - Name: "CIS Controls Capability Assessor"
105
-
106
- 2. **Add Custom Actions**
107
- - In your copilot, go to **Actions**
108
- - Click **+ Add action**
109
- - Select **Custom connector**
110
- - Choose your Framework MCP connector
111
-
112
- ## Data Actions Configuration
113
-
114
- ### Primary Action: Get Safeguard Details
115
-
116
- Configure this action for detailed CIS safeguard information:
117
-
118
- **Action Details:**
119
- - **Name**: Get CIS Safeguard Details
120
- - **Description**: Retrieve detailed CIS Controls safeguard breakdown for LLM analysis
121
- - **Connector**: Framework MCP Custom Connector
122
- - **Operation**: `getSafeguardDetails`
123
-
124
- **Parameters:**
125
- - **safeguardId** (Required): CIS safeguard ID (e.g., "1.1", "5.1", "12.8")
126
- - **include_examples** (Optional): Include implementation examples (true/false)
127
-
128
- **Sample Input:**
129
- ```json
130
- {
131
- "safeguardId": "5.1",
132
- "include_examples": true
133
- }
134
- ```
135
-
136
- ### Secondary Action: List All Safeguards
137
-
138
- Configure this action for safeguard discovery:
139
-
140
- **Action Details:**
141
- - **Name**: List CIS Safeguards
142
- - **Description**: Get complete list of available CIS Controls safeguards
143
- - **Connector**: Framework MCP Custom Connector
144
- - **Operation**: `listSafeguards`
145
-
146
- **No Parameters Required**
147
-
148
- ## Example User Interactions
149
-
150
- Once configured, users can interact with your Copilot using natural language for LLM-driven analysis:
151
-
152
- ### LLM-Driven Capability Assessment
153
-
154
- ```
155
- "Get details for safeguard 1.1, then analyze Microsoft Defender for Endpoint's capability: 'Microsoft Defender provides comprehensive asset discovery, detailed hardware and software inventory, and real-time asset status monitoring across all enterprise endpoints.' Determine appropriate capability role and provide confidence assessment."
156
-
157
- "Retrieve safeguard 5.1 requirements and assess CrowdStrike Falcon: 'We offer centralized identity integration with detailed user account tracking and automated access reviews.' Consider enterprise environment and provide implementation recommendations."
158
-
159
- "Get safeguard 6.3 details. For a financial services organization with strict compliance requirements, analyze Okta's MFA solution: 'We provide comprehensive multi-factor authentication for all external applications with adaptive authentication policies, risk-based assessments, and SSO integration.'"
160
- ```
161
-
162
- ### Comparative Analysis Examples
163
-
164
- ```
165
- "Get safeguard 7.1 details and compare these vulnerability management solutions: 1) Rapid7 InsightVM: 'Continuous vulnerability scanning with automated remediation guidance' 2) Qualys VMDR: 'Cloud-based vulnerability scanning with patch prioritization' 3) Tenable Nessus: 'Comprehensive vulnerability assessment with compliance reporting'. Rank by implementation completeness."
166
-
167
- "Retrieve details for safeguards 13.1, 13.3, and 13.6. Analyze how SentinelOne addresses these network monitoring requirements and identify any coverage gaps."
168
- ```
169
-
170
- ### Data Discovery Requests
171
-
172
- ```
173
- "Show me all available CIS safeguards related to identity and access management."
174
-
175
- "Get detailed breakdown of safeguard 11.1 including all sub-elements and governance requirements."
176
-
177
- "List all Implementation Group 1 (IG1) safeguards for a small business security assessment."
178
- ```
179
-
180
- ## Advanced Configuration
181
-
182
- ### Custom Prompts and Instructions
183
-
184
- Add these instructions to your Copilot's system message:
185
-
186
- ```
187
- You are a CIS Controls expert powered by Framework MCP's Pure Data Provider architecture. You help security professionals perform sophisticated vendor capability analysis using authoritative CIS Controls data.
188
-
189
- Key Principles:
190
- 1. Always retrieve detailed safeguard data using get_safeguard_details before analysis
191
- 2. Apply context-aware analysis considering industry, risk profile, and organizational needs
192
- 3. Explain the 5 capability roles: FULL, PARTIAL, FACILITATES, GOVERNANCE, VALIDATES
193
- 4. Provide transparent reasoning with evidence-based assessments
194
- 5. Offer implementation recommendations and deployment guidance
195
-
196
- Analysis Approach:
197
- 1. Get safeguard requirements using the data actions
198
- 2. Analyze vendor capabilities against detailed CIS requirements
199
- 3. Consider organizational context (industry, size, risk tolerance)
200
- 4. Provide capability role determination with confidence assessment
201
- 5. Include implementation recommendations and integration considerations
202
- 6. For comparative analysis, rank vendors and explain reasoning
203
-
204
- Flexibility: Adapt analysis methodology based on user needs - strict compliance, risk-based assessment, technology integration, or custom evaluation criteria.
205
- ```
206
-
207
- ### Environment Variables
208
-
209
- If deploying with authentication or custom configuration:
210
-
211
- ```bash
212
- # Optional environment variables for your deployed API
213
- NODE_ENV=production
214
- PORT=8080
215
- ALLOWED_ORIGINS=https://copilotstudio.microsoft.com,https://your-domain.com
216
- ```
217
-
218
- ### Rate Limiting and Monitoring
219
-
220
- Monitor your API usage through the `/api/metrics` endpoint:
221
-
222
- ```bash
223
- curl https://your-api-url.com/api/metrics
224
- ```
225
-
226
- Response includes:
227
- - Request counts by endpoint
228
- - Error rates
229
- - Uptime statistics
230
- - Performance metrics
231
-
232
- ## Troubleshooting
233
-
234
- ### Common Issues
235
-
236
- **Connection Failed:**
237
- - Verify your API URL is accessible publicly
238
- - Check that the `/health` endpoint returns 200
239
- - Ensure no authentication is required
240
-
241
- **Operations Not Working:**
242
- - Verify the `swagger.json` was imported correctly
243
- - Check that operation IDs match: `validateVendorMapping`, `analyzeVendorResponse`, etc.
244
- - Test operations individually in the connector test interface
245
-
246
- **Invalid Responses:**
247
- - Check request body format matches the schema
248
- - Verify required fields are provided
249
- - Review API error messages for validation details
250
-
251
- **Performance Issues:**
252
- - Monitor the `/api/metrics` endpoint for high error rates
253
- - Consider upgrading your cloud deployment plan
254
- - Check network connectivity from Microsoft's servers
255
-
256
- ### Testing Checklist
257
-
258
- Before going live, test these scenarios:
259
-
260
- - [ ] Health check endpoint responds correctly
261
- - [ ] Validate vendor mapping with FULL claim (domain match)
262
- - [ ] Validate vendor mapping with FULL claim (domain mismatch - should auto-downgrade)
263
- - [ ] Analyze vendor response for capability determination
264
- - [ ] Get safeguard details for a specific safeguard
265
- - [ ] List all available safeguards
266
- - [ ] Handle invalid inputs gracefully
267
-
268
- ### Support Resources
269
-
270
- - **Framework MCP Issues**: [GitHub Issues](https://github.com/therealcybermattlee/FrameworkMCP/issues)
271
- - **Microsoft Copilot Studio Docs**: [Microsoft Documentation](https://docs.microsoft.com/en-us/microsoft-copilot-studio/)
272
- - **OpenAPI Connector Guide**: [Custom Connectors in Copilot Studio](https://docs.microsoft.com/en-us/connectors/custom-connectors/)
273
-
274
- ## Best Practices
275
-
276
- ### Security Considerations
277
-
278
- 1. **Deploy with HTTPS**: Always use HTTPS for production deployments
279
- 2. **Monitor Usage**: Regularly check the `/api/metrics` endpoint
280
- 3. **Rate Limiting**: Consider implementing rate limiting for high-traffic scenarios
281
- 4. **Access Logs**: Enable logging in production for audit trails
282
-
283
- ### Performance Optimization
284
-
285
- 1. **CDN**: Consider using a CDN for global deployments
286
- 2. **Caching**: The API includes intelligent caching for repeated requests
287
- 3. **Scaling**: Use cloud platform auto-scaling for high demand
288
- 4. **Monitoring**: Set up health check monitoring with your cloud provider
289
-
290
- ### User Experience
291
-
292
- 1. **Clear Instructions**: Provide users with example prompts
293
- 2. **Error Handling**: Explain what to do when validation fails
294
- 3. **Capability Education**: Help users understand the 5 capability roles
295
- 4. **Domain Validation**: Explain why auto-downgrades occur
296
-
297
- ## Success Metrics
298
-
299
- Track these metrics to measure integration success:
300
-
301
- - **Request Volume**: Total API calls per day/week
302
- - **Error Rate**: Percentage of failed requests (target: <5%)
303
- - **Response Time**: Average API response time (target: <500ms)
304
- - **User Satisfaction**: Feedback on capability assessment accuracy
305
- - **Coverage**: Number of unique safeguards being assessed
306
-
307
- ## Next Steps
308
-
309
- After successful integration:
310
-
311
- 1. **Train Users**: Provide training on the 5 capability roles and domain validation
312
- 2. **Create Templates**: Develop standard prompts for common assessment scenarios
313
- 3. **Integration Workflows**: Connect with existing GRC tools and processes
314
- 4. **Feedback Loop**: Collect user feedback for continuous improvement
315
- 5. **Scale Usage**: Expand to additional teams and use cases
316
-
317
- ---
318
-
319
- **Questions or Issues?**
320
- - 📧 Create an issue: [GitHub Issues](https://github.com/therealcybermattlee/FrameworkMCP/issues)
321
- - 💬 Start a discussion: [GitHub Discussions](https://github.com/therealcybermattlee/FrameworkMCP/discussions)
322
- - 🐦 Follow updates: [@cybermattlee](https://twitter.com/cybermattlee)