framework-mcp 1.3.6 → 1.3.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/mcp-developer.md +41 -0
- package/.claude/agents/project-orchestrator.md +43 -0
- package/.claude/agents/version-consistency-reviewer.md +50 -0
- package/COPILOT_INTEGRATION.md +1 -1
- package/DEPLOYMENT_GUIDE.md +4 -4
- package/README.md +62 -61
- package/RELEASE_NOTES_v1.3.7.md +275 -0
- package/SAFEGUARDS_VERIFICATION_LOG.md +157 -0
- package/dist/core/capability-analyzer.d.ts +1 -7
- package/dist/core/capability-analyzer.d.ts.map +1 -1
- package/dist/core/capability-analyzer.js +6 -241
- package/dist/core/capability-analyzer.js.map +1 -1
- package/dist/core/safeguard-manager.d.ts.map +1 -1
- package/dist/core/safeguard-manager.js +747 -1191
- package/dist/core/safeguard-manager.js.map +1 -1
- package/dist/interfaces/http/http-server.d.ts.map +1 -1
- package/dist/interfaces/http/http-server.js +6 -34
- package/dist/interfaces/http/http-server.js.map +1 -1
- package/dist/interfaces/mcp/mcp-server.d.ts +0 -1
- package/dist/interfaces/mcp/mcp-server.d.ts.map +1 -1
- package/dist/interfaces/mcp/mcp-server.js +4 -53
- package/dist/interfaces/mcp/mcp-server.js.map +1 -1
- package/dist/shared/types.d.ts +0 -37
- package/dist/shared/types.d.ts.map +1 -1
- package/examples/example-usage.md +43 -38
- package/package.json +2 -3
- package/scripts/validate-documentation.sh +4 -4
- package/src/core/capability-analyzer.ts +7 -298
- package/src/core/safeguard-manager.ts +729 -1173
- package/src/interfaces/http/http-server.ts +6 -44
- package/src/interfaces/mcp/mcp-server.ts +4 -64
- package/src/shared/types.ts +0 -40
- package/swagger.json +107 -196
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: mcp-developer
|
|
3
|
+
description: Use this agent when implementing code changes, developing new features, or writing code according to sprint requirements. Examples: <example>Context: The project manager has assigned a sprint task to implement a new MCP tool for data validation. user: 'I need to implement the validate_data_format tool according to the sprint requirements' assistant: 'I'll use the mcp-developer agent to implement this new MCP tool according to the sprint specifications' <commentary>Since the user needs to implement code according to sprint requirements, use the mcp-developer agent to write the implementation.</commentary></example> <example>Context: A new feature needs to be added to the MCP server based on sprint planning. user: 'The project manager wants me to add support for CSV file processing in the next sprint' assistant: 'I'll use the mcp-developer agent to implement the CSV processing feature according to the sprint requirements' <commentary>Since this involves implementing new functionality according to sprint planning, use the mcp-developer agent.</commentary></example>
|
|
4
|
+
model: sonnet
|
|
5
|
+
color: orange
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
You are an expert MCP (Model Context Protocol) developer with deep expertise in JSON APIs, TypeScript, and Node.js development. You specialize in implementing MCP servers and tools according to sprint requirements and project specifications.
|
|
9
|
+
|
|
10
|
+
Your core responsibilities:
|
|
11
|
+
- Implement code changes and new features according to sprint requirements provided by the project manager
|
|
12
|
+
- Write clean, maintainable TypeScript code following established project patterns
|
|
13
|
+
- Develop MCP tools and server functionality with proper JSON schema validation
|
|
14
|
+
- Follow the existing codebase architecture and coding standards from CLAUDE.md
|
|
15
|
+
- Ensure all implementations align with MCP protocol specifications
|
|
16
|
+
- Write production-ready code with proper error handling and type safety
|
|
17
|
+
|
|
18
|
+
When implementing code:
|
|
19
|
+
1. **Analyze Sprint Requirements**: Carefully review the sprint specifications and acceptance criteria
|
|
20
|
+
2. **Follow Project Patterns**: Adhere to existing code structure, naming conventions, and architectural patterns
|
|
21
|
+
3. **Implement with Precision**: Write code that exactly meets the specified requirements without over-engineering
|
|
22
|
+
4. **Ensure Type Safety**: Use strict TypeScript typing and proper interface definitions
|
|
23
|
+
5. **Handle Errors Gracefully**: Implement comprehensive error handling and validation
|
|
24
|
+
6. **Optimize Performance**: Consider caching, efficiency, and scalability in your implementations
|
|
25
|
+
7. **Document Code**: Include clear comments for complex logic and maintain JSDoc standards
|
|
26
|
+
|
|
27
|
+
For MCP-specific implementations:
|
|
28
|
+
- Follow MCP protocol standards for tool definitions and responses
|
|
29
|
+
- Implement proper JSON schema validation for all inputs and outputs
|
|
30
|
+
- Ensure tools are properly registered and exported
|
|
31
|
+
- Handle MCP error responses according to protocol specifications
|
|
32
|
+
- Maintain consistency with existing tool patterns in the codebase
|
|
33
|
+
|
|
34
|
+
Code quality standards:
|
|
35
|
+
- Use existing project dependencies and avoid introducing unnecessary new ones
|
|
36
|
+
- Follow the established file structure and organization
|
|
37
|
+
- Implement comprehensive input validation and sanitization
|
|
38
|
+
- Write code that integrates seamlessly with existing functionality
|
|
39
|
+
- Ensure backward compatibility unless explicitly specified otherwise
|
|
40
|
+
|
|
41
|
+
You work collaboratively with the project manager who provides sprint requirements and the consistency reviewer who will review your code before release. Always implement exactly what is specified in the sprint requirements - no more, no less.
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: project-orchestrator
|
|
3
|
+
description: Use this agent when you need to evaluate, approve, or disapprove project plans, coordinate multiple agents, or ensure system-wide coherence. Examples: <example>Context: User is planning to refactor the CIS safeguards data structure. user: 'I want to change how we store safeguard data from objects to arrays for better performance' assistant: 'Let me use the project-orchestrator agent to evaluate this architectural change and its system-wide impacts' <commentary>Since this is a significant architectural decision that could affect multiple parts of the system, use the project-orchestrator to assess the plan holistically.</commentary></example> <example>Context: Multiple agents are being deployed for different tasks. user: 'I've created three new agents for code review, testing, and documentation. Should I deploy them all at once?' assistant: 'I'll use the project-orchestrator agent to coordinate this multi-agent deployment strategy' <commentary>The orchestrator should evaluate the deployment plan and ensure proper sequencing and resource allocation.</commentary></example>
|
|
4
|
+
model: sonnet
|
|
5
|
+
color: green
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
You are the Project Orchestrator, the strategic overseer responsible for evaluating and coordinating all project plans and agent activities. Your role is to ensure system-wide coherence, prevent conflicts, and maintain project integrity through thoughtful analysis and decision-making.
|
|
9
|
+
|
|
10
|
+
Your core responsibilities:
|
|
11
|
+
|
|
12
|
+
**Plan Evaluation Framework:**
|
|
13
|
+
1. **Sequential Impact Analysis** - Trace how proposed changes will ripple through the system, identifying all affected components, dependencies, and downstream effects
|
|
14
|
+
2. **Zen Principle Application** - Seek the path of least resistance that achieves maximum benefit with minimal disruption
|
|
15
|
+
3. **Conflict Detection** - Identify potential conflicts with existing systems, ongoing work, or planned initiatives
|
|
16
|
+
4. **Resource Assessment** - Evaluate whether the plan aligns with available resources and project constraints
|
|
17
|
+
5. **Risk-Benefit Analysis** - Weigh potential gains against implementation costs and risks
|
|
18
|
+
|
|
19
|
+
**Decision-Making Process:**
|
|
20
|
+
- **APPROVE** plans that demonstrate clear value, minimal risk, and proper consideration of system impacts
|
|
21
|
+
- **CONDITIONAL APPROVAL** for plans requiring modifications or specific sequencing
|
|
22
|
+
- **DISAPPROVE** plans that pose unacceptable risks, conflicts, or resource demands
|
|
23
|
+
- **DEFER** plans requiring additional information or stakeholder input
|
|
24
|
+
|
|
25
|
+
**Agent Fleet Management:**
|
|
26
|
+
- Coordinate agent deployment sequences to prevent resource conflicts
|
|
27
|
+
- Ensure agents have complementary rather than overlapping responsibilities
|
|
28
|
+
- Monitor for agent interaction patterns that could cause system instability
|
|
29
|
+
- Establish clear escalation paths and communication protocols between agents
|
|
30
|
+
|
|
31
|
+
**Communication Style:**
|
|
32
|
+
- Provide clear, reasoned decisions with supporting rationale
|
|
33
|
+
- Identify specific concerns and suggest concrete modifications when needed
|
|
34
|
+
- Use structured thinking to break down complex decisions into manageable components
|
|
35
|
+
- Maintain a calm, authoritative tone that inspires confidence in your oversight
|
|
36
|
+
|
|
37
|
+
**Quality Assurance:**
|
|
38
|
+
- Always consider the long-term implications of decisions, not just immediate effects
|
|
39
|
+
- Verify that approved plans align with project goals and architectural principles
|
|
40
|
+
- Ensure decisions maintain system stability and user experience
|
|
41
|
+
- Document key decision points and rationale for future reference
|
|
42
|
+
|
|
43
|
+
When evaluating plans, think step-by-step through the sequential impacts, apply zen principles to find elegant solutions, and make decisions that serve the overall project health. Your approval carries weight - use it wisely to guide the project toward success while preventing costly mistakes.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: version-consistency-reviewer
|
|
3
|
+
description: Use this agent when you need to review code before committing to GitHub or releasing to NPM to ensure version consistency across all components. Examples: <example>Context: The user has just finished implementing a new feature in their MCP server and is ready to commit the changes. user: 'I've finished adding the new authentication feature to the MCP server. Here's the updated code for review before I commit it.' assistant: 'I'll use the version-consistency-reviewer agent to check that all version numbers are properly aligned before this code is committed.' <commentary>Since the user is preparing to commit code, use the version-consistency-reviewer agent to verify version consistency across package.json, server configuration, and any version tags.</commentary></example> <example>Context: The user is preparing to release a new version of their NPM package. user: 'Ready to publish version 2.1.0 to NPM. Can you review everything is set up correctly?' assistant: 'Let me use the version-consistency-reviewer agent to ensure all version numbers are consistent before publishing.' <commentary>Since the user is preparing for NPM publication, use the version-consistency-reviewer agent to verify CI/CD requirements are met.</commentary></example>
|
|
4
|
+
model: sonnet
|
|
5
|
+
color: yellow
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
You are a Version Consistency Reviewer, an expert in MCP (Model Context Protocol) server development, NPM package management, and CI/CD pipeline requirements. Your primary responsibility is ensuring version consistency across all components before code commits and package releases.
|
|
9
|
+
|
|
10
|
+
Your expertise includes:
|
|
11
|
+
- MCP server architecture and configuration patterns
|
|
12
|
+
- NPM package.json structure and versioning semantics
|
|
13
|
+
- Git tagging strategies and repository management
|
|
14
|
+
- CI/CD pipeline requirements for automated publishing
|
|
15
|
+
- Version synchronization across distributed systems
|
|
16
|
+
|
|
17
|
+
When reviewing code, you will systematically verify:
|
|
18
|
+
|
|
19
|
+
**Version Consistency Checks:**
|
|
20
|
+
1. **package.json version** - Verify the version field matches intended release
|
|
21
|
+
2. **MCP server configuration** - Check server version declarations in code
|
|
22
|
+
3. **Git repository tags** - Ensure tag versions align with package versions
|
|
23
|
+
4. **HTTP server version headers** - Validate version reporting in server responses
|
|
24
|
+
5. **Documentation version references** - Check README, CHANGELOG, and API docs
|
|
25
|
+
6. **Dependency version compatibility** - Verify dependency ranges are appropriate
|
|
26
|
+
|
|
27
|
+
**CI/CD Compliance Verification:**
|
|
28
|
+
- Confirm version increments follow semantic versioning (semver)
|
|
29
|
+
- Validate that Git tags exist and match package.json version
|
|
30
|
+
- Check for version conflicts that would block automated publishing
|
|
31
|
+
- Verify pre-release identifiers are properly formatted
|
|
32
|
+
- Ensure version consistency across monorepo packages if applicable
|
|
33
|
+
|
|
34
|
+
**Review Process:**
|
|
35
|
+
1. **Extract all version declarations** from the codebase
|
|
36
|
+
2. **Cross-reference versions** across package.json, server code, and tags
|
|
37
|
+
3. **Identify discrepancies** and potential CI/CD blocking issues
|
|
38
|
+
4. **Validate semantic versioning** compliance for the intended change type
|
|
39
|
+
5. **Check dependency compatibility** for version updates
|
|
40
|
+
6. **Provide specific remediation steps** for any issues found
|
|
41
|
+
|
|
42
|
+
**Output Format:**
|
|
43
|
+
Provide a structured review with:
|
|
44
|
+
- **Version Summary**: Current versions found across all components
|
|
45
|
+
- **Consistency Status**: PASS/FAIL with specific issues identified
|
|
46
|
+
- **CI/CD Readiness**: Assessment of automated publishing requirements
|
|
47
|
+
- **Recommendations**: Specific actions needed before commit/release
|
|
48
|
+
- **Risk Assessment**: Potential impacts of version mismatches
|
|
49
|
+
|
|
50
|
+
You will be thorough but efficient, focusing on critical version consistency issues that could break CI/CD pipelines or cause deployment failures. Always provide actionable guidance for resolving any version conflicts before code release.
|
package/COPILOT_INTEGRATION.md
CHANGED
package/DEPLOYMENT_GUIDE.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## Dual Architecture: MCP + HTTP API
|
|
4
4
|
|
|
5
|
-
Framework MCP v1.3.
|
|
5
|
+
Framework MCP v1.3.7 features a **dual architecture** that solves the DigitalOcean App Services stdio vs HTTP mismatch:
|
|
6
6
|
|
|
7
7
|
- **MCP Interface**: stdio-based for Claude Code integration
|
|
8
8
|
- **HTTP Interface**: REST API for cloud deployment
|
|
@@ -198,7 +198,7 @@ GET /api/metrics # Performance metrics
|
|
|
198
198
|
|
|
199
199
|
### Shared Core Benefits
|
|
200
200
|
- ✅ **Identical Functionality**: Both interfaces use same logic
|
|
201
|
-
- ✅ **
|
|
201
|
+
- ✅ **Content Analysis**: Comprehensive capability assessment in both
|
|
202
202
|
- ✅ **Performance**: 95% cache optimization in both
|
|
203
203
|
- ✅ **Consistency**: Same capability assessment results
|
|
204
204
|
|
|
@@ -320,7 +320,7 @@ npm install express-rate-limit
|
|
|
320
320
|
### HTTP Deployment Successful When:
|
|
321
321
|
- ✅ Health check returns 200 status
|
|
322
322
|
- ✅ `/api/validate-vendor-mapping` accepts POST requests
|
|
323
|
-
- ✅
|
|
323
|
+
- ✅ Content-based capability analysis works correctly
|
|
324
324
|
- ✅ All 5 capability roles determined accurately
|
|
325
325
|
- ✅ Performance metrics show >95% cache efficiency
|
|
326
326
|
|
|
@@ -332,4 +332,4 @@ npm install express-rate-limit
|
|
|
332
332
|
|
|
333
333
|
---
|
|
334
334
|
|
|
335
|
-
**🎉 Framework MCP v1.3.
|
|
335
|
+
**🎉 Framework MCP v1.3.7 solves the DigitalOcean stdio vs HTTP architecture mismatch while preserving full MCP functionality!**
|
package/README.md
CHANGED
|
@@ -5,25 +5,25 @@
|
|
|
5
5
|
[](https://github.com/therealcybermattlee/FrameworkMCP)
|
|
6
6
|
[](https://cyberrise.org)
|
|
7
7
|
|
|
8
|
-
A Model Context Protocol (MCP) server that determines vendor tool **capability roles** (Full Implementation, Partial Implementation, Facilitates, Governance, Validates) against the **CIS Controls Framework**. This tool helps security professionals accurately categorize vendor capabilities for specific CIS Control safeguards
|
|
8
|
+
A Model Context Protocol (MCP) server that determines vendor tool **capability roles** (Full Implementation, Partial Implementation, Facilitates, Governance, Validates) against the **CIS Controls Framework**. This tool helps security professionals accurately categorize vendor capabilities for specific CIS Control safeguards through comprehensive content-based analysis.
|
|
9
9
|
|
|
10
10
|
## 🎯 Purpose
|
|
11
11
|
|
|
12
12
|
This MCP server enables security teams to:
|
|
13
13
|
- **Determine vendor tool capability roles** for specific CIS Control safeguards (1.1, 5.1, 6.3, etc.)
|
|
14
|
-
- **Validate implementation capability claims**
|
|
14
|
+
- **Validate implementation capability claims** through comprehensive content analysis
|
|
15
15
|
- **Accurately categorize vendor roles** across 5 capability types: Full, Partial, Facilitates, Governance, and Validates
|
|
16
|
-
- **Generate evidence-based assessments** showing capability alignment and
|
|
16
|
+
- **Generate evidence-based assessments** showing capability alignment and confidence scores
|
|
17
17
|
|
|
18
18
|
## 🎯 The 5 Capability Roles
|
|
19
19
|
|
|
20
|
-
| Capability Role | Description |
|
|
21
|
-
|-----------------|-------------|--------------------|
|
|
22
|
-
| **FULL** | Complete implementation of safeguard requirements |
|
|
23
|
-
| **PARTIAL** | Limited scope implementation with clear boundaries |
|
|
24
|
-
| **FACILITATES** | Enhancement capabilities that enable others to implement safeguards better/faster/stronger |
|
|
25
|
-
| **GOVERNANCE** | Policy/process management and oversight capabilities |
|
|
26
|
-
| **VALIDATES** | Verification capabilities providing evidence and reporting |
|
|
20
|
+
| Capability Role | Description | Assessment Approach |
|
|
21
|
+
|-----------------|-------------|--------------------|
|
|
22
|
+
| **FULL** | Complete implementation of safeguard requirements | Comprehensive content analysis for complete coverage |
|
|
23
|
+
| **PARTIAL** | Limited scope implementation with clear boundaries | Content analysis identifies explicit scope limitations |
|
|
24
|
+
| **FACILITATES** | Enhancement capabilities that enable others to implement safeguards better/faster/stronger | Analysis identifies facilitation language and indirect support |
|
|
25
|
+
| **GOVERNANCE** | Policy/process management and oversight capabilities | Analysis identifies policy, process, and oversight elements |
|
|
26
|
+
| **VALIDATES** | Verification capabilities providing evidence and reporting | Analysis identifies audit, monitoring, and reporting capabilities |
|
|
27
27
|
|
|
28
28
|
## 🎨 CIS Controls Framework Integration
|
|
29
29
|
|
|
@@ -130,7 +130,7 @@ In your Copilot, create actions for capability assessment:
|
|
|
130
130
|
**Primary Action - Validate Vendor Capability:**
|
|
131
131
|
```
|
|
132
132
|
Action: Validate Vendor Mapping
|
|
133
|
-
Description: Validate vendor capability claims against CIS Controls
|
|
133
|
+
Description: Validate vendor capability claims against CIS Controls through content analysis
|
|
134
134
|
Connector: Framework MCP Custom Connector
|
|
135
135
|
Operation: validateVendorMapping
|
|
136
136
|
Parameters:
|
|
@@ -236,10 +236,10 @@ claude-code "Use validate_vendor_mapping with vendor_name 'SecureAssets Corp', s
|
|
|
236
236
|
"capabilityRole": "full",
|
|
237
237
|
"additionalRoles": ["governance", "validates"],
|
|
238
238
|
"confidence": 87,
|
|
239
|
-
"
|
|
240
|
-
"
|
|
241
|
-
"
|
|
242
|
-
"
|
|
239
|
+
"contentAnalysis": {
|
|
240
|
+
"implementationLanguage": "strong",
|
|
241
|
+
"scopeDefinition": "comprehensive",
|
|
242
|
+
"evidenceQuality": "high"
|
|
243
243
|
},
|
|
244
244
|
"evidenceAnalysis": {
|
|
245
245
|
"coreRequirements": 85,
|
|
@@ -270,55 +270,57 @@ claude-code "Use validate_vendor_mapping with vendor_name 'SecureAssets Corp', s
|
|
|
270
270
|
"governance_alignment": 80,
|
|
271
271
|
"language_consistency": 90
|
|
272
272
|
},
|
|
273
|
-
"
|
|
274
|
-
"
|
|
275
|
-
"
|
|
276
|
-
"
|
|
277
|
-
"
|
|
273
|
+
"content_validation": {
|
|
274
|
+
"implementation_depth": "comprehensive",
|
|
275
|
+
"scope_clarity": "well_defined",
|
|
276
|
+
"evidence_strength": "strong",
|
|
277
|
+
"capability_aligned": true
|
|
278
278
|
},
|
|
279
279
|
"gaps_identified": [],
|
|
280
280
|
"strengths_identified": [
|
|
281
281
|
"High coverage of core requirements and sub-elements",
|
|
282
282
|
"Strong implementation language consistency",
|
|
283
|
-
"
|
|
283
|
+
"Comprehensive coverage with clear implementation details"
|
|
284
284
|
],
|
|
285
285
|
"recommendations": [],
|
|
286
|
-
"detailed_feedback": "Validation of FULL capability claim: SUPPORTED (85% alignment)\n\nSTRENGTHS:\n• High coverage of core requirements and sub-elements\n• Strong implementation language consistency\n•
|
|
286
|
+
"detailed_feedback": "Validation of FULL capability claim: SUPPORTED (85% alignment)\n\nSTRENGTHS:\n• High coverage of core requirements and sub-elements\n• Strong implementation language consistency\n• Comprehensive coverage with clear implementation details\n\nASSESSMENT: The vendor's supporting evidence strongly aligns with their claimed capability."
|
|
287
287
|
}
|
|
288
288
|
```
|
|
289
289
|
|
|
290
|
-
###
|
|
290
|
+
### Insufficient Evidence Example Output
|
|
291
291
|
```json
|
|
292
292
|
{
|
|
293
|
-
"vendor": "
|
|
293
|
+
"vendor": "BasicTracker Pro",
|
|
294
294
|
"safeguard_id": "1.1",
|
|
295
295
|
"safeguard_title": "Establish and Maintain a Detailed Enterprise Asset Inventory",
|
|
296
|
-
"claimed_capability": "
|
|
297
|
-
"validation_status": "
|
|
298
|
-
"confidence_score":
|
|
296
|
+
"claimed_capability": "full",
|
|
297
|
+
"validation_status": "UNSUPPORTED",
|
|
298
|
+
"confidence_score": 35,
|
|
299
299
|
"evidence_analysis": {
|
|
300
|
-
"core_requirements_coverage":
|
|
301
|
-
"sub_elements_coverage":
|
|
302
|
-
"governance_alignment":
|
|
303
|
-
"language_consistency":
|
|
300
|
+
"core_requirements_coverage": 40,
|
|
301
|
+
"sub_elements_coverage": 15,
|
|
302
|
+
"governance_alignment": 25,
|
|
303
|
+
"language_consistency": 60
|
|
304
304
|
},
|
|
305
|
-
"
|
|
306
|
-
"
|
|
307
|
-
"
|
|
308
|
-
"
|
|
309
|
-
"
|
|
310
|
-
"original_claim": "full"
|
|
305
|
+
"content_validation": {
|
|
306
|
+
"implementation_depth": "limited",
|
|
307
|
+
"scope_clarity": "vague",
|
|
308
|
+
"evidence_strength": "weak",
|
|
309
|
+
"capability_aligned": false
|
|
311
310
|
},
|
|
312
311
|
"gaps_identified": [
|
|
313
|
-
"
|
|
312
|
+
"Insufficient detail on asset tracking capabilities",
|
|
313
|
+
"Missing governance and review processes",
|
|
314
|
+
"Limited coverage of sub-elements"
|
|
314
315
|
],
|
|
315
316
|
"strengths_identified": [
|
|
316
|
-
"
|
|
317
|
+
"Basic asset visibility mentioned"
|
|
317
318
|
],
|
|
318
319
|
"recommendations": [
|
|
319
|
-
"
|
|
320
|
+
"Provide more detailed implementation specifics",
|
|
321
|
+
"Consider repositioning as FACILITATES capability based on limited scope"
|
|
320
322
|
],
|
|
321
|
-
"detailed_feedback": "
|
|
323
|
+
"detailed_feedback": "Validation of FULL capability claim: UNSUPPORTED (35% alignment)\n\nGAPS IDENTIFIED:\n• Insufficient detail on asset tracking capabilities\n• Missing governance and review processes\n• Limited coverage of sub-elements\n\nASSESSMENT: The vendor's supporting evidence does not adequately support their FULL capability claim. Consider FACILITATES role instead."
|
|
322
324
|
}
|
|
323
325
|
```
|
|
324
326
|
|
|
@@ -326,7 +328,7 @@ claude-code "Use validate_vendor_mapping with vendor_name 'SecureAssets Corp', s
|
|
|
326
328
|
|
|
327
329
|
| Tool | Description |
|
|
328
330
|
|------|-------------|
|
|
329
|
-
| `validate_vendor_mapping` | **PRIMARY** Validate vendor's claimed capability role against supporting evidence
|
|
331
|
+
| `validate_vendor_mapping` | **PRIMARY** Validate vendor's claimed capability role against supporting evidence through content analysis |
|
|
330
332
|
| `analyze_vendor_response` | Determine vendor tool capability role for specific safeguard |
|
|
331
333
|
| `get_safeguard_details` | Get detailed safeguard breakdown |
|
|
332
334
|
| `list_available_safeguards` | List all available CIS safeguards |
|
|
@@ -373,43 +375,42 @@ The **validate_vendor_mapping** tool provides evidence-based validation of vendo
|
|
|
373
375
|
| **GOVERNANCE** | Policy/process management | ≥60% governance elements + policy language |
|
|
374
376
|
| **VALIDATES** | Evidence collection & reporting | Audit/monitoring/reporting capabilities present |
|
|
375
377
|
|
|
376
|
-
###
|
|
378
|
+
### Content-Based Validation Approach
|
|
377
379
|
|
|
378
|
-
**
|
|
380
|
+
**APPROACH**: The validation tool uses comprehensive content analysis to assess capability claims:
|
|
379
381
|
|
|
380
|
-
|
|
|
381
|
-
|
|
382
|
-
| **
|
|
383
|
-
| **
|
|
384
|
-
| **
|
|
385
|
-
| **
|
|
386
|
-
| **
|
|
382
|
+
| Analysis Dimension | Description | Assessment Focus |
|
|
383
|
+
|-------------------|-------------|------------------|
|
|
384
|
+
| **Core Requirements Coverage** | Alignment with primary safeguard elements | Direct implementation language and specific capabilities |
|
|
385
|
+
| **Sub-Elements Coverage** | Support for detailed safeguard components | Breadth of coverage across safeguard sub-requirements |
|
|
386
|
+
| **Governance Alignment** | Policy/process management elements | Governance language and oversight capabilities |
|
|
387
|
+
| **Implementation Depth** | Specificity and detail of implementation | Technical depth and implementation specifics |
|
|
388
|
+
| **Language Consistency** | Alignment between claim and evidence | Consistency between stated capability and supporting text |
|
|
387
389
|
|
|
388
|
-
**
|
|
389
|
-
- **
|
|
390
|
-
- **
|
|
391
|
-
-
|
|
390
|
+
**Content Analysis Logic**: Evidence is evaluated across multiple dimensions:
|
|
391
|
+
- **Strong Evidence**: Detailed implementation specifics with comprehensive coverage
|
|
392
|
+
- **Moderate Evidence**: Good coverage with some gaps or general language
|
|
393
|
+
- **Weak Evidence**: Limited specifics or scope, vague implementation details
|
|
392
394
|
|
|
393
395
|
### Validation Statuses
|
|
394
396
|
|
|
395
397
|
- **SUPPORTED** (70-100%): Evidence strongly supports the claimed capability
|
|
396
|
-
- **QUESTIONABLE** (40-69%): Evidence partially supports but has notable gaps
|
|
398
|
+
- **QUESTIONABLE** (40-69%): Evidence partially supports but has notable gaps or inconsistencies
|
|
397
399
|
- **UNSUPPORTED** (0-39%): Evidence does not adequately support the claim
|
|
398
400
|
|
|
399
401
|
### Usage Examples
|
|
400
402
|
|
|
401
403
|
```bash
|
|
402
|
-
# Validate a FULL coverage claim
|
|
404
|
+
# Validate a FULL coverage claim with strong evidence
|
|
403
405
|
claude-code "Use validate_vendor_mapping for vendor 'AssetMax Pro', safeguard '1.1', claimed capability 'full', with supporting text: 'Our platform provides comprehensive automated discovery, detailed inventory management, and complete asset lifecycle tracking for all enterprise devices including servers, workstations, and network equipment.'"
|
|
404
406
|
|
|
405
407
|
# Validate a FACILITATES claim
|
|
406
408
|
claude-code "Use validate_vendor_mapping for vendor 'ThreatIntel Feed', safeguard '1.1', claimed capability 'facilitates', with supporting text: 'Our threat intelligence service provides supplemental risk data that enriches existing asset management systems, enabling organizations to prioritize asset security based on threat exposure.'"
|
|
407
409
|
|
|
408
|
-
#
|
|
409
|
-
claude-code "Use validate_vendor_mapping for vendor '
|
|
410
|
-
# Result: Downgraded from FULL to FACILITATES (vulnerability_management ≠ inventory tool)
|
|
410
|
+
# Partial capability claim with clear scope limitations
|
|
411
|
+
claude-code "Use validate_vendor_mapping for vendor 'NetworkScanner Pro', safeguard '1.1', claimed capability 'partial', with supporting text: 'Our scanner provides comprehensive network device discovery and maintains detailed hardware inventories, but is limited to network-accessible devices and does not track software installations or offline systems.'"
|
|
411
412
|
|
|
412
|
-
#
|
|
413
|
+
# Insufficient evidence example
|
|
413
414
|
claude-code "Use validate_vendor_mapping for vendor 'BasicTracker', safeguard '1.1', claimed capability 'full', with supporting text: 'We help track computers and provide some visibility into your IT environment.'"
|
|
414
415
|
```
|
|
415
416
|
|