docworks 0.15.0 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +98 -6
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -61,6 +61,28 @@ provider: openai
|
|
|
61
61
|
model: gpt-4o-mini
|
|
62
62
|
```
|
|
63
63
|
|
|
64
|
+
### Configurable Thresholds
|
|
65
|
+
|
|
66
|
+
Set pass/fail criteria to match your documentation maturity:
|
|
67
|
+
|
|
68
|
+
```yaml
|
|
69
|
+
source: https://docs.yourcompany.com
|
|
70
|
+
|
|
71
|
+
# Global threshold - fail if less than 85% questions are answerable
|
|
72
|
+
threshold: 85
|
|
73
|
+
|
|
74
|
+
journeys:
|
|
75
|
+
authentication:
|
|
76
|
+
threshold: 100 # Critical path - must be perfect
|
|
77
|
+
questions:
|
|
78
|
+
- How do I get API keys?
|
|
79
|
+
- How do I authenticate?
|
|
80
|
+
|
|
81
|
+
examples: # Uses global 85% threshold
|
|
82
|
+
- Where are code samples?
|
|
83
|
+
- Are there tutorials?
|
|
84
|
+
```
|
|
85
|
+
|
|
64
86
|
## Rich Feedback
|
|
65
87
|
|
|
66
88
|
Instead of simple YES/NO, get actionable insights:
|
|
@@ -72,11 +94,52 @@ Instead of simple YES/NO, get actionable insights:
|
|
|
72
94
|
Missing:
|
|
73
95
|
- API key generation steps
|
|
74
96
|
- Token refresh documentation
|
|
97
|
+
|
|
98
|
+
Journey: authentication
|
|
99
|
+
Score: 75% (Threshold: 100%)
|
|
100
|
+
❌ FAILED - Below required threshold
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## CI/CD Integration
|
|
104
|
+
|
|
105
|
+
### Progressive Validation
|
|
106
|
+
|
|
107
|
+
Different thresholds for different environments:
|
|
108
|
+
|
|
109
|
+
```yaml
|
|
110
|
+
# .github/workflows/docs.yml
|
|
111
|
+
name: Documentation Validation
|
|
112
|
+
on: [pull_request, push]
|
|
113
|
+
|
|
114
|
+
jobs:
|
|
115
|
+
validate-pr:
|
|
116
|
+
if: github.event_name == 'pull_request'
|
|
117
|
+
runs-on: ubuntu-latest
|
|
118
|
+
steps:
|
|
119
|
+
- uses: actions/checkout@v3
|
|
120
|
+
|
|
121
|
+
- name: Validate PR docs (lenient)
|
|
122
|
+
run: npx docworks check --threshold 70
|
|
123
|
+
env:
|
|
124
|
+
DOCWORKS_SOURCE: https://preview-${{ github.event.pull_request.number }}.docs.example.com
|
|
125
|
+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
126
|
+
|
|
127
|
+
validate-production:
|
|
128
|
+
if: github.ref == 'refs/heads/main'
|
|
129
|
+
runs-on: ubuntu-latest
|
|
130
|
+
steps:
|
|
131
|
+
- uses: actions/checkout@v3
|
|
132
|
+
|
|
133
|
+
- name: Validate production docs (strict)
|
|
134
|
+
run: npx docworks check --threshold 95
|
|
135
|
+
env:
|
|
136
|
+
DOCWORKS_SOURCE: https://docs.example.com
|
|
137
|
+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
75
138
|
```
|
|
76
139
|
|
|
77
|
-
|
|
140
|
+
### Multi-Model Testing
|
|
78
141
|
|
|
79
|
-
Test against multiple AI models
|
|
142
|
+
Test against multiple AI models:
|
|
80
143
|
|
|
81
144
|
```yaml
|
|
82
145
|
# .github/workflows/docs.yml
|
|
@@ -90,15 +153,18 @@ jobs:
|
|
|
90
153
|
include:
|
|
91
154
|
- provider: openai
|
|
92
155
|
model: gpt-4o
|
|
156
|
+
threshold: 90
|
|
93
157
|
- provider: openai
|
|
94
158
|
model: gpt-4o-mini
|
|
159
|
+
threshold: 85
|
|
95
160
|
- provider: anthropic
|
|
96
161
|
model: claude-3-opus
|
|
162
|
+
threshold: 90
|
|
97
163
|
|
|
98
164
|
runs-on: ubuntu-latest
|
|
99
165
|
steps:
|
|
100
166
|
- uses: actions/checkout@v3
|
|
101
|
-
- run: npx docworks check
|
|
167
|
+
- run: npx docworks check --threshold ${{ matrix.threshold }}
|
|
102
168
|
env:
|
|
103
169
|
PROVIDER: ${{ matrix.provider }}
|
|
104
170
|
MODEL: ${{ matrix.model }}
|
|
@@ -117,21 +183,47 @@ jobs:
|
|
|
117
183
|
# Initialize config
|
|
118
184
|
docworks init
|
|
119
185
|
|
|
120
|
-
# Validate all journeys
|
|
186
|
+
# Validate all journeys (uses config threshold or defaults to 100%)
|
|
121
187
|
docworks check
|
|
122
188
|
|
|
189
|
+
# Override threshold via command line
|
|
190
|
+
docworks check --threshold 80
|
|
191
|
+
|
|
123
192
|
# Test specific journey
|
|
124
193
|
docworks check --journey authentication
|
|
125
194
|
|
|
126
|
-
# Output as JSON
|
|
195
|
+
# Output as JSON (includes threshold data)
|
|
127
196
|
docworks check --format json
|
|
128
197
|
```
|
|
129
198
|
|
|
199
|
+
## Exit Codes
|
|
200
|
+
|
|
201
|
+
- `0` - All thresholds met ✅
|
|
202
|
+
- `1` - Below threshold ❌
|
|
203
|
+
|
|
204
|
+
```bash
|
|
205
|
+
# Strict validation - fail on any missing docs
|
|
206
|
+
docworks check --threshold 100
|
|
207
|
+
|
|
208
|
+
# Allow 20% missing during initial setup
|
|
209
|
+
docworks check --threshold 80
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
## Progressive Adoption
|
|
213
|
+
|
|
214
|
+
Start lenient and increase strictness as your documentation improves:
|
|
215
|
+
|
|
216
|
+
1. **Initial Setup** - `threshold: 60` (allow gaps while building)
|
|
217
|
+
2. **Development** - `threshold: 80` (most questions answered)
|
|
218
|
+
3. **Staging** - `threshold: 90` (nearly complete)
|
|
219
|
+
4. **Production** - `threshold: 95-100` (comprehensive docs)
|
|
220
|
+
|
|
130
221
|
## Why DocWorks?
|
|
131
222
|
|
|
132
223
|
- **Real-world testing** - AI navigates docs like developers do
|
|
224
|
+
- **Configurable strictness** - Match your documentation maturity
|
|
133
225
|
- **Actionable feedback** - Know exactly what to fix
|
|
134
|
-
- **CI/CD ready** -
|
|
226
|
+
- **CI/CD ready** - Different thresholds for PR vs production
|
|
135
227
|
- **Progressive** - Start simple, add complexity as needed
|
|
136
228
|
|
|
137
229
|
## License
|