@sidhxntt/vibe-lint 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE.txt +21 -0
- package/README.md +143 -0
- package/bad-prompt.txt +1 -0
- package/good-prompt.txt +1 -0
- package/package.json +14 -0
- package/src/index.js +515 -0
- package/src/tempCodeRunnerFile.js +225 -0
package/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Sid.
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,143 @@
|
|
|
1
|
+
# vibe-lint
|
|
2
|
+
|
|
3
|
+
A linter for your prompts — flags weak, vague, and contradictory instructions
|
|
4
|
+
|
|
5
|
+

|
|
6
|
+

|
|
7
|
+

|
|
8
|
+
|
|
9
|
+
## Features
|
|
10
|
+
|
|
11
|
+
- **Comprehensive rule engine** with 15+ lint rules covering common prompt engineering pitfalls
|
|
12
|
+
- **Severity levels** — errors for critical issues, warnings for style problems, info for suggestions
|
|
13
|
+
- **Colored terminal output** with line numbers and context highlighting
|
|
14
|
+
- **Actionable suggestions** — specific alternatives for each flagged pattern
|
|
15
|
+
- **Documentation links** to OpenAI and prompt engineering best practices
|
|
16
|
+
- **Interactive and file-based modes** — lint from stdin or file arguments
|
|
17
|
+
|
|
18
|
+
## Prerequisites
|
|
19
|
+
|
|
20
|
+
- Node.js 14+ (uses ES modules)
|
|
21
|
+
|
|
22
|
+
## Installation
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
# Clone the repository
|
|
26
|
+
git clone <repository-url>
|
|
27
|
+
cd vibe_linter
|
|
28
|
+
|
|
29
|
+
# Install as global CLI tool
|
|
30
|
+
npm install -g .
|
|
31
|
+
|
|
32
|
+
# Or run directly with npm
|
|
33
|
+
npm start
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Usage
|
|
37
|
+
|
|
38
|
+
### Interactive Mode (stdin)
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
# Type your prompt, then Ctrl+D to lint
|
|
42
|
+
vibe-lint
|
|
43
|
+
|
|
44
|
+
# Or pipe content
|
|
45
|
+
echo "You are a helpful AI assistant. Please be helpful and improve the code." | vibe-lint
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
### File Mode
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
# Lint a single prompt file
|
|
52
|
+
vibe-lint my-prompt.txt
|
|
53
|
+
|
|
54
|
+
# Lint multiple files
|
|
55
|
+
vibe-lint prompt1.txt prompt2.txt
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### Example Output
|
|
59
|
+
|
|
60
|
+
Running `vibe-lint bad-prompt.txt`:
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
┌─ bad-prompt.txt ─────────────────────────────────────────────────
|
|
64
|
+
│
|
|
65
|
+
│ 1 │ You are a helpful AI assistant. Please try to be more helpful and improve the code as needed.
|
|
66
|
+
│ │ ·─────┬─────· ·────────┬────────·
|
|
67
|
+
│ │ │ │
|
|
68
|
+
│ │ W002 W001
|
|
69
|
+
│ 2 │ Make it better and more professional. Feel free to handle the above code and do your best.
|
|
70
|
+
│ │ ·─────┬─────· ·─────────┬─────────·
|
|
71
|
+
│ │ │ │
|
|
72
|
+
│ │ W001 E001
|
|
73
|
+
|
|
74
|
+
Errors: 1, Warnings: 3, Info: 0
|
|
75
|
+
|
|
76
|
+
W002 "Be helpful" gives the model no optimization target
|
|
77
|
+
└ Try: 'Answer in ≤3 sentences unless the topic requires more depth'
|
|
78
|
+
|
|
79
|
+
W001 Vague improvement request — "better" is unmeasurable
|
|
80
|
+
└ Try: Specify the axis: 'reduce cyclomatic complexity below 10'
|
|
81
|
+
|
|
82
|
+
E001 Ambiguous delegation — model will hallucinate scope boundaries
|
|
83
|
+
└ Try: List exact subtasks: '1. Parse input 2. Validate schema 3. Return JSON'
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
## Rule Categories
|
|
87
|
+
|
|
88
|
+
| Category | Description | Example Issues |
|
|
89
|
+
|----------|-------------|----------------|
|
|
90
|
+
| `vague-quality` | Unmeasurable quality descriptors | "make it better", "be helpful", "high-quality" |
|
|
91
|
+
| `ambiguous-scope` | Unclear task boundaries | "handle it", "as needed", "where appropriate" |
|
|
92
|
+
| `output-format` | Unspecified response format | "good format", "respond appropriately" |
|
|
93
|
+
| `role-confusion` | Redundant AI identity statements | "you are an AI assistant" |
|
|
94
|
+
| `verbosity-conflict` | Contradictory length requirements | "don't be verbose but be comprehensive" |
|
|
95
|
+
|
|
96
|
+
## Project Structure
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
vibe_linter/
|
|
100
|
+
├── bad-prompt.txt # Example of a poorly-written prompt
|
|
101
|
+
├── good-prompt.txt # Example of a well-structured prompt
|
|
102
|
+
├── package.json # NPM package configuration
|
|
103
|
+
└── src/
|
|
104
|
+
└── index.js # Main CLI application with rule engine
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
### Key Components
|
|
108
|
+
|
|
109
|
+
- **Rule Engine**: 15+ regex-based rules with severity levels and suggestions
|
|
110
|
+
- **Linter Logic**: Multi-line context analysis with precise error positioning
|
|
111
|
+
- **CLI Interface**: Supports both interactive stdin and file-based input
|
|
112
|
+
- **Colored Output**: ANSI terminal formatting for improved readability
|
|
113
|
+
|
|
114
|
+
## Adding Custom Rules
|
|
115
|
+
|
|
116
|
+
Rules are defined in the `RULES` array in `src/index.js`:
|
|
117
|
+
|
|
118
|
+
```javascript
|
|
119
|
+
{
|
|
120
|
+
id: "W001",
|
|
121
|
+
severity: "warn", // "error" | "warn" | "info"
|
|
122
|
+
category: "vague-quality",
|
|
123
|
+
pattern: /\b(make it better)\b/gi,
|
|
124
|
+
message: "Vague improvement request",
|
|
125
|
+
suggestions: [
|
|
126
|
+
"Specify the axis: 'reduce complexity'",
|
|
127
|
+
"Target a metric: 'cut latency by 30%'"
|
|
128
|
+
],
|
|
129
|
+
docs: "https://platform.openai.com/docs/guides/prompt-engineering"
|
|
130
|
+
}
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
## Contributing
|
|
134
|
+
|
|
135
|
+
1. Fork the repository
|
|
136
|
+
2. Create a feature branch: `git checkout -b feature-name`
|
|
137
|
+
3. Add your rule to the `RULES` array with test cases
|
|
138
|
+
4. Test with both example files: `npm start good-prompt.txt bad-prompt.txt`
|
|
139
|
+
5. Submit a pull request
|
|
140
|
+
|
|
141
|
+
## License
|
|
142
|
+
|
|
143
|
+
MIT
|
package/bad-prompt.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
You are a helpful AI assistant. Please try to be more helpful and improve the code as needed. Make it better and more professional. Feel free to handle the above code and do your best. Don't be verbose but be comprehensive. Be an expert and respond appropriately.
|
package/good-prompt.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
You are a senior TypeScript engineer specializing in React performance optimization. Your user is a mid-level frontend developer who understands JSX but may not know advanced React internals. When reviewing code: 1. Flag all components that re-render on every parent render (missing memo/useMemo/useCallback) 2. Identify prop drilling deeper than 3 levels — suggest Context or state lifting 3. Mark any useEffect with missing or incorrect dependency arrays 4. Note bundle-size risks: dynamic imports missing, large deps imported fully. Respond with: - A JSON array of findings: { line, severity: "error"|"warn"|"info", message, fix } - Then a markdown summary under the heading "## What to fix first". Do not rewrite the entire file. Only show changed lines with 3 lines of context above and below.
|
package/package.json
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@sidhxntt/vibe-lint",
|
|
3
|
+
"version": "1.0.0",
|
|
4
|
+
"description": "A linter for your prompts — flags weak, vague, and contradictory instructions",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"@sidhxntt/vibe-lint": "./index.js"
|
|
8
|
+
},
|
|
9
|
+
"scripts": {
|
|
10
|
+
"start": "node ./src/index.js"
|
|
11
|
+
},
|
|
12
|
+
"keywords": ["prompt", "linter", "ai", "llm", "prompt-engineering"],
|
|
13
|
+
"license": "MIT"
|
|
14
|
+
}
|
package/src/index.js
ADDED
|
@@ -0,0 +1,515 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
import readline from "readline";
|
|
4
|
+
|
|
5
|
+
// ─── ANSI Colors ─────────────────────────────────────────────────────────────
|
|
6
|
+
const c = {
|
|
7
|
+
reset: "\x1b[0m",
|
|
8
|
+
bold: "\x1b[1m",
|
|
9
|
+
dim: "\x1b[2m",
|
|
10
|
+
red: "\x1b[38;5;203m",
|
|
11
|
+
yellow: "\x1b[38;5;220m",
|
|
12
|
+
green: "\x1b[38;5;114m",
|
|
13
|
+
cyan: "\x1b[38;5;117m",
|
|
14
|
+
magenta: "\x1b[38;5;213m",
|
|
15
|
+
gray: "\x1b[38;5;245m",
|
|
16
|
+
white: "\x1b[97m",
|
|
17
|
+
bg_red: "\x1b[48;5;52m",
|
|
18
|
+
bg_yellow: "\x1b[48;5;58m",
|
|
19
|
+
bg_green: "\x1b[48;5;22m",
|
|
20
|
+
};
|
|
21
|
+
|
|
22
|
+
const bold = (s) => `${c.bold}${s}${c.reset}`;
|
|
23
|
+
const dim = (s) => `${c.dim}${s}${c.reset}`;
|
|
24
|
+
const red = (s) => `${c.red}${s}${c.reset}`;
|
|
25
|
+
const yellow = (s) => `${c.yellow}${s}${c.reset}`;
|
|
26
|
+
const green = (s) => `${c.green}${s}${c.reset}`;
|
|
27
|
+
const cyan = (s) => `${c.cyan}${s}${c.reset}`;
|
|
28
|
+
const magenta = (s) => `${c.magenta}${s}${c.reset}`;
|
|
29
|
+
const gray = (s) => `${c.gray}${s}${c.reset}`;
|
|
30
|
+
const white = (s) => `${c.white}${s}${c.reset}`;
|
|
31
|
+
|
|
32
|
+
// ─── Rule Engine ─────────────────────────────────────────────────────────────
|
|
33
|
+
|
|
34
|
+
const RULES = [
|
|
35
|
+
// ── Vague quality descriptors ──────────────────────────────────────────────
|
|
36
|
+
{
|
|
37
|
+
id: "W001",
|
|
38
|
+
severity: "warn",
|
|
39
|
+
category: "vague-quality",
|
|
40
|
+
pattern: /\b(make it better|improve (it|this|the code|the output))\b/gi,
|
|
41
|
+
message: 'Vague improvement request — "better" is unmeasurable',
|
|
42
|
+
suggestions: [
|
|
43
|
+
"Specify the axis: 'reduce cyclomatic complexity below 10'",
|
|
44
|
+
"Target a metric: 'cut response latency by 30%'",
|
|
45
|
+
"Name the smell: 'eliminate magic numbers, extract named constants'",
|
|
46
|
+
"Describe the reader: 'a junior dev should understand without inline comments'",
|
|
47
|
+
],
|
|
48
|
+
docs: "https://platform.openai.com/docs/guides/prompt-engineering#specify-the-desired-output-format",
|
|
49
|
+
},
|
|
50
|
+
{
|
|
51
|
+
id: "W002",
|
|
52
|
+
severity: "warn",
|
|
53
|
+
category: "vague-quality",
|
|
54
|
+
pattern: /\b(be (more )?(helpful|useful|good|better|clearer|concise))\b/gi,
|
|
55
|
+
message: '"Be helpful/useful/good" gives the model no optimization target',
|
|
56
|
+
suggestions: [
|
|
57
|
+
"'Answer in ≤3 sentences unless the topic requires more depth'",
|
|
58
|
+
"'If unsure, list 2 options with trade-offs rather than picking one'",
|
|
59
|
+
"'Prefer code examples over prose explanations for how-to questions'",
|
|
60
|
+
],
|
|
61
|
+
docs: null,
|
|
62
|
+
},
|
|
63
|
+
{
|
|
64
|
+
id: "W003",
|
|
65
|
+
severity: "warn",
|
|
66
|
+
category: "vague-quality",
|
|
67
|
+
pattern: /\b(high[- ]quality|professional|polished|nice|clean)\b/gi,
|
|
68
|
+
message: '"High-quality/professional" is subjective without a rubric',
|
|
69
|
+
suggestions: [
|
|
70
|
+
"For code: 'passes ESLint strict, has JSDoc on public functions, no TODOs'",
|
|
71
|
+
"For prose: 'Flesch-Kincaid grade ≤ 10, active voice, no filler phrases'",
|
|
72
|
+
"For UI: 'WCAG AA contrast, touch targets ≥ 44px, no layout shift'",
|
|
73
|
+
],
|
|
74
|
+
docs: null,
|
|
75
|
+
},
|
|
76
|
+
{
|
|
77
|
+
id: "E001",
|
|
78
|
+
severity: "error",
|
|
79
|
+
category: "ambiguous-scope",
|
|
80
|
+
pattern: /\b(do (the|your) (best|thing)|handle (it|this|everything))\b/gi,
|
|
81
|
+
message: "Ambiguous delegation — model will hallucinate scope boundaries",
|
|
82
|
+
suggestions: [
|
|
83
|
+
"List exact subtasks: '1. Parse input 2. Validate schema 3. Return JSON'",
|
|
84
|
+
"Set a ceiling: 'Only modify files in /src/components, leave tests untouched'",
|
|
85
|
+
"Define done: 'Complete when all existing tests pass with no new warnings'",
|
|
86
|
+
],
|
|
87
|
+
docs: null,
|
|
88
|
+
},
|
|
89
|
+
{
|
|
90
|
+
id: "E002",
|
|
91
|
+
severity: "error",
|
|
92
|
+
category: "ambiguous-scope",
|
|
93
|
+
pattern: /\b(as needed|where (appropriate|necessary)|if (applicable|needed|relevant))\b/gi,
|
|
94
|
+
message: 'Conditional hedges let the model decide scope — it will decide wrong',
|
|
95
|
+
suggestions: [
|
|
96
|
+
"Replace with explicit conditions: 'Add error handling if the function throws'",
|
|
97
|
+
"Use always/never: 'Always add type annotations. Never use `any`.'",
|
|
98
|
+
"Enumerate the cases: 'Add comments above functions longer than 20 lines'",
|
|
99
|
+
],
|
|
100
|
+
docs: null,
|
|
101
|
+
},
|
|
102
|
+
{
|
|
103
|
+
id: "W004",
|
|
104
|
+
severity: "warn",
|
|
105
|
+
category: "output-format",
|
|
106
|
+
pattern: /\b(in (a |an )?(good|nice|clear|readable|proper) format)\b/gi,
|
|
107
|
+
message: '"Good format" is ambiguous — specify the exact structure',
|
|
108
|
+
suggestions: [
|
|
109
|
+
"'Return a JSON object: { summary: string, tags: string[], confidence: 0-1 }'",
|
|
110
|
+
"'Use markdown with H2 sections: Overview, Usage, Examples, Caveats'",
|
|
111
|
+
"'Plain text only, no markdown. Max 80 chars per line.'",
|
|
112
|
+
],
|
|
113
|
+
docs: null,
|
|
114
|
+
},
|
|
115
|
+
{
|
|
116
|
+
id: "W005",
|
|
117
|
+
severity: "warn",
|
|
118
|
+
category: "output-format",
|
|
119
|
+
pattern: /\b(respond (appropriately|accordingly|as you see fit))\b/gi,
|
|
120
|
+
message: "Deferred format decision will produce inconsistent outputs",
|
|
121
|
+
suggestions: [
|
|
122
|
+
"Specify mime type: 'Respond with application/json'",
|
|
123
|
+
"Give a template: 'Use this structure: [ANALYSIS]\\n[CODE]\\n[CAVEATS]'",
|
|
124
|
+
"Set length bounds: 'Between 100 and 300 words'",
|
|
125
|
+
],
|
|
126
|
+
docs: null,
|
|
127
|
+
},
|
|
128
|
+
{
|
|
129
|
+
id: "E003",
|
|
130
|
+
severity: "error",
|
|
131
|
+
category: "role-confusion",
|
|
132
|
+
pattern: /\b(you (are|will be) an? (AI|assistant|language model|LLM|bot))\b/gi,
|
|
133
|
+
message: "Restating model identity wastes tokens and adds no behavioral constraint",
|
|
134
|
+
suggestions: [
|
|
135
|
+
"Replace with a domain expert persona: 'You are a senior Rust compiler engineer'",
|
|
136
|
+
"Or a constraint: 'You have access only to information in the provided context'",
|
|
137
|
+
"Describe the audience: 'Your user is a first-year CS student'",
|
|
138
|
+
],
|
|
139
|
+
docs: null,
|
|
140
|
+
},
|
|
141
|
+
{
|
|
142
|
+
id: "W006",
|
|
143
|
+
severity: "warn",
|
|
144
|
+
category: "role-confusion",
|
|
145
|
+
pattern: /\b(act (as|like) (a |an )?(helpful|smart|intelligent|knowledgeable) (AI|assistant))\b/gi,
|
|
146
|
+
message: '"Act as a helpful assistant" adds noise, not signal',
|
|
147
|
+
suggestions: [
|
|
148
|
+
"'Act as a PostgreSQL performance consultant reviewing slow queries'",
|
|
149
|
+
"'Act as a skeptical code reviewer who prioritizes security over brevity'",
|
|
150
|
+
"'Act as the user\'s rubber duck — ask clarifying questions before answering'",
|
|
151
|
+
],
|
|
152
|
+
docs: null,
|
|
153
|
+
},
|
|
154
|
+
{
|
|
155
|
+
id: "W007",
|
|
156
|
+
severity: "warn",
|
|
157
|
+
category: "no-examples",
|
|
158
|
+
pattern: /\b(for example[,:]?\s*(\.\.\.)?$|e\.g\.\s*(\.\.\.)?$|such as[,:]?\s*(\.\.\.)?$)/gim,
|
|
159
|
+
message: "Trailing example placeholder — fill it in or remove it",
|
|
160
|
+
suggestions: [
|
|
161
|
+
"Concrete few-shot: show an input→output pair the model should emulate",
|
|
162
|
+
"Negative example: show what NOT to produce (often more powerful)",
|
|
163
|
+
"Edge case: show the tricky case, not the happy path",
|
|
164
|
+
],
|
|
165
|
+
docs: null,
|
|
166
|
+
},
|
|
167
|
+
{
|
|
168
|
+
id: "E004",
|
|
169
|
+
severity: "error",
|
|
170
|
+
category: "contradiction",
|
|
171
|
+
pattern: /\b(be (brief|concise|short)).{0,120}(be (comprehensive|thorough|detailed|exhaustive))\b/gis,
|
|
172
|
+
message: "Contradictory length constraints — model will average them badly",
|
|
173
|
+
suggestions: [
|
|
174
|
+
"Pick one and qualify the other: 'Be concise. Expand only on error handling.'",
|
|
175
|
+
"Use section-level rules: 'Summary: 1 sentence. Implementation: as long as needed.'",
|
|
176
|
+
"Set word counts: 'Under 150 words total, but include all code in full'",
|
|
177
|
+
],
|
|
178
|
+
docs: null,
|
|
179
|
+
},
|
|
180
|
+
{
|
|
181
|
+
id: "W008",
|
|
182
|
+
severity: "warn",
|
|
183
|
+
category: "politeness-bloat",
|
|
184
|
+
pattern: /\b(please (please )?(try to |attempt to |do your best to )?|kindly|feel free to|don't hesitate to)\b/gi,
|
|
185
|
+
message: "Politeness tokens consume context budget and dilute instruction weight",
|
|
186
|
+
suggestions: [
|
|
187
|
+
"Drop courtesy words entirely — models don't have feelings",
|
|
188
|
+
"Convert to imperative: 'Return X' not 'Please try to return X'",
|
|
189
|
+
"Every token should carry constraint or context",
|
|
190
|
+
],
|
|
191
|
+
docs: null,
|
|
192
|
+
},
|
|
193
|
+
{
|
|
194
|
+
id: "W009",
|
|
195
|
+
severity: "warn",
|
|
196
|
+
category: "negation-only",
|
|
197
|
+
pattern: /\b(don't (be|use|include|add|make|write|do|say)|avoid being|never (be|sound))\b/gi,
|
|
198
|
+
message: 'Negation-only rules are weak — models anchor on the forbidden concept',
|
|
199
|
+
suggestions: [
|
|
200
|
+
"Pair every DON'T with a DO: 'Don't use passive voice → use active constructions'",
|
|
201
|
+
"Positive constraint: 'Use direct assertions' instead of 'Don't be wishy-washy'",
|
|
202
|
+
"Show an example of what you WANT instead",
|
|
203
|
+
],
|
|
204
|
+
docs: null,
|
|
205
|
+
},
|
|
206
|
+
{
|
|
207
|
+
id: "E005",
|
|
208
|
+
severity: "error",
|
|
209
|
+
category: "missing-context",
|
|
210
|
+
pattern: /\b(the (code|file|document|data|text|above|previous|earlier))\b(?![\s\S]*?```)/gi,
|
|
211
|
+
message: "Reference to context not present in the prompt — model will hallucinate",
|
|
212
|
+
suggestions: [
|
|
213
|
+
"Paste the actual code/data inline in a fenced block",
|
|
214
|
+
"Use explicit variable names: define it earlier in the prompt",
|
|
215
|
+
"If using a system with retrieval, confirm the context is injected at runtime",
|
|
216
|
+
],
|
|
217
|
+
docs: null,
|
|
218
|
+
},
|
|
219
|
+
{
|
|
220
|
+
id: "W010",
|
|
221
|
+
severity: "warn",
|
|
222
|
+
category: "no-success-criteria",
|
|
223
|
+
pattern: /\b(until (it('s| is) (good|right|working|correct|done))|when (you('re| are) happy|satisfied|done))\b/gi,
|
|
224
|
+
message: "Subjective termination condition — model can't self-evaluate accurately",
|
|
225
|
+
suggestions: [
|
|
226
|
+
"Objective criterion: 'Stop when all 5 test cases produce correct output'",
|
|
227
|
+
"Countable: 'Generate exactly 10 variants, ranked by estimated click-through'",
|
|
228
|
+
"Verifiable: 'Complete when the function has <5 lines and passes mypy strict'",
|
|
229
|
+
],
|
|
230
|
+
docs: null,
|
|
231
|
+
},
|
|
232
|
+
{
|
|
233
|
+
id: "I001",
|
|
234
|
+
severity: "info",
|
|
235
|
+
category: "chain-of-thought",
|
|
236
|
+
pattern: /\b(answer (this|the question|directly)|just (give|tell|show) (me|us) (the )?(answer|result|output))\b/gi,
|
|
237
|
+
message: "Skipping reasoning may reduce accuracy on complex tasks",
|
|
238
|
+
suggestions: [
|
|
239
|
+
"Add: 'Think step by step before giving the final answer'",
|
|
240
|
+
"Use scratchpad: 'Reason in <thinking> tags, then output in <answer> tags'",
|
|
241
|
+
"If speed matters, note it explicitly: 'Skip reasoning, latency is critical'",
|
|
242
|
+
],
|
|
243
|
+
docs: null,
|
|
244
|
+
},
|
|
245
|
+
{
|
|
246
|
+
id: "W011",
|
|
247
|
+
severity: "warn",
|
|
248
|
+
category: "vague-persona",
|
|
249
|
+
pattern: /\b(expert|specialist|professional|guru|master|wizard|ninja)\b(?! in| at| of| with)/gi,
|
|
250
|
+
message: 'Bare "expert" persona lacks domain specificity',
|
|
251
|
+
suggestions: [
|
|
252
|
+
"'Expert' → 'Staff engineer with 10yr distributed systems experience'",
|
|
253
|
+
"Add the skepticism level: 'Expert who defaults to the simplest solution'",
|
|
254
|
+
"Scope the knowledge: 'Expert in Python async, unfamiliar with Rust'",
|
|
255
|
+
],
|
|
256
|
+
docs: null,
|
|
257
|
+
},
|
|
258
|
+
];
|
|
259
|
+
|
|
260
|
+
// ─── Analyzer ─────────────────────────────────────────────────────────────────
|
|
261
|
+
|
|
262
|
+
function analyzePrompt(text) {
|
|
263
|
+
const findings = [];
|
|
264
|
+
|
|
265
|
+
for (const rule of RULES) {
|
|
266
|
+
const regex = new RegExp(rule.pattern.source, rule.pattern.flags);
|
|
267
|
+
let match;
|
|
268
|
+
while ((match = regex.exec(text)) !== null) {
|
|
269
|
+
const lineNum = text.slice(0, match.index).split("\n").length;
|
|
270
|
+
const colNum = match.index - text.lastIndexOf("\n", match.index - 1);
|
|
271
|
+
findings.push({
|
|
272
|
+
rule,
|
|
273
|
+
match: match[0],
|
|
274
|
+
index: match.index,
|
|
275
|
+
line: lineNum,
|
|
276
|
+
col: colNum,
|
|
277
|
+
});
|
|
278
|
+
}
|
|
279
|
+
}
|
|
280
|
+
|
|
281
|
+
// Deduplicate same rule on same line
|
|
282
|
+
const seen = new Set();
|
|
283
|
+
return findings.filter((f) => {
|
|
284
|
+
const key = `${f.rule.id}:${f.line}`;
|
|
285
|
+
if (seen.has(key)) return false;
|
|
286
|
+
seen.add(key);
|
|
287
|
+
return true;
|
|
288
|
+
});
|
|
289
|
+
}
|
|
290
|
+
|
|
291
|
+
// ─── Renderer ─────────────────────────────────────────────────────────────────
|
|
292
|
+
|
|
293
|
+
function severityIcon(sev) {
|
|
294
|
+
return { error: red("✖"), warn: yellow("⚠"), info: cyan("ℹ") }[sev];
|
|
295
|
+
}
|
|
296
|
+
|
|
297
|
+
function severityLabel(sev) {
|
|
298
|
+
return {
|
|
299
|
+
error: red(bold("error")),
|
|
300
|
+
warn: yellow(bold("warn ")),
|
|
301
|
+
info: cyan(bold("info ")),
|
|
302
|
+
}[sev];
|
|
303
|
+
}
|
|
304
|
+
|
|
305
|
+
function renderFindings(findings, sourceText, opts = {}) {
|
|
306
|
+
if (findings.length === 0) {
|
|
307
|
+
console.log(`\n ${green("✔")} ${bold("No issues found.")} Your prompt is tight.\n`);
|
|
308
|
+
return;
|
|
309
|
+
}
|
|
310
|
+
|
|
311
|
+
const lines = sourceText.split("\n");
|
|
312
|
+
|
|
313
|
+
for (const f of findings) {
|
|
314
|
+
const { rule, match, line, col } = f;
|
|
315
|
+
|
|
316
|
+
console.log(
|
|
317
|
+
`\n ${severityIcon(rule.severity)} ${severityLabel(rule.severity)} ` +
|
|
318
|
+
gray(`[${rule.id}]`) + ` ` +
|
|
319
|
+
bold(white(rule.message))
|
|
320
|
+
);
|
|
321
|
+
console.log(` ${gray(`${line}:${col}`)} ${gray(`"${match}"`)}`);
|
|
322
|
+
|
|
323
|
+
// Source context
|
|
324
|
+
const srcLine = lines[line - 1] || "";
|
|
325
|
+
const highlighted = srcLine.replace(
|
|
326
|
+
match,
|
|
327
|
+
`${c.red}${c.bold}${match}${c.reset}`
|
|
328
|
+
);
|
|
329
|
+
console.log(` ${dim("│")} ${highlighted}`);
|
|
330
|
+
|
|
331
|
+
// Suggestions
|
|
332
|
+
if (!opts.noSuggestions && rule.suggestions.length > 0) {
|
|
333
|
+
console.log(` ${dim("│")}`);
|
|
334
|
+
console.log(` ${dim("│")} ${magenta("→ Try instead:")}`);
|
|
335
|
+
for (const s of rule.suggestions) {
|
|
336
|
+
console.log(` ${dim("│")} ${green("•")} ${s}`);
|
|
337
|
+
}
|
|
338
|
+
}
|
|
339
|
+
}
|
|
340
|
+
}
|
|
341
|
+
|
|
342
|
+
function renderSummary(findings) {
|
|
343
|
+
const errors = findings.filter((f) => f.rule.severity === "error").length;
|
|
344
|
+
const warns = findings.filter((f) => f.rule.severity === "warn").length;
|
|
345
|
+
const infos = findings.filter((f) => f.rule.severity === "info").length;
|
|
346
|
+
|
|
347
|
+
const total = findings.length;
|
|
348
|
+
console.log(`\n ${dim("─".repeat(58))}`);
|
|
349
|
+
|
|
350
|
+
if (total === 0) return;
|
|
351
|
+
|
|
352
|
+
const parts = [];
|
|
353
|
+
if (errors) parts.push(red(`${errors} error${errors !== 1 ? "s" : ""}`));
|
|
354
|
+
if (warns) parts.push(yellow(`${warns} warning${warns !== 1 ? "s" : ""}`));
|
|
355
|
+
if (infos) parts.push(cyan(`${infos} hint${infos !== 1 ? "s" : ""}`));
|
|
356
|
+
|
|
357
|
+
console.log(` ${bold("Found:")} ${parts.join(gray(" │ "))}`);
|
|
358
|
+
|
|
359
|
+
// Score
|
|
360
|
+
const score = Math.max(0, 100 - errors * 20 - warns * 8 - infos * 2);
|
|
361
|
+
const scoreColor = score >= 80 ? green : score >= 50 ? yellow : red;
|
|
362
|
+
const bar = renderScoreBar(score);
|
|
363
|
+
console.log(` ${bold("Score:")} ${scoreColor(score + "/100")} ${bar}`);
|
|
364
|
+
console.log();
|
|
365
|
+
}
|
|
366
|
+
|
|
367
|
+
function renderScoreBar(score) {
|
|
368
|
+
const filled = Math.round(score / 5);
|
|
369
|
+
const empty = 20 - filled;
|
|
370
|
+
const color = score >= 80 ? c.green : score >= 50 ? c.yellow : c.red;
|
|
371
|
+
return (
|
|
372
|
+
gray("[") +
|
|
373
|
+
color + "█".repeat(filled) + c.reset +
|
|
374
|
+
gray("░".repeat(empty)) +
|
|
375
|
+
gray("]")
|
|
376
|
+
);
|
|
377
|
+
}
|
|
378
|
+
|
|
379
|
+
function renderHeader(filename) {
|
|
380
|
+
console.log();
|
|
381
|
+
console.log(
|
|
382
|
+
` ${bold(cyan("vibe-lint"))} ${gray("v1.0.0")} ${dim("─")} ${gray(filename || "stdin")}`
|
|
383
|
+
);
|
|
384
|
+
console.log(` ${gray("Flagging weak, vague, and counterproductive prompt instructions")}`);
|
|
385
|
+
}
|
|
386
|
+
|
|
387
|
+
// ─── JSON reporter ────────────────────────────────────────────────────────────
|
|
388
|
+
|
|
389
|
+
function toJson(findings, sourceText, file) {
|
|
390
|
+
const errors = findings.filter((f) => f.rule.severity === "error").length;
|
|
391
|
+
const warns = findings.filter((f) => f.rule.severity === "warn").length;
|
|
392
|
+
const score = Math.max(0, 100 - errors * 20 - warns * 8);
|
|
393
|
+
return JSON.stringify(
|
|
394
|
+
{
|
|
395
|
+
file: file || "stdin",
|
|
396
|
+
score,
|
|
397
|
+
summary: { errors, warnings: warns, hints: findings.filter((f) => f.rule.severity === "info").length },
|
|
398
|
+
findings: findings.map((f) => ({
|
|
399
|
+
id: f.rule.id,
|
|
400
|
+
severity: f.rule.severity,
|
|
401
|
+
category: f.rule.category,
|
|
402
|
+
message: f.rule.message,
|
|
403
|
+
match: f.match,
|
|
404
|
+
line: f.line,
|
|
405
|
+
col: f.col,
|
|
406
|
+
suggestions: f.rule.suggestions,
|
|
407
|
+
})),
|
|
408
|
+
},
|
|
409
|
+
null,
|
|
410
|
+
2
|
|
411
|
+
);
|
|
412
|
+
}
|
|
413
|
+
|
|
414
|
+
// ─── REPL Entry ───────────────────────────────────────────────────────────────
|
|
415
|
+
|
|
416
|
+
function listRules() {
|
|
417
|
+
console.log(`\n ${bold(cyan("vibe-lint rules"))}\n`);
|
|
418
|
+
const byCategory = {};
|
|
419
|
+
for (const r of RULES) {
|
|
420
|
+
if (!byCategory[r.category]) byCategory[r.category] = [];
|
|
421
|
+
byCategory[r.category].push(r);
|
|
422
|
+
}
|
|
423
|
+
for (const [cat, rules] of Object.entries(byCategory)) {
|
|
424
|
+
console.log(` ${bold(magenta(cat))}`);
|
|
425
|
+
for (const r of rules) {
|
|
426
|
+
console.log(` ${gray(r.id)} ${severityIcon(r.severity)} ${r.message}`);
|
|
427
|
+
}
|
|
428
|
+
console.log();
|
|
429
|
+
}
|
|
430
|
+
}
|
|
431
|
+
|
|
432
|
+
function printWelcome() {
|
|
433
|
+
console.clear();
|
|
434
|
+
console.log();
|
|
435
|
+
console.log(` ${bold(cyan("vibe-lint"))} ${gray("v1.0.0")}`);
|
|
436
|
+
console.log(` ${gray("Flags weak, vague, and contradictory prompt instructions")}`);
|
|
437
|
+
console.log();
|
|
438
|
+
console.log(` ${dim("Paste your prompt below and press")} ${bold("Enter twice")} ${dim("to lint it.")}`);
|
|
439
|
+
console.log(` ${dim("Commands:")} ${cyan(":rules")} ${dim("list all rules")} ${cyan(":clear")} ${dim("clear screen")} ${cyan(":quit")} ${dim("or Ctrl+C to exit")}`);
|
|
440
|
+
console.log(` ${dim("─".repeat(58))}`);
|
|
441
|
+
console.log();
|
|
442
|
+
}
|
|
443
|
+
|
|
444
|
+
async function runRepl() {
|
|
445
|
+
printWelcome();
|
|
446
|
+
|
|
447
|
+
const rl = readline.createInterface({
|
|
448
|
+
input: process.stdin,
|
|
449
|
+
output: process.stdout,
|
|
450
|
+
terminal: true,
|
|
451
|
+
prompt: ` ${cyan("›")} `,
|
|
452
|
+
});
|
|
453
|
+
|
|
454
|
+
let buffer = [];
|
|
455
|
+
|
|
456
|
+
const flush = () => {
|
|
457
|
+
const source = buffer.join("\n").trim();
|
|
458
|
+
buffer = [];
|
|
459
|
+
|
|
460
|
+
if (!source) return;
|
|
461
|
+
|
|
462
|
+
if (source === ":quit" || source === ":q") {
|
|
463
|
+
console.log(`\n ${gray("bye.\n")}`);
|
|
464
|
+
process.exit(0);
|
|
465
|
+
}
|
|
466
|
+
if (source === ":rules") { listRules(); promptNext(); return; }
|
|
467
|
+
if (source === ":clear") { printWelcome(); promptNext(); return; }
|
|
468
|
+
|
|
469
|
+
const findings = analyzePrompt(source);
|
|
470
|
+
console.log();
|
|
471
|
+
console.log(` ${dim("─".repeat(58))}`);
|
|
472
|
+
renderFindings(findings, source);
|
|
473
|
+
renderSummary(findings);
|
|
474
|
+
promptNext();
|
|
475
|
+
};
|
|
476
|
+
|
|
477
|
+
const promptNext = () => {
|
|
478
|
+
console.log(` ${dim("─".repeat(58))}`);
|
|
479
|
+
console.log(` ${dim("Paste next prompt (Enter twice to lint) or")} ${cyan(":quit")}`);
|
|
480
|
+
console.log();
|
|
481
|
+
rl.prompt();
|
|
482
|
+
};
|
|
483
|
+
|
|
484
|
+
rl.prompt();
|
|
485
|
+
|
|
486
|
+
rl.on("line", (line) => {
|
|
487
|
+
if (line.trim() === "" && buffer.length > 0) {
|
|
488
|
+
flush();
|
|
489
|
+
} else {
|
|
490
|
+
buffer.push(line);
|
|
491
|
+
}
|
|
492
|
+
});
|
|
493
|
+
|
|
494
|
+
rl.on("close", () => {
|
|
495
|
+
if (buffer.length > 0) flush();
|
|
496
|
+
console.log(`\n ${gray("bye.\n")}`);
|
|
497
|
+
process.exit(0);
|
|
498
|
+
});
|
|
499
|
+
|
|
500
|
+
rl.on("SIGINT", () => {
|
|
501
|
+
if (buffer.length > 0) {
|
|
502
|
+
buffer = [];
|
|
503
|
+
console.log(`\n ${yellow("⚠")} ${gray("Input cleared.")}\n`);
|
|
504
|
+
promptNext();
|
|
505
|
+
} else {
|
|
506
|
+
console.log(`\n ${gray("bye.\n")}`);
|
|
507
|
+
process.exit(0);
|
|
508
|
+
}
|
|
509
|
+
});
|
|
510
|
+
}
|
|
511
|
+
|
|
512
|
+
runRepl().catch((e) => {
|
|
513
|
+
console.error(red(` ✖ ${e.message}`));
|
|
514
|
+
process.exit(1);
|
|
515
|
+
});
|
|
@@ -0,0 +1,225 @@
|
|
|
1
|
+
const RULES = [
|
|
2
|
+
// ── Vague quality descriptors ──────────────────────────────────────────────
|
|
3
|
+
{
|
|
4
|
+
id: "W001",
|
|
5
|
+
severity: "warn",
|
|
6
|
+
category: "vague-quality",
|
|
7
|
+
pattern: /\b(make it better|improve (it|this|the code|the output))\b/gi,
|
|
8
|
+
message: 'Vague improvement request — "better" is unmeasurable',
|
|
9
|
+
suggestions: [
|
|
10
|
+
"Specify the axis: 'reduce cyclomatic complexity below 10'",
|
|
11
|
+
"Target a metric: 'cut response latency by 30%'",
|
|
12
|
+
"Name the smell: 'eliminate magic numbers, extract named constants'",
|
|
13
|
+
"Describe the reader: 'a junior dev should understand without inline comments'",
|
|
14
|
+
],
|
|
15
|
+
docs: "https://platform.openai.com/docs/guides/prompt-engineering#specify-the-desired-output-format",
|
|
16
|
+
},
|
|
17
|
+
{
|
|
18
|
+
id: "W002",
|
|
19
|
+
severity: "warn",
|
|
20
|
+
category: "vague-quality",
|
|
21
|
+
pattern: /\b(be (more )?(helpful|useful|good|better|clearer|concise))\b/gi,
|
|
22
|
+
message: '"Be helpful/useful/good" gives the model no optimization target',
|
|
23
|
+
suggestions: [
|
|
24
|
+
"'Answer in ≤3 sentences unless the topic requires more depth'",
|
|
25
|
+
"'If unsure, list 2 options with trade-offs rather than picking one'",
|
|
26
|
+
"'Prefer code examples over prose explanations for how-to questions'",
|
|
27
|
+
],
|
|
28
|
+
docs: null,
|
|
29
|
+
},
|
|
30
|
+
{
|
|
31
|
+
id: "W003",
|
|
32
|
+
severity: "warn",
|
|
33
|
+
category: "vague-quality",
|
|
34
|
+
pattern: /\b(high[- ]quality|professional|polished|nice|clean)\b/gi,
|
|
35
|
+
message: '"High-quality/professional" is subjective without a rubric',
|
|
36
|
+
suggestions: [
|
|
37
|
+
"For code: 'passes ESLint strict, has JSDoc on public functions, no TODOs'",
|
|
38
|
+
"For prose: 'Flesch-Kincaid grade ≤ 10, active voice, no filler phrases'",
|
|
39
|
+
"For UI: 'WCAG AA contrast, touch targets ≥ 44px, no layout shift'",
|
|
40
|
+
],
|
|
41
|
+
docs: null,
|
|
42
|
+
},
|
|
43
|
+
{
|
|
44
|
+
id: "E001",
|
|
45
|
+
severity: "error",
|
|
46
|
+
category: "ambiguous-scope",
|
|
47
|
+
pattern: /\b(do (the|your) (best|thing)|handle (it|this|everything))\b/gi,
|
|
48
|
+
message: "Ambiguous delegation — model will hallucinate scope boundaries",
|
|
49
|
+
suggestions: [
|
|
50
|
+
"List exact subtasks: '1. Parse input 2. Validate schema 3. Return JSON'",
|
|
51
|
+
"Set a ceiling: 'Only modify files in /src/components, leave tests untouched'",
|
|
52
|
+
"Define done: 'Complete when all existing tests pass with no new warnings'",
|
|
53
|
+
],
|
|
54
|
+
docs: null,
|
|
55
|
+
},
|
|
56
|
+
{
|
|
57
|
+
id: "E002",
|
|
58
|
+
severity: "error",
|
|
59
|
+
category: "ambiguous-scope",
|
|
60
|
+
pattern: /\b(as needed|where (appropriate|necessary)|if (applicable|needed|relevant))\b/gi,
|
|
61
|
+
message: 'Conditional hedges let the model decide scope — it will decide wrong',
|
|
62
|
+
suggestions: [
|
|
63
|
+
"Replace with explicit conditions: 'Add error handling if the function throws'",
|
|
64
|
+
"Use always/never: 'Always add type annotations. Never use `any`.'",
|
|
65
|
+
"Enumerate the cases: 'Add comments above functions longer than 20 lines'",
|
|
66
|
+
],
|
|
67
|
+
docs: null,
|
|
68
|
+
},
|
|
69
|
+
{
|
|
70
|
+
id: "W004",
|
|
71
|
+
severity: "warn",
|
|
72
|
+
category: "output-format",
|
|
73
|
+
pattern: /\b(in (a |an )?(good|nice|clear|readable|proper) format)\b/gi,
|
|
74
|
+
message: '"Good format" is ambiguous — specify the exact structure',
|
|
75
|
+
suggestions: [
|
|
76
|
+
"'Return a JSON object: { summary: string, tags: string[], confidence: 0-1 }'",
|
|
77
|
+
"'Use markdown with H2 sections: Overview, Usage, Examples, Caveats'",
|
|
78
|
+
"'Plain text only, no markdown. Max 80 chars per line.'",
|
|
79
|
+
],
|
|
80
|
+
docs: null,
|
|
81
|
+
},
|
|
82
|
+
{
|
|
83
|
+
id: "W005",
|
|
84
|
+
severity: "warn",
|
|
85
|
+
category: "output-format",
|
|
86
|
+
pattern: /\b(respond (appropriately|accordingly|as you see fit))\b/gi,
|
|
87
|
+
message: "Deferred format decision will produce inconsistent outputs",
|
|
88
|
+
suggestions: [
|
|
89
|
+
"Specify mime type: 'Respond with application/json'",
|
|
90
|
+
"Give a template: 'Use this structure: [ANALYSIS]\\n[CODE]\\n[CAVEATS]'",
|
|
91
|
+
"Set length bounds: 'Between 100 and 300 words'",
|
|
92
|
+
],
|
|
93
|
+
docs: null,
|
|
94
|
+
},
|
|
95
|
+
{
|
|
96
|
+
id: "E003",
|
|
97
|
+
severity: "error",
|
|
98
|
+
category: "role-confusion",
|
|
99
|
+
pattern: /\b(you (are|will be) an? (AI|assistant|language model|LLM|bot))\b/gi,
|
|
100
|
+
message: "Restating model identity wastes tokens and adds no behavioral constraint",
|
|
101
|
+
suggestions: [
|
|
102
|
+
"Replace with a domain expert persona: 'You are a senior Rust compiler engineer'",
|
|
103
|
+
"Or a constraint: 'You have access only to information in the provided context'",
|
|
104
|
+
"Describe the audience: 'Your user is a first-year CS student'",
|
|
105
|
+
],
|
|
106
|
+
docs: null,
|
|
107
|
+
},
|
|
108
|
+
{
|
|
109
|
+
id: "W006",
|
|
110
|
+
severity: "warn",
|
|
111
|
+
category: "role-confusion",
|
|
112
|
+
pattern: /\b(act (as|like) (a |an )?(helpful|smart|intelligent|knowledgeable) (AI|assistant))\b/gi,
|
|
113
|
+
message: '"Act as a helpful assistant" adds noise, not signal',
|
|
114
|
+
suggestions: [
|
|
115
|
+
"'Act as a PostgreSQL performance consultant reviewing slow queries'",
|
|
116
|
+
"'Act as a skeptical code reviewer who prioritizes security over brevity'",
|
|
117
|
+
"'Act as the user\'s rubber duck — ask clarifying questions before answering'",
|
|
118
|
+
],
|
|
119
|
+
docs: null,
|
|
120
|
+
},
|
|
121
|
+
{
|
|
122
|
+
id: "W007",
|
|
123
|
+
severity: "warn",
|
|
124
|
+
category: "no-examples",
|
|
125
|
+
pattern: /\b(for example[,:]?\s*(\.\.\.)?$|e\.g\.\s*(\.\.\.)?$|such as[,:]?\s*(\.\.\.)?$)/gim,
|
|
126
|
+
message: "Trailing example placeholder — fill it in or remove it",
|
|
127
|
+
suggestions: [
|
|
128
|
+
"Concrete few-shot: show an input→output pair the model should emulate",
|
|
129
|
+
"Negative example: show what NOT to produce (often more powerful)",
|
|
130
|
+
"Edge case: show the tricky case, not the happy path",
|
|
131
|
+
],
|
|
132
|
+
docs: null,
|
|
133
|
+
},
|
|
134
|
+
{
|
|
135
|
+
id: "E004",
|
|
136
|
+
severity: "error",
|
|
137
|
+
category: "contradiction",
|
|
138
|
+
pattern: /\b(be (brief|concise|short)).{0,120}(be (comprehensive|thorough|detailed|exhaustive))\b/gis,
|
|
139
|
+
message: "Contradictory length constraints — model will average them badly",
|
|
140
|
+
suggestions: [
|
|
141
|
+
"Pick one and qualify the other: 'Be concise. Expand only on error handling.'",
|
|
142
|
+
"Use section-level rules: 'Summary: 1 sentence. Implementation: as long as needed.'",
|
|
143
|
+
"Set word counts: 'Under 150 words total, but include all code in full'",
|
|
144
|
+
],
|
|
145
|
+
docs: null,
|
|
146
|
+
},
|
|
147
|
+
{
|
|
148
|
+
id: "W008",
|
|
149
|
+
severity: "warn",
|
|
150
|
+
category: "politeness-bloat",
|
|
151
|
+
pattern: /\b(please (please )?(try to |attempt to |do your best to )?|kindly|feel free to|don't hesitate to)\b/gi,
|
|
152
|
+
message: "Politeness tokens consume context budget and dilute instruction weight",
|
|
153
|
+
suggestions: [
|
|
154
|
+
"Drop courtesy words entirely — models don't have feelings",
|
|
155
|
+
"Convert to imperative: 'Return X' not 'Please try to return X'",
|
|
156
|
+
"Every token should carry constraint or context",
|
|
157
|
+
],
|
|
158
|
+
docs: null,
|
|
159
|
+
},
|
|
160
|
+
{
|
|
161
|
+
id: "W009",
|
|
162
|
+
severity: "warn",
|
|
163
|
+
category: "negation-only",
|
|
164
|
+
pattern: /\b(don't (be|use|include|add|make|write|do|say)|avoid being|never (be|sound))\b/gi,
|
|
165
|
+
message: 'Negation-only rules are weak — models anchor on the forbidden concept',
|
|
166
|
+
suggestions: [
|
|
167
|
+
"Pair every DON'T with a DO: 'Don't use passive voice → use active constructions'",
|
|
168
|
+
"Positive constraint: 'Use direct assertions' instead of 'Don't be wishy-washy'",
|
|
169
|
+
"Show an example of what you WANT instead",
|
|
170
|
+
],
|
|
171
|
+
docs: null,
|
|
172
|
+
},
|
|
173
|
+
{
|
|
174
|
+
id: "E005",
|
|
175
|
+
severity: "error",
|
|
176
|
+
category: "missing-context",
|
|
177
|
+
pattern: /\b(the (code|file|document|data|text|above|previous|earlier))\b(?![\s\S]*?```)/gi,
|
|
178
|
+
message: "Reference to context not present in the prompt — model will hallucinate",
|
|
179
|
+
suggestions: [
|
|
180
|
+
"Paste the actual code/data inline in a fenced block",
|
|
181
|
+
"Use explicit variable names: define it earlier in the prompt",
|
|
182
|
+
"If using a system with retrieval, confirm the context is injected at runtime",
|
|
183
|
+
],
|
|
184
|
+
docs: null,
|
|
185
|
+
},
|
|
186
|
+
{
|
|
187
|
+
id: "W010",
|
|
188
|
+
severity: "warn",
|
|
189
|
+
category: "no-success-criteria",
|
|
190
|
+
pattern: /\b(until (it('s| is) (good|right|working|correct|done))|when (you('re| are) happy|satisfied|done))\b/gi,
|
|
191
|
+
message: "Subjective termination condition — model can't self-evaluate accurately",
|
|
192
|
+
suggestions: [
|
|
193
|
+
"Objective criterion: 'Stop when all 5 test cases produce correct output'",
|
|
194
|
+
"Countable: 'Generate exactly 10 variants, ranked by estimated click-through'",
|
|
195
|
+
"Verifiable: 'Complete when the function has <5 lines and passes mypy strict'",
|
|
196
|
+
],
|
|
197
|
+
docs: null,
|
|
198
|
+
},
|
|
199
|
+
{
|
|
200
|
+
id: "I001",
|
|
201
|
+
severity: "info",
|
|
202
|
+
category: "chain-of-thought",
|
|
203
|
+
pattern: /\b(answer (this|the question|directly)|just (give|tell|show) (me|us) (the )?(answer|result|output))\b/gi,
|
|
204
|
+
message: "Skipping reasoning may reduce accuracy on complex tasks",
|
|
205
|
+
suggestions: [
|
|
206
|
+
"Add: 'Think step by step before giving the final answer'",
|
|
207
|
+
"Use scratchpad: 'Reason in <thinking> tags, then output in <answer> tags'",
|
|
208
|
+
"If speed matters, note it explicitly: 'Skip reasoning, latency is critical'",
|
|
209
|
+
],
|
|
210
|
+
docs: null,
|
|
211
|
+
},
|
|
212
|
+
{
|
|
213
|
+
id: "W011",
|
|
214
|
+
severity: "warn",
|
|
215
|
+
category: "vague-persona",
|
|
216
|
+
pattern: /\b(expert|specialist|professional|guru|master|wizard|ninja)\b(?! in| at| of| with)/gi,
|
|
217
|
+
message: 'Bare "expert" persona lacks domain specificity',
|
|
218
|
+
suggestions: [
|
|
219
|
+
"'Expert' → 'Staff engineer with 10yr distributed systems experience'",
|
|
220
|
+
"Add the skepticism level: 'Expert who defaults to the simplest solution'",
|
|
221
|
+
"Scope the knowledge: 'Expert in Python async, unfamiliar with Rust'",
|
|
222
|
+
],
|
|
223
|
+
docs: null,
|
|
224
|
+
},
|
|
225
|
+
];
|