@codacy/tools-opengrep-1 0.23.0 → 0.24.0

package/README.md

@@ -24,20 +24,47 @@ Multi-language SAST (Static Application Security Testing) scanner using the [Ope
 | Strategy         | CLI                                                                |
 | Languages        | 30+ (Python, JavaScript, TypeScript, Java, Go, Ruby, C, C++, etc.) |
 | Binary           | `opengrep`                                                         |
-| File patterns    | `*` (auto-detects by language)                                     |
+| File patterns    | ~70 extensions derived from the languages above, plus `Dockerfile` |
 | Pattern count    | ~2500                                                              |
 | Extra dependency | `yaml` (for YAML rule config parsing/generation)                   |
 
 ## Rule bundling
 
-Opengrep uses YAML rule files for detection. The adapter bundles ~2200 rules into `src/rules.yaml`, sourced from:
+Opengrep uses YAML rule files for detection. The adapter bundles ~2500 rules into `src/rules.yaml`, sourced from:
 
-1. **semgrep/semgrep-rules** — community rules (pinned at commit `4ccd3b9`)
-2. **GitLab SAST rules** — MIT-licensed rules from `gitlab.com/gitlab-org/security-products/sast-rules`
-3. **Codacy custom rules** — from `codacy/codacy-opengrep` wrapper repo
+1. **opengrep/opengrep-rules** — the registry rules, pinned at the same commit as the Codacy image
+2. **GitLab SAST rules** — MIT-licensed, from `gitlab.com/gitlab-org/security-products/sast-rules`
+3. **Codacy custom rules** — `docs/codacy-rules*.yaml` in `codacy/codacy-opengrep`
 
 The bundled `src/rules.yaml` is **committed to the repository** for consistency and reproducibility.
 
+### Keeping it in step with the Codacy image
+
+All three sources, their pinned commits and their file filters are mirrored from
+`codacy/codacy-opengrep`'s `internal/docgen/parsing.go`. Keep them that way: the image is the
+reference implementation, and `scripts/fetch-rules.mjs` says which upstream function each piece
+came from.
+
+Three mistakes in that mirroring cost **296 of 2534 patterns (12%)**, every one of them silent —
+the pattern stays in `patterns.json`, `init --auto` enables it, the config claims it is active, and
+it simply never fires:
+
+- **Rule ids were derived from the file path.** GitLab ids are written in the file and must be
+  taken verbatim. Path derivation looks right for the top-level tree (`java/crypto/rule-X.yml`
+  really does hold `java_crypto_rule-X`) and is wrong for the licence-tiered one, where
+  `rules/lgpl-cc/properties/spring/accesscontrol/rule-Y.yml` holds `properties_spring_rule-Y`. The
+  sharpest case is `rule-JwtDecodeWithoutVerify.yml`, whose id uses an underscore where the
+  filename uses a hyphen. 265 of 588 GitLab rules were lost this way.
+- **The registry source was upstream `semgrep/semgrep-rules`**, not Opengrep's fork. The two share
+  most ids, so the output looks plausible while individual rules go missing.
+- **YAML was read and rewritten line by line.** That corrupted multi-line flow scalars — the
+  generated file stopped parsing, and opengrep then reported _zero findings for the entire run_
+  rather than an error — and it skipped every `tests/` directory, where some shipped rules live.
+
+`test/rules-coverage.test.ts` is the guard: it fails if any catalogued pattern has no rule, and if
+the generated file does not parse. Run `pnpm fetch-rules` when it does; the script prints the
+orphaned ids.
+
 ## Updating patterns
 
 ```bash
@@ -57,7 +84,8 @@ Pattern IDs follow the format `Semgrep_python.lang.security.audit.exec-detected.
 pnpm fetch-rules
 
 # This clones the rule repos, extracts matching rules, and writes src/rules.yaml
-# Only rules whose IDs match src/patterns.json are included
+# Only rules whose IDs match src/patterns.json are included — and the script now
+# warns loudly about the reverse, a catalogued pattern with no rule to run.
 
 # Commit the result
 git add src/rules.yaml

package/dist/index.js

@@ -30722,14 +30722,18 @@ var pattern_mappings_default = {
 // src/metadata.ts
 var descriptor = {
   id: "Semgrep",
+  shortName: "opengrep",
   displayName: "Opengrep",
   preferredVersion: "1.26.0",
+  // Canonical `KNOWN_LANGUAGES` names, which is what `init --auto` matches against
+  // (`init.ts:882`). The engine's own names differ and must be translated here:
+  // Bash/Shell and Vue/Javascript collapse onto one entry each, since `.sh` and
+  // `.vue` are already those languages. Guarded by `descriptor-languages.test.ts`.
   languages: [
     "Apex",
-    "Bash",
     "C",
-    "C++",
-    "C#",
+    "CPP",
+    "CSharp",
     "Clojure",
     "Dart",
     "Dockerfile",
@@ -30737,7 +30741,7 @@ var descriptor = {
     "Go",
     "HTML",
     "Java",
-    "JavaScript",
+    "Javascript",
     "JSON",
     "Kotlin",
     "Lua",
@@ -30750,12 +30754,110 @@ var descriptor = {
     "Swift",
     "Terraform",
     "TypeScript",
-    "Visual Basic",
-    "Vue",
+    "VisualBasic",
     "XML",
     "YAML"
   ],
-  filePatterns: ["*"],
+  // Derived from `languages` above, via `LANGUAGE_CONFIG`, plus the extensions
+  // sibling adapters already route for those same languages (`.cjs`, `.hh`,
+  // `.jsonc`, `.json5`, `.pyi`, `.dash`, `.ksh`) — `LANGUAGE_CONFIG` is a
+  // detection map and does not claim to be exhaustive per tool.
+  //
+  // This was `["*"]`, which `routeFilesToTool` short-circuits to "every file", so
+  // opengrep was handed the whole repository: on one 1774-file repo, 358 of them
+  // (20%) were png/jpg/font assets. Opengrep resolves a language by extension and
+  // skips what it cannot parse, so findings were never affected — but the paths
+  // still cost a realpath+stat each, inflated the reported file count, and ate the
+  // 200 KB argv budget, splitting one `opengrep scan` into two.
+  //
+  // Keep this a superset of `LANGUAGE_CONFIG`'s extensions for every declared
+  // language; `opengrep-1/test/file-patterns.test.ts` fails if a language gains one
+  // that is not routed here, because a missed extension is a silent coverage loss.
+  filePatterns: [
+    "**/*.bash",
+    "**/*.c",
+    "**/*.cc",
+    "**/*.cjs",
+    "**/*.clj",
+    "**/*.cljc",
+    "**/*.cljs",
+    "**/*.cls",
+    "**/*.cpp",
+    "**/*.cs",
+    "**/*.cxx",
+    "**/*.dart",
+    "**/*.dash",
+    "**/*.dockerfile",
+    "**/*.edn",
+    "**/*.env",
+    "**/*.ex",
+    "**/*.exs",
+    "**/*.gemspec",
+    "**/*.go",
+    "**/*.h",
+    "**/*.hh",
+    "**/*.hpp",
+    "**/*.htm",
+    "**/*.html",
+    "**/*.ino",
+    "**/*.java",
+    "**/*.jbuilder",
+    "**/*.js",
+    "**/*.jsm",
+    "**/*.json",
+    "**/*.json5",
+    "**/*.jsonc",
+    "**/*.jsx",
+    "**/*.ksh",
+    "**/*.kt",
+    "**/*.kts",
+    "**/*.lua",
+    "**/*.luau",
+    "**/*.mjs",
+    "**/*.opal",
+    "**/*.php",
+    "**/*.podspec",
+    "**/*.pom",
+    "**/*.py",
+    "**/*.pyi",
+    "**/*.rake",
+    "**/*.rb",
+    "**/*.rlib",
+    "**/*.rs",
+    "**/*.scala",
+    "**/*.sh",
+    "**/*.swift",
+    "**/*.tf",
+    "**/*.trigger",
+    "**/*.ts",
+    "**/*.tsx",
+    "**/*.vb",
+    "**/*.vue",
+    "**/*.wsdl",
+    "**/*.xhtml",
+    "**/*.xml",
+    "**/*.xsl",
+    "**/*.yaml",
+    "**/*.yml",
+    // Source files with no extension. `LANGUAGE_CONFIG.files` is a *detection*
+    // list — it is mostly lockfiles (`yarn.lock`, `go.sum`, `poetry.lock`), which
+    // identify a language without being source in it — so only the entries
+    // opengrep can actually parse are routed here: Dockerfiles, and Ruby's
+    // extensionless DSL files.
+    "**/Dockerfile",
+    "**/Dockerfile.*",
+    "**/Berksfile",
+    "**/Capfile",
+    "**/Cheffile",
+    "**/Fastfile",
+    "**/Gemfile",
+    "**/Guardfile",
+    "**/Podfile",
+    "**/Rakefile",
+    "**/Thorfile",
+    "**/Vagrantfile",
+    "**/config.ru"
+  ],
   supportedOS: ["linux", "darwin"],
   supportedArch: ["x64", "arm64"],
   supportedRunners: ["local", "container"],
@@ -30845,9 +30947,11 @@ function stripCheckIdPrefix(checkId, configPrefix) {
   }
   return stripped;
 }
-function truncate(str, maxLen) {
-  if (str.length <= maxLen) return str;
-  return str.slice(0, maxLen - 3) + "...";
+function firstSentence(message) {
+  const oneLine = message.replace(/\n/g, " ");
+  const match = /(^.*?[a-z]{2,}[.!?])\s+\W*[A-Z]/.exec(oneLine);
+  if (match) return match[1];
+  return oneLine.length > 500 ? oneLine.slice(0, 500) : oneLine;
 }
 var ARGV_BYTE_BUDGET = 200 * 1024;
 function chunkFilesByArgvBudget(files) {
@@ -31220,7 +31324,10 @@ var adapter = {
             kind: errorType,
             message,
             level,
-            filePath
+            filePath,
+            // Opengrep only reports a path on errors that stopped it scanning
+            // that file, so a path here means the file was not analysed.
+            ...filePath ? { fileNotAnalyzed: true } : {}
           });
         }
       }
@@ -31257,7 +31364,7 @@ var adapter = {
             column: finding.start.col,
             endLine: finding.end.line,
             endColumn: finding.end.col,
-            message: truncate(finding.extra.message, 250),
+            message: firstSentence(finding.extra.message ?? ""),
             lineContent: (0, import_tooling.getLineContent)(lines, finding.start.line),
             multilineContent: (0, import_tooling.getMultilineContent)(lines, finding.start.line, finding.end.line),
             category: meta.category,

package/dist/metadata.js

@@ -255,14 +255,18 @@ var pattern_mappings_default = {
 // src/metadata.ts
 var descriptor = {
   id: "Semgrep",
+  shortName: "opengrep",
   displayName: "Opengrep",
   preferredVersion: "1.26.0",
+  // Canonical `KNOWN_LANGUAGES` names, which is what `init --auto` matches against
+  // (`init.ts:882`). The engine's own names differ and must be translated here:
+  // Bash/Shell and Vue/Javascript collapse onto one entry each, since `.sh` and
+  // `.vue` are already those languages. Guarded by `descriptor-languages.test.ts`.
   languages: [
     "Apex",
-    "Bash",
     "C",
-    "C++",
-    "C#",
+    "CPP",
+    "CSharp",
     "Clojure",
     "Dart",
     "Dockerfile",
@@ -270,7 +274,7 @@ var descriptor = {
     "Go",
     "HTML",
     "Java",
-    "JavaScript",
+    "Javascript",
     "JSON",
     "Kotlin",
     "Lua",
@@ -283,12 +287,110 @@ var descriptor = {
     "Swift",
     "Terraform",
     "TypeScript",
-    "Visual Basic",
-    "Vue",
+    "VisualBasic",
     "XML",
     "YAML"
   ],
-  filePatterns: ["*"],
+  // Derived from `languages` above, via `LANGUAGE_CONFIG`, plus the extensions
+  // sibling adapters already route for those same languages (`.cjs`, `.hh`,
+  // `.jsonc`, `.json5`, `.pyi`, `.dash`, `.ksh`) — `LANGUAGE_CONFIG` is a
+  // detection map and does not claim to be exhaustive per tool.
+  //
+  // This was `["*"]`, which `routeFilesToTool` short-circuits to "every file", so
+  // opengrep was handed the whole repository: on one 1774-file repo, 358 of them
+  // (20%) were png/jpg/font assets. Opengrep resolves a language by extension and
+  // skips what it cannot parse, so findings were never affected — but the paths
+  // still cost a realpath+stat each, inflated the reported file count, and ate the
+  // 200 KB argv budget, splitting one `opengrep scan` into two.
+  //
+  // Keep this a superset of `LANGUAGE_CONFIG`'s extensions for every declared
+  // language; `opengrep-1/test/file-patterns.test.ts` fails if a language gains one
+  // that is not routed here, because a missed extension is a silent coverage loss.
+  filePatterns: [
+    "**/*.bash",
+    "**/*.c",
+    "**/*.cc",
+    "**/*.cjs",
+    "**/*.clj",
+    "**/*.cljc",
+    "**/*.cljs",
+    "**/*.cls",
+    "**/*.cpp",
+    "**/*.cs",
+    "**/*.cxx",
+    "**/*.dart",
+    "**/*.dash",
+    "**/*.dockerfile",
+    "**/*.edn",
+    "**/*.env",
+    "**/*.ex",
+    "**/*.exs",
+    "**/*.gemspec",
+    "**/*.go",
+    "**/*.h",
+    "**/*.hh",
+    "**/*.hpp",
+    "**/*.htm",
+    "**/*.html",
+    "**/*.ino",
+    "**/*.java",
+    "**/*.jbuilder",
+    "**/*.js",
+    "**/*.jsm",
+    "**/*.json",
+    "**/*.json5",
+    "**/*.jsonc",
+    "**/*.jsx",
+    "**/*.ksh",
+    "**/*.kt",
+    "**/*.kts",
+    "**/*.lua",
+    "**/*.luau",
+    "**/*.mjs",
+    "**/*.opal",
+    "**/*.php",
+    "**/*.podspec",
+    "**/*.pom",
+    "**/*.py",
+    "**/*.pyi",
+    "**/*.rake",
+    "**/*.rb",
+    "**/*.rlib",
+    "**/*.rs",
+    "**/*.scala",
+    "**/*.sh",
+    "**/*.swift",
+    "**/*.tf",
+    "**/*.trigger",
+    "**/*.ts",
+    "**/*.tsx",
+    "**/*.vb",
+    "**/*.vue",
+    "**/*.wsdl",
+    "**/*.xhtml",
+    "**/*.xml",
+    "**/*.xsl",
+    "**/*.yaml",
+    "**/*.yml",
+    // Source files with no extension. `LANGUAGE_CONFIG.files` is a *detection*
+    // list — it is mostly lockfiles (`yarn.lock`, `go.sum`, `poetry.lock`), which
+    // identify a language without being source in it — so only the entries
+    // opengrep can actually parse are routed here: Dockerfiles, and Ruby's
+    // extensionless DSL files.
+    "**/Dockerfile",
+    "**/Dockerfile.*",
+    "**/Berksfile",
+    "**/Capfile",
+    "**/Cheffile",
+    "**/Fastfile",
+    "**/Gemfile",
+    "**/Guardfile",
+    "**/Podfile",
+    "**/Rakefile",
+    "**/Thorfile",
+    "**/Vagrantfile",
+    "**/config.ru"
+  ],
   supportedOS: ["linux", "darwin"],
   supportedArch: ["x64", "arm64"],
   supportedRunners: ["local", "container"],

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@codacy/tools-opengrep-1",
-  "version": "0.23.0",
+  "version": "0.24.0",
   "description": "Opengrep adapter — CLI-mode multi-language SAST scanner",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -10,7 +10,7 @@
   ],
   "dependencies": {
     "yaml": "^2.6.0",
-    "@codacy/tooling": "0.23.0"
+    "@codacy/tooling": "0.24.0"
   },
   "license": "MIT",
   "author": "Codacy <support@codacy.com> (https://www.codacy.com)",