monocart-coverage-reports 2.5.3 → 2.5.5

package/README.md

@@ -65,10 +65,10 @@ console.log(coverageResults.summary);
 ## Available Reports
 > V8 build-in reports (V8 data only):
 - `v8`
-    - Browser: Build with webpack [V8](https://cenfun.github.io/monocart-coverage-reports/v8) and [V8 Minify](https://cenfun.github.io/monocart-coverage-reports/v8-minify); Build with [Rollup](https://cenfun.github.io/monocart-coverage-reports/v8-rollup) and [Esbuild](https://cenfun.github.io/monocart-coverage-reports/v8-esbuild); Collect with [puppeteer](https://cenfun.github.io/monocart-coverage-reports/puppeteer/); [anonymous](https://cenfun.github.io/monocart-coverage-reports/anonymous/) and [css](https://cenfun.github.io/monocart-coverage-reports/css/)
-    - Node.js: Collect with [env](https://cenfun.github.io/monocart-coverage-reports/v8-node-env), and also V8 [API](https://cenfun.github.io/monocart-coverage-reports/v8-node-api), [Inspector](https://cenfun.github.io/monocart-coverage-reports/v8-node-ins) and [CDP](https://cenfun.github.io/monocart-coverage-reports/v8-node-cdp); Web server example: [koa](https://cenfun.github.io/monocart-coverage-reports/v8-node-koa/)
+    - Browser: Build with webpack [V8](https://cenfun.github.io/monocart-coverage-reports/v8) and [Minify](https://cenfun.github.io/monocart-coverage-reports/minify); Build with [Rollup](https://cenfun.github.io/monocart-coverage-reports/rollup) and [Esbuild](https://cenfun.github.io/monocart-coverage-reports/esbuild); Collect with [puppeteer](https://cenfun.github.io/monocart-coverage-reports/puppeteer/); [anonymous](https://cenfun.github.io/monocart-coverage-reports/anonymous/) and [css](https://cenfun.github.io/monocart-coverage-reports/css/)
+    - Node.js: Collect with [env](https://cenfun.github.io/monocart-coverage-reports/node-env), and also V8 [API](https://cenfun.github.io/monocart-coverage-reports/node-api), [Inspector](https://cenfun.github.io/monocart-coverage-reports/node-ins) and [CDP](https://cenfun.github.io/monocart-coverage-reports/node-cdp); Web server example: [koa](https://cenfun.github.io/monocart-coverage-reports/node-koa/)
 
-![](test/v8.gif)
+![](./assets/v8.gif)
 
 - `v8-json`
     - [V8 coverage-report.json](https://cenfun.github.io/monocart-coverage-reports/v8-and-istanbul/coverage-report.json)
@@ -76,7 +76,7 @@ console.log(coverageResults.summary);
     - coverage data for [Codecov](https://docs.codecov.com/docs/codecov-custom-coverage-format), see [example](https://app.codecov.io/github/cenfun/monocart-coverage-reports) 
 - `console-details` Show file coverage and uncovered lines in the console. Like `text`, but for V8. For Github actions, we can enforce color with env: `FORCE_COLOR: true`.
 
-![](test/console-details.png)
+![](./assets/console-details.png)
 
 > Istanbul build-in reports (both V8 and istanbul data):
 - `clover`
@@ -101,7 +101,7 @@ console.log(coverageResults.summary);
 > Other reports:
 - `console-summary` shows coverage summary in the console
 
-![](test/console-summary.png)
+![](./assets/console-summary.png)
 
 - `raw` only keep all original data, which can be used for other reports input with `inputDir`
     - see [Merge Coverage Reports](#merge-coverage-reports)
@@ -597,32 +597,12 @@ const a = tf ? 'true' : 'false';
 ```
 > `m1` and `m2` are two consecutive mappings, `p` is the position we looking for. However, we can only get the position of the `m1` or `m2` if we don't fix it to `p`. Especially the generated code is different from the original code, such as the code was minified, compressed or converted, it is difficult to find the exact position.
 
-- 2, The coverage of functions and branches is incorrect. V8 only provided coverage at functions and it's blocks. But if a function is uncovered (count = 0), there is no information for it's blocks and sub-level functions.
-And also there are some problems about counting the functions and branches:
-```js
-// Problem: When the function or parent-level function is uncovered, then its sub-level functions will never be counted.
-functions.forEach(block => {
-    block.ranges.forEach((range, i) => {
-        if (block.isBlockCoverage) {
-            // v8-to-istanbul: new CovBranch() 
-            // Problem: not every block is branch, and the first block could be function.
-            if (block.functionName && i === 0) {
-                // v8-to-istanbul: new CovFunction()
-                // Problem: no anonymous function
-            }
-        } else if (block.functionName) {
-            // v8-to-istanbul: new CovFunction()
-            // Problem: no anonymous function
-        }
-    }
-});
-```
-see source code [v8-to-istanbul.js](https://github.com/istanbuljs/v8-to-istanbul/blob/master/lib/v8-to-istanbul.js)
+- 2, The coverage of functions and branches is incorrect. V8 only provided coverage at functions and it's blocks. But if a function is uncovered (count = 0), there is no information for it's blocks and sub-level functions. And also there are some problems about counting the functions and branches.
 
 ### How Monocart Works
-We implemented new converter instead of v8-to-istanbul:
+We implemented new converter:
 - 1, Trying to fix the middle position if not found the exact mapping for the position.
-- 2, Finding all functions, statements and branches by parsing the source code [AST](https://github.com/acornjs/acorn), however the V8 cannot provide effective branch coverage information for `AssignmentPattern`.
+- 2, Finding all functions, statements and branches by parsing the source code [AST](https://github.com/acornjs/acorn). However, there's a small issue, which is the V8 cannot provide effective branch coverage information for `AssignmentPattern`.
 
 | AST                   | V8             | 
 | :---------------------| :------------- | 
@@ -645,11 +625,11 @@ const coverageOptions = {
 };
 ```
 When `logging` is `debug`, the raw report data will be preserved in `[outputDir]/.cache` or `[outputDir]/raw` if `raw` report is used. And the dist file will be preserved in the V8 list, and by opening the browser's devtool, it makes data verification visualization effortless.
-![](./test/debug-coverage.png)
+![](./assets/debug-coverage.png)
 
 - Check sourcemap with [Source Map Visualization](https://evanw.github.io/source-map-visualization/)
 
-![](./test/debug-sourcemap.png)
+![](./assets/debug-sourcemap.png)
 
 ## Integration
 ### [Playwright](https://github.com/microsoft/playwright)

package/lib/converter/ast.js

@@ -148,7 +148,9 @@ const getFakeAstInfo = (coverageList) => {
 
 const getJsAstInfo = (item, coverageList) => {
 
-    const { source, fake } = item;
+    const {
+        source, fake, empty
+    } = item;
 
     if (fake) {
         return getFakeAstInfo(coverageList);
@@ -169,7 +171,11 @@ const getJsAstInfo = (item, coverageList) => {
     try {
         ast = acorn.parse(source, options);
     } catch (e) {
-        Util.logInfo(`failed to parse source file: ${item.sourcePath} ${e.message}`);
+
+        // empty = untested file, could be .ts, .vue and so on, that can not be parsed normally
+        if (!empty) {
+            Util.logInfo(`failed to parse source file: ${item.sourcePath} ${e.message}`);
+        }
         // https://github.com/acornjs/acorn/tree/master/acorn-loose
         // It is recommended to always try a parse with the regular acorn parser first,
         // and only fall back to this parser when that one finds syntax errors.
@@ -208,18 +214,20 @@ const addRule = (item, coverageList, coveredRanges, count = 0) => {
 
 };
 
-const addAtRule = (item, coverageList, coveredRanges) => {
+const addAtRule = (item, coverageList, coveredRanges, empty) => {
     const { name, nodes } = item;
 
+    const defaultCount = empty ? 0 : 1;
+
     if (['charset', 'import', 'namespace'].includes(name)) {
-        addRule(item, coverageList, coveredRanges, 1);
+        addRule(item, coverageList, coveredRanges, defaultCount);
         return;
     }
 
     if (['media', 'supports', 'container', 'layer'].includes(name)) {
 
         const childCoverageList = [];
-        const count = addCssRules(nodes, childCoverageList, coveredRanges);
+        const count = addCssRules(nodes, childCoverageList, coveredRanges, empty);
         if (count) {
 
             coverageList.push({
@@ -249,7 +257,7 @@ const addAtRule = (item, coverageList, coveredRanges) => {
 
 };
 
-const addCssRules = (list, coverageList, coveredRanges) => {
+const addCssRules = (list, coverageList, coveredRanges, empty) => {
 
     let count = 0;
 
@@ -287,7 +295,7 @@ const addCssRules = (list, coverageList, coveredRanges) => {
         // });
 
         if (type === 'atrule') {
-            addAtRule(item, coverageList, coveredRanges);
+            addAtRule(item, coverageList, coveredRanges, empty);
         }
 
     });
@@ -298,7 +306,7 @@ const addCssRules = (list, coverageList, coveredRanges) => {
 const getCssAstInfo = (item, coverageList) => {
 
     const {
-        source, sourcePath, ranges
+        source, ranges, empty
     } = item;
 
     // sort covered ranges
@@ -312,13 +320,9 @@ const getCssAstInfo = (item, coverageList) => {
         };
     });
 
-    const time_start_parse = new Date();
     const ast = parseCss(source);
-    Util.logTime(`parse css: ${sourcePath}`, time_start_parse);
 
-    // if (sourcePath.endsWith('style.css')) {
-    addCssRules(ast.nodes, coverageList, coveredRanges);
-    // }
+    addCssRules(ast.nodes, coverageList, coveredRanges, empty);
 
     return {
         functions: [],

package/lib/converter/collect-source-maps.js

@@ -1,4 +1,5 @@
 const fs = require('fs');
+const path = require('path');
 const EC = require('eight-colors');
 const { fileURLToPath, pathToFileURL } = require('url');
 const Concurrency = require('../platform/concurrency.js');
@@ -66,16 +67,49 @@ const getSourceMapUrl = (content, url) => {
     }
 };
 
-const resolveSourceMap = (data) => {
-    if (data) {
-        const {
-            sources, sourcesContent, mappings
-        } = data;
-        if (!sources || !sourcesContent || !mappings) {
-            return;
+const resolveSourcesContent = async (sources, url) => {
+    const list = [];
+    for (const file of sources) {
+        const sourceUrl = Util.resolveUrl(file, url);
+        if (sourceUrl) {
+            const sourcePath = sourceUrl.toString();
+            if (sourcePath.startsWith('file:')) {
+                const content = await Util.readFile(path.resolve(fileURLToPath(sourcePath)));
+                list.push(content);
+                continue;
+            }
         }
+        Util.logDebug(EC.red(`failed to load source content ${file}`));
+    }
+    if (list.length === sources.length) {
+        return list;
+    }
+};
+
+const resolveSourceMap = async (data, url) => {
+    if (!data) {
+        return;
+    }
+    const {
+        sources, sourcesContent, mappings
+    } = data;
+
+    if (!sources || !mappings) {
+        return;
+    }
+
+    // exists sources content
+    if (sourcesContent && sourcesContent.length === sources.length) {
         return data;
     }
+
+    // load sources content by sources
+    const contents = await resolveSourcesContent(sources, url);
+    if (contents) {
+        data.sourcesContent = contents;
+        return data;
+    }
+
 };
 
 const saveSourceFile = async (filePath, data) => {
@@ -92,7 +126,7 @@ const saveSourceFile = async (filePath, data) => {
 
 const collectSourceMaps = async (v8list, options) => {
 
-    let count = 0;
+    const smList = [];
     const concurrency = new Concurrency();
     for (const item of v8list) {
 
@@ -130,8 +164,11 @@ const collectSourceMaps = async (v8list, options) => {
             // from inline sync
             const smc = convertSourceMap.fromSource(source);
             if (smc) {
-                sourceData.sourceMap = resolveSourceMap(smc.sourcemap);
-                count += 1;
+                sourceData.sourceMap = await resolveSourceMap(smc.sourcemap, url);
+                smList.push({
+                    url,
+                    inline: true
+                });
                 await saveSourceFile(sourcePath, sourceData);
                 continue;
             }
@@ -154,16 +191,20 @@ const collectSourceMaps = async (v8list, options) => {
 
     // from url concurrency
     await concurrency.start(async (item) => {
-        const sourceData = item.sourceData;
-        const data = await loadSourceMap(item.sourceMapUrl);
+        const { sourceMapUrl, sourceData } = item;
+        const { url } = sourceData;
+        const data = await loadSourceMap(sourceMapUrl);
         if (data) {
-            sourceData.sourceMap = resolveSourceMap(data);
-            count += 1;
+            sourceData.sourceMap = await resolveSourceMap(data, url);
+            smList.push({
+                url,
+                sourceMapUrl
+            });
         }
         await saveSourceFile(item.sourcePath, sourceData);
     });
 
-    return count;
+    return smList;
 
 };
 

package/lib/converter/converter.js

@@ -1300,6 +1300,7 @@ const convertV8List = (v8list, options) => {
         }
 
         // ============================
+        const time_start_ast = Date.now();
         // move functions and ranges to coverageList
         let coverageList = [];
         let astInfo;
@@ -1318,6 +1319,7 @@ const convertV8List = (v8list, options) => {
                 delete item.ranges;
             }
         }
+        Util.logTime(`- parsed ast: ${sourcePath}`, time_start_ast);
 
         // console.log(sourcePath, astInfo.statements.length);
         // ============================
@@ -1345,14 +1347,19 @@ const convertV8List = (v8list, options) => {
             v8Data: item
         };
 
+        const time_start_unpack = Date.now();
         unpackDistFile(item, state, options);
+        Util.logTime(`- unpacked dist: ${sourcePath}`, time_start_unpack);
 
         stateList.push(state);
 
     }
 
-    return generateV8DataList(stateList, options);
+    const time_start_convert = Date.now();
+    const dataList = generateV8DataList(stateList, options);
+    Util.logTime('- generated v8 data list', time_start_convert);
 
+    return dataList;
 };
 
 module.exports = {

package/lib/generate.js

@@ -247,7 +247,7 @@ const getCoverageResults = async (dataList, sourceCache, options) => {
         const {
             v8DataList, coverageData, fileSources
         } = results;
-        Util.logTime(`converted v8 data: ${v8DataList.length}`, time_start);
+        Util.logTime(`converted v8 list: ${v8DataList.length}`, time_start);
 
         return generateV8ListReports(v8DataList, coverageData, fileSources, options);
     }