cc4-embedded-system 3.1.10 → 3.2.1

package/README.md

@@ -1,31 +1,40 @@
 # CC4EmbeddedSystem V3
-- This version based on [html-minifier-next](https://github.com/j9t/html-minifier-next) and rewrite [lwIP makefsdata](https://github.com/m-labs/lwip/tree/master/src/apps/httpd/makefsdata), run on localhost, default port: ```3000```.
+- V3.2.1 is a TypeScript/Node.js rewrite of [lwIP makefsdata](https://github.com/m-labs/lwip/tree/master/src/apps/httpd/makefsdata). GUI mode runs on localhost, default port: `3000`.
+- Assets are handled by type: [html-minifier-next](https://github.com/j9t/html-minifier-next) processes HTML, [Lightning CSS](https://lightningcss.dev/) processes CSS, [esbuild](https://esbuild.github.io/) processes JavaScript, and [SVGO](https://svgo.dev/) processes SVG.
+- Optional deterministic gzip storage reduces the Flash payload for text assets. The MCU sends the stored bytes unchanged; a gzip-capable HTTP client, normally a browser, decompresses the response.
 - This tool is deployed on [npm package](https://www.npmjs.com/package/cc4-embedded-system).
+- The last successful source/destination paths and gzip setting are automatically saved as `cc4es_configs.json` in the operating system's local application-data directory. On Windows this is `%LOCALAPPDATA%\\cc4-embedded-system\\cc4es_configs.json`.
+- The config file is created only after the first successful build, and its complete path is printed at that time. Use `cc4es --config-path` to print it later.
 
 ## Demo
-- Select input directory
-
-![](./Screenshot/inputs.png)
-- Select output file path
-
-![](./Screenshot/output.png)
-- Result
-
-![](./Screenshot/v3.1.10.png)
+### CLI mode
+![](./Screenshot/cli.png)
+### Select input directory
+![](./Screenshot/src.png)
+### Select output file path
+![](./Screenshot/dst.png)
+### Results
+![](./Screenshot/v3.2.1_result1.png)
+![](./Screenshot/v3.2.1_result2.png)
 
 ## Structure
 ```text
 CC4EmbeddedSystem/
 ├── src/
-│   ├── gui.ts        // Express server & CLI entry point
-│   │   └── utils.ts  // To get latest tool version
-│   └── makefsdata.ts // Core C code generation & minification logic
+│   ├── gui.ts            // CLI entry point
+│   ├── cli.ts            // Command-line parsing and help text
+│   ├── server.ts         // Express GUI server
+│   ├── config.ts         // Persistent paths and gzip options
+│   ├── gzip-options.ts   // Shared gzip defaults and validation
+│   ├── minify-options.ts // Shared HTML minifier options
+│   ├── makefsdata.ts     // Asset routing, gzip, and C code generation
+│   └── utils.ts          // Package version lookup
 ├── public/
 │   └── index.html    // Web GUI dashboard
-├── Sereenshot/
-│   ├── inputs.png    // Demo image
-│   ├── output.png    // Demo image
-│   └── v3.1.10.png   // Demo image
+├── test/
+│   └── cc4es.test.mjs // Native Node.js regression tests
+├── Screenshot/
+│   └── ...            // GUI and CLI screenshots
 ├── node_modules/     // Required submodules during development
 ├── dist/             // Compiled JavaScript output (Auto-generated)
 ├── package.json      // Project configuration & dependencies
@@ -33,7 +42,108 @@ CC4EmbeddedSystem/
 └── tsconfig.json     // TypeScript configuration
 ```
 
+## Build architecture
+
+CC4ES never modifies the source directory. For each file it produces one final response body, writes the HTTP header and body into `fsdata.c`, and keeps lwIP's `FS_FILE_FLAGS_HEADER_INCLUDED` model.
+
+```text
+source file
+  -> extension-based optimizer
+       .html/.htm  html-minifier-next
+       .css        Lightning CSS
+       .js/.mjs    esbuild (or raw with --no-minify-js)
+       .svg        SVGO
+       other       raw copy
+  -> final response body
+  -> optional deterministic gzip when it is smaller
+  -> HTTP header + fsdata.c byte array
+```
+
+HTML-only options such as `--remove-comments` are not applied to external JavaScript. This prevents an external `.js` file from being interpreted as an HTML comment. `--no-minify-js` and `--no-minify-css` keep their respective external assets unchanged, while HTML can still use its own minification options.
+
+### Gzip storage
+
+Gzip is optional and disabled by default. `--gzip` considers `.html`, `.htm`, `.css`, `.js`, `.mjs`, `.json`, `.svg`, `.xml`, `.txt`, and `.map`; binary and already-compressed resource types remain raw. A candidate is stored as gzip only when its gzip body is strictly smaller than its final raw body.
+
+For gzip bodies, CC4ES writes `Content-Encoding: gzip` before the HTTP header terminator. The gzip mtime and OS metadata are normalized so the same input and options produce byte-identical output. A raw fallback has no `Content-Encoding` header.
+
+This is a fixed storage representation, not runtime content negotiation: a gzip-enabled `fsdata.c` does not also store a raw duplicate and does not inspect `Accept-Encoding`. Enable gzip only when the target clients support it; use `--no-gzip` for legacy clients. No MCU decompressor, zlib dependency, lwIP change, or application-code change is required.
+
+#### Measurement example
+
+The following measured an example app build used `--gzip --gzip-level 9`. The exact result depends on the frontend assets, compiler settings, and firmware image format; treat it as an example rather than a universal saving guarantee.
+
+| File | Without gzip | gzip level 9 | Reduction |
+|---|---:|---:|---:|
+| `fsdata.c` | 652,029 B | 197,764 B | 454,265 B (69.67%) |
+| `app.elf` | 2,641,144 B | 2,555,120 B | 86,024 B (3.26%) |
+| `app.hex` | 1,001,729 B | 751,917 B | 249,812 B (24.94%) |
+| `app.img` | 356,108 B | 267,292 B | 88,816 B (24.94%) |
+
+The flashed memory regions show where the saving occurs:
+
+| Region | Without gzip | gzip level 9 | Reduction |
+|---|---:|---:|---:|
+| `text` | 355,420 B | 266,604 B | 88,816 B |
+| `data` | 680 B | 680 B | 0 B |
+| `bss` | 83,280 B | 83,280 B | 0 B |
+
+In this build, gzip saved **88,816 bytes** of Flash, approximately **86.73 KiB**. The ELF file includes debug symbols, so its on-disk size decreases by only 3.26%; use the `.img` size or the `text + data` regions when assessing flashed capacity.
+
+## Path Configs
+After the first successful build, CC4EmbeddedSystem saves the most recently used source/destination paths and gzip settings in `cc4es_configs.json`.
+
+The file is stored in the local application-data directory by default:
+- Windows: `%LOCALAPPDATA%\cc4-embedded-system\cc4es_configs.json`
+- macOS: `~/Library/Application Support/cc4-embedded-system/cc4es_configs.json`
+- Linux: `$XDG_STATE_HOME/cc4-embedded-system/cc4es_configs.json` (or `~/.local/state/...`)
+
+Run `cc4es --config-path` to print the exact path on the current computer.
+
+```json
+{
+  "schemaVersion": 2,
+  "lastBuild": {
+    "src": "C:\\YourProjects\\src",
+    "dst": "C:\\YourProjects\\dst\\fsdata.c"
+  },
+  "gzipOptions": {
+    "gzip": false,
+    "gzipLevel": 9
+  }
+}
+```
+
 ## Commands
+### CLI Help Texts
+```text
+Usage: cc4es [options]
+
+Without options, starts the browser GUI.
+
+  -h, --help                         Show this help and exit
+  -V, --version                      Show version and exit
+  --config-path                      Print the full config file path and exit
+  --headless                         Build without server or browser
+  -s, --src <directory>              Source directory (headless)
+  -d, --dst <file|directory>         Output C file or directory (default: fsdata.c)
+  -p, --port <1-65535>               GUI server port
+  --[no-]optimize-svg                Enable or disable SVGO (default: enabled)
+  --svgo-multipass                   Run SVGO optimization repeatedly
+  --[no-]gzip                        Enable or disable gzip resource storage (default: disabled)
+  --gzip-level <1-9>                 gzip compression level (default: 9)
+
+HTML compression options:
+  --[no-]collapse-whitespace         --[no-]remove-comments
+  --[no-]minify-js                   --[no-]minify-css
+  --[no-]conditional-comments        --[no-]decode-entities
+  --[no-]remove-attribute-quotes     --[no-]remove-empty-attributes
+  --[no-]remove-redundant-attributes --[no-]use-short-doctype
+
+Headless mode requires --src or --dst and never reruns the last build with
+no path options. A missing counterpart uses the last saved path; if no
+destination is saved, fsdata.c is created in the current working directory.
+```
 ### Normal Use
 - Global installation
     ```bash
@@ -48,19 +158,57 @@ CC4EmbeddedSystem/
     ```bash
     cc4es --port 3002
     ```
+- Headless build (does not start a server or open a browser)
+    ```bash
+    cc4es --headless --src C:\Project\web --dst C:\Project\fsdata.c
+    ```
+- Headless build with the default output filename. An existing output directory uses `fsdata.c` inside it.
+    ```bash
+    cc4es --headless --src C:\Project\web
+    cc4es --headless --src C:\Project\web --dst C:\Project\output
+    ```
+- Headless safety: a parameterless command stops instead of rerunning the last build
+    ```bash
+    cc4es --headless
+    ```
+- CLI help and version
+    ```bash
+    cc4es --help
+    cc4es --version
+    cc4es --config-path
+    ```
+- Compression flags can be supplied in headless mode. Every flag accepts a `--no-` form.
+    ```bash
+    cc4es --headless --no-minify-js --remove-attribute-quotes --svgo-multipass
+    ```
+- Gzip is optional and disabled by default. Only text resources that become smaller are stored as gzip; their response header contains `Content-Encoding: gzip`.
+    ```bash
+    cc4es --headless --src C:\Project\web --dst C:\Project\fsdata.c --gzip --gzip-level 9
+    ```
+  A gzip-enabled image stores one fixed gzip representation for eligible resources. It does not keep a raw duplicate or negotiate `Accept-Encoding`; use `--no-gzip` when a consumer requires non-gzip clients. Test it with a browser or `curl --compressed http://DEVICE_IP/app.js`.
+- Files are processed by type: HTML uses html-minifier-next, CSS uses Lightning CSS, JavaScript uses esbuild, and SVG uses SVGO. `--no-minify-js` and `--no-minify-css` preserve their respective external assets.
 - Update
   ```bash
   npm install -g cc4-embedded-system@latest
   ```
 ### Development
 ```bash
-# build
+# install locked dependencies
+npm ci
+
+# run regression tests
+npm test
+
+# compile TypeScript and copy public GUI assets to dist/
 npm run build
 
 # mimic global installation
 npm link
 cc4es
 
+# headless development test
+npm run dev -- --headless --src C:\Project\web --dst C:\Project\fsdata.c
+
 # publish
 npm whoami # verify
 npm login
@@ -68,4 +216,4 @@ npm version patch --no-git-tag-version # if increase version
 npm run dev # dev mode for testing
 npm run build # build the newest
 npm publish --access public
-```
+```

package/dist/cli.js

@@ -0,0 +1,171 @@
+import { DEFAULT_HTML_MINIFY_OPTIONS } from './minify-options.js';
+import { DEFAULT_GZIP_LEVEL, isGzipLevel } from './gzip-options.js';
+export class CliUsageError extends Error {
+    constructor(message) {
+        super(message);
+        this.name = 'CliUsageError';
+    }
+}
+const MINIFY_FLAGS = {
+    '--collapse-whitespace': 'collapseWhitespace',
+    '--no-collapse-whitespace': 'collapseWhitespace',
+    '--remove-comments': 'removeComments',
+    '--no-remove-comments': 'removeComments',
+    '--minify-js': 'minifyJS',
+    '--no-minify-js': 'minifyJS',
+    '--minify-css': 'minifyCSS',
+    '--no-minify-css': 'minifyCSS',
+    '--conditional-comments': 'processConditionalComments',
+    '--no-conditional-comments': 'processConditionalComments',
+    '--decode-entities': 'decodeEntities',
+    '--no-decode-entities': 'decodeEntities',
+    '--remove-attribute-quotes': 'removeAttributeQuotes',
+    '--no-remove-attribute-quotes': 'removeAttributeQuotes',
+    '--remove-empty-attributes': 'removeEmptyAttributes',
+    '--no-remove-empty-attributes': 'removeEmptyAttributes',
+    '--remove-redundant-attributes': 'removeRedundantAttributes',
+    '--no-remove-redundant-attributes': 'removeRedundantAttributes',
+    '--use-short-doctype': 'useShortDoctype',
+    '--no-use-short-doctype': 'useShortDoctype'
+};
+const getRequiredValue = (args, index, option) => {
+    const value = args[index + 1];
+    if (!value || value.startsWith('-'))
+        throw new CliUsageError(`${option} requires a value.`);
+    return value;
+};
+const parsePort = (value) => {
+    if (!/^\d+$/.test(value))
+        throw new CliUsageError('--port must be an integer from 1 to 65535.');
+    const port = Number.parseInt(value, 10);
+    if (port < 1 || port > 65535)
+        throw new CliUsageError('--port must be an integer from 1 to 65535.');
+    return port;
+};
+const parseGzipLevel = (value) => {
+    if (!/^\d+$/.test(value))
+        throw new CliUsageError('--gzip-level must be an integer from 1 to 9.');
+    const gzipLevel = Number.parseInt(value, 10);
+    if (!isGzipLevel(gzipLevel))
+        throw new CliUsageError('--gzip-level must be an integer from 1 to 9.');
+    return gzipLevel;
+};
+export const parseCliArguments = (args) => {
+    const parsed = {
+        showHelp: false,
+        showVersion: false,
+        showConfigPath: false,
+        headless: false,
+        minifyOpts: { ...DEFAULT_HTML_MINIFY_OPTIONS },
+        optimizeSvg: true,
+        svgoMultipass: false
+    };
+    for (let index = 0; index < args.length; index++) {
+        const argument = args[index];
+        if (!argument)
+            continue;
+        const minifyOption = MINIFY_FLAGS[argument];
+        if (minifyOption) {
+            parsed.minifyOpts[minifyOption] = !argument.startsWith('--no-');
+            continue;
+        }
+        switch (argument) {
+            case '-h':
+            case '--help':
+                parsed.showHelp = true;
+                break;
+            case '-V':
+            case '--version':
+                parsed.showVersion = true;
+                break;
+            case '--config-path':
+                parsed.showConfigPath = true;
+                break;
+            case '--headless':
+                parsed.headless = true;
+                break;
+            case '-s':
+            case '--src':
+            case '--input':
+                parsed.inputPath = getRequiredValue(args, index, argument);
+                index++;
+                break;
+            case '-d':
+            case '--dst':
+            case '--output':
+                parsed.outputPath = getRequiredValue(args, index, argument);
+                index++;
+                break;
+            case '-p':
+            case '--port':
+                parsed.port = parsePort(getRequiredValue(args, index, argument));
+                index++;
+                break;
+            case '--optimize-svg':
+                parsed.optimizeSvg = true;
+                break;
+            case '--no-optimize-svg':
+                parsed.optimizeSvg = false;
+                break;
+            case '--svgo-multipass':
+                parsed.svgoMultipass = true;
+                break;
+            case '--gzip':
+                parsed.gzip = true;
+                break;
+            case '--no-gzip':
+                parsed.gzip = false;
+                break;
+            case '--gzip-level': {
+                const value = args[index + 1];
+                if (!value || value.startsWith('--'))
+                    throw new CliUsageError('--gzip-level requires a value.');
+                parsed.gzipLevel = parseGzipLevel(value);
+                index++;
+                break;
+            }
+            default:
+                throw new CliUsageError(`Unknown option: ${argument}`);
+        }
+    }
+    if (parsed.headless && parsed.port !== undefined) {
+        throw new CliUsageError('--port is only available when starting the GUI.');
+    }
+    if (parsed.headless && !parsed.inputPath && !parsed.outputPath) {
+        throw new CliUsageError('Headless mode requires --src or --dst. It will not automatically rerun the last build.');
+    }
+    if (!parsed.headless && (parsed.inputPath || parsed.outputPath)) {
+        throw new CliUsageError('--src and --dst require --headless.');
+    }
+    return parsed;
+};
+export const getHelpText = () => {
+    return [
+        'Usage: cc4es [options]',
+        '',
+        'Without options, starts the browser GUI.',
+        '',
+        '  -h, --help                         Show this help and exit',
+        '  -V, --version                      Show version and exit',
+        '  --config-path                      Print the full config file path and exit',
+        '  --headless                         Build without server or browser',
+        '  -s, --src <directory>              Source directory (headless)',
+        '  -d, --dst <file|directory>         Output C file or directory (default: fsdata.c)',
+        '  -p, --port <1-65535>               GUI server port',
+        '  --[no-]optimize-svg                Enable or disable SVGO (default: enabled)',
+        '  --svgo-multipass                   Run SVGO optimization repeatedly',
+        '  --[no-]gzip                        Enable or disable gzip resource storage (default: disabled)',
+        `  --gzip-level <1-9>                 gzip compression level (default: ${DEFAULT_GZIP_LEVEL})`,
+        '',
+        'HTML compression options:',
+        '  --[no-]collapse-whitespace         --[no-]remove-comments',
+        '  --[no-]minify-js                   --[no-]minify-css',
+        '  --[no-]conditional-comments        --[no-]decode-entities',
+        '  --[no-]remove-attribute-quotes     --[no-]remove-empty-attributes',
+        '  --[no-]remove-redundant-attributes --[no-]use-short-doctype',
+        '',
+        'Headless mode requires --src or --dst and never reruns the last build with',
+        'no path options. A missing counterpart uses the last saved path; if no',
+        'destination is saved, fsdata.c is created in the current working directory.'
+    ].join('\n');
+};

package/dist/config.js

@@ -0,0 +1,90 @@
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { isGzipLevel } from './gzip-options.js';
+export const CONFIG_FILE_NAME = 'cc4es_configs.json';
+const getStateDirectory = () => {
+    const homeDirectory = os.homedir();
+    if (process.platform === 'win32') {
+        return process.env.LOCALAPPDATA ?? path.join(homeDirectory, 'AppData', 'Local');
+    }
+    if (process.platform === 'darwin') {
+        return path.join(homeDirectory, 'Library', 'Application Support');
+    }
+    return process.env.XDG_STATE_HOME ?? path.join(homeDirectory, '.local', 'state');
+};
+export const getConfigFilePath = () => {
+    return path.join(getStateDirectory(), 'cc4-embedded-system', CONFIG_FILE_NAME);
+};
+const isRecord = (value) => {
+    return typeof value === 'object' && value !== null;
+};
+const getGzipOptions = (value) => {
+    if (!isRecord(value))
+        return undefined;
+    const gzip = value.gzip;
+    const gzipLevel = value.gzipLevel;
+    if (typeof gzip !== 'boolean' || !isGzipLevel(gzipLevel))
+        return undefined;
+    return { gzip, gzipLevel };
+};
+const readConfig = () => {
+    try {
+        const parsedConfig = JSON.parse(fs.readFileSync(getConfigFilePath(), 'utf8'));
+        if (!isRecord(parsedConfig))
+            return { schemaVersion: 2 };
+        const config = { schemaVersion: 2 };
+        const gzipOptions = getGzipOptions(parsedConfig.gzipOptions);
+        if (gzipOptions)
+            config.gzipOptions = gzipOptions;
+        if (!isRecord(parsedConfig.lastBuild))
+            return config;
+        const src = parsedConfig.lastBuild.src;
+        const dst = parsedConfig.lastBuild.dst;
+        if (typeof src !== 'string' || typeof dst !== 'string' || !src || !dst) {
+            return config;
+        }
+        config.lastBuild = { src, dst };
+        return config;
+    }
+    catch {
+        return { schemaVersion: 2 };
+    }
+};
+export const getLastBuildPaths = () => {
+    return readConfig().lastBuild;
+};
+export const getLastGzipOptions = () => {
+    return readConfig().gzipOptions;
+};
+export const saveLastBuildPaths = (src, dst, gzipOptions) => {
+    const configFilePath = getConfigFilePath();
+    const temporaryFilePath = `${configFilePath}.${process.pid}.tmp`;
+    const configAlreadyExists = fs.existsSync(configFilePath);
+    const config = {
+        schemaVersion: 2,
+        lastBuild: { src, dst }
+    };
+    if (gzipOptions)
+        config.gzipOptions = gzipOptions;
+    try {
+        fs.mkdirSync(path.dirname(configFilePath), { recursive: true });
+        fs.writeFileSync(temporaryFilePath, `${JSON.stringify(config, null, 2)}\n`, 'utf8');
+        fs.renameSync(temporaryFilePath, configFilePath);
+        if (!configAlreadyExists)
+            console.log(`📝 Created config: ${configFilePath}`);
+        return true;
+    }
+    catch (error) {
+        const message = error instanceof Error ? error.message : String(error);
+        console.warn(`⚠️ Could not save ${CONFIG_FILE_NAME}: ${message}`);
+        try {
+            if (fs.existsSync(temporaryFilePath))
+                fs.unlinkSync(temporaryFilePath);
+        }
+        catch {
+            // Ignore cleanup errors because the build itself succeeded.
+        }
+        return false;
+    }
+};