tsc-ts 0.1.1 → 0.1.3

package/README.md

@@ -33,7 +33,7 @@ npx tsc-ts --noEmit src/index.ts
 
 `tsc-ts --api` serves the TypeScript API over stdin and stdout, like `tsc --api`. On Windows, `--api --pipe \\.\pipe\<name>` serves it over a named pipe it creates, as `tsc --api --pipe` does, for clients that connect to the pipe (such as scriptc under Node.js); only the native executable supports `--pipe`. On macOS and Linux, `--api --listen <path>` serves it on a Unix socket it creates, with mode 0600, to one client of the same user at a time, each until the client closes its end. Each client gets the answers a new `tsc-ts --api` process would give, except that symbol ids continue from earlier clients, and the server reads every file through the client again, but it keeps parsed files between clients and reuses those whose text is unchanged, along with the encoding `getSourceFile` sends for each, so a client that opens the same project again loads it faster. The server exits when no client connects for 60 seconds (`--idle-exit <seconds>` sets the time), or after a client when its resident memory is above 2.5 times what it held after its first client (`TSC_TS_LISTEN_MEMORY` sets the limit in megabytes). To stop, it removes the socket file, serves the clients that had connected, and exits. It replaces a socket file that no server answers on, such as one left by a server that was killed. Only the native executable supports `--listen`. With `--default-libraries-from-disk`, the server reads the files in its default library directory (`TSC_TS_LIB_DIR`, or the executable's directory) from the disk itself, even when the client handles file system calls (`--callbacks`), which saves a round trip to the client for each library file.
 
-`--checkers N` checks the program on N threads. Unless `--checkers` or `--singleThreaded` is given, the native executable uses one checker per CPU it may run on (up to 16), and at least tsgo's four; it uses fewer than the CPUs, but never fewer than four, when the estimated peak memory of more checkers would not fit 85% of the machine's memory (`TSC_TS_CHECKERS=fixed` keeps tsgo's four), and checkers beyond the fourth stop taking files, one at a time, when the process's memory reaches that share; it shares one program between its threads and also parses and binds the program's files on them. Unless `--singleThreaded` is given, the program's files are parsed and bound on threads also with `--checkers 1` and in the first build of `--watch`, with or without build info, as tsgo does. Files that share costly declarations, such as a large generated schema and the few files importing it, are checked together on one checker. The checker threads share the results they compute where the run expects that to pay: at 12 or more checkers, when 40% of the program's files are declarations or come from `node_modules` and the checkers' ranges mostly resolve the same large libraries. `TSC_TS_SHARE=1` turns sharing on for any program and `TSC_TS_SHARE=0` keeps it off, and a run whose sharing stops paying stops sharing its results as it goes. A sharing run may also bound how many results it posts at a time, which costs a few percent on a project that does not gain by it and saves more on one that does: the learned choice among a project's runs, off until its record shows it faster, and `TSC_TS_SHARE_BOUNDED=1` turns it on for any program while `TSC_TS_SHARE_BOUNDED=0` keeps it off. `TSC_TS_AUTO=0` keeps the choices of a run before automatic selection (fewer checkers where the ranges resolve large libraries, no sharing), and `TSC_TS_AUTO=static` keeps the rules and the in-run switches but reads and writes no records, and `TSC_TS_AUTO=explore` adds the experimental exploration described below. Unless `--singleThreaded` is given, the files are parsed on the CPUs the process may run on, but one, whatever the checker count, and `TSC_TS_PARSE_THREADS` sets how many. `--extendedDiagnostics` prints the checker threads, the sharing and the parse threads after the statistics table, under `Suggestions:`, with where each value came from (`pinned: --checkers`, `default`, `automatic: ...`, `learned: check 14% faster over 3 runs`, `exploring`, `control`) and how to pin it; `--build` prints one section per project it builds. The section also names the run's record under `Records:`. Each run that checks a program writes what it measured to `~/.cache/tsc-ts/runs-v1/` (`TSC_TS_CACHE_DIR` sets the directory, `TSC_TS_CACHE=0` turns records off), for later runs of the same project on the same machine to read. A later run chooses from that record where the rules left a setting open: a configuration the samples show faster than the static choice — by 3% or twice the standard error of the gains each of its samples makes against the nearest run of the other arm, on three samples of each taken while the machine was not busy — is adopted, every sixteenth run (and the first after two weeks) re-measures the static choice as a control run, and an adopted configuration whose last three samples all fall behind the control runs reverts at once. A run never tries a configuration that could make itself slower: it adopts only what the samples already show faster, from `tsc-ts doctor`, from a pinned run or from a run that already used it. `TSC_TS_AUTO=explore` turns on an experimental alternative in which a run also tries a setting the signals leave uncertain: one run in eight tries three quarters of the count where the library text puts it near the cut, sharing at eight checkers near the rule's limits, or half the parse threads, and such a run is slower than the configuration it would have used. A program whose shape changed by more than 15% discards the record's samples. Watch mode, a `--build` of several projects at once, `--singleThreaded`, Node builds and `TSC_TS_AUTO=0` or `static` keep no record, and the section says why; a run with `CI` set reads and adopts but never explores. `tsc-ts doctor [-p <tsconfig>] [--runs N] [--budget SECONDS] [--emit] [--json] [--no-save]` measures these settings on this machine: it runs the project's check under the default configuration, with sharing on, with up to 16 checkers, with both, and with half the parse threads, compares the phase times by the same rule automatic selection uses, and prints what it recommends with the evidence for it. Each measured run is a child process, so it runs as a user's run would, and it writes what it measured to the file named by `TSC_TS_DOCTOR_REPORT`; runs without the variable are unaffected. Doctor stores its samples in the record with role `doctor`, unless `--no-save` is given. The command is experimental: the configurations it compares and what it recommends may change. Under Node.js, where every thread builds its own copy of the program, tsc-ts checks on one thread unless `--checkers` is given. `--incremental`, `--watch` and `--build` always check on one thread. The diagnostics are the same as with one checker.
+`--checkers N` checks the program on N threads. Unless `--checkers` or `--singleThreaded` is given, the native executable uses one checker per CPU it may run on (up to 16), and at least tsgo's four; it uses fewer than the CPUs, but never fewer than four, when the estimated peak memory of more checkers would not fit 85% of the machine's memory (`TSC_TS_CHECKERS=fixed` keeps tsgo's four), and checkers beyond the fourth stop taking files, one at a time, when the process's memory reaches that share; it shares one program between its threads and also parses and binds the program's files on them. Unless `--singleThreaded` is given, the program's files are parsed and bound on threads also with `--checkers 1` and in the first build of `--watch`, with or without build info, as tsgo does. Files that share costly declarations, such as a large generated schema and the few files importing it, are checked together on one checker. The checker threads share the results they compute where the run expects that to pay: at 12 or more checkers, when 40% of the program's files are declarations or come from `node_modules` and the checkers' ranges mostly resolve the same large libraries. `TSC_TS_SHARE=1` turns sharing on for any program and `TSC_TS_SHARE=0` keeps it off, and a run whose sharing stops paying stops sharing its results as it goes. A sharing run may also bound how many results it posts at a time, which costs a few percent on a project that does not gain by it and saves more on one that does: the learned choice among a project's runs, off until its record shows it faster, and `TSC_TS_SHARE_BOUNDED=1` turns it on for any program while `TSC_TS_SHARE_BOUNDED=0` keeps it off. `TSC_TS_AUTO=0` keeps the choices of a run before automatic selection (fewer checkers where the ranges resolve large libraries, no sharing), and `TSC_TS_AUTO=static` keeps the rules and the in-run switches but reads and writes no records, and `TSC_TS_AUTO=explore` adds the experimental exploration described below. Unless `--singleThreaded` is given, the files are parsed on the CPUs the process may run on, but one, whatever the checker count, and `TSC_TS_PARSE_THREADS` sets how many. `--extendedDiagnostics` prints the checker threads, the sharing and the parse threads after the statistics table, under `Suggestions:`, with where each value came from (`pinned: --checkers`, `default`, `automatic: ...`, `learned: check 14% faster over 3 runs`, `exploring`, `control`) and how to pin it; `--build` prints one section per project it builds. The section also names the run's record under `Records:`. Each run that checks a program writes what it measured to `~/.cache/tsc-ts/runs-v1/` (`TSC_TS_CACHE_DIR` sets the directory, `TSC_TS_CACHE=0` turns records off), for later runs of the same project on the same machine to read. A later run chooses from that record where the rules left a setting open: a configuration the samples show faster than the static choice — by 3% or twice the standard error of the gains each of its samples makes against the nearest run of the other arm, on three samples of each taken while the machine was not busy — is adopted, every sixteenth run (and the first after two weeks) re-measures the static choice as a control run, and an adopted configuration whose last three samples all fall behind the control runs reverts at once. A run never tries a configuration that could make itself slower: it adopts only what the samples already show faster, from `tsc-ts doctor`, from a pinned run or from a run that already used it. `TSC_TS_AUTO=explore` turns on an experimental alternative in which a run also tries a setting the signals leave uncertain: one run in eight tries three quarters of the count where the library text puts it near the cut, sharing at eight checkers near the rule's limits, or half the parse threads, and such a run is slower than the configuration it would have used. A program whose shape changed by more than 15% discards the record's samples. Watch mode, a `--build` of several projects at once, `--singleThreaded`, Node builds and `TSC_TS_AUTO=0` or `static` keep no record, and the section says why; a run with `CI` set reads and adopts but never explores. `tsc-ts doctor [-p <tsconfig>] [--runs N] [--budget SECONDS] [--emit] [--json] [--no-save]` measures these settings on this machine: it runs the project's check under the default configuration, with sharing on, with up to 16 checkers, with both, and with half the parse threads, compares the phase times by the same rule automatic selection uses, and prints what it recommends with the evidence for it. Each measured run is a child process, so it runs as a user's run would, and it writes what it measured to the file named by `TSC_TS_DOCTOR_REPORT`; runs without the variable are unaffected. Doctor stores its samples in the record with role `doctor`, unless `--no-save` is given. The command is experimental: the configurations it compares and what it recommends may change. Under Node.js, where every thread builds its own copy of the program, tsc-ts checks on one thread unless `--checkers` is given. `--incremental`, `--watch` and `--build` always check on one thread. The diagnostics are the same as with one checker, except for the two a check reports when a type instantiation reaches the depth limit: TS5115 on the instantiations, and TS2344 on the constraint check that stops there. Whether a file reaches the limit depends on the declarations its checker has already resolved, and which files a checker has is decided while checking runs, so those two diagnostics are best effort with more than one checker: one run can report them in different files than another run, and a run with one checker can report them where a run with several does not.
 
 The native executable never frees syntax trees, types, signatures or symbols: it keeps them until it exits, which saves the cost of tracking them during a build. A one-shot build is unaffected, but a long-running process grows with each rebuild or project update. Native `--watch` (also with `--build`) runs in a child process, and when a change arrives while that process holds more than 2.5 times the memory it held after its first build, a new process replaces it and builds the change, reporting what the old process would have. `TSC_TS_WATCH_MEMORY` sets that limit in megabytes, and `TSC_TS_WATCH_MEMORY=0` turns the replacement off. A `--api` process grows with each project update; restart it to reclaim the memory.
 
@@ -84,3 +84,9 @@ The program's diagnostics, with their text and positions, are those `tsc --api`
 An exception the compiler throws while answering a call reaches the caller as an `InternalCompilerError`, with the exception as its `cause`; the program's checker may be inconsistent after one, so dispose of the program. `program.dispose()` frees the program's checkers; calls on the program or its checker after it throw a `DisposedProgramError`.
 
 The package has a JavaScript build with declaration files for Node.js, and the TypeScript sources of the API and the compiler (the `source` export condition) for compilers that build it from source. The sources compile with scriptc: a test builds a program on `tsc-ts/api` natively and checks that it answers as the Node build.
+
+The API imports `@scriptc/threads`, an optional peer dependency that the command line doesn't use. Install it next to `tsc-ts` to use `tsc-ts/api`:
+
+```bash
+npm install -D tsc-ts @scriptc/threads
+```

package/dist/api/compiler/ast.d.ts

@@ -2327,6 +2327,12 @@ export declare class SourceFile extends Node {
     patternAmbientModules: readonly PatternAmbientModule[];
     /** UMD exports from `export as namespace N`. */
     globalExports: SymbolTable | undefined;
+    /** The diagnostics the parse thread's grammar walk reported for this file. */
+    grammarDiagnostics: Diagnostic[] | undefined;
+    /** The nodes of this file whose grammar check needs the checker, in tree order. */
+    grammarResidualNodes: Node[] | undefined;
+    /** The blocks and statements that reported a statement in an ambient context. */
+    grammarAmbientReports: readonly number[];
     constructor(fileName: string, path: string, text: string, externalModuleIndicatorOptions: ExternalModuleIndicatorOptions, statements: NodeList, endOfFileToken: Token);
     forEachChild(v: NodeVisitor): boolean;
     setParentOfChildren(): void;

package/dist/api/compiler/ast.js

@@ -52,9 +52,14 @@ export class NodeVisitor {
         return this.visitList(list);
     }
 }
-// Native builds store `pos`, `end` and `id` as 32-bit integers when every write provably stores an
-// int32, so writes of values the compiler can't bound (parameters, the id counter) are coerced
-// with `| 0`; the values are always int32.
+// Native builds store `pos`, `end` and `id` as 32-bit integers when every write provably stores
+// an int32, so writes of values the compiler can't bound (parameters, the id counter) are
+// coerced with `| 0`; the values are always int32.
+//
+// The fields of a node that the public API's node types declare are stored fields, not values
+// computed from or packed into another field: a native build copies the stored fields of a node
+// into a record when API code narrows one (`node as StringLiteralLike`), and a field behind an
+// accessor is not copied.
 export class Node {
     kind;
     flags;
@@ -6626,6 +6631,7 @@ export const emptyCommentDirectives = [];
 export const emptyPragmas = [];
 export const emptyFileReferences = [];
 export const emptyPatternAmbientModules = [];
+const emptyNumbers = [];
 /** `list` with `item` appended: a new array when `list` is empty (it may be shared), else `list` itself. */
 export function appendToList(list, item) {
     if (list.length === 0)
@@ -6698,6 +6704,13 @@ export class SourceFile extends Node {
     patternAmbientModules;
     /** UMD exports from `export as namespace N`. */
     globalExports;
+    // Set by the grammar checks of a parse thread (checker-grammar.ts).
+    /** The diagnostics the parse thread's grammar walk reported for this file. */
+    grammarDiagnostics;
+    /** The nodes of this file whose grammar check needs the checker, in tree order. */
+    grammarResidualNodes;
+    /** The blocks and statements that reported a statement in an ambient context. */
+    grammarAmbientReports;
     constructor(fileName, path, text, externalModuleIndicatorOptions, statements, endOfFileToken) {
         super(308 /* SyntaxKind.SourceFile */);
         this.fileName = fileName;
@@ -6744,6 +6757,9 @@ export class SourceFile extends Node {
         this.symbolCount = 0;
         this.patternAmbientModules = emptyPatternAmbientModules;
         this.globalExports = undefined;
+        this.grammarDiagnostics = undefined;
+        this.grammarResidualNodes = undefined;
+        this.grammarAmbientReports = emptyNumbers;
     }
     forEachChild(v) {
         return v.visitList(this.statements) || v.visitNode(this.endOfFileToken);

package/dist/api/compiler/checker-functions.js

@@ -33,7 +33,7 @@ import { couldContainTypeVariables, instantiateSignature } from './instantiation
 import { getContextualSignatureForFunctionLikeDeclaration, getInferenceContext, instantiateContextualType, } from './checker-contextual.js';
 import { checkJSDocFullSignature, checkJSDocFullSignatureType, checkUnmatchedJSDocParameters, getFullSignatureTypeNode, } from './checker-jsdoc.js';
 import { getEffectiveCheckNode, isFunctionLike, nodeIsMissing } from './ast-utilities.js';
-import { getFlowTypeOfReferenceAt, isConstantReference, isReachableFlowNode, isSymbolAssigned, } from './checker-narrowing.js';
+import { endFlowAnalysisStop, getFlowTypeOfReferenceAt, isConstantReference, isReachableFlowNode, isSymbolAssigned, } from './checker-narrowing.js';
 import { FlowNode } from './flow.js';
 import { checkAwaitedType, checkGeneratorInstantiationAssignabilityToReturnType, checkIteratedTypeOrElementType, createGeneratorType, createPromiseReturnType, createPromiseType, getAwaitedTypeEx, getAwaitedTypeNoAlias, getGlobalPromiseTypeChecked, getIterationTypeOfGeneratorFunctionReturnType, getIterationTypeOfIterable, getIterationTypesOfGeneratorFunctionReturnType, getIterationTypesOfIterable, getPromisedTypeOfPromise, isReferenceToType, unwrapAwaitedType, unwrapReturnType, } from './checker-iteration.js';
 import { typeToString } from './type-printer.js';
@@ -274,6 +274,10 @@ export function checkFunctionLikeBody(c, node) {
         if (returnType !== undefined)
             checkReturnExpression(c, node, unwrapReturnType(c, returnType, functionFlags), body, body, exprType, false);
     }
+    // A body too large for control flow analysis stops narrowing in the rest of that body only
+    // (flowAnalysisDisabled, checker-narrowing.ts); what follows the body starts over. Inferring the
+    // return type above checks the body's return expressions, which can reach the limit.
+    endFlowAnalysisStop(c, body);
 }
 /** Checks a generator's or async function's return type annotation. */
 function checkSignatureReturnType(c, node, returnTypeNode, functionFlags) {

package/dist/api/compiler/checker-grammar.d.ts

@@ -1,10 +1,17 @@
 import { type Node, type NodeList, NodeVisitor, type ObjectLiteralExpression, type SourceFile } from './ast.js';
 import { type Checker } from './checker.js';
-import { ModuleKind } from './options.js';
+import { Diagnostic, type DiagnosticMessage } from './diagnostics.js';
+import { type CompilerOptions, ModuleKind } from './options.js';
 import { SourceFileMetadata } from './program-utilities.js';
 import { Scanner } from './scanner.js';
 /** Runs the grammar checks of every node of `file`. */
 export declare function checkGrammarSourceFile(c: Checker, file: SourceFile): void;
+/**
+ * Runs the grammar checks of `file` in a parse thread, before the file is published: the checks that
+ * don't need the checker report now, and the nodes of those that do are kept for the checker, which
+ * runs them in the same tree order (`checkGrammarSourceFile`).
+ */
+export declare function checkGrammarSourceFileInParseThread(file: SourceFile, options: CompilerOptions, metadata: SourceFileMetadata): void;
 /** Checks the grammar of an object literal each time it is checked, as tsgo does. */
 export declare function checkGrammarObjectLiteralOfExpression(c: Checker, node: ObjectLiteralExpression, inDestructuring: boolean): void;
 /** Reports a trailing comma after the rest element of a destructuring assignment. */
@@ -17,7 +24,10 @@ export declare function checkGrammarAwaitExpression(c: Checker, node: Node): voi
 export declare function checkGrammarYieldOfExpression(c: Checker, node: Node): void;
 /** Per-file state of the grammar walk. */
 export declare class GrammarChecker extends NodeVisitor {
-    readonly c: Checker;
+    /** The checker of a walk in the checker thread; undefined for a walk in a parse thread. */
+    readonly c: Checker | undefined;
+    readonly compilerOptions: CompilerOptions;
+    readonly languageVersion: number;
     readonly file: SourceFile;
     /** Grammar errors are only reported in files without parse errors. */
     readonly hasParseErrors: boolean;
@@ -25,10 +35,35 @@ export declare class GrammarChecker extends NodeVisitor {
     readonly metadata: SourceFileMetadata;
     readonly legacyDecorators: boolean;
     /** Blocks and statements that already reported a statement in an ambient context. */
-    readonly reportedAmbientStatements: Set<number>;
+    reportedAmbientStatements: Set<number>;
+    /**
+     * The IDs `reportedAmbientStatements` gained in a parse thread's walk, in the order it gained them, so
+     * that the reports of a node whose check the checker runs are undone without copying the set.
+     * Undefined in the checker thread, where no check is undone.
+     */
+    private readonly ambientReportLog;
+    /**
+     * The diagnostics of a walk in a parse thread, which has no checker, for the checker to add to its
+     * collection; undefined in the checker thread, where they are reported as they are found.
+     */
+    readonly diagnostics: Diagnostic[] | undefined;
+    /** The nodes of a parse thread's walk whose check needs the checker, in tree order. */
+    readonly residualNodes: Node[] | undefined;
+    /** Set when the check of a parse thread's walk needs the checker: the checker runs that node. */
+    needsChecker: boolean;
     private scanner;
-    constructor(c: Checker, file: SourceFile);
+    constructor(c: Checker | undefined, file: SourceFile, options?: CompilerOptions, metadata?: SourceFileMetadata, 
+    /** `file.grammarAmbientReports`: what a parse thread's walk already reported. */
+    ambientReports?: readonly number[]);
     visitNode(node: Node): boolean;
+    /** Records that `id` reported a statement in an ambient context. */
+    reportAmbientStatement(id: number): void;
+    /** Undoes the ambient-context reports added since the log had `mark` entries. */
+    private undoAmbientReports;
+    /** Reports an error at `node`, as `Checker.error`; returns the diagnostic. */
+    error(node: Node | undefined, message: DiagnosticMessage, ...args: string[]): Diagnostic;
+    /** Reports `diagnostic`, as `Checker.addDiagnostic`. */
+    report(diagnostic: Diagnostic): void;
     /** A scanner over the file's text, for rescanning tokens. */
     getScanner(): Scanner;
 }