zorya-classifier 0.2.2 → 0.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +30 -16
- package/dist/app/classifier-contracts.d.ts +8 -8
- package/dist/app/composed-classifier.d.ts +17 -0
- package/dist/app/composed-classifier.js +1 -0
- package/dist/app/create-classifier.d.ts +7 -1
- package/dist/app/create-classifier.js +24 -16
- package/dist/cli.js +1 -1
- package/dist/entities/classification.d.ts +2 -17
- package/dist/entities/image.d.ts +2 -11
- package/dist/entities/image.js +1 -4
- package/dist/entities/localization.d.ts +6 -21
- package/dist/entities/object-proposals.d.ts +1 -1
- package/dist/entities/representation.d.ts +0 -183
- package/dist/entities/representation.js +1 -33
- package/dist/entities/segmentation-verification.d.ts +1 -1
- package/dist/entities/segmentation.d.ts +5 -20
- package/dist/entities/whole-object.d.ts +6 -21
- package/dist/infrastructure/segmentation/grabcut/index.d.ts +2 -2
- package/dist/infrastructure/segmentation/grabcut/index.js +1 -1
- package/dist/infrastructure/segmentation/mobile-sam/object-proposals.d.ts +1 -1
- package/dist/infrastructure/segmentation/mobile-sam/object-proposals.js +1 -1
- package/dist/infrastructure/segmentation/mobile-sam/options.d.ts +2 -2
- package/dist/infrastructure/segmentation/mobile-sam/segmenter.d.ts +1 -1
- package/dist/infrastructure/segmentation/mobile-sam/segmenter.js +1 -1
- package/docs/README.ru.md +37 -23
- package/docs/acceptance.md +6 -4
- package/docs/agents.ru.md +14 -12
- package/docs/api.md +4 -4
- package/docs/architecture.md +2 -2
- package/docs/architecture.ru.md +10 -10
- package/docs/migrations/experimental-text-image-removal.ru.md +9 -0
- package/docs/provenance.json +283 -0
- package/docs/v02-usage.md +8 -8
- package/docs/vision-results.md +1 -1
- package/package.json +2 -2
- package/dist/app/classifier-representation.d.ts +0 -5
- package/dist/app/classifier-representation.js +0 -33
package/README.md
CHANGED
|
@@ -2,7 +2,21 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/zorya-classifier)
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
> **More complete object boxes without fine-tuning in this DEV pilot: H8a 7/9 vs native Jev 1.14 3/9**
|
|
6
|
+
>
|
|
7
|
+
> One exposed pilot compares a historical H8a run with one new Jev run: 12 tasks across 9 unique source images (the four UI targets share one screenshot), 9 tasks with the target and 3 without. **H8a** uses existing mask candidates with an explicitly configured OpenAI **gpt-6-luna** vision route; no fine-tuning was done for this comparison. **Jev 1.14** uses native Box+Noul and returns rectangles, not segmentation masks. The strict box criterion requires IoU ≥0.5 **and** at least 90% coverage of the same reference box. This is a development pilot, not a heldout qualification.
|
|
8
|
+
>
|
|
9
|
+
> | Measure | H8a | Jev 1.14 |
|
|
10
|
+
> | ------------------------------------------------- | ---------------- | ------------- |
|
|
11
|
+
> | Strict box criterion | 7/9 | 3/9 |
|
|
12
|
+
> | IoU ≥0.5 only (secondary, not the gate) | 7/9 | 8/9 |
|
|
13
|
+
> | False outputs on 3 absent cases (lower is better) | 2/3 (Bowl, Undo) | 1/3 (Fighter) |
|
|
14
|
+
>
|
|
15
|
+
> [Full report (RU)](https://github.com/zoomplus/zorya-classifier/blob/v0.2.3/docs/research/v03-jev-native-box-results.ru.md) · [Evidence JSON](https://github.com/zoomplus/zorya-classifier/blob/v0.2.3/docs/evidence/v03/jev-native-box-dev12-20261025.json) · [Download visual comparison ZIP](https://github.com/zoomplus/zorya-classifier/releases/download/v0.2.3/jev-native-box-dev12-comparison.zip)
|
|
16
|
+
>
|
|
17
|
+
> H8a is a research/benchmark recipe only: it is not the default `locate`/`extract` path and is not an H8 strategy shipped in npm. This small pilot makes no general accuracy claim.
|
|
18
|
+
|
|
19
|
+
Provider-independent text/image classification for Node.js and TypeScript, with available spatial localization and object extraction APIs. MIT application code. The library never selects or connects a model for you: create a client with an explicit `DecisionPort`, either injected by your application or built with an adapter you configure. The model and provider shown in examples are examples, not defaults. Ordinary vision language models and native decision APIs are both supported for closed classification.
|
|
6
20
|
|
|
7
21
|
Use an explicitly configured image-capable model through a supported adapter, or implement the small `DecisionPort` for another protocol. OpenRouter is an optional adapter, not a mandatory service. A more capable model may help on a task, but model strength does not guarantee a monotonic accuracy improvement. Compatibility and quality vary; see the exact [tested models/results](docs/vision-results.md), [acceptance](docs/acceptance.md), and historical [Jev baseline](docs/results.md).
|
|
8
22
|
|
|
@@ -22,21 +36,21 @@ Node.js 22.12+ is required. Source: [zoomplus/zorya-classifier](https://github.c
|
|
|
22
36
|
npm install zorya-classifier
|
|
23
37
|
```
|
|
24
38
|
|
|
25
|
-
The release archive also installs with `npm install ./zorya-classifier-0.2.
|
|
39
|
+
The release archive also installs with `npm install ./zorya-classifier-0.2.3.tgz` (filename follows `package.json` version). Runtime dependencies are Zod and Sharp; model weights, Electron, a database and background services are not included. Optional local segmentation backends and their model assets are a separate responsibility of the caller; the library does not download or choose them.
|
|
26
40
|
|
|
27
41
|
## Capability and status
|
|
28
42
|
|
|
29
|
-
|
|
43
|
+
Spatial APIs are implemented and documented; the [H8a vs Jev DEV12 box comparison](https://github.com/zoomplus/zorya-classifier/blob/v0.2.3/docs/research/v03-jev-native-box-results.ru.md) is one research recipe, not a promoted default. Historical metrics below and in `docs/evidence/**` are preserved as recorded.
|
|
30
44
|
|
|
31
|
-
| Capability | Status
|
|
32
|
-
| --------------------------------------------------------------- |
|
|
33
|
-
| `classify` with an explicit `DecisionPort` / supported adapters | Supported
|
|
34
|
-
| Default `locate` (region grid search) |
|
|
35
|
-
| `RefinementPort` |
|
|
36
|
-
| `locate({ mode: 'whole-object' })` |
|
|
37
|
-
| `extract`, `SegmentationPort`, box-check composition |
|
|
38
|
-
| `objectProposals` / native segmentation factories |
|
|
39
|
-
| `representation: { type: 'experimental-text', … }` |
|
|
45
|
+
| Capability | Status | Notes |
|
|
46
|
+
| --------------------------------------------------------------- | --------- | ------------------------------------------------------------------------------------ |
|
|
47
|
+
| `classify` with an explicit `DecisionPort` / supported adapters | Supported | Closed class ID; no universal model accuracy claim |
|
|
48
|
+
| Default `locate` (region grid search) | Available | Hypothesized grid cell, not whole-object geometry or proof of absence |
|
|
49
|
+
| `RefinementPort` | Available | Optional legacy hook; caller-owned |
|
|
50
|
+
| `locate({ mode: 'whole-object' })` | Available | Interval or caller segmentation paths |
|
|
51
|
+
| `extract`, `SegmentationPort`, box-check composition | Available | Structural checks ≠ semantic object or mask quality |
|
|
52
|
+
| `objectProposals` / native segmentation factories | Available | Caller-owned; model/runtime assets remain caller-supplied |
|
|
53
|
+
| `representation: { type: 'experimental-text', … }` | Removed | Public input fails strict validation (`INVALID_INPUT`); encoders stay benchmark-only |
|
|
40
54
|
|
|
41
55
|
Heldout region-seeded metrics (one documented recipe, not universal): 26/60 positive coarse seed hits; 8/60 complete whole-object boxes; 0 false seeds on 20 negatives. Among 30 exact-canvas masks: 8 reached IoU 0.8, 3 failed, 19 had no predicted mask. H8 DEV reported 7/9 positives with 2/3 false outputs and did not qualify. A narrow historical admission pass is not broad spatial qualification.
|
|
42
56
|
|
|
@@ -117,9 +131,9 @@ try {
|
|
|
117
131
|
|
|
118
132
|
Supply the **full trusted chat-completions URL**, such as your local `http://localhost:8000/v1/chat/completions`, and the actual deployed model ID. Anonymous local servers can omit the key; custom headers support other authentication. These are integration examples, not a claim that every provider/model has been live-qualified. Other wire protocols can implement the exported `DecisionPort`. Keep keys on a trusted server, outside frontend bundles and Git.
|
|
119
133
|
|
|
120
|
-
## Locate an object
|
|
134
|
+
## Locate an object
|
|
121
135
|
|
|
122
|
-
Default region `locate` is
|
|
136
|
+
Default region `locate` is available; a returned cell is not whole-object geometry. With either configured client:
|
|
123
137
|
|
|
124
138
|
```ts
|
|
125
139
|
const result = await client.locate({
|
|
@@ -133,9 +147,9 @@ if (result.status === 'located') console.log(result.region)
|
|
|
133
147
|
|
|
134
148
|
The locator uses up to three sequential calls: coarse 4×4 regions, fine grids in four padded regions, then four enlarged candidates. Coordinates come from code, never free-form model coordinates. Ordinary-model rankings select candidates; independent present/absent/unknown answers govern abstention. Multiple matching physical instances are allowed; one candidate is returned. Localization requires the model/endpoint to accept multiple images per call (up to nine); single-image-only endpoints can still support `classify`.
|
|
135
149
|
|
|
136
|
-
## Extract a mask-derived object
|
|
150
|
+
## Extract a mask-derived object
|
|
137
151
|
|
|
138
|
-
Extraction
|
|
152
|
+
Extraction uses caller-owned segmentation; mask quality and box checks measure different things. For a visible object box, center, or transparent cutout, use `extract` with an explicitly configured caller-owned segmenter. The existing region locator supplies a seed; the segmenter receives that seed while processing the full oriented source image. The library derives the returned box and center from actual mask pixels only. After segmentation, the caller's decision port checks the full source with the mask-derived box and original seed anchor: `complete` yields the result (optional cutout); `needs-expansion` allows at most one enclosing-mask expansion and one final check (a final `complete` can still produce the result); unresolved checks, missing expansion support, no valid enclosing mask, or a failed final check yield `unknown` with no second expansion. Crops, resize and padding preserve source/mask alignment; box-check rendering uses injected `RasterPort.imageState` on the full canvas. This adds one or two closed checks to the normal region requests, without model-generated coordinates or an extent search. See the [0.2 usage guide](docs/v02-usage.md) and runnable [`extract-object.ts`](https://github.com/zoomplus/zorya-classifier/blob/main/examples/extract-object.ts). The classifier and segmenter are separately caller-owned and both must be disposed.
|
|
139
153
|
|
|
140
154
|
## Development and limits
|
|
141
155
|
|
|
@@ -9,24 +9,24 @@ export interface ClassifierOptions {
|
|
|
9
9
|
concurrency?: number;
|
|
10
10
|
maxQueue?: number;
|
|
11
11
|
maxInputBytes?: number;
|
|
12
|
-
/**
|
|
12
|
+
/** Optional legacy region refinement. */
|
|
13
13
|
refinement?: RefinementPort;
|
|
14
|
-
/** Caller-owned; classifier disposal never disposes a shared segmenter.
|
|
14
|
+
/** Caller-owned; classifier disposal never disposes a shared segmenter. Segmentation adapter; structural checks do not establish semantic mask quality. */
|
|
15
15
|
segmentation?: SegmentationPort;
|
|
16
|
-
/** Optional caller-owned containment checker; by default reuses the injected decision port.
|
|
16
|
+
/** Optional caller-owned containment checker; by default reuses the injected decision port. Box-check override; verdicts are not universal spatial guarantees. */
|
|
17
17
|
segmentationBoxCheck?: SegmentationBoxCheckPort;
|
|
18
|
-
/** Explicit, caller-owned pixel-mask proposal strategy for whole-object operations.
|
|
18
|
+
/** Explicit, caller-owned pixel-mask proposal strategy for whole-object operations. */
|
|
19
19
|
objectProposals?: ObjectProposalPort;
|
|
20
20
|
}
|
|
21
21
|
export interface Classifier {
|
|
22
22
|
classify(input: ClassifyInput): Promise<ClassificationResult>;
|
|
23
|
-
/**
|
|
23
|
+
/** Spatial localization; see {@link LocateInput} and {@link WholeObjectLocateInput}. */
|
|
24
24
|
locate(input: WholeObjectLocateInput): Promise<WholeObjectLocalizationResult>;
|
|
25
|
-
/**
|
|
25
|
+
/** Coarse region search; returns a hypothesized grid cell, not whole-object geometry. */
|
|
26
26
|
locate(input: LocateInput): Promise<LocalizationResult>;
|
|
27
|
-
/**
|
|
27
|
+
/** Spatial localization. */
|
|
28
28
|
locate(input: AnyLocateInput): Promise<LocalizationResult | WholeObjectLocalizationResult>;
|
|
29
|
-
/**
|
|
29
|
+
/** Region-seeded extraction; mask and box checks measure different things than semantic object accuracy. */
|
|
30
30
|
extract(input: ExtractInput): Promise<ExtractionResult>;
|
|
31
31
|
dispose(): Promise<void>;
|
|
32
32
|
}
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
import type { ClassificationResult, ClassifyInput, LocalizationResult, LocateInput, Representation } from '../entities/contracts.js';
|
|
2
|
+
import type { ExtractInput, ExtractionResult } from '../entities/segmentation.js';
|
|
3
|
+
import type { AnyLocateInput, WholeObjectLocateInput, WholeObjectLocalizationResult } from '../entities/whole-object.js';
|
|
4
|
+
export type WithRepresentationField<T extends {
|
|
5
|
+
representation?: unknown;
|
|
6
|
+
}, R> = Omit<T, 'representation'> & {
|
|
7
|
+
representation?: R | undefined;
|
|
8
|
+
};
|
|
9
|
+
/** Internal composition surface; not exported from the public package entry. */
|
|
10
|
+
export type ComposedClassifier<R = Representation> = {
|
|
11
|
+
classify(input: WithRepresentationField<ClassifyInput, R>): Promise<ClassificationResult>;
|
|
12
|
+
locate(input: WithRepresentationField<WholeObjectLocateInput, R>): Promise<WholeObjectLocalizationResult>;
|
|
13
|
+
locate(input: WithRepresentationField<LocateInput, R>): Promise<LocalizationResult>;
|
|
14
|
+
locate(input: WithRepresentationField<AnyLocateInput, R>): Promise<LocalizationResult | WholeObjectLocalizationResult>;
|
|
15
|
+
extract(input: WithRepresentationField<ExtractInput, R>): Promise<ExtractionResult>;
|
|
16
|
+
dispose(): Promise<void>;
|
|
17
|
+
};
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export {};
|
|
@@ -1,8 +1,14 @@
|
|
|
1
|
+
import type { Representation } from '../entities/contracts.js';
|
|
1
2
|
import type { WholeObjectIntervalPolicy } from '../processes/whole-object.js';
|
|
2
3
|
import type { ImageRepresentationPort } from '../entities/representation.js';
|
|
3
4
|
import type { BoundaryViewPort } from '../entities/boundary-view.js';
|
|
4
5
|
import type { Classifier, ClassifierOptions } from './classifier-contracts.js';
|
|
6
|
+
import type { ComposedClassifier } from './composed-classifier.js';
|
|
5
7
|
export type { Classifier, ClassifierOptions } from './classifier-contracts.js';
|
|
8
|
+
interface ResearchRepresentationHook {
|
|
9
|
+
representationPort: ImageRepresentationPort;
|
|
10
|
+
acceptsRepresentation(raw: unknown): boolean;
|
|
11
|
+
}
|
|
6
12
|
export declare function createClassifier(options: ClassifierOptions): Classifier;
|
|
7
13
|
/** Internal composition seam for benchmark helpers; not re-exported from public entry points. */
|
|
8
|
-
export declare function composeClassifier(options: ClassifierOptions,
|
|
14
|
+
export declare function composeClassifier<R = Representation>(options: ClassifierOptions, research?: ResearchRepresentationHook, boundaryView?: BoundaryViewPort, intervalPolicy?: WholeObjectIntervalPolicy): ComposedClassifier<R>;
|
|
@@ -10,14 +10,23 @@ import { createSharpObjectProposalPresentation } from '../infrastructure/segment
|
|
|
10
10
|
import { createSegmentationBoxCheck } from '../features/segmentation/box-check.js';
|
|
11
11
|
import { createSharpSegmentationBoxPresentation } from '../infrastructure/segmentation/box-presentation.js';
|
|
12
12
|
import { validateClassifierOptions } from './classifier-config.js';
|
|
13
|
-
import { createClassifierRepresentation } from './classifier-representation.js';
|
|
14
13
|
import { createClassifierRuntime, parseOperationSignal } from './classifier-runtime.js';
|
|
15
14
|
import { createClassifierSpatial } from './classifier-spatial.js';
|
|
15
|
+
function prepareResearchOperationInput(rest, research) {
|
|
16
|
+
if (!research || rest.representation === undefined)
|
|
17
|
+
return { rest, port: undefined };
|
|
18
|
+
if (!research.acceptsRepresentation(rest.representation))
|
|
19
|
+
return { rest, port: undefined };
|
|
20
|
+
return {
|
|
21
|
+
rest: { ...rest, representation: { type: 'vision' } },
|
|
22
|
+
port: research.representationPort,
|
|
23
|
+
};
|
|
24
|
+
}
|
|
16
25
|
export function createClassifier(options) {
|
|
17
26
|
return composeClassifier(options);
|
|
18
27
|
}
|
|
19
28
|
/** Internal composition seam for benchmark helpers; not re-exported from public entry points. */
|
|
20
|
-
export function composeClassifier(options,
|
|
29
|
+
export function composeClassifier(options, research, boundaryView, intervalPolicy) {
|
|
21
30
|
const { config, ports } = validateClassifierOptions(options);
|
|
22
31
|
const { segmenter, objectProposals } = ports;
|
|
23
32
|
const runtime = createClassifierRuntime(config);
|
|
@@ -40,7 +49,6 @@ export function composeClassifier(options, experimental, boundaryView, intervalP
|
|
|
40
49
|
presentation: createSharpObjectProposalPresentation(),
|
|
41
50
|
}
|
|
42
51
|
: undefined;
|
|
43
|
-
const { representationFor } = createClassifierRepresentation(experimental);
|
|
44
52
|
const spatial = createClassifierSpatial({
|
|
45
53
|
getDecision: () => options.decision,
|
|
46
54
|
raster,
|
|
@@ -57,14 +65,13 @@ export function composeClassifier(options, experimental, boundaryView, intervalP
|
|
|
57
65
|
throw new ClassifierError('INVALID_INPUT', 'Expected locate input object');
|
|
58
66
|
const { signal, ...rest } = input;
|
|
59
67
|
const parent = parseOperationSignal(signal);
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
const
|
|
63
|
-
return runtime.withOwnedInput(parsed.image.byteLength, parent, () => ({ ...parsed, image: new Uint8Array(parsed.image) }), (snapshot, s) => spatial.locateWhole(snapshot, s,
|
|
68
|
+
const { rest: rewritten, port } = prepareResearchOperationInput(rest, research);
|
|
69
|
+
if (rewritten.mode === 'whole-object') {
|
|
70
|
+
const parsed = parse(wholeObjectSchema, rewritten);
|
|
71
|
+
return runtime.withOwnedInput(parsed.image.byteLength, parent, () => ({ ...parsed, image: new Uint8Array(parsed.image) }), (snapshot, s) => spatial.locateWhole(snapshot, s, port));
|
|
64
72
|
}
|
|
65
|
-
const parsed = parse(locateSchema,
|
|
66
|
-
|
|
67
|
-
return runtime.withOwnedInput(parsed.image.byteLength, parent, () => ({ ...parsed, image: new Uint8Array(parsed.image) }), (snapshot, s) => locateInput(snapshot, options.decision, raster, s, options.refinement, representation));
|
|
73
|
+
const parsed = parse(locateSchema, rewritten);
|
|
74
|
+
return runtime.withOwnedInput(parsed.image.byteLength, parent, () => ({ ...parsed, image: new Uint8Array(parsed.image) }), (snapshot, s) => locateInput(snapshot, options.decision, raster, s, options.refinement, port));
|
|
68
75
|
}
|
|
69
76
|
return {
|
|
70
77
|
async classify(input) {
|
|
@@ -73,9 +80,10 @@ export function composeClassifier(options, experimental, boundaryView, intervalP
|
|
|
73
80
|
throw new ClassifierError('INVALID_INPUT', 'Expected classify input object');
|
|
74
81
|
const { signal, ...rest } = input;
|
|
75
82
|
const parent = parseOperationSignal(signal);
|
|
76
|
-
const
|
|
83
|
+
const { rest: rewritten, port } = prepareResearchOperationInput(rest, research);
|
|
84
|
+
const parsed = parse(classifySchema, rewritten);
|
|
77
85
|
const hasImage = typeof parsed.input !== 'string' && parsed.input.image !== undefined;
|
|
78
|
-
const representation = hasImage ?
|
|
86
|
+
const representation = hasImage ? port : undefined;
|
|
79
87
|
const byteCount = typeof parsed.input === 'string'
|
|
80
88
|
? Buffer.byteLength(parsed.input)
|
|
81
89
|
: (parsed.input.image?.byteLength ?? 0) + Buffer.byteLength(parsed.input.text ?? '');
|
|
@@ -97,14 +105,14 @@ export function composeClassifier(options, experimental, boundaryView, intervalP
|
|
|
97
105
|
throw new ClassifierError('INVALID_INPUT', 'Expected extract input object');
|
|
98
106
|
const { signal, ...rest } = input;
|
|
99
107
|
const parent = parseOperationSignal(signal);
|
|
100
|
-
const
|
|
108
|
+
const { rest: rewritten, port } = prepareResearchOperationInput(rest, research);
|
|
109
|
+
const parsed = parse(extractSchema, rewritten);
|
|
101
110
|
if (!segmenter && !proposalDependencies)
|
|
102
111
|
throw new ClassifierError('UNSUPPORTED', 'extract requires an explicit SegmentationPort or ObjectProposalPort');
|
|
103
|
-
const representation = representationFor(parsed.representation);
|
|
104
112
|
return runtime.withOwnedInput(parsed.image.byteLength, parent, () => ({ ...parsed, image: new Uint8Array(parsed.image) }), (snapshot, s) => {
|
|
105
113
|
if (segmenter)
|
|
106
|
-
return spatial.extractRegion(snapshot, s,
|
|
107
|
-
return spatial.extractWithProposals(snapshot, s,
|
|
114
|
+
return spatial.extractRegion(snapshot, s, port);
|
|
115
|
+
return spatial.extractWithProposals(snapshot, s, port);
|
|
108
116
|
});
|
|
109
117
|
},
|
|
110
118
|
dispose: () => runtime.dispose(),
|
package/dist/cli.js
CHANGED
|
@@ -8,7 +8,7 @@ import { createMobileSamSegmenter, createGrabCutSegmenter } from './segmentation
|
|
|
8
8
|
async function main() {
|
|
9
9
|
const args = parseCliArguments(process.argv.slice(2));
|
|
10
10
|
if (args.command === 'help') {
|
|
11
|
-
console.log('zorya-classifier classify-text <text> <classes.json>\nzorya-classifier classify-image <image> <classes.json>\nzorya-classifier locate <image> <target> [--mode region|whole-object] (
|
|
11
|
+
console.log('zorya-classifier classify-text <text> <classes.json>\nzorya-classifier classify-image <image> <classes.json>\nzorya-classifier locate <image> <target> [--mode region|whole-object] (spatial search; grid cells are not whole-object boxes)\nzorya-classifier extract <image> <target> --output <prefix> --backend mobile-sam --encoder <path> --decoder <path> [--provider cpu|dml] (extraction with caller-owned backend)\nzorya-classifier extract <image> <target> --output <prefix> --backend grabcut --opencv <path> (extraction with caller-owned backend)\nSet ZORYA_MODEL. Default ZORYA_MODE=decisions; use chat for ordinary vision models/custom chat endpoints. OpenRouter: OPENROUTER_API_KEY + ZORYA_PROVIDER. Direct: ZORYA_ENDPOINT + optional ZORYA_API_KEY. Optional ZORYA_REASONING_EFFORT and ZORYA_RESPONSE_FORMAT. classify is supported; locate/extract/object backends are available—model, provider and native assets remain caller-supplied. One classify request; up to three region or six whole-object requests. Extraction additionally runs the explicit local backend. Output files are never overwritten; native assets are never downloaded automatically.');
|
|
12
12
|
return;
|
|
13
13
|
}
|
|
14
14
|
const { command, value, target } = args;
|
|
@@ -44,22 +44,13 @@ export declare const classifySchema: z.ZodObject<{
|
|
|
44
44
|
description: string;
|
|
45
45
|
}[]>;
|
|
46
46
|
instructions: z.ZodOptional<z.ZodString>;
|
|
47
|
-
representation: z.ZodOptional<z.
|
|
47
|
+
representation: z.ZodOptional<z.ZodObject<{
|
|
48
48
|
type: z.ZodLiteral<"vision">;
|
|
49
49
|
}, "strict", z.ZodTypeAny, {
|
|
50
50
|
type: "vision";
|
|
51
51
|
}, {
|
|
52
52
|
type: "vision";
|
|
53
|
-
}
|
|
54
|
-
type: z.ZodLiteral<"experimental-text">;
|
|
55
|
-
profile: z.ZodLiteral<"text-image-v1">;
|
|
56
|
-
}, "strict", z.ZodTypeAny, {
|
|
57
|
-
type: "experimental-text";
|
|
58
|
-
profile: "text-image-v1";
|
|
59
|
-
}, {
|
|
60
|
-
type: "experimental-text";
|
|
61
|
-
profile: "text-image-v1";
|
|
62
|
-
}>]>>;
|
|
53
|
+
}>>;
|
|
63
54
|
}, "strict", z.ZodTypeAny, {
|
|
64
55
|
input: string | {
|
|
65
56
|
text?: string | undefined;
|
|
@@ -72,9 +63,6 @@ export declare const classifySchema: z.ZodObject<{
|
|
|
72
63
|
instructions?: string | undefined;
|
|
73
64
|
representation?: {
|
|
74
65
|
type: "vision";
|
|
75
|
-
} | {
|
|
76
|
-
type: "experimental-text";
|
|
77
|
-
profile: "text-image-v1";
|
|
78
66
|
} | undefined;
|
|
79
67
|
}, {
|
|
80
68
|
input: string | {
|
|
@@ -88,9 +76,6 @@ export declare const classifySchema: z.ZodObject<{
|
|
|
88
76
|
instructions?: string | undefined;
|
|
89
77
|
representation?: {
|
|
90
78
|
type: "vision";
|
|
91
|
-
} | {
|
|
92
|
-
type: "experimental-text";
|
|
93
|
-
profile: "text-image-v1";
|
|
94
79
|
} | undefined;
|
|
95
80
|
}>;
|
|
96
81
|
export type ClassifyInput = z.input<typeof classifySchema> & {
|
package/dist/entities/image.d.ts
CHANGED
|
@@ -9,22 +9,13 @@ export declare const limits: Readonly<{
|
|
|
9
9
|
maxImages: 9;
|
|
10
10
|
}>;
|
|
11
11
|
export declare const imageSchema: z.ZodEffects<z.ZodType<Uint8Array<ArrayBufferLike>, z.ZodTypeDef, Uint8Array<ArrayBufferLike>>, Uint8Array<ArrayBufferLike>, Uint8Array<ArrayBufferLike>>;
|
|
12
|
-
export declare const representationSchema: z.
|
|
12
|
+
export declare const representationSchema: z.ZodObject<{
|
|
13
13
|
type: z.ZodLiteral<"vision">;
|
|
14
14
|
}, "strict", z.ZodTypeAny, {
|
|
15
15
|
type: "vision";
|
|
16
16
|
}, {
|
|
17
17
|
type: "vision";
|
|
18
|
-
}
|
|
19
|
-
type: z.ZodLiteral<"experimental-text">;
|
|
20
|
-
profile: z.ZodLiteral<"text-image-v1">;
|
|
21
|
-
}, "strict", z.ZodTypeAny, {
|
|
22
|
-
type: "experimental-text";
|
|
23
|
-
profile: "text-image-v1";
|
|
24
|
-
}, {
|
|
25
|
-
type: "experimental-text";
|
|
26
|
-
profile: "text-image-v1";
|
|
27
|
-
}>]>;
|
|
18
|
+
}>;
|
|
28
19
|
export type Representation = z.infer<typeof representationSchema>;
|
|
29
20
|
export interface RasterImage {
|
|
30
21
|
bytes: Uint8Array;
|
package/dist/entities/image.js
CHANGED
|
@@ -9,7 +9,4 @@ export const limits = Object.freeze({
|
|
|
9
9
|
export const imageSchema = z
|
|
10
10
|
.custom((value) => value instanceof Uint8Array)
|
|
11
11
|
.refine((v) => v.byteLength > 0 && v.byteLength <= limits.imageBytes, 'Expected 1..16MiB image bytes');
|
|
12
|
-
export const representationSchema = z.
|
|
13
|
-
z.object({ type: z.literal('vision') }).strict(),
|
|
14
|
-
z.object({ type: z.literal('experimental-text'), profile: z.literal('text-image-v1') }).strict(),
|
|
15
|
-
]);
|
|
12
|
+
export const representationSchema = z.object({ type: z.literal('vision') }).strict();
|
|
@@ -22,31 +22,19 @@ export declare const locateSchema: z.ZodObject<{
|
|
|
22
22
|
height: number;
|
|
23
23
|
}>>;
|
|
24
24
|
mode: z.ZodOptional<z.ZodLiteral<"region">>;
|
|
25
|
-
representation: z.ZodOptional<z.
|
|
25
|
+
representation: z.ZodOptional<z.ZodObject<{
|
|
26
26
|
type: z.ZodLiteral<"vision">;
|
|
27
27
|
}, "strict", z.ZodTypeAny, {
|
|
28
28
|
type: "vision";
|
|
29
29
|
}, {
|
|
30
30
|
type: "vision";
|
|
31
|
-
}
|
|
32
|
-
type: z.ZodLiteral<"experimental-text">;
|
|
33
|
-
profile: z.ZodLiteral<"text-image-v1">;
|
|
34
|
-
}, "strict", z.ZodTypeAny, {
|
|
35
|
-
type: "experimental-text";
|
|
36
|
-
profile: "text-image-v1";
|
|
37
|
-
}, {
|
|
38
|
-
type: "experimental-text";
|
|
39
|
-
profile: "text-image-v1";
|
|
40
|
-
}>]>>;
|
|
31
|
+
}>>;
|
|
41
32
|
}, "strict", z.ZodTypeAny, {
|
|
42
33
|
image: Uint8Array<ArrayBufferLike>;
|
|
43
34
|
target: string;
|
|
44
35
|
exclusions: string[];
|
|
45
36
|
representation?: {
|
|
46
37
|
type: "vision";
|
|
47
|
-
} | {
|
|
48
|
-
type: "experimental-text";
|
|
49
|
-
profile: "text-image-v1";
|
|
50
38
|
} | undefined;
|
|
51
39
|
roi?: {
|
|
52
40
|
left: number;
|
|
@@ -60,9 +48,6 @@ export declare const locateSchema: z.ZodObject<{
|
|
|
60
48
|
target: string;
|
|
61
49
|
representation?: {
|
|
62
50
|
type: "vision";
|
|
63
|
-
} | {
|
|
64
|
-
type: "experimental-text";
|
|
65
|
-
profile: "text-image-v1";
|
|
66
51
|
} | undefined;
|
|
67
52
|
exclusions?: string[] | undefined;
|
|
68
53
|
roi?: {
|
|
@@ -73,7 +58,7 @@ export declare const locateSchema: z.ZodObject<{
|
|
|
73
58
|
} | undefined;
|
|
74
59
|
mode?: "region" | undefined;
|
|
75
60
|
}>;
|
|
76
|
-
/**
|
|
61
|
+
/** Coarse region search; a located grid cell is a hypothesis, not reliable whole-object geometry or proof of absence. */
|
|
77
62
|
export type LocateInput = z.input<typeof locateSchema> & {
|
|
78
63
|
signal?: AbortSignal;
|
|
79
64
|
};
|
|
@@ -88,7 +73,7 @@ export interface LocalizationMetadata {
|
|
|
88
73
|
knownCostUsd: number;
|
|
89
74
|
unknownCostCount: number;
|
|
90
75
|
}
|
|
91
|
-
/**
|
|
76
|
+
/** Outcome of spatial localization; grid regions are not semantic object boxes or masks. */
|
|
92
77
|
export type LocalizationResult = ({
|
|
93
78
|
status: 'located';
|
|
94
79
|
region: PixelRect;
|
|
@@ -105,7 +90,7 @@ export type LocalizationResult = ({
|
|
|
105
90
|
}) & {
|
|
106
91
|
metadata: LocalizationMetadata;
|
|
107
92
|
};
|
|
108
|
-
/**
|
|
93
|
+
/** Optional legacy region refinement. */
|
|
109
94
|
export type RefinementResult = {
|
|
110
95
|
status: 'refined';
|
|
111
96
|
region: PixelRect;
|
|
@@ -114,7 +99,7 @@ export type RefinementResult = {
|
|
|
114
99
|
status: 'unknown';
|
|
115
100
|
reason: string;
|
|
116
101
|
};
|
|
117
|
-
/**
|
|
102
|
+
/** Caller-owned refinement hook. */
|
|
118
103
|
export interface RefinementPort {
|
|
119
104
|
refine(input: {
|
|
120
105
|
image: Uint8Array;
|
|
@@ -35,7 +35,7 @@ export interface ObjectProposalBatch {
|
|
|
35
35
|
}
|
|
36
36
|
/**
|
|
37
37
|
* Caller-owned. propose settles only after physical work closes; no model calls here.
|
|
38
|
-
*
|
|
38
|
+
* Mask proposal strategy; no general quality qualification is implied.
|
|
39
39
|
*/
|
|
40
40
|
export interface ObjectProposalPort {
|
|
41
41
|
propose(input: ObjectProposalInput): Promise<ObjectProposalBatch>;
|
|
@@ -1,5 +1,4 @@
|
|
|
1
1
|
import type { DecisionState, GridCell, PixelRect, RasterImage } from './contracts.js';
|
|
2
|
-
import { z } from 'zod';
|
|
3
2
|
/** Canonical oriented RGBA8. Alpha remains meaningful; transparent is not black. */
|
|
4
3
|
export interface RgbaRaster {
|
|
5
4
|
data: Uint8Array;
|
|
@@ -41,185 +40,3 @@ export interface ImageRepresentationPort {
|
|
|
41
40
|
metadata: RepresentationMetadata;
|
|
42
41
|
}>;
|
|
43
42
|
}
|
|
44
|
-
export declare const representationLimits: Readonly<{
|
|
45
|
-
totalBytes: number;
|
|
46
|
-
partCharacters: 64000;
|
|
47
|
-
maxSide: 64;
|
|
48
|
-
maxViews: 9;
|
|
49
|
-
}>;
|
|
50
|
-
export declare const preparedTextSchema: z.ZodObject<{
|
|
51
|
-
state: z.ZodArray<z.ZodString, "many">;
|
|
52
|
-
metadata: z.ZodObject<{
|
|
53
|
-
profile: z.ZodString;
|
|
54
|
-
bytes: z.ZodNumber;
|
|
55
|
-
views: z.ZodArray<z.ZodObject<{
|
|
56
|
-
id: z.ZodString;
|
|
57
|
-
sourceRegion: z.ZodObject<{
|
|
58
|
-
left: z.ZodNumber;
|
|
59
|
-
top: z.ZodNumber;
|
|
60
|
-
width: z.ZodNumber;
|
|
61
|
-
height: z.ZodNumber;
|
|
62
|
-
}, "strict", z.ZodTypeAny, {
|
|
63
|
-
left: number;
|
|
64
|
-
top: number;
|
|
65
|
-
width: number;
|
|
66
|
-
height: number;
|
|
67
|
-
}, {
|
|
68
|
-
left: number;
|
|
69
|
-
top: number;
|
|
70
|
-
width: number;
|
|
71
|
-
height: number;
|
|
72
|
-
}>;
|
|
73
|
-
width: z.ZodNumber;
|
|
74
|
-
height: z.ZodNumber;
|
|
75
|
-
scaleX: z.ZodNumber;
|
|
76
|
-
scaleY: z.ZodNumber;
|
|
77
|
-
content: z.ZodOptional<z.ZodObject<{
|
|
78
|
-
left: z.ZodNumber;
|
|
79
|
-
top: z.ZodNumber;
|
|
80
|
-
width: z.ZodNumber;
|
|
81
|
-
height: z.ZodNumber;
|
|
82
|
-
}, "strict", z.ZodTypeAny, {
|
|
83
|
-
left: number;
|
|
84
|
-
top: number;
|
|
85
|
-
width: number;
|
|
86
|
-
height: number;
|
|
87
|
-
}, {
|
|
88
|
-
left: number;
|
|
89
|
-
top: number;
|
|
90
|
-
width: number;
|
|
91
|
-
height: number;
|
|
92
|
-
}>>;
|
|
93
|
-
}, "strict", z.ZodTypeAny, {
|
|
94
|
-
width: number;
|
|
95
|
-
height: number;
|
|
96
|
-
id: string;
|
|
97
|
-
sourceRegion: {
|
|
98
|
-
left: number;
|
|
99
|
-
top: number;
|
|
100
|
-
width: number;
|
|
101
|
-
height: number;
|
|
102
|
-
};
|
|
103
|
-
scaleX: number;
|
|
104
|
-
scaleY: number;
|
|
105
|
-
content?: {
|
|
106
|
-
left: number;
|
|
107
|
-
top: number;
|
|
108
|
-
width: number;
|
|
109
|
-
height: number;
|
|
110
|
-
} | undefined;
|
|
111
|
-
}, {
|
|
112
|
-
width: number;
|
|
113
|
-
height: number;
|
|
114
|
-
id: string;
|
|
115
|
-
sourceRegion: {
|
|
116
|
-
left: number;
|
|
117
|
-
top: number;
|
|
118
|
-
width: number;
|
|
119
|
-
height: number;
|
|
120
|
-
};
|
|
121
|
-
scaleX: number;
|
|
122
|
-
scaleY: number;
|
|
123
|
-
content?: {
|
|
124
|
-
left: number;
|
|
125
|
-
top: number;
|
|
126
|
-
width: number;
|
|
127
|
-
height: number;
|
|
128
|
-
} | undefined;
|
|
129
|
-
}>, "many">;
|
|
130
|
-
}, "strict", z.ZodTypeAny, {
|
|
131
|
-
profile: string;
|
|
132
|
-
bytes: number;
|
|
133
|
-
views: {
|
|
134
|
-
width: number;
|
|
135
|
-
height: number;
|
|
136
|
-
id: string;
|
|
137
|
-
sourceRegion: {
|
|
138
|
-
left: number;
|
|
139
|
-
top: number;
|
|
140
|
-
width: number;
|
|
141
|
-
height: number;
|
|
142
|
-
};
|
|
143
|
-
scaleX: number;
|
|
144
|
-
scaleY: number;
|
|
145
|
-
content?: {
|
|
146
|
-
left: number;
|
|
147
|
-
top: number;
|
|
148
|
-
width: number;
|
|
149
|
-
height: number;
|
|
150
|
-
} | undefined;
|
|
151
|
-
}[];
|
|
152
|
-
}, {
|
|
153
|
-
profile: string;
|
|
154
|
-
bytes: number;
|
|
155
|
-
views: {
|
|
156
|
-
width: number;
|
|
157
|
-
height: number;
|
|
158
|
-
id: string;
|
|
159
|
-
sourceRegion: {
|
|
160
|
-
left: number;
|
|
161
|
-
top: number;
|
|
162
|
-
width: number;
|
|
163
|
-
height: number;
|
|
164
|
-
};
|
|
165
|
-
scaleX: number;
|
|
166
|
-
scaleY: number;
|
|
167
|
-
content?: {
|
|
168
|
-
left: number;
|
|
169
|
-
top: number;
|
|
170
|
-
width: number;
|
|
171
|
-
height: number;
|
|
172
|
-
} | undefined;
|
|
173
|
-
}[];
|
|
174
|
-
}>;
|
|
175
|
-
}, "strict", z.ZodTypeAny, {
|
|
176
|
-
metadata: {
|
|
177
|
-
profile: string;
|
|
178
|
-
bytes: number;
|
|
179
|
-
views: {
|
|
180
|
-
width: number;
|
|
181
|
-
height: number;
|
|
182
|
-
id: string;
|
|
183
|
-
sourceRegion: {
|
|
184
|
-
left: number;
|
|
185
|
-
top: number;
|
|
186
|
-
width: number;
|
|
187
|
-
height: number;
|
|
188
|
-
};
|
|
189
|
-
scaleX: number;
|
|
190
|
-
scaleY: number;
|
|
191
|
-
content?: {
|
|
192
|
-
left: number;
|
|
193
|
-
top: number;
|
|
194
|
-
width: number;
|
|
195
|
-
height: number;
|
|
196
|
-
} | undefined;
|
|
197
|
-
}[];
|
|
198
|
-
};
|
|
199
|
-
state: string[];
|
|
200
|
-
}, {
|
|
201
|
-
metadata: {
|
|
202
|
-
profile: string;
|
|
203
|
-
bytes: number;
|
|
204
|
-
views: {
|
|
205
|
-
width: number;
|
|
206
|
-
height: number;
|
|
207
|
-
id: string;
|
|
208
|
-
sourceRegion: {
|
|
209
|
-
left: number;
|
|
210
|
-
top: number;
|
|
211
|
-
width: number;
|
|
212
|
-
height: number;
|
|
213
|
-
};
|
|
214
|
-
scaleX: number;
|
|
215
|
-
scaleY: number;
|
|
216
|
-
content?: {
|
|
217
|
-
left: number;
|
|
218
|
-
top: number;
|
|
219
|
-
width: number;
|
|
220
|
-
height: number;
|
|
221
|
-
} | undefined;
|
|
222
|
-
}[];
|
|
223
|
-
};
|
|
224
|
-
state: string[];
|
|
225
|
-
}>;
|