ugcinc 4.20.0 → 4.22.0

package/README.md

@@ -321,3 +321,62 @@ of original upload quality or watermark removal is made.
 Both operations require an organization and `content:analyze` for scoped keys.
 Errors may include `errorCode` and `retryable`. Existing scraper credit/rate safeguards
 apply. Storage/provider errors do not imply that a post or profile was deleted.
+
+## Video inspection
+
+```ts
+const frames = await client.media.extractFrames({url: videoUrl, timestampsMs: [0, 1500, 3000]});
+const scenes = await client.media.detectScenes({url: videoUrl});
+// Recover a pending job without submitting another analysis:
+const status = await client.media.getMediaAnalysis({jobId});
+```
+
+These methods use authenticated `POST /media/analysis/extract-frames`,
+`POST /media/analysis/detect-scenes` and `POST /media/analysis/status`.
+They require an organization and `content:analyze`. Agent tools are `extract_frames`,
+`detect_scenes` and the supporting `get_media_analysis` status operation.
+
+Extraction accepts either `timestampsMs` (1–48 values, including timestamps from other
+analysis results) or `range: {startMs,endMs,intervalMs}` (end exclusive, maximum 48
+samples). `selection` is `nearest` (default; ties choose the earlier frame), `before`
+or `after`. Times are milliseconds from the first displayed video frame. Out-of-range
+requests fail; `after` fails if no later frame exists. Returned frames include requested
+and actual times, original integer PTS as a string, decoded frame index, dimensions
+and owned PNG URL. Rotation and sample aspect ratio are applied; outputs fit within
+1280 pixels on each axis without upscaling. `contactSheet` defaults to true and returns
+a labeled sheet with cell boxes linked to the individual images. Duplicate requests
+retain their order and may reference the same decoded frame.
+
+Scene detection reuses the existing adaptive and brightness/pHash detector and returns
+source-time ranges, frame indices and detector settings. It does not export split MP4s.
+Ranges end exclusively. The last scene ends at video duration; its endPts is null when
+there is no following frame. Boundaries are detector estimates, especially for fades,
+not a guarantee of semantic scenes or an exact creative edit point.
+
+Both operations run as bounded worker jobs. SDK/tool calls make up to 180 status polls, spaced two seconds apart;
+check `data.status`: pending/processing includes `jobId` for subsequent status calls,
+completed includes `result`, and failed includes an explicit error. A successful HTTP
+response does not imply job completion. Status handles belong to their organization
+and expire after seven days. Each submission analyzes the current source bytes rather
+than caching by URL. Results include source SHA-256, encoded/display dimensions,
+rotation, time base, source starting PTS, decoded frame count and an owned manifest.
+Video URLs must be public HTTPS sources. Limits: 100 MB, 30 minutes, 4096 pixels per
+encoded dimension, 120,000 decoded frames. Unsupported/malformed media fails explicitly.
+
+### Image and video OCR
+
+Quick reference: `client.media.readImageText({url})`, `client.media.readVideoText({url, mode, range?, intervalMs?})`; agent tools `read_image_text` and `read_video_text`. Both use the authenticated analysis job/status flow above.
+
+```ts
+const imageText = await client.media.readImageText({url: imageUrl});
+const sampledText = await client.media.readVideoText({url: videoUrl, mode: 'fast', intervalMs: 1000});
+const everyFrame = await client.media.readVideoText({url: videoUrl, mode: 'frame_scan', range: {startMs: 0, endMs: 5000}});
+```
+
+`POST /media/analysis/read-image-text` accepts an HTTPS image URL. Results contain `regions` (raw recognized text, four-point polygon, enclosing box, model confidence, readable/unreadable status), inferred `lines` with region IDs and reading order, source dimensions/hash, model details, `previewUrl` and `manifestUrl`. Coordinates are pixels after EXIF orientation; transparent pixels are composited on white. English recognition only; no guaranteed reading order for columns or inverted text. Low-confidence candidates retain `rawText` but return `text: null`. Undetected text cannot be marked unreadable, and empty output does not prove absence. Images must be static, at most 4096 pixels per axis /16 megapixels, and at most 100 MiB.
+
+`POST /media/analysis/read-video-text` accepts a video URL, `mode` (`fast` default or `frame_scan`), optional `{startMs,endMs}` range and optional `intervalMs` (fast only, default1000, minimum100). Range end is exclusive; range defaults to the whole clip. A request may cover at most120 seconds and inspect at most240 frames; split longer frame scans into ranges. Invalid/excessive requests fail explicitly, never silently truncate. Times are relative to the first displayed frame; original `sourcePts` and time base are retained. The frame displayed at a partial range's start is included, so its source timestamp can precede `startMs`.
+
+Results include per-frame regions/lines, up to eight annotated frame previews, heuristic text `tracks` with observed boxes, and `coverage.inspected` plus `coverage.uninspected`. `coverage.complete` means all displayed frames within the range were inspected, not that OCR found every word. A track's `firstObservedMs` / `lastObservedEndMs` bounds observations; it does not prove uninterrupted visibility between sampled frames. Fast mode can miss brief text entirely. Fully scanned recognized single-frame text retains its decoded frame interval. Recognition scores are not calibrated probabilities. Track identity, scene-change rejection and line grouping are estimates; use individual observations and polygons for evidence.
+
+Both methods return `ApiResponse<MediaAnalysisJob>`, waiting through up to180 two-second status polls. A pending/processing response retains `jobId`; resume with `getMediaAnalysis({jobId})`. Job results include model/runtime versions and SHA-256 checksums. These CPU jobs share the existing bounded analysis queue. A failed job does not return partial OCR as complete.

package/dist/automations/nodes/index.d.ts

@@ -333,18 +333,18 @@ export declare const nodeDefinitions: {
         __TOutputs: import("./screenshot-animation").ScreenshotAnimationNodeOutputs;
     };
     readonly 'remotion-composition': NodeDefinition<"remotion-composition", "generator", {
+        width: number;
+        height: number;
         source: string;
         outputMode: "per-input" | "single";
         selectionMode: null;
         outputType: "video" | "image";
         inputs: {
-            type: "number" | "boolean" | "object" | "video" | "image" | "audio" | "text";
+            type: "number" | "boolean" | "object" | "video" | "image" | "text" | "audio";
             id: string;
             isArray: boolean;
             required: boolean;
         }[];
-        width: number;
-        height: number;
         fps: number;
         durationInFrames: number;
         inputProps: Record<string, unknown>;

package/dist/automations/nodes/remotion-composition.d.ts

@@ -4,35 +4,35 @@ type Outputs = {
     output: VideoValue;
 };
 declare const defaults: {
+    width: number;
+    height: number;
     source: string;
     outputMode: "per-input" | "single";
     selectionMode: null;
     outputType: "video" | "image";
     inputs: {
-        type: "number" | "boolean" | "object" | "video" | "image" | "audio" | "text";
+        type: "number" | "boolean" | "object" | "video" | "image" | "text" | "audio";
         id: string;
         isArray: boolean;
         required: boolean;
     }[];
-    width: number;
-    height: number;
     fps: number;
     durationInFrames: number;
     inputProps: Record<string, unknown>;
 };
 declare const _default: import("./types").NodeDefinition<"remotion-composition", "generator", {
+    width: number;
+    height: number;
     source: string;
     outputMode: "per-input" | "single";
     selectionMode: null;
     outputType: "video" | "image";
     inputs: {
-        type: "number" | "boolean" | "object" | "video" | "image" | "audio" | "text";
+        type: "number" | "boolean" | "object" | "video" | "image" | "text" | "audio";
         id: string;
         isArray: boolean;
         required: boolean;
     }[];
-    width: number;
-    height: number;
     fps: number;
     durationInFrames: number;
     inputProps: Record<string, unknown>;

package/dist/index.d.ts

@@ -86,3 +86,5 @@ export type { SuccessResponse, ErrorResponse, ApiResponse, } from './types';
 export type { CropBoundary, CropAxisConfig, DynamicCropConfig, BorderRadiusConfig, VerticalAnchor, HorizontalAnchor, HorizontalSelfAnchor, VerticalSelfAnchor, RelativePositionConfigX, RelativePositionConfigY, } from './render/types';
 export type { PublicPostFailureCode, PublicPostRecovery, PublicPostNextAction, PublicPostFailure } from './post-failure';
 export type { ListProfilePostsParams, ListProfilePostsResult, DownloadSocialPostParams, DownloadSocialPostResult, SocialReferencePost, SocialReferenceAsset } from './social-acquisition';
+export type { ReadImageTextParams, ReadVideoTextParams, ImageTextResult, VideoTextResult, ExtractFramesParams, DetectScenesParams, FramesResult, ScenesResult, MediaAnalysisResult, MediaAnalysisJob } from "./media-analysis";
+export { readImageTextSchema, readVideoTextSchema, imageTextResultSchema, videoTextResultSchema, extractFramesSchema, detectScenesSchema, mediaAnalysisStatusSchema, mediaAnalysisResultSchema } from "./media-analysis";

package/dist/index.js

@@ -7,7 +7,7 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.hasMissingTriggerError = exports.getPortErrorsForNode = exports.getErrorNodeIds = exports.validateWorkflow = exports.getCapturedNodes = exports.getNodesDownstreamOfSet = exports.getDownstreamNodes = exports.checkCrossContextViolation = exports.getForEachContext = exports.getConnectedSource = exports.cleanupStaleConnections = exports.removeNodeConnections = exports.removeConnection = exports.addConnection = exports.deriveConnections = exports.createNode = exports.getOutputSchema = exports.getNodeByType = exports.getAllNodes = exports.getInputPreviewValue = exports.computeAllNodePortsWithPreviews = exports.computeAllNodePorts = exports.computePortsWithPreviews = exports.computePortsForNode = exports.areTypesCompatible = exports.parseRemotionPropsJson = exports.REMOTION_COMPOSITION_IMPORTS = exports.remotionCompositionPortSchema = exports.remotionCompositionNodeSchema = exports.remotionCompositionSchema = exports.submitIMessageDmRenderJob = exports.submitInstagramDmRenderJob = exports.submitAutoCaptionRenderJob = exports.submitRemotionCompositionRenderJob = exports.submitScreenshotAnimationRenderJob = exports.submitSceneSplitJob = exports.submitDeduplicationJob = exports.getRenderJobStatus = exports.submitVideoRenderJob = exports.submitImageRenderJob = exports.BillingClient = exports.CommentsClient = exports.MediaClient = exports.AutomationsClient = exports.OrganizationClient = exports.StatsClient = exports.PostsClient = exports.TasksClient = exports.AccountsClient = exports.UGCClient = void 0;
 exports.parseOpenAPIOutputPath = exports.parseOpenAPIInputs = exports.mapOpenAPIType = exports.nodeConfigToCaptionStyle = exports.HttpMethods = exports.prepareImageComposerInput = exports.substituteVariables = exports.processTemplate = exports.extractTemplateVariables = exports.PortIdSchema = exports.normalizeToPortId = exports.portIdToTitle = exports.isValidPortId = exports.portId = exports.selectFromPool = exports.getModelDurations = exports.getModelAspectRatios = exports.ALL_VIDEO_MODELS = exports.getAvailableInputModes = exports.hasReferenceVariant = exports.getReferenceModelId = exports.hasFirstLastFrameVariant = exports.getFirstLastFrameModelId = exports.hasImageToVideoVariant = exports.getImageToVideoModelId = exports.isImageToVideoModel = exports.IMAGE_ASPECT_RATIOS = exports.ALL_IMAGE_MODELS = exports.getEditModelId = exports.isEditModel = exports.getCapturedSourceNodeTypes = exports.getFlowControlNodeTypes = exports.getAllNodeDefinitions = exports.getNodeDefinition = exports.isPortType = exports.formatPortType = exports.isAsyncExecutor = exports.internalNodeTypes = exports.nodeDefinitions = exports.extractWorkflowConfig = exports.generateNodeName = exports.shuffleNodePreview = exports.getPreviewValue = exports.removePreviewForConnection = exports.updatePreviewMapForConnection = exports.computePreviewMap = exports.computeAllNodePreviews = exports.resolveNodePreview = exports.getWorkflowOutputSchema = exports.hasMissingTerminalError = void 0;
-exports.prepareVideoComposerInput = exports.formatDateTag = exports.outputFieldsToZod = exports.outputFieldToZod = exports.LLMProviders = exports.applyLogicOperator = exports.IfLogicOperators = exports.resolvePath = exports.indexExpressionToString = exports.indexExpressionToIndexes = exports.resolveUnknownPath = exports.getOutputTypeFromCategory = void 0;
+exports.mediaAnalysisResultSchema = exports.mediaAnalysisStatusSchema = exports.detectScenesSchema = exports.extractFramesSchema = exports.videoTextResultSchema = exports.imageTextResultSchema = exports.readVideoTextSchema = exports.readImageTextSchema = exports.prepareVideoComposerInput = exports.formatDateTag = exports.outputFieldsToZod = exports.outputFieldToZod = exports.LLMProviders = exports.applyLogicOperator = exports.IfLogicOperators = exports.resolvePath = exports.indexExpressionToString = exports.indexExpressionToIndexes = exports.resolveUnknownPath = exports.getOutputTypeFromCategory = void 0;
 // =============================================================================
 // Client Exports
 // =============================================================================
@@ -174,3 +174,12 @@ var save_to_media_1 = require("./automations/nodes/save-to-media");
 Object.defineProperty(exports, "formatDateTag", { enumerable: true, get: function () { return save_to_media_1.formatDateTag; } });
 var video_composer_1 = require("./automations/nodes/video-composer");
 Object.defineProperty(exports, "prepareVideoComposerInput", { enumerable: true, get: function () { return video_composer_1.prepareVideoComposerInput; } });
+var media_analysis_1 = require("./media-analysis");
+Object.defineProperty(exports, "readImageTextSchema", { enumerable: true, get: function () { return media_analysis_1.readImageTextSchema; } });
+Object.defineProperty(exports, "readVideoTextSchema", { enumerable: true, get: function () { return media_analysis_1.readVideoTextSchema; } });
+Object.defineProperty(exports, "imageTextResultSchema", { enumerable: true, get: function () { return media_analysis_1.imageTextResultSchema; } });
+Object.defineProperty(exports, "videoTextResultSchema", { enumerable: true, get: function () { return media_analysis_1.videoTextResultSchema; } });
+Object.defineProperty(exports, "extractFramesSchema", { enumerable: true, get: function () { return media_analysis_1.extractFramesSchema; } });
+Object.defineProperty(exports, "detectScenesSchema", { enumerable: true, get: function () { return media_analysis_1.detectScenesSchema; } });
+Object.defineProperty(exports, "mediaAnalysisStatusSchema", { enumerable: true, get: function () { return media_analysis_1.mediaAnalysisStatusSchema; } });
+Object.defineProperty(exports, "mediaAnalysisResultSchema", { enumerable: true, get: function () { return media_analysis_1.mediaAnalysisResultSchema; } });