runbios-sdk 0.3.0 → 0.4.0-rc.334

package/CHANGELOG.md

@@ -5,6 +5,72 @@ surface is added, renamed or removed, the patch for a fix inside an unchanged
 surface. The SDK, the Python SDK (`runbios-sdk` on PyPI) and the MCP server
 (`runbios-mcp`) are released together, at one version.
 
+## 0.4.0
+
+Added
+
+- Conscious Loop, saved datasets: `loop.listDatasets`, `getDataset`,
+  `createDataset`, `updateDataset`, `deleteDataset` and `previewDataset`. A
+  dataset is a saved set of conditions (tags, any or all of them; feedback;
+  where samples came from; the model that answered; when they were recorded)
+  and a live selection. `createDataset` takes `LoopDatasetCreate`, whose
+  `name` is required, as the API requires it; `updateDataset` takes
+  `LoopDatasetWrite`. A pipeline trains on its tags or the `datasets` it
+  adds, narrowed by its `conditions` (both on `createPipeline` and
+  `updatePipeline`, and read back on the pipeline); an attempt and a run
+  carry `trained_on`, what it trained on as it was then. `listTraces` takes
+  `dataset`.
+- Conscious Loop, the judge's scoring of unreviewed samples, where the
+  platform scores samples (a pipeline's `auto_scoring` is not null): a
+  pipeline's `auto_scoring` (`status`, `picked`, `below_bar`, `unscored`,
+  `message`) and `minimums.picked_by_judge`; `auto_scoring: { enabled }` on
+  the pipeline writers; `listTraces({ judge: 'picked' | 'below_bar' |
+  'unscored' })`; a sample's `judge` (`score`, `picked`) and, on the judge's
+  own signal, `picked` and `reason_unavailable`; on attempts, versions and
+  the attempt in flight, `sample_scoring_cents` (inside `cost_cents`),
+  `sample_scored`, `sample_picked` and `train_rows_by_judge`; the run
+  detail's `cost.sample_scoring_cents`, and `sample_scoring_cents` on a run,
+  a runs-list row and a metrics-history point.
+- Conscious Loop, what the console reads: a pipeline's `samples_total` and
+  `samples_recent` (the last 48 hours) and `live_deployment.deployment_id`;
+  the run detail's `explanation` and `traces_filter_attempt`;
+  `listTraces({ attempt, split })`, one attempt's own samples; each row of
+  `listTraces({ pipeline })` and each pipeline of `getTracePipelines` says what
+  happens to the sample there (`state`, a `LoopSampleStateName`: `held_back`,
+  `learned`, `trained`, `waiting` or `left_out`, with `attempt_no`,
+  `version_no` and `reason`, the sentence to show); the import answer's
+  `refused_reasons` (`LoopImportRefusal`: `code`, `count`, `sentence`);
+  `getMetricsHistory({ rule_id })`'s `samples`, the pipeline's samples per day
+  by feedback; the workspace's totals on `loop.stats()` (`samples_recent`,
+  `feedback`, a `LoopFeedbackCounts`, and `pipelines`, a
+  `LoopPipelineCounts`); and `listTrainingRuns` takes `result`, with each row
+  carrying `attempt`, the attempt as its pipeline's own list says it. The new
+  fields on existing answers are optional on their types.
+- Conscious Loop, since 0.3.0 was published: `comparison_returned_cents` on
+  `TrainingRun`, `TrainingRunSummary`, `TrainingRunCost`, `PipelineAttempt`
+  and `PipelineActiveRun`, what was taken off for a comparison machine that
+  never started (it is not charged, and the figures beside it are already
+  net of it); `mean_change_needed` on `EvaluationMargin` and
+  `LoopMetricPoint`, the average score change an attempt actually had to
+  clear; and `billed` on `LoopStorage`, whether that storage figure is
+  charged (only the part in the platform's storage, and only where the
+  platform charges for it).
+
+Changed
+
+- `listTraces`' `signal`, `signalled` and `unsignalled` read a person's
+  feedback: the judge's score of an unreviewed sample is not feedback, so a
+  sample it scored stays unsignalled. `verdict_score` no longer reads as good
+  feedback above zero; whether a scored sample trains is `judge.picked`.
+- `createPipeline` says what saving one consents to: the GPU time its
+  attempts use and, where the platform scores samples, the judge's scoring of
+  unreviewed samples, shown on the attempt it was for.
+- `PipelineAttemptOutcome`, and `result` on `listPipelineAttempts` and
+  `listTrainingRuns`, gain `not_tested`: an attempt that ended before it was
+  tested (its comparison machine never started, or it expired first).
+  `inconclusive` is now only an attempt that was tested and could not be
+  decided, or compared nothing.
+
 ## 0.3.0
 
 Added

package/dist/client.js

@@ -378,7 +378,7 @@ export class HttpClient {
     constructor(config) {
         // Default host stays api.runbios.ai for now; cutover to api.runbios.ai is
         // planned once its DNS exists.
-        this.baseUrl = (config.baseUrl || envBaseUrl() || 'https://api.runbios.ai').replace(/\/+$/, '');
+        this.baseUrl = (config.baseUrl || envBaseUrl() || 'https://api-staging.runbios.ai').replace(/\/+$/, '');
         this.apiKey = config.apiKey ?? envApiKey();
         this.accessToken = config.accessToken;
         this.orgId = config.orgId;

package/dist/index.d.ts

@@ -36,7 +36,7 @@ import { Loop } from './resources/loop.js';
  * SDK version. Sent as part of the User-Agent header.
  * Must match package.json "version" -- enforced by a contract test.
  */
-export declare const VERSION = "0.3.0";
+export declare const VERSION = "0.4.0-rc.334";
 export declare class RunBiOS {
     /** Search models, fetch configs, check adapter compatibility. */
     readonly models: Models;
@@ -75,7 +75,7 @@ export { Training } from './resources/training.js';
 export { Wallet } from './resources/wallet.js';
 export { GPU, type GPURecommendation } from './resources/gpu.js';
 export { Inference, validateChatRequest, parseSSE, buildInferenceRequest, contextSizingBasis, type ContextSizingBasis, CONTEXT_DEFAULT_CEILING, CONTEXT_EDITABLE_FLOOR, type ChatMessage, type FunctionTool, type ChatCompletionParams, type ChatCompletionResponse, type ChatCompletionChunk, type InferenceVerbParams, type CompletionParams, type EmbeddingParams, type RerankParams, type AnthropicMessageParams, type AnthropicMessageResponse, type LoopCaptureOption, type LoopSampleIdOnResult, LOOP_PIPELINE_HEADER, LOOP_SAMPLE_ID_HEADER, type ServerlessUsageWindow, type ServerlessTimeseriesMetric, type ServerlessWorkspaceLimits, type ServerlessUsageEnvelope, type ServerlessUsageOverviewResponse, type ServerlessUsageRowsResponse, type ServerlessUsageTimeseriesResponse, type ServerlessUsageDailyResponse, type ServerlessSavingsTotals, type ServerlessSavingsPeriod, type ServerlessUsageSavingsResponse, } from './resources/inference.js';
-export type { BiOSConfig, PaginatedResponse, ApiErrorBody, AvailableGpuAlternative, CapacityMinimumRequirement, InferenceBookingAccepted, Model, ModelSearchParams, ModelSearchResponse, ModelDetailResponse, InferenceModel, InferenceModelListResponse, ModelConfig, Dataset, DatasetListParams, DatasetListResponse, DatasetUploadParams, DatasetPreview, DatasetPreviewParams, DatasetImportHFParams, DatasetRegisterHFParams, DatasetHubSearchParams, DatasetHubPreviewParams, DatasetValidation, DatasetFormatVariant, DatasetFormatSpec, DatasetFormatSpecs, DatasetStorageUsage, TrainingMethod, RLHFAlgorithm, AdapterType, TrainingJobStatus, TrainingStatusFilter, TrainingCreateParams, TrainingStartFromParams, TrainingStartFrom, TrainingExportStep, TrainingListParams, TrainingListResponse, TrainingJob, TrainingMetrics, TrainingResourceMetrics, TrainingDeviceMetrics, MetricPoint, MetricGraphConfig, TrainingCheckpoint, TrainingLogs, TrainingLogEntry, TrainingStopResponse, TrainingResumeResponse, CanonicalTrainingRequest, TrainingPreflightDataset, TrainingPreflightWarning, TrainingPreflightResponse, TrainingCapabilityChoice, TrainingConfigFieldCapability, TrainingCapabilities, TrainingFeature, GPUChoice, WalletBalance, Transaction, TransactionListResponse, TransactionListParams, GPUInfo, GPUPricingResponse, GPUOptionsParams, TrainingRecommendParams, TrainingAdvisorVerdict, TrainingAdvisorRecommendation, TrainingAdvisorUserShape, TrainingAdvisorJustification, TrainingAdvisorBasis, GPUOption, GPUOptionSuggestion, GPUOptionsResponse, InferenceStatus, InferenceCreateParams, InferenceUpdateParams, InferenceDeployment, InferenceDeploymentSummary, InferenceCreateResponse, InferenceListResponse, InferenceUpdateResponse, InferenceLifecycleResponse, InferenceDeleteResponse, InferenceGPUOptionMarket, InferenceGPUOption, InferenceGPUAlternative, InferenceGPUOptionsParams, InferenceGPUOptionsResponse, InferenceCanonicalRequest, InferencePreflightWarning, InferencePreflightResponse, AdapterCompatibility, AdapterCompatibilityResponse, AdapterCompatibilityParams, SupportedArchitecture, ArchitectureScope, SupportedArchitecturesResponse, ApiKey, ApiKeyScope, ApiKeyIntrospection, Organization, OrgMember, OrgInvite, Workspace, Integration, IntegrationCreateParams, StorageUsage, StorageObject, PromptTokensDetails, ChatCompletionUsage, ServingKind, TrainingRulePausedReason, TrainingTrigger, TrainingRecipe, TrainingRunState, TrainingVerdict, TrainingDecision, EvaluationStatus, EvaluationItemStatus, ItemErrorCode, EvaluationWinner, EvaluationItemWinnerFilter, ReportStatusFilter, ReportSort, EvaluationWarningCode, TrainingToolCall, TrainingMessage, TrainingRuleServing, TrainingRunPromoteRequest, TrainingRunRejectRequest, TrainingRunRollbackRequest, TrainingRunCancelRequest, TrainingRunListParams, TrainingRun, TrainingRunSummary, TrainingRunEvent, TrainingRunLinks, TrainingRunActions, EvaluationRef, EvaluationDecoding, EvaluationJudgeDimension, EvaluationDimensionScore, EvaluationWarning, EvaluationMargin, Evaluation, EvaluationItemJudge, EvaluationReferenceKind, EvaluationReference, EvaluationPairwiseOrder, EvaluationItemPairwise, EvaluationItemSide, EvaluationItem, EvaluationItemListParams, BenchmarkStatus, BenchmarkSourceKind, BenchmarkRunStatus, BenchmarkScoring, BenchmarkCheck, BenchmarkDimensionScore, Benchmark, BenchmarkItem, BenchmarkRun, BenchmarkRunItemSide, BenchmarkRunItem, BenchmarkRunItemListParams, BenchmarkHistoryPoint, BenchmarkSource, BenchmarkCreateParams, BenchmarkListParams, BenchmarkItemListParams, BenchmarkHistoryParams, BenchmarkRetireParams, InferenceAlias, InferenceAliasRequest, TrainingRuleDeleteResponse, TrainingRunListResponse, TrainingRunBenchmarkRun, TrainingRunResponse, TrainingRunActionResponse, EvaluationItemsResponse, BenchmarkRunItemsResponse, InferenceAliasListResponse, InferenceAliasResponse, InferenceAliasDeleteResponse, BenchmarkListResponse, BenchmarkItemsResponse, BenchmarkHistoryResponse, TrainingRuleRunRefusalCode, TrainingRuleSaveRefusalCode, TrainingRuleMonthlyLimitRefusal, PipelineStatus, PipelineActiveRun, PipelineVersion, PipelineAttempt, AttemptExplanation, PipelineAttemptListParams, PipelineAttemptListResponse, PipelineAttemptModelFilter, PipelineAttemptResultFilter, PipelineAttemptTrigger, Lineage, PipelineChain, PipelineChainRebuild, TrainingRunRollbackRefusalCode, LoopTraceRebuild, LoopTraceDeleteResponse, PipelineAttemptOutcome, PipelineLiveDeploymentStatus, PipelineLiveDeployment, LoopStorageWhere, LoopStorage, Pipeline, PipelineListParams, PipelineListStatus, PipelineListResponse, PipelineTrainingSettings, PipelineTrainingDefaults, PipelineTrainingRequest, PipelineResponse, PipelineTrainWhen, PipelineSchedule, PipelineWriteRequest, PipelineMutationResponse, PipelineImportParams, PipelineFloors, HeldBackFloors, LoopModel, LoopModelsResponse, LoopUnavailableModel, LoopUnavailableReasonCode, LoopTracePipeline, LoopTracePipelinesResponse, LoopMetricsHistoryParams, LoopMetricPoint, LoopSignalDay, LoopMetricsHistory, LoopCaptureFeedback, LoopFeedbackWord, LoopCaptureHealth, LoopCaptureMiss, LoopSignalManyItem, LoopSignalManyResult, PromotionPolicyState, RuleBenchmark, PipelineBenchmarksResponse, PipelineBenchmarkAttachParams, PipelineBenchmarkMoveParams, CompareBenchmark, VersionBenchmarkScore, PromotionMeasurement, PromotionOutcome, VersionHoldout, VersionScores, VersionScoreDifference, VersionCompareResponse, VersionCompareParams, TrainingRunPromotion, VerdictCheck, VerdictCheckKey, VerdictChecksWithheld, TrainingRunCost, AttemptDirection, AttemptHoldout, AttemptCompareSide, AttemptSharedSample, AttemptSharedFigures, AttemptBenchmarkSide, AttemptCompareBenchmark, AttemptCompareItemSide, AttemptCompareItem, AttemptCompareResponse, AttemptCompareParams, } from './types.js';
+export type { BiOSConfig, PaginatedResponse, ApiErrorBody, AvailableGpuAlternative, CapacityMinimumRequirement, InferenceBookingAccepted, Model, ModelSearchParams, ModelSearchResponse, ModelDetailResponse, InferenceModel, InferenceModelListResponse, ModelConfig, Dataset, DatasetListParams, DatasetListResponse, DatasetUploadParams, DatasetPreview, DatasetPreviewParams, DatasetImportHFParams, DatasetRegisterHFParams, DatasetHubSearchParams, DatasetHubPreviewParams, DatasetValidation, DatasetFormatVariant, DatasetFormatSpec, DatasetFormatSpecs, DatasetStorageUsage, TrainingMethod, RLHFAlgorithm, AdapterType, TrainingJobStatus, TrainingStatusFilter, TrainingCreateParams, TrainingStartFromParams, TrainingStartFrom, TrainingExportStep, TrainingListParams, TrainingListResponse, TrainingJob, TrainingMetrics, TrainingResourceMetrics, TrainingDeviceMetrics, MetricPoint, MetricGraphConfig, TrainingCheckpoint, TrainingLogs, TrainingLogEntry, TrainingStopResponse, TrainingResumeResponse, CanonicalTrainingRequest, TrainingPreflightDataset, TrainingPreflightWarning, TrainingPreflightResponse, TrainingCapabilityChoice, TrainingConfigFieldCapability, TrainingCapabilities, TrainingFeature, GPUChoice, WalletBalance, Transaction, TransactionListResponse, TransactionListParams, GPUInfo, GPUPricingResponse, GPUOptionsParams, TrainingRecommendParams, TrainingAdvisorVerdict, TrainingAdvisorRecommendation, TrainingAdvisorUserShape, TrainingAdvisorJustification, TrainingAdvisorBasis, GPUOption, GPUOptionSuggestion, GPUOptionsResponse, InferenceStatus, InferenceCreateParams, InferenceUpdateParams, InferenceDeployment, InferenceDeploymentSummary, InferenceCreateResponse, InferenceListResponse, InferenceUpdateResponse, InferenceLifecycleResponse, InferenceDeleteResponse, InferenceGPUOptionMarket, InferenceGPUOption, InferenceGPUAlternative, InferenceGPUOptionsParams, InferenceGPUOptionsResponse, InferenceCanonicalRequest, InferencePreflightWarning, InferencePreflightResponse, AdapterCompatibility, AdapterCompatibilityResponse, AdapterCompatibilityParams, SupportedArchitecture, ArchitectureScope, SupportedArchitecturesResponse, ApiKey, ApiKeyScope, ApiKeyIntrospection, Organization, OrgMember, OrgInvite, Workspace, Integration, IntegrationCreateParams, StorageUsage, StorageObject, PromptTokensDetails, ChatCompletionUsage, ServingKind, TrainingRulePausedReason, TrainingTrigger, TrainingRecipe, TrainingRunState, TrainingVerdict, TrainingDecision, EvaluationStatus, EvaluationItemStatus, ItemErrorCode, EvaluationWinner, EvaluationItemWinnerFilter, ReportStatusFilter, ReportSort, EvaluationWarningCode, TrainingToolCall, TrainingMessage, TrainingRuleServing, TrainingRunPromoteRequest, TrainingRunRejectRequest, TrainingRunRollbackRequest, TrainingRunCancelRequest, TrainingRunListParams, TrainingRun, TrainingRunSummary, TrainingRunEvent, TrainingRunLinks, TrainingRunActions, EvaluationRef, EvaluationDecoding, EvaluationJudgeDimension, EvaluationDimensionScore, EvaluationWarning, EvaluationMargin, Evaluation, EvaluationItemJudge, EvaluationReferenceKind, EvaluationReference, EvaluationPairwiseOrder, EvaluationItemPairwise, EvaluationItemSide, EvaluationItem, EvaluationItemListParams, BenchmarkStatus, BenchmarkSourceKind, BenchmarkRunStatus, BenchmarkScoring, BenchmarkCheck, BenchmarkDimensionScore, Benchmark, BenchmarkItem, BenchmarkRun, BenchmarkRunItemSide, BenchmarkRunItem, BenchmarkRunItemListParams, BenchmarkHistoryPoint, BenchmarkSource, BenchmarkCreateParams, BenchmarkListParams, BenchmarkItemListParams, BenchmarkHistoryParams, BenchmarkRetireParams, InferenceAlias, InferenceAliasRequest, TrainingRuleDeleteResponse, TrainingRunListResponse, TrainingRunBenchmarkRun, TrainingRunResponse, TrainingRunActionResponse, EvaluationItemsResponse, BenchmarkRunItemsResponse, InferenceAliasListResponse, InferenceAliasResponse, InferenceAliasDeleteResponse, BenchmarkListResponse, BenchmarkItemsResponse, BenchmarkHistoryResponse, TrainingRuleRunRefusalCode, TrainingRuleSaveRefusalCode, TrainingRuleMonthlyLimitRefusal, PipelineStatus, PipelineActiveRun, PipelineVersion, PipelineAttempt, AttemptExplanation, PipelineAttemptListParams, PipelineAttemptListResponse, PipelineAttemptModelFilter, PipelineAttemptResultFilter, PipelineAttemptTrigger, Lineage, PipelineChain, PipelineChainRebuild, TrainingRunRollbackRefusalCode, LoopTraceRebuild, LoopTraceDeleteResponse, PipelineAttemptOutcome, PipelineLiveDeploymentStatus, PipelineLiveDeployment, LoopStorageWhere, LoopStorage, Pipeline, PipelineListParams, PipelineListStatus, PipelineListResponse, PipelineTrainingSettings, PipelineTrainingDefaults, PipelineTrainingRequest, PipelineResponse, PipelineTrainWhen, PipelineSchedule, PipelineWriteRequest, PipelineMutationResponse, PipelineImportParams, PipelineFloors, HeldBackFloors, LoopModel, LoopModelsResponse, LoopUnavailableModel, LoopUnavailableReasonCode, LoopTracePipeline, LoopTracePipelinesResponse, LoopMetricsHistoryParams, LoopMetricPoint, LoopSignalDay, LoopMetricsHistory, LoopCaptureFeedback, LoopFeedbackWord, LoopCaptureHealth, LoopCaptureMiss, LoopSignalManyItem, LoopSignalManyResult, PromotionPolicyState, RuleBenchmark, PipelineBenchmarksResponse, PipelineBenchmarkAttachParams, PipelineBenchmarkMoveParams, CompareBenchmark, VersionBenchmarkScore, PromotionMeasurement, PromotionOutcome, VersionHoldout, VersionScores, VersionScoreDifference, VersionCompareResponse, VersionCompareParams, TrainingRunPromotion, VerdictCheck, VerdictCheckKey, VerdictChecksWithheld, TrainingRunCost, AttemptDirection, AttemptHoldout, AttemptCompareSide, AttemptSharedSample, AttemptSharedFigures, AttemptBenchmarkSide, AttemptCompareBenchmark, AttemptCompareItemSide, AttemptCompareItem, AttemptCompareResponse, AttemptCompareParams, LoopTraceJudge, LoopJudgeFilter, LoopSampleState, LoopSampleStateName, LoopSampleDay, LoopImportRefusal, PipelineAutoScoring, PipelineAutoScoringStatus, LoopDatasetFeedback, LoopSampleOrigin, LoopFeedbackCounts, LoopPipelineCounts, LoopDatasetPipelineRef, LoopDataset, LoopDatasetWrite, LoopDatasetCreate, LoopDatasetPreviewParams, LoopDatasetPreviewModel, LoopDatasetPreview, LoopDatasetListParams, LoopDatasetListResponse, LoopDatasetDeleteResponse, PipelineDatasetRef, PipelineConditions, PipelineConditionsWrite, TrainedOnDataset, TrainedOn, } from './types.js';
 /** The closed set of run states a training run never leaves. */
 export { TERMINAL_RUN_STATES } from './types.js';
 /** The most versions a pipeline may be set to make. */

package/dist/index.js

@@ -36,7 +36,7 @@ import { Loop } from './resources/loop.js';
  * SDK version. Sent as part of the User-Agent header.
  * Must match package.json "version" -- enforced by a contract test.
  */
-export const VERSION = '0.3.0';
+export const VERSION = '0.4.0-rc.334';
 export class RunBiOS {
     /** Search models, fetch configs, check adapter compatibility. */
     models;

package/dist/resources/inference.js

@@ -373,7 +373,7 @@ export class Inference {
         this.key = config.inferenceKey || envInferenceKey() || envApiKey();
         // Same default host as HttpClient (api.runbios.ai); api.runbios.ai cutover
         // is planned once its DNS exists — update both call sites together.
-        this.baseUrl = (config.baseUrl || envBaseUrl() || 'https://api.runbios.ai').replace(/\/+$/, '');
+        this.baseUrl = (config.baseUrl || envBaseUrl() || 'https://api-staging.runbios.ai').replace(/\/+$/, '');
         this.timeout = config.timeout ?? 900_000;
         this._http = http;
     }

package/dist/resources/loop.d.ts

@@ -1,5 +1,5 @@
 import { type HttpClient } from '../client.js';
-import type { LoopTrace, LoopTraceDeleteResponse, LoopTraceListResponse, LoopTraceListParams, LoopCaptureParams, LoopCaptureResult, LoopImportParams, LoopImportResult, LoopSignal, LoopSignalParams, LoopSignalManyItem, LoopSignalManyResult, LoopLabel, LoopLabelCount, LoopConfig, LoopConfigParams, LoopStats, LoopTracePipelinesResponse, LoopMetricsHistory, LoopMetricsHistoryParams, TrainingRuleDeleteResponse, TrainingRunListParams, TrainingRunListResponse, TrainingRunResponse, TrainingRunActionResponse, TrainingRunPromoteRequest, TrainingRunRejectRequest, TrainingRunRollbackRequest, TrainingRunCancelRequest, Evaluation, EvaluationItemListParams, EvaluationItemsResponse, Benchmark, BenchmarkCreateParams, BenchmarkHistoryParams, BenchmarkHistoryResponse, BenchmarkItemListParams, BenchmarkItemsResponse, BenchmarkListParams, BenchmarkListResponse, BenchmarkRetireParams, BenchmarkRun, BenchmarkRunItemListParams, BenchmarkRunItemsResponse, Pipeline, PipelineAttemptListParams, PipelineAttemptListResponse, PipelineListParams, PipelineListResponse, PipelineWriteRequest, PipelineMutationResponse, LoopModelsResponse, PipelineImportParams, PipelineBenchmarksResponse, PipelineBenchmarkAttachParams, PipelineBenchmarkMoveParams, VersionCompareResponse, VersionCompareParams, AttemptCompareResponse, AttemptCompareParams } from '../types.js';
+import type { LoopTrace, LoopTraceDeleteResponse, LoopTraceListResponse, LoopTraceListParams, LoopCaptureParams, LoopCaptureResult, LoopImportParams, LoopImportResult, LoopSignal, LoopSignalParams, LoopSignalManyItem, LoopSignalManyResult, LoopLabel, LoopLabelCount, LoopConfig, LoopConfigParams, LoopStats, LoopTracePipelinesResponse, LoopMetricsHistory, LoopMetricsHistoryParams, TrainingRuleDeleteResponse, TrainingRunListParams, TrainingRunListResponse, TrainingRunResponse, TrainingRunActionResponse, TrainingRunPromoteRequest, TrainingRunRejectRequest, TrainingRunRollbackRequest, TrainingRunCancelRequest, Evaluation, EvaluationItemListParams, EvaluationItemsResponse, Benchmark, BenchmarkCreateParams, BenchmarkHistoryParams, BenchmarkHistoryResponse, BenchmarkItemListParams, BenchmarkItemsResponse, BenchmarkListParams, BenchmarkListResponse, BenchmarkRetireParams, BenchmarkRun, BenchmarkRunItemListParams, BenchmarkRunItemsResponse, Pipeline, PipelineAttemptListParams, PipelineAttemptListResponse, PipelineListParams, PipelineListResponse, PipelineWriteRequest, PipelineMutationResponse, LoopModelsResponse, PipelineImportParams, PipelineBenchmarksResponse, PipelineBenchmarkAttachParams, PipelineBenchmarkMoveParams, VersionCompareResponse, VersionCompareParams, AttemptCompareResponse, AttemptCompareParams, LoopDataset, LoopDatasetWrite, LoopDatasetCreate, LoopDatasetListParams, LoopDatasetListResponse, LoopDatasetDeleteResponse, LoopDatasetPreview, LoopDatasetPreviewParams } from '../types.js';
 export declare class Loop {
     private readonly _http;
     /** @internal */
@@ -66,7 +66,9 @@ export declare class Loop {
      * an earlier import already had, and the sentences in `notes` are written
      * from those counts, so the numbers and the prose describe the same call and
      * a re-send cannot report a reviewed conversation as waiting for review.
-     * `refused` is rows whose shape could not be read, which the file fixes;
+     * `refused` is rows whose shape could not be read, which the file fixes,
+     * and `refused_reasons` says why, one plain `sentence` per reason with its
+     * `count` (show those, never the codes);
      * `not_saved` is rows that were understood and then could not be stored,
      * which the file cannot fix.
      *
@@ -98,11 +100,33 @@ export declare class Loop {
      * in every message, the answer and its tool calls. Samples kept in your own
      * storage cannot be searched; a search answers how many were left out as
      * `unsearched`.
+     *
+     * `signal` (`good`, `bad`, `corrected`), `signalled` and `unsignalled` read
+     * a PERSON's feedback: the judge's score of an unreviewed sample is not
+     * feedback, so a sample it scored stays in `unsignalled`. `judge` splits
+     * those unreviewed samples: `picked` (scored 0.8 or more, trained on),
+     * `below_bar` (not trained on) and `unscored`; the three add up to
+     * `unsignalled`. A row the judge scored carries `judge` (`score`,
+     * `picked`).
+     *
+     * `pipeline` lists what one pipeline trains on (its tags or its datasets,
+     * narrowed by its conditions), each row with `state`: what happens to it in
+     * that pipeline (`held_back`, `learned`, `trained`, `waiting` or
+     * `left_out`, with `reason`, the sentence to show). `dataset` lists what
+     * one saved dataset selects; not with `pipeline`. `attempt` (a run id)
+     * with `split` (`train` or `held_back`) lists the samples one attempt
+     * trained on or held back.
      */
     listTraces(params?: LoopTraceListParams): Promise<LoopTraceListResponse>;
     /**
      * Read one conversation, with every verdict and tag recorded on it.
      *
+     * When the judge scored it, `judge` carries the score and whether it was
+     * picked, and `signals` holds the judge's score (source `judge`, verdict
+     * `scored`) with `picked`, its per-aspect `metadata.scores` and its
+     * `reason`; `reason_unavailable` means the reason could not be read from
+     * storage just now. A person's feedback decides over it.
+     *
      * A conversation kept in your own storage carries `payload_stored_externally`.
      * When its text could not be read just now it also carries
      * `payload_unavailable`, with `messages` null and `completion` empty: it is
@@ -199,13 +223,20 @@ export declare class Loop {
     /** Every verdict on one conversation, oldest first. */
     listSignals(traceId: string): Promise<LoopSignal[]>;
     /**
-     * The pipelines one conversation feeds: every live pipeline whose tags
-     * select it, and whether each is switched on.
+     * The pipelines one conversation feeds: every live pipeline whose tags or
+     * datasets select it (and whose conditions keep it), and whether each is
+     * switched on.
      *
      * Being listed means the pipeline would select it, not that it will be
      * trained on -- a set still drops answers it cannot use and duplicates, and
-     * holds some back to judge by; `note` says so. An empty list is a real
-     * answer: the conversation trains nothing until it carries a pipeline's tag.
+     * holds some back to judge by; `note` says so. Each pipeline's `state` says
+     * what happens to it there: `held_back` (to test its attempts),
+     * `learned` (by version `version_no`), `trained` (by attempt `attempt_no`,
+     * which did not go live), `waiting` (for the next attempt) or `left_out`;
+     * `reason` is the sentence to show (for `left_out`, why). An empty list is
+     * a real answer: the conversation
+     * trains nothing until it carries a pipeline's tag or joins one of its
+     * datasets.
      */
     getTracePipelines(traceId: string): Promise<LoopTracePipelinesResponse>;
     /**
@@ -292,6 +323,12 @@ export declare class Loop {
      * every call was recorded. Rows are per tag: a call naming two tags is
      * counted under each, so never add them up as calls. One pipeline's share
      * is `capture` on its detail.
+     *
+     * The workspace's totals: `samples_recent` (recorded in the last 48
+     * hours), `feedback` (every sample by the feedback a person gave it: good,
+     * bad, corrected or unreviewed; they add up to `traces`) and `pipelines`
+     * (how many there are, and how many are training, waiting on you
+     * (`needs_you`), serving or paused).
      */
     stats(): Promise<LoopStats>;
     /**
@@ -299,11 +336,18 @@ export declare class Loop {
      * its attempt number, the version it became (null when it did not), its
      * base model, its win rate, what it cost -- and each day's feedback counted
      * by verdict. `rule_id` narrows the attempts to one pipeline; the feedback
-     * counts are the workspace's. `truncated` says older attempts exist beyond
-     * `limit`.
+     * counts are the workspace's. `samples` is the samples recorded each day
+     * over `days`, by their feedback today (`good`, `bad`, `corrected`,
+     * `unreviewed`): with `rule_id`, that pipeline's. `truncated` says older
+     * attempts exist beyond `limit`.
      */
     getMetricsHistory(params?: LoopMetricsHistoryParams): Promise<LoopMetricsHistory>;
-    /** Attempts, newest first. Filter by pipeline (`rule_id`) or by the state they are sitting in. */
+    /**
+     * Every pipeline's attempts, newest first. Filter by pipeline (`rule_id`),
+     * by the state they are sitting in, or by what became of them (`result`,
+     * as {@link listPipelineAttempts} takes it). Each row carries `attempt`:
+     * the attempt as its pipeline's own list says it, `explanation` included.
+     */
     listTrainingRuns(params?: TrainingRunListParams): Promise<TrainingRunListResponse>;
     /**
      * One run with its timeline, the addresses of everything it created, and
@@ -312,7 +356,11 @@ export declare class Loop {
      * `timeline` is written in the same transaction as each state change, so it
      * is the record of what actually happened rather than a reconstruction.
      * `available_actions` is the honest answer to "can I promote this": a button
-     * that cannot work should never be offered.
+     * that cannot work should never be offered. `explanation` is what became
+     * of it in plain words, as on the pipeline's attempts; `run.trained_on` is
+     * what it trained on, as it was then (its tags, conditions and each
+     * dataset at its revision); `traces_filter_attempt` true means
+     * `listTraces({ attempt: id, split })` lists its samples.
      */
     getTrainingRun(id: string): Promise<TrainingRunResponse>;
     /**
@@ -353,8 +401,9 @@ export declare class Loop {
      * The workspace's pipelines, a page at a time: its versions (the attempts
      * that became the live model), its latest 5 attempts (`attempts_total` says
      * how many; {@link listPipelineAttempts} pages through every one), which
-     * version is live, the attempt in flight, and what the pipeline is waiting
-     * for.
+     * version is live, the attempt in flight, what the pipeline is waiting
+     * for (`minimums`), and how many samples it has recorded (`samples_total`,
+     * `samples_recent` in the last 48 hours).
      *
      * `limit` is 100 when absent and at most 100; `q` matches the name, the tag
      * or the model (case-insensitive); `status` keeps the pipelines `training`,
@@ -375,6 +424,12 @@ export declare class Loop {
      * somebody does (`paused`, with `paused_reason` saying why; `complete` at
      * `max_versions`). The monthly safety limit is the platform's and has no
      * field: when it holds the pipeline, `next_reason` says so.
+     *
+     * What it trains on: its `tags`, the `datasets` it adds, and the
+     * `conditions` that narrow both. `auto_scoring` is the judge's scoring of
+     * its unreviewed samples (null where the platform does not score them):
+     * `message` is the service's sentence for it, and `minimums.picked_by_judge`
+     * how many of the samples it has the judge picked.
      */
     getPipeline(id: string): Promise<Pipeline>;
     /**
@@ -382,7 +437,8 @@ export declare class Loop {
      * it made, winner or not, each the shape `attempts` has on the pipeline
      * (with its `explanation` in plain words). `limit` is 50 when absent and at
      * most 200. Narrow by `result` (`became_version`, `not_better`,
-     * `inconclusive`, `failed`, `stopped`, `running` -- which includes one
+     * `inconclusive` -- tested, could not decide --, `not_tested` -- ended
+     * before it was tested --, `failed`, `stopped`, `running` -- which includes one
      * waiting for a decision -- or `waiting_for_review`) and by `model` (`main`,
      * or `second` for the second model's attempts); `total` follows the
      * filters.
@@ -423,12 +479,21 @@ export declare class Loop {
      * version, on what it has not learned plus a replay of what it already
      * learned (about half of what it trains on, picked at random); a winner
      * is merged in.
-     * What is charged is the GPU time the runs actually use; the platform's own
-     * per-run and monthly limits stop a run that goes wrong. `holdout.percent`
+     * What is charged is the GPU time the attempts actually use and, where the
+     * platform scores samples, the judge's scoring of unreviewed samples, shown
+     * on the attempt it was for; the platform's own per-run and monthly limits
+     * stop a run that goes wrong. `holdout.percent`
      * (5 to 20, default 5) is the share of its samples held back to judge each
      * attempt and never trained on. `training` sets any of its training
      * settings (`epochs`, `learning_rate`, `lora_rank`, `lora_alpha`,
      * `replay_percent`); left out, or null, every one is automatic.
+     * `datasets` (ids from {@link listDatasets}) adds their samples to what its
+     * tags select, and `conditions` narrows both. The judge's scoring of
+     * unreviewed samples is on unless `auto_scoring: { enabled: false }`.
+     * Where the platform scores samples (the pipeline's `auto_scoring` is not
+     * null), while the pipeline is short of the samples its next attempt
+     * needs, the judge scores samples nobody reviewed and the ones scoring 0.8
+     * or more are trained on; what it cost shows on the attempt it was for.
      */
     createPipeline(params: PipelineWriteRequest): Promise<PipelineMutationResponse>;
     /**
@@ -442,7 +507,12 @@ export declare class Loop {
      * `training: null` puts every one back to automatic; the next attempt
      * trains with them, and earlier attempts keep the settings they trained
      * with. Out of range is 422 `TRAINING_SETTING_OUT_OF_RANGE`, naming the
-     * field. Saving is the authorization, as on a create.
+     * field. `datasets` (`[]` takes every one off) and `conditions` (null
+     * clears them) change what it trains on from the next attempt: nothing is
+     * trained again, versions already made keep what they learned, and to make
+     * the live version forget samples, delete them ({@link deleteTrace}).
+     * `auto_scoring: { enabled: false }` stops the judge's scoring of
+     * unreviewed samples. Saving is the authorization, as on a create.
      */
     updatePipeline(id: string, params: PipelineWriteRequest): Promise<PipelineMutationResponse>;
     /**
@@ -479,6 +549,47 @@ export declare class Loop {
      * that brought no verdict of its own as a good example to learn from.
      */
     importIntoPipeline(id: string, params: PipelineImportParams): Promise<LoopImportResult>;
+    /** The workspace's datasets, the most recently changed first: `{ datasets, total, limit, offset }`. */
+    listDatasets(params?: LoopDatasetListParams): Promise<LoopDatasetListResponse>;
+    /**
+     * One dataset: its conditions, how many samples it selects now (`samples`,
+     * the same number `listTraces({ dataset: id })` totals) by feedback, how
+     * many are in a benchmark (never trained on), and the pipelines that train
+     * on it (`used_by`).
+     */
+    getDataset(id: string): Promise<LoopDataset>;
+    /**
+     * Save a dataset: a `name` (unique in the workspace, whatever its case:
+     * 409 `DATASET_NAME_TAKEN`) and any of its conditions. Nothing trains on
+     * it until a pipeline adds it.
+     */
+    createDataset(params: LoopDatasetCreate): Promise<LoopDataset>;
+    /**
+     * Change a dataset: only the fields sent change (null clears `origin`,
+     * `model`, `from` or `to`; `[]` clears `tags` or `feedback`). Pass
+     * `expected_revision` to be refused 409 `REVISION_MISMATCH` rather than
+     * overwrite someone else's edit. Every pipeline in `used_by` trains on the
+     * new selection from its next attempt: samples that join count as new
+     * samples, nothing is trained again, and versions already made keep what
+     * they learned.
+     */
+    updateDataset(id: string, params: LoopDatasetWrite): Promise<LoopDataset>;
+    /**
+     * Delete a dataset. Its samples stay. Every pipeline that trained on it
+     * (`removed_from`) stops taking its samples from its next attempt; versions
+     * already made keep what they learned. It cannot be undone.
+     */
+    deleteDataset(id: string): Promise<LoopDatasetDeleteResponse>;
+    /**
+     * Count what a set of conditions selects, before saving anything: the
+     * samples, by feedback, how many are in a benchmark, and the models that
+     * answered them. `datasets` adds saved datasets' samples, as a pipeline
+     * does, so a pipeline's tags, datasets and conditions can be counted in
+     * one call. The counts are before duplicates and over-long conversations
+     * are set aside; a pipeline's own `minimums` are the exact training
+     * numbers. It writes nothing.
+     */
+    previewDataset(params?: LoopDatasetPreviewParams): Promise<LoopDatasetPreview>;
     /** The pipeline's benchmarks in priority order (1 is the primary), the ones it stopped using, and its promotion policy. */
     listPipelineBenchmarks(id: string): Promise<PipelineBenchmarksResponse>;
     /** Add a benchmark to a pipeline, last unless `priority` says where. Every version from now on is replayed on it. */
@@ -504,7 +615,7 @@ export declare class Loop {
      * and `.b` are each attempt's mean and pass rate over those same
      * conversations -- the like-for-like figures; each side's `holdout` is over
      * its own, possibly larger, set. When the two were scored differently
-     * (another scoring model or rubric), `shared_sample.comparable` is false,
+     * (another judge or rubric), `shared_sample.comparable` is false,
      * `why` says so, no direction is given and `filter` is refused (400); a
      * conversation whose right answer changed between them has no direction
      * either. Numbers and ids only: read an answer with `listEvaluationItems`.