completion-kit 0.29.0 → 0.29.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d1da0c4306c55de2ce239393e22ce0512a051e9c85285165fa363b746bbf4a09
-  data.tar.gz: e9f2adb520737b42f5b9a51841cf32b0e5249bb74b5c13bedc3758c17dbca29a
+  metadata.gz: f314f2989362808e829e3f1bbbfc5718eca193aa4602ae4b4b3e72bc6c9251c1
+  data.tar.gz: b9322db963ff8bb00660e09309940f692cab0fff289c8069b3be68ecf81db429
 SHA512:
-  metadata.gz: 226c925a5fa2e048acbb20d0c71bf63c02af14102073875d7c42858da7e9a36c6a6f4ad87eb570a5a18d977e25a16e104ec95e37f2a1424f1ddadd3a7a3e0c69
-  data.tar.gz: 64638c834849310c7f818bd2db14bf0a682447e4b52409bd5f9656ec0fe14b732a08dea84d8c414adb36e509e807b7869945271aa6ee8aca8c7396c99a78c324
+  metadata.gz: bff1f44883f8a652ffde54da844bd626db029e432e91f5f79648b926b34ada993599b0a04176bfc842bd1c7da786dc9a05af5744779ba6e3fdca677e213639d1
+  data.tar.gz: bff0d10f9a3f7ccc857d52b5038ac06e4ccbc6e15e73195ff18c95a2f0223c4d223bdd17749f36d6933d7bb5925e3dd9e522cf4e012b26ac76a9c9b78c40a0c6

data/README.md

@@ -71,10 +71,33 @@ The engine mounts at `/completion_kit`. Generate / judge flows enqueue Active Jo
 
 ```erb
 <%= stylesheet_link_tag "completion_kit/application", media: "all" %>
-<%= javascript_include_tag "completion_kit/application", defer: true %>
+<%= javascript_include_tag "completion_kit/application", defer: true, nonce: true %>
 ```
 
-Without the JavaScript include, in-page behaviours silently fail: live tag-breadcrumb updates, relative-time ticking, CSV row hover-expand, model-refresh progress, focus-first-error, and local-time formatting.
+Without the JavaScript include, in-page behaviours silently fail: clickable table rows, dialogs, copy buttons, the API token reveal, the run form, live run status, live tag-breadcrumb updates, relative-time ticking, CSV row hover-expand, model-refresh progress, focus-first-error, and local-time formatting.
+
+**Content Security Policy.** The engine's pages have no inline event handlers or inline scripts, and the engine layout puts the request nonce on its script tags, so the pages work under a nonce-based `script-src`. The views still use inline `style` attributes, so leave `style-src` out of the nonce directives and allow `'unsafe-inline'` there for now.
+
+Use a nonce that lasts the whole session, not a fresh random one per request. Turbo Drive re-creates the scripts in each new page body with the nonce from the `csp-nonce` meta tag, but the browser keeps enforcing the policy header of the first page it loaded, so a per-request nonce blocks nonce scripts after the first Drive visit.
+
+Keep a random nonce in the session, and skip it for API controllers so cookie-less API and MCP calls get no session cookie, as the example below does. The Rails default, `->(request) { request.session.id.to_s }`, also lasts the session, but it is empty on a brand-new visitor's first response, so a policy that admits scripts only by nonce blocks that first page's scripts until the session exists.
+
+```ruby
+Rails.application.configure do
+  config.content_security_policy do |policy|
+    policy.default_src :self
+    policy.script_src  :self
+    policy.style_src   :self, :unsafe_inline
+  end
+
+  config.content_security_policy_nonce_generator = lambda do |request|
+    next if request.controller_instance.is_a?(ActionController::API)
+
+    request.session[:csp_nonce] ||= SecureRandom.base64(16)
+  end
+  config.content_security_policy_nonce_directives = %w(script-src)
+end
+```
 
 ## Providers
 
@@ -84,12 +107,14 @@ CompletionKit discovers available models from each provider's API automatically.
 |----------|----------|----------------|
 | **OpenAI** | `OPENAI_API_KEY` | GPT-5, GPT-4.1, GPT-4o, etc. |
 | **Anthropic** | `ANTHROPIC_API_KEY` | Claude Opus, Sonnet, Haiku |
-| **Ollama / local endpoint** | `OLLAMA_API_ENDPOINT` (default: `http://localhost:11434/v1`) | Any model you've `ollama pull`-ed, or any OpenAI-compatible local server (vLLM, LM Studio, llama.cpp) |
+| **Ollama / local endpoint** | `OLLAMA_API_ENDPOINT` (default: `http://localhost:11434/v1`), `OLLAMA_API_KEY` (optional) | Any model you've `ollama pull`-ed, or any OpenAI-compatible local server (vLLM, LM Studio, llama.cpp) |
 | **OpenRouter** | `OPENROUTER_API_KEY` | 100+ models from 30+ providers through one API key |
 | **Azure AI Foundry** | No env var — add via the Providers UI / a `ProviderCredential` (`api_key` + `api_endpoint` required, `api_version` optional) | Azure OpenAI deployments and Foundry projects |
 
 Set these as environment variables or configure them in the generated initializer. You can also add provider credentials through the web UI under Settings.
 
+The Ollama key from `OLLAMA_API_KEY` or `config.ollama_api_key` goes only to the Ollama endpoint from `OLLAMA_API_ENDPOINT` or `config.ollama_api_endpoint`, or to `http://localhost:11434/v1` when neither is set and `config.allow_loopback_endpoints` is on, which is the default. A provider credential with no key of its own gets that key only while it points at that same endpoint. Point it anywhere else and CompletionKit sends no key, so store the key on the credential itself. A key stored on a credential always goes to that credential's endpoint. Two endpoints match when their scheme, host, port and path match. The case of the scheme and host, a default port written out, a trailing slash and a trailing `/v1` make no difference. A blank stored key counts as no key.
+
 ### Encryption
 
 Provider API keys are encrypted at rest using [Active Record encryption](https://guides.rubyonrails.org/active_record_encryption.html). You need three encryption keys configured before the app will boot in production.
@@ -169,6 +194,26 @@ end
 
 Limiting uses `Rails.cache`. A shared cache store (Solid Cache, Redis) throttles accurately across multiple app instances; a per-process store still throttles each instance independently.
 
+## Input limits
+
+Uploads, prompt templates, and regex check patterns have limits, so one large or hostile input cannot tie up a worker. Tune them in the initializer:
+
+```ruby
+CompletionKit.configure do |c|
+  c.max_upload_bytes = 25 * 1024 * 1024  # largest dataset CSV or import file (default 25 MB)
+  c.max_template_bytes = 1024 * 1024     # largest prompt template (default 1 MB)
+  c.regex_timeout = 1.0                  # seconds a regex check may spend matching one row (default 1.0)
+end
+```
+
+A prompt whose template is over `max_template_bytes` fails validation when its template is saved. Prompts saved before a lower limit keep working until their template changes.
+
+The first line of a dataset's CSV is its header row. A dataset fails validation when its CSV is saved with a header row that is blank or longer than 64 KB. The 64 KB limit is fixed, not a setting.
+
+A dataset also fails validation when its CSV is saved with bytes that are not valid UTF-8, so export spreadsheets as UTF-8 CSV.
+
+A regex check that runs past its time limit fails that row and says it timed out. The limit is `regex_timeout`, or your app's `Regexp.timeout` when that is shorter, so the engine never loosens a stricter limit your app set. The engine sets the limit on each pattern and never changes the process-wide `Regexp.timeout`. Set `regex_timeout` to a finite number of seconds of at least `0.001`, or to `nil` to use only `Regexp.timeout`. Any other value, such as `0` or `Float::INFINITY`, raises an `ArgumentError` when the initializer runs.
+
 ## How it works
 
 <p align="center">

data/app/assets/javascripts/completion_kit/application.js

@@ -483,3 +483,386 @@ function ckInstallConfirm() {
 }
 document.addEventListener("turbo:load", ckInstallConfirm);
 ckInstallConfirm();
+
+var ckRowIgnoreSelector = "a, button, input, select, textarea, label, summary, form, dialog, [data-ck-row-ignore]";
+
+function ckRowHref(e) {
+  var target = e.target;
+  if (!target || !target.closest) return null;
+  var row = target.closest("[data-ck-href]");
+  if (!row) return null;
+  var inner = target.closest(ckRowIgnoreSelector);
+  if (inner && row.contains(inner)) return null;
+  return row.getAttribute("data-ck-href");
+}
+
+document.addEventListener("click", function(e) {
+  var href = ckRowHref(e);
+  if (!href) return;
+  if (e.metaKey || e.ctrlKey || e.shiftKey) {
+    window.open(href, "_blank");
+  } else if (window.Turbo && typeof window.Turbo.visit === "function") {
+    window.Turbo.visit(href);
+  } else {
+    window.location.assign(href);
+  }
+});
+
+document.addEventListener("auxclick", function(e) {
+  if (e.button !== 1) return;
+  var href = ckRowHref(e);
+  if (href) window.open(href, "_blank");
+});
+
+document.addEventListener("click", function(e) {
+  if (e.target && e.target.closest && e.target.closest("[data-ck-stop-propagation]")) e.stopPropagation();
+});
+
+function ckShowDialog(dialog) {
+  if (dialog && typeof dialog.showModal === "function" && !dialog.open) dialog.showModal();
+}
+
+document.addEventListener("click", function(e) {
+  var target = e.target;
+  if (!target || !target.closest) return;
+  var opener = target.closest("[data-ck-dialog-open]");
+  if (opener) {
+    ckShowDialog(document.getElementById(opener.getAttribute("data-ck-dialog-open")));
+    return;
+  }
+  var closer = target.closest("[data-ck-dialog-close]");
+  if (closer) {
+    var dialog = closer.closest("dialog");
+    if (dialog) dialog.close();
+    return;
+  }
+  if (target.matches("dialog[data-ck-dialog]")) target.close();
+});
+
+document.addEventListener("turbo:load", function() {
+  ckShowDialog(document.querySelector("dialog[data-ck-dialog-autoopen]"));
+});
+
+function ckFlashCopied(el) {
+  el.classList.add("ck-api-copy--done");
+  setTimeout(function() { el.classList.remove("ck-api-copy--done"); }, 1500);
+}
+
+function ckCopyText(el, text) {
+  navigator.clipboard.writeText(text).then(function() {
+    if (el.hasAttribute("data-ck-copy-feedback")) ckFlashCopied(el);
+  });
+}
+
+function ckToggleToken(el) {
+  var showing = el.textContent === el.dataset.masked;
+  el.textContent = showing ? el.dataset.real : el.dataset.masked;
+  el.setAttribute("aria-pressed", showing ? "true" : "false");
+  el.setAttribute("aria-label", showing ? "Hide API token" : "Reveal API token");
+  navigator.clipboard.writeText(el.dataset.real).then(function() { ckFlashCopied(el); });
+}
+
+function ckCopyExample(btn) {
+  var text = btn.closest(".ck-api-example").querySelector("pre").textContent;
+  var realToken = btn.dataset.realToken;
+  var displayToken = btn.dataset.displayToken;
+  if (realToken && displayToken) text = text.split(displayToken).join(realToken);
+  navigator.clipboard.writeText(text).then(function() { ckFlashCopied(btn); });
+}
+
+document.addEventListener("click", function(e) {
+  var target = e.target;
+  if (!target || !target.closest) return;
+  var toggle = target.closest("[data-ck-token-toggle]");
+  if (toggle) { ckToggleToken(toggle); return; }
+  var example = target.closest("[data-ck-copy-example]");
+  if (example) { ckCopyExample(example); return; }
+  var literal = target.closest("[data-ck-copy-text]");
+  if (literal) { ckCopyText(literal, literal.getAttribute("data-ck-copy-text")); return; }
+  var copier = target.closest("[data-ck-copy]");
+  if (!copier) return;
+  var source = document.getElementById(copier.getAttribute("data-ck-copy"));
+  if (source) ckCopyText(copier, source.textContent);
+});
+
+document.addEventListener("click", function(e) {
+  var target = e.target;
+  if (!target || !target.closest) return;
+  if (target.closest("[data-ck-refresh-models]")) {
+    ckRefreshModels();
+    return;
+  }
+  var refresher = target.closest("[data-ck-provider-refresh]");
+  if (!refresher) return;
+  e.preventDefault();
+  fetch(refresher.getAttribute("data-ck-provider-refresh"), {
+    method: "POST",
+    headers: { "X-CSRF-Token": document.querySelector('meta[name="csrf-token"]').getAttribute("content") }
+  });
+  ckStartDiscoveryPolling(8000);
+});
+
+document.addEventListener("click", function(e) {
+  var toggle = e.target && e.target.closest && e.target.closest("[data-ck-prompt-toggle]");
+  if (!toggle) return;
+  var text = document.getElementById(toggle.getAttribute("aria-controls"));
+  if (!text) return;
+  var expanded = text.classList.toggle("ck-prompt-preview__text--expanded");
+  toggle.firstChild.textContent = expanded ? "Show less" : "Show more";
+  toggle.setAttribute("aria-expanded", expanded ? "true" : "false");
+});
+
+function ckToggleMetricGroupFilter(btn) {
+  btn.classList.toggle("tag-mark--off");
+  var bar = btn.closest("[data-metric-tag-filter]");
+  var grid = bar && bar.parentElement.querySelector("[data-metric-checkboxes]");
+  if (!grid) return;
+  var active = Array.prototype.map.call(
+    bar.querySelectorAll(".ck-metric-tag-filter__chip:not(.tag-mark--off)"),
+    function(b) { return b.getAttribute("data-filter-tag"); }
+  );
+  grid.querySelectorAll(".ck-checkbox-label").forEach(function(lbl) {
+    var tags = (lbl.getAttribute("data-metric-tags") || "").split(",").filter(Boolean);
+    var visible = active.length === 0 || tags.some(function(t) { return active.indexOf(t) !== -1; });
+    lbl.style.display = visible ? "" : "none";
+  });
+}
+
+document.addEventListener("click", function(e) {
+  var target = e.target;
+  if (!target || !target.closest) return;
+  var chip = target.closest("[data-metric-tag-filter] [data-filter-tag]");
+  if (chip) {
+    ckToggleMetricGroupFilter(chip);
+    return;
+  }
+  var group = target.closest("[data-metric-group]");
+  if (group) ckToggleMetricGroup(group);
+});
+
+function ckUpdateRunForm() {
+  var promptEl = document.getElementById('run_prompt_id');
+  var judgeEl = document.getElementById('run_judge_model');
+  var judgeOnlyEl = document.getElementById('run_judge_only');
+  var judgeOnly = !!(judgeOnlyEl && judgeOnlyEl.checked);
+  var promptField = document.getElementById('prompt-field');
+  var outputColumnField = document.getElementById('output-column-field');
+  var outputColumnEl = document.getElementById('run_output_column');
+  if (promptField) promptField.hidden = judgeOnly;
+  if (outputColumnField) outputColumnField.hidden = !judgeOnly;
+  if (judgeOnly && promptEl) promptEl.value = '';
+
+  var prompt = promptEl ? promptEl.value : '';
+  var judge = judgeEl ? judgeEl.value : '';
+  var judgeListed = !!judgeEl && judgeEl.type !== 'hidden';
+  var metrics = document.querySelectorAll('input[name="run[metric_ids][]"]:checked');
+  var judgeField = document.getElementById('judge-field');
+  var metricsField = document.getElementById('metrics-field');
+  var judgeHint = document.getElementById('judge-hint');
+  var metricsHint = document.getElementById('metrics-hint');
+  var submitBtn = document.getElementById('run-submit');
+
+  if (judgeField) judgeField.className = 'ck-field';
+  if (metricsField) metricsField.className = 'ck-field';
+  if (judgeHint) judgeHint.textContent = '';
+  if (metricsHint) metricsHint.textContent = '';
+
+  var judgedMetrics = Array.prototype.filter.call(metrics, function(cb) {
+    var lbl = cb.closest('[data-needs-judge]');
+    return !!lbl && lbl.dataset.needsJudge === '1';
+  });
+  var judgeUnpaired = judgeListed && !!judge && metrics.length === 0;
+  var metricsUnpaired = judgeListed && !judge && judgedMetrics.length > 0;
+  if (judgeListed && judge && metrics.length > 0 && judgedMetrics.length === 0) {
+    if (judgeHint) judgeHint.textContent = 'Checks need no judge, so this run will not use it.';
+  } else if (judgeListed && judge && metrics.length > 0) {
+    if (judgeHint) judgeHint.textContent = judgeOnly ? 'Existing outputs will be judged automatically.' : 'Responses will be generated then judged automatically.';
+  } else if (judgeUnpaired) {
+    if (judgeField) judgeField.className = 'ck-field ck-field--info';
+    if (metricsField) metricsField.className = 'ck-field ck-field--info';
+    if (metricsHint) metricsHint.textContent = 'Select at least one metric or group to enable judging.';
+  } else if (metricsUnpaired) {
+    if (judgeField) judgeField.className = 'ck-field ck-field--info';
+    if (judgeHint) judgeHint.textContent = 'Select a judge model to enable judging.';
+  }
+
+  var datasetEl = document.getElementById('run_dataset_id');
+  var datasetHint = document.getElementById('dataset-hint');
+  var datasetField = document.getElementById('dataset-field');
+  var dataset = datasetEl ? datasetEl.value : '';
+  var selectedOption = promptEl ? promptEl.options[promptEl.selectedIndex] : null;
+  var hasVars = selectedOption && selectedOption.dataset.hasVariables === '1';
+  var promptVars = (selectedOption && selectedOption.dataset.variables ? selectedOption.dataset.variables.split(/,\s*/) : []).filter(Boolean);
+
+  var missingVars = [];
+  if (hasVars && dataset && datasetEl) {
+    var datasetOption = datasetEl.options[datasetEl.selectedIndex];
+    var headers = (datasetOption && datasetOption.dataset.headers ? datasetOption.dataset.headers.split(/,\s*/) : []).filter(Boolean);
+    missingVars = promptVars.filter(function(v) { return headers.indexOf(v) === -1; });
+  }
+
+  if (datasetField) datasetField.className = 'ck-field';
+  if (datasetHint) datasetHint.textContent = '';
+  if (missingVars.length > 0) {
+    if (datasetField) datasetField.className = 'ck-field ck-field--error';
+    if (datasetHint) datasetHint.textContent = 'Dataset is missing ' + (missingVars.length === 1 ? 'column' : 'columns') + ' the prompt needs: ' + missingVars.join(', ');
+  } else if (hasVars && !dataset) {
+    if (datasetField) datasetField.className = 'ck-field ck-field--info';
+    if (datasetHint) datasetHint.textContent = 'This prompt uses variables. Select a dataset to provide values.';
+  }
+
+  var expectedWarn = document.getElementById('expected-column-warn');
+  var expectedField = document.getElementById('expected-column-field');
+  var expectedColEl = document.getElementById('run_expected_column');
+  if (expectedField) expectedField.className = 'ck-field';
+  if (expectedWarn) expectedWarn.textContent = '';
+  var wantsExpected = false;
+  document.querySelectorAll('input[name="run[metric_ids][]"]:checked').forEach(function(cb) {
+    var lbl = cb.closest('[data-compare-expected]');
+    if (lbl && lbl.dataset.compareExpected === '1') wantsExpected = true;
+  });
+  if (wantsExpected && dataset && datasetEl) {
+    var expOption = datasetEl.options[datasetEl.selectedIndex];
+    var expHeaders = (expOption && expOption.dataset.headers ? expOption.dataset.headers.split(/,\s*/) : []).filter(Boolean);
+    var expCol = ((expectedColEl && expectedColEl.value) || 'expected_output').trim();
+    if (expHeaders.indexOf(expCol) === -1) {
+      if (expectedField) expectedField.className = 'ck-field ck-field--error';
+      if (expectedWarn) expectedWarn.textContent = "A selected check grades against each row's expected value, but this dataset has no \"" + expCol + "\" column, so every row would fail. Add the column or point this at the right one.";
+    }
+  }
+
+  var summary = document.getElementById('prompt-summary');
+  if (summary) {
+    if (selectedOption && selectedOption.value) {
+      var desc = selectedOption.dataset.description || '';
+      var tmpl = selectedOption.dataset.templatePreview || '';
+      var descEl = document.getElementById('prompt-summary-description');
+      descEl.textContent = desc;
+      descEl.hidden = !desc;
+      document.getElementById('prompt-summary-template').textContent = tmpl;
+      summary.hidden = false;
+    } else {
+      summary.hidden = true;
+    }
+  }
+
+  var valid;
+  if (judgeOnly) {
+    valid = !!dataset;
+    if (dataset && datasetEl && outputColumnEl) {
+      var headersJudge = (datasetEl.options[datasetEl.selectedIndex] && datasetEl.options[datasetEl.selectedIndex].dataset.headers ? datasetEl.options[datasetEl.selectedIndex].dataset.headers.split(/,\s*/) : []).filter(Boolean);
+      var col = (outputColumnEl.value || 'actual_output').trim();
+      if (col === '' || headersJudge.indexOf(col) === -1) {
+        valid = false;
+        if (datasetField) datasetField.className = 'ck-field ck-field--error';
+        if (datasetHint) datasetHint.textContent = "Dataset has no \"" + col + "\" column. Pick a different output column or dataset.";
+      }
+    } else if (!dataset) {
+      if (datasetField) datasetField.className = 'ck-field ck-field--info';
+      if (datasetHint) datasetHint.textContent = 'Skip generation and score responses you already have from a dataset column. Works with rubric metrics or deterministic checks.';
+    }
+  } else {
+    valid = prompt !== '';
+    if (hasVars && !dataset) valid = false;
+    if (missingVars.length > 0) valid = false;
+  }
+  if (judgeUnpaired || metricsUnpaired) valid = false;
+  if (submitBtn) submitBtn.disabled = !valid;
+
+  ckUpdateMetricGroupsState();
+}
+
+function ckToggleMetricGroup(button) {
+  var ids = (button.getAttribute('data-metric-ids') || '').split(',').filter(Boolean);
+  var allChecked = ids.every(function(id) {
+    var cb = document.getElementById('run_metric_' + id);
+    return cb && cb.checked;
+  });
+  ids.forEach(function(id) {
+    var cb = document.getElementById('run_metric_' + id);
+    if (cb) cb.checked = !allChecked;
+  });
+  ckUpdateRunForm();
+}
+
+function ckUpdateMetricGroupsState() {
+  document.querySelectorAll('[data-metric-group]').forEach(function(btn) {
+    var ids = (btn.getAttribute('data-metric-ids') || '').split(',').filter(Boolean);
+    if (ids.length === 0) return;
+    var allChecked = ids.every(function(id) {
+      var cb = document.getElementById('run_metric_' + id);
+      return cb && cb.checked;
+    });
+    btn.classList.toggle('ck-metric-group-pill--active', allChecked);
+  });
+}
+
+var ckRunFormChangeIds = ["run_judge_model", "run_prompt_id", "run_dataset_id", "run_judge_only"];
+
+document.addEventListener("change", function(e) {
+  var target = e.target;
+  if (!target) return;
+  if (ckRunFormChangeIds.indexOf(target.id) !== -1 || (target.type === "checkbox" && target.name === "run[metric_ids][]")) ckUpdateRunForm();
+});
+
+document.addEventListener("input", function(e) {
+  if (e.target && e.target.id === "run_output_column") ckUpdateRunForm();
+});
+
+function ckInitRunForm() {
+  if (document.getElementById("run_judge_only")) ckUpdateRunForm();
+}
+
+function ckDatasetHeaderPreview() {
+  var area = document.getElementById("dataset_csv_data");
+  var box = document.getElementById("dataset-header-preview");
+  var chips = document.getElementById("dataset-header-chips");
+  if (!area || !box || !chips) return;
+  var recognized = { expected_output: "answer key", actual_output: "pre-made output" };
+  var firstLine = (area.value.split(/\r?\n/).find(function(l) { return l.trim() !== ""; }) || "");
+  var headers = firstLine.split(",").map(function(h) { return h.trim().replace(/^"|"$/g, ""); }).filter(Boolean);
+  if (headers.length === 0) { box.hidden = true; chips.textContent = ""; return; }
+  box.hidden = false;
+  chips.textContent = "";
+  headers.forEach(function(h) {
+    var role = recognized[h];
+    var chip = document.createElement("span");
+    chip.className = "ck-header-chip" + (role ? " ck-header-chip--special" : "");
+    chip.textContent = role ? h + " · " + role : h;
+    chips.appendChild(chip);
+  });
+}
+
+document.addEventListener("input", function(e) {
+  if (e.target && e.target.id === "dataset_csv_data") ckDatasetHeaderPreview();
+});
+
+function ckInitPageForms() {
+  ckInitRunForm();
+  ckDatasetHeaderPreview();
+}
+
+document.addEventListener("turbo:load", ckInitPageForms);
+document.addEventListener("turbo:render", ckInitPageForms);
+
+function ckStartRunStatusPoll() {
+  clearTimeout(window.ckRunStatusKick);
+  clearInterval(window.ckRunStatusPoll);
+  window.ckRunStatusKick = null;
+  window.ckRunStatusPoll = null;
+  var marker = document.querySelector("[data-ck-run-status-url]");
+  if (!marker) return;
+  var url = marker.getAttribute("data-ck-run-status-url");
+  var refresh = function() {
+    fetch(url, { headers: { Accept: "text/vnd.turbo-stream.html" }, credentials: "same-origin" })
+      .then(function(r) { return r.ok ? r.text() : null; })
+      .then(function(html) { if (html) Turbo.renderStreamMessage(html); })
+      .catch(function() {});
+  };
+  window.ckRunStatusKick = setTimeout(refresh, 1000);
+  window.ckRunStatusPoll = setInterval(refresh, 15000);
+}
+
+document.addEventListener("turbo:load", ckStartRunStatusPoll);
+
+ckInitPageForms();

data/app/controllers/completion_kit/mcp_controller.rb

@@ -3,6 +3,7 @@ module CompletionKit
     PUBLIC_METHODS = %w[initialize notifications/initialized tools/list].freeze
 
     FOREIGN_KEY_MESSAGE = "This change references a record that does not exist or is still in use.".freeze
+    TIMEOUT_MESSAGE = "The input was too large or too complex to process.".freeze
 
     skip_before_action :authenticate_api!, only: [:stream, :handle], raise: false
 
@@ -47,6 +48,9 @@ module CompletionKit
     rescue ActiveRecord::InvalidForeignKey => e
       Rails.error.report(e, handled: true, context: { controller: "CompletionKit::McpController" })
       render json: jsonrpc_error(request_body.dig("id"), -32602, FOREIGN_KEY_MESSAGE), status: :ok
+    rescue Regexp::TimeoutError => e
+      Rails.error.report(e, handled: true, context: { controller: "CompletionKit::McpController" })
+      render json: jsonrpc_error(request_body.dig("id"), -32602, TIMEOUT_MESSAGE), status: :ok
     rescue StandardError => e
       Rails.error.report(e, handled: true, context: { controller: "CompletionKit::McpController" })
       render json: jsonrpc_error(request_body.dig("id"), -32603, "Internal error"), status: :ok

data/app/models/completion_kit/dataset.rb

@@ -2,14 +2,17 @@ module CompletionKit
   class Dataset < ApplicationRecord
     include CompletionKit::Taggable
 
+    HEADER_ROW_LIMIT = 64 * 1024
+
     has_many :runs, dependent: :destroy
 
     validates :name, presence: true
-    validates :csv_data, presence: true
+    validate :csv_data_present
     validate :csv_data_within_size_limit
+    validate :header_row_usable, if: :will_save_change_to_csv_data?
 
     def csv_data_within_size_limit
-      return if csv_data.blank?
+      return if csv_data_blank?
       return if csv_data.bytesize <= CompletionKit.config.max_upload_bytes
 
       errors.add(:csv_data, "is too large (limit #{CompletionKit.config.max_upload_bytes / (1024 * 1024)} MB)")
@@ -37,12 +40,58 @@ module CompletionKit
     end
 
     def headers
-      return [] if csv_data.blank?
+      row = header_row
+      return [] if row.nil?
 
       require "csv"
-      ::CSV.parse(csv_data.lines.first.to_s).first.to_a.map(&:to_s).map(&:strip)
+      ::CSV.parse(row).first.to_a.map(&:to_s).map(&:strip)
     rescue ::CSV::MalformedCSVError
       []
     end
+
+    def any_rows?
+      stop = header_row_end
+      return false unless stop
+
+      stop += 1 if csv_data.byteslice(stop, 2) == "\r\n"
+      csv_data.bytesize > stop + 1
+    end
+
+    private
+
+    def csv_data_present
+      errors.add(:csv_data, :blank) if csv_data_blank?
+    end
+
+    def csv_data_blank?
+      csv_data.to_s.empty? || (csv_data.valid_encoding? && header_row.blank? && csv_data.lstrip.empty?)
+    end
+
+    def header_row
+      return if csv_data.to_s.empty?
+
+      stop = header_row_end
+      return csv_data.byteslice(0, stop) if stop
+
+      csv_data if csv_data.bytesize <= HEADER_ROW_LIMIT
+    end
+
+    def header_row_end
+      head = csv_data.to_s.byteslice(0, HEADER_ROW_LIMIT + 1)
+      [head.byteindex("\n"), head.byteindex("\r")].compact.min
+    end
+
+    def header_row_usable
+      return if csv_data_blank?
+
+      row = header_row
+      if !csv_data.valid_encoding?
+        errors.add(:csv_data, "is not valid UTF-8")
+      elsif row.nil?
+        errors.add(:csv_data, "has a header row longer than #{HEADER_ROW_LIMIT / 1024} KB")
+      elsif row.blank?
+        errors.add(:csv_data, "has a blank header row")
+      end
+    end
   end
 end

data/app/models/completion_kit/metric.rb

@@ -205,7 +205,7 @@ module CompletionKit
     end
 
     def validate_check_pattern(config)
-      Regexp.new(config["pattern"].to_s)
+      CompletionKit::Checks::Regex.build(config)
     rescue RegexpError
       add_check_error("needs #{check_label("pattern")} to be a valid regular expression", "pattern")
     end

data/app/models/completion_kit/prompt.rb

@@ -9,6 +9,7 @@ module CompletionKit
 
     validates :name, presence: true
     validates :template, presence: true
+    validate :template_within_size_limit, if: :will_save_change_to_template?
     validates :llm_model, presence: true
     validate :llm_model_usable_for_generation, if: :llm_model_newly_selected?
     validates :family_key, presence: true
@@ -97,6 +98,13 @@ module CompletionKit
 
     private
 
+    def template_within_size_limit
+      limit = CompletionKit.config.max_template_bytes
+      return if template.to_s.bytesize <= limit
+
+      errors.add(:template, "is too large (limit #{ActiveSupport::NumberHelper.number_to_human_size(limit)})")
+    end
+
     def llm_model_newly_selected?
       return false if llm_model.blank?
       return llm_model != cloned_from_llm_model if new_record? && cloned_from_llm_model.present?

data/app/models/completion_kit/run.rb

@@ -336,7 +336,7 @@ module CompletionKit
         return fail_with_summary!("Cannot start a run in state \"#{status}\". Use rerun to create a fresh copy, or retry_failures / regrade to work with the existing responses.")
       end
 
-      return fail_with_summary!("Dataset has no rows") if dataset && dataset.row_count.zero?
+      return fail_with_summary!("Dataset has no rows") if dataset && !dataset.any_rows?
 
       if judge_only?
         column = output_column.presence || "actual_output"

data/app/services/completion_kit/api_config.rb

@@ -10,6 +10,7 @@ module CompletionKit
     def self.for_provider(provider_name)
       provider = provider_name.to_s
       stored = ProviderCredential.find_by(provider: provider)&.config_hash || {}
+      stored = stored.except(:api_key) if provider == "ollama" && stored[:api_key].blank?
 
       defaults = if CompletionKit.config.tenant_scope
                    PROVIDERS.include?(provider) ? { provider: provider } : {}
@@ -22,7 +23,7 @@ module CompletionKit
                    when "ollama"
                      {
                        provider: "ollama",
-                       api_key: CompletionKit.config.ollama_api_key || ENV["OLLAMA_API_KEY"],
+                       api_key: OllamaClient.fallback_api_key(stored[:api_endpoint] || OllamaClient.configured_endpoint),
                        api_endpoint: CompletionKit.config.ollama_api_endpoint || ENV["OLLAMA_API_ENDPOINT"]
                      }
                    when "openrouter"

data/app/services/completion_kit/checks/regex.rb

@@ -1,17 +1,27 @@
 module CompletionKit
   module Checks
     class Regex
-      def call(target, config)
+      def self.build(config)
         options = 0
         options |= Regexp::IGNORECASE unless Flag.on?(config, "case_sensitive")
         options |= Regexp::MULTILINE if Flag.on?(config, "multiline")
-        pattern = Regexp.new(config["pattern"].to_s, options)
+        Regexp.new(config["pattern"].to_s, options, timeout: timeout)
+      end
+
+      def self.timeout
+        [CompletionKit.config.regex_timeout, Regexp.timeout].compact.min
+      end
+
+      def call(target, config)
+        pattern = self.class.build(config)
 
         if pattern.match?(target.to_s)
           Result.new(passed: true, detail: "matched /#{config["pattern"]}/")
         else
           Result.new(passed: false, detail: "no match for /#{config["pattern"]}/")
         end
+      rescue Regexp::TimeoutError
+        Result.new(passed: false, detail: "timed out after #{pattern.timeout || Regexp.timeout}s matching /#{config["pattern"]}/")
       rescue RegexpError => e
         Result.new(passed: false, detail: "invalid pattern: #{e.message}")
       end