completion-kit 0.5.29 → 0.5.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +41 -4
- data/app/assets/javascripts/completion_kit/application.js +6 -0
- data/app/assets/stylesheets/completion_kit/application.css +95 -0
- data/app/controllers/completion_kit/application_controller.rb +7 -0
- data/app/controllers/completion_kit/dashboard_controller.rb +22 -0
- data/app/controllers/completion_kit/onboarding_controller.rb +4 -6
- data/app/services/completion_kit/onboarding/concepts.rb +32 -0
- data/app/views/completion_kit/dashboard/show.html.erb +91 -0
- data/app/views/completion_kit/onboarding/_concept.html.erb +3 -0
- data/app/views/completion_kit/onboarding/show.html.erb +1 -0
- data/app/views/layouts/completion_kit/application.html.erb +8 -8
- data/config/routes.rb +1 -0
- data/lib/completion_kit/version.rb +1 -1
- metadata +5 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 3edbdacce1ca9737b2e7c3c93f2f7cfd27fbea96b2d55d470c8f5dfa7299d247
|
|
4
|
+
data.tar.gz: 8dc9c09b93800fbb7262015a9cf340030bb8e90dad8bda5ca1a97579a1987173
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 21a65c71afb3769bdbe8434ea71bf9ee6101ac0f47377080f4bfda4b8eb2acb7e4d2631d41202493eec619e3db44a13827fc46a568a1e134fcd836d2182ae5a6
|
|
7
|
+
data.tar.gz: 77c8534e4842727910628c18672918983d21c655ebdcb3998a70a3bfe0578257c3d41aeee200dc4841f505d74d2b097aa0ce1b493456e015f47c9438e5e8d622
|
data/README.md
CHANGED
|
@@ -18,19 +18,19 @@ It's the difference between "this prompt seems to work" and "this prompt scores
|
|
|
18
18
|
|
|
19
19
|
> **Just want to use it?** [CompletionKit Cloud](https://completionkit.com) is the same engine, fully hosted — zero install, no Rails ops, plans at [completionkit.com/pricing](https://completionkit.com/pricing).
|
|
20
20
|
|
|
21
|
-

|
|
22
22
|
|
|
23
23
|
## Three ways to run it
|
|
24
24
|
|
|
25
|
-
Same engine, same UI, same REST API and MCP server — pick the deployment that fits.
|
|
25
|
+
Same engine, same UI, same REST API and MCP server — pick the deployment that fits. The first two are stack-agnostic: you run CompletionKit as a product and talk to it over HTTP and MCP, whatever language your own app is written in. The third is for teams already building on Rails.
|
|
26
26
|
|
|
27
27
|
### 1. Hosted — [completionkit.com](https://completionkit.com) (recommended)
|
|
28
28
|
|
|
29
29
|
The fastest path. Sign up and you're running on the same engine you'd self-host, without touching a Rails app. No `db:migrate`, no Puma, no Solid Queue, no provider key management — multi-tenant workspaces, your team logs in, you go. Plans at [completionkit.com/pricing](https://completionkit.com/pricing).
|
|
30
30
|
|
|
31
|
-
### 2. Self-hosted — the bundled standalone
|
|
31
|
+
### 2. Self-hosted — the bundled standalone app
|
|
32
32
|
|
|
33
|
-
Run it on your own infra.
|
|
33
|
+
Run it on your own infra as a self-contained product. There's nothing to integrate and no Ruby to write — once it's up, you drive everything through the web UI, the REST API, and the MCP server, from whatever stack your own app is built in. It needs Postgres and a host that can run the app (Fly, Render, Heroku, Docker, …).
|
|
34
34
|
|
|
35
35
|
```bash
|
|
36
36
|
git clone https://github.com/homemade-software-inc/completion-kit.git
|
|
@@ -147,6 +147,12 @@ Limiting uses `Rails.cache`. A shared cache store (Solid Cache, Redis) throttles
|
|
|
147
147
|
|
|
148
148
|
## How it works
|
|
149
149
|
|
|
150
|
+
<p align="center">
|
|
151
|
+
<img src="https://raw.githubusercontent.com/homemade-software-inc/completion-kit/main/docs/diagrams/workflow.png" alt="CompletionKit workflow: a prompt and a dataset feed a run against a model, an LLM judge scores each output on your rubric, low scores drive an AI-suggested rewrite, and the new prompt version re-runs so you can compare" width="820" />
|
|
152
|
+
</p>
|
|
153
|
+
|
|
154
|
+
It's a loop. Each pass leaves you with a score you can compare against the last one.
|
|
155
|
+
|
|
150
156
|
1. **Create a prompt** with `{{variable}}` placeholders
|
|
151
157
|
2. **Upload a dataset.** A CSV where column headers match the variable names.
|
|
152
158
|
3. **Run it** against a model and score outputs with an LLM judge against criteria you define.
|
|
@@ -217,6 +223,37 @@ bin/rails db:migrate
|
|
|
217
223
|
git add db/migrate/ && git commit -m "install new engine migration"
|
|
218
224
|
```
|
|
219
225
|
|
|
226
|
+
### Docker
|
|
227
|
+
|
|
228
|
+
The standalone app ships a `Dockerfile`, so you can self-host it without a Ruby toolchain on the host. Build with the **repository root** as the context — the app depends on the engine source alongside it:
|
|
229
|
+
|
|
230
|
+
```bash
|
|
231
|
+
docker build -f standalone/Dockerfile -t completion-kit .
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
CompletionKit needs a Rails secret (`SECRET_KEY_BASE`) and three Active Record encryption keys. With Docker there's no Rails toolchain on the host to run `bin/rails db:encryption:init`, so generate them with `openssl`. Generate them **once** and keep them stable — if the encryption keys change, provider credentials already stored in the database can no longer be decrypted. Write everything to an env file:
|
|
235
|
+
|
|
236
|
+
```bash
|
|
237
|
+
cat > completion-kit.env <<EOF
|
|
238
|
+
DATABASE_URL=postgres://user:pass@host/completionkit
|
|
239
|
+
SECRET_KEY_BASE=$(openssl rand -hex 64)
|
|
240
|
+
COMPLETION_KIT_ENCRYPTION_PRIMARY_KEY=$(openssl rand -hex 32)
|
|
241
|
+
COMPLETION_KIT_ENCRYPTION_DETERMINISTIC_KEY=$(openssl rand -hex 32)
|
|
242
|
+
COMPLETION_KIT_ENCRYPTION_KEY_DERIVATION_SALT=$(openssl rand -hex 32)
|
|
243
|
+
EOF
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
`openssl rand` runs as the file is written, so each line gets a real random value. Keep `completion-kit.env` out of version control and back it up somewhere safe.
|
|
247
|
+
|
|
248
|
+
Run the web process and a job worker from the same image, both pointed at that file:
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
docker run -d -p 3000:3000 --env-file completion-kit.env completion-kit
|
|
252
|
+
docker run -d --env-file completion-kit.env completion-kit ./bin/jobs
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
Both processes must share the same `SECRET_KEY_BASE` and encryption keys — the single env file guarantees that. The web container runs `db:prepare` on boot, so migrations apply on first start and on every deploy.
|
|
256
|
+
|
|
220
257
|
## Multi-tenant host apps (advanced)
|
|
221
258
|
|
|
222
259
|
For hosts that mount CompletionKit in a multi-tenant app, two optional hooks scope engine records per tenant without forking the engine:
|
|
@@ -158,3 +158,9 @@ document.addEventListener("turbo:before-stream-render", function(event) {
|
|
|
158
158
|
if (status) { status.textContent = 'Models updated.'; setTimeout(function() { status.textContent = ' '; }, 3000); }
|
|
159
159
|
}
|
|
160
160
|
});
|
|
161
|
+
|
|
162
|
+
document.addEventListener("click", function(e) {
|
|
163
|
+
document.querySelectorAll("details.ck-nav-menu[open], details.ck-settings-menu[open], details.ck-flyout[open]").forEach(function(menu) {
|
|
164
|
+
if (!menu.contains(e.target)) menu.removeAttribute("open");
|
|
165
|
+
});
|
|
166
|
+
});
|
|
@@ -118,6 +118,13 @@ form.button_to {
|
|
|
118
118
|
padding: 0.75rem 0;
|
|
119
119
|
}
|
|
120
120
|
|
|
121
|
+
@media (min-width: 641px) {
|
|
122
|
+
.ck-topbar__inner .ck-nav {
|
|
123
|
+
position: relative;
|
|
124
|
+
top: 8px;
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
|
|
121
128
|
/* Mobile navigation — a <details> hamburger. On desktop the trigger is
|
|
122
129
|
hidden and the nav shows inline; at the mobile breakpoint the nav
|
|
123
130
|
collapses behind the trigger and opens as a dropdown panel. The two
|
|
@@ -4345,6 +4352,16 @@ a.tag-mark {
|
|
|
4345
4352
|
.ck-settings-menu__trigger::-webkit-details-marker {
|
|
4346
4353
|
display: none;
|
|
4347
4354
|
}
|
|
4355
|
+
.ck-settings-menu__trigger svg {
|
|
4356
|
+
display: block;
|
|
4357
|
+
width: 18px;
|
|
4358
|
+
height: 18px;
|
|
4359
|
+
}
|
|
4360
|
+
.ck-settings-menu__trigger:focus-visible {
|
|
4361
|
+
outline: none;
|
|
4362
|
+
border-color: var(--ck-accent);
|
|
4363
|
+
color: var(--ck-accent);
|
|
4364
|
+
}
|
|
4348
4365
|
.ck-settings-menu__panel {
|
|
4349
4366
|
position: absolute;
|
|
4350
4367
|
right: 0;
|
|
@@ -4374,6 +4391,16 @@ a.tag-mark {
|
|
|
4374
4391
|
.ck-settings-menu__item:hover {
|
|
4375
4392
|
background: var(--ck-surface-hover);
|
|
4376
4393
|
}
|
|
4394
|
+
.ck-settings-menu__panel form {
|
|
4395
|
+
margin: 0;
|
|
4396
|
+
}
|
|
4397
|
+
.ck-settings-menu__panel button.ck-settings-menu__item {
|
|
4398
|
+
width: 100%;
|
|
4399
|
+
text-align: left;
|
|
4400
|
+
background: none;
|
|
4401
|
+
border: 0;
|
|
4402
|
+
cursor: pointer;
|
|
4403
|
+
}
|
|
4377
4404
|
|
|
4378
4405
|
/* Settings page kicker — small label above the page title on settings pages */
|
|
4379
4406
|
.ck-settings-kicker {
|
|
@@ -4961,3 +4988,71 @@ a.tag-mark {
|
|
|
4961
4988
|
max-height: 60vh;
|
|
4962
4989
|
}
|
|
4963
4990
|
}
|
|
4991
|
+
|
|
4992
|
+
.ck-concept {
|
|
4993
|
+
position: relative;
|
|
4994
|
+
display: inline;
|
|
4995
|
+
}
|
|
4996
|
+
.ck-concept__toggle {
|
|
4997
|
+
margin: 0 0 0 0.15rem;
|
|
4998
|
+
padding: 0;
|
|
4999
|
+
border: 0;
|
|
5000
|
+
background: none;
|
|
5001
|
+
font: inherit;
|
|
5002
|
+
line-height: 0;
|
|
5003
|
+
color: var(--ck-dim);
|
|
5004
|
+
cursor: pointer;
|
|
5005
|
+
}
|
|
5006
|
+
.ck-concept__toggle:hover,
|
|
5007
|
+
.ck-concept__toggle:focus {
|
|
5008
|
+
color: var(--ck-accent);
|
|
5009
|
+
}
|
|
5010
|
+
.ck-concept__icon {
|
|
5011
|
+
width: 0.92em;
|
|
5012
|
+
height: 0.92em;
|
|
5013
|
+
vertical-align: -0.12em;
|
|
5014
|
+
}
|
|
5015
|
+
.ck-concept__pop {
|
|
5016
|
+
display: none;
|
|
5017
|
+
position: absolute;
|
|
5018
|
+
top: calc(100% + 0.4rem);
|
|
5019
|
+
left: 0;
|
|
5020
|
+
z-index: 40;
|
|
5021
|
+
width: 19rem;
|
|
5022
|
+
max-width: calc(100vw - 1.5rem);
|
|
5023
|
+
padding: 0.7rem 0.8rem;
|
|
5024
|
+
background: var(--ck-bg-strong);
|
|
5025
|
+
border: 1px solid var(--ck-line-strong);
|
|
5026
|
+
border-radius: var(--ck-radius);
|
|
5027
|
+
box-shadow: 0 16px 34px rgba(0, 0, 0, 0.5);
|
|
5028
|
+
white-space: normal;
|
|
5029
|
+
}
|
|
5030
|
+
.ck-concept__toggle:hover + .ck-concept__pop,
|
|
5031
|
+
.ck-concept__toggle:focus + .ck-concept__pop,
|
|
5032
|
+
.ck-concept__pop:hover {
|
|
5033
|
+
display: block;
|
|
5034
|
+
}
|
|
5035
|
+
.ck-concept__name {
|
|
5036
|
+
display: block;
|
|
5037
|
+
margin-bottom: 0.3rem;
|
|
5038
|
+
font-family: var(--ck-mono);
|
|
5039
|
+
font-size: 0.72rem;
|
|
5040
|
+
font-weight: 700;
|
|
5041
|
+
letter-spacing: 0.06em;
|
|
5042
|
+
text-transform: uppercase;
|
|
5043
|
+
color: var(--ck-text);
|
|
5044
|
+
}
|
|
5045
|
+
.ck-concept__body {
|
|
5046
|
+
display: block;
|
|
5047
|
+
font-size: 0.8rem;
|
|
5048
|
+
line-height: 1.5;
|
|
5049
|
+
color: var(--ck-muted);
|
|
5050
|
+
}
|
|
5051
|
+
@media (max-width: 640px) {
|
|
5052
|
+
.ck-concept__pop {
|
|
5053
|
+
position: fixed;
|
|
5054
|
+
inset: auto 0.75rem 0.75rem 0.75rem;
|
|
5055
|
+
width: auto;
|
|
5056
|
+
max-width: none;
|
|
5057
|
+
}
|
|
5058
|
+
}
|
|
@@ -3,12 +3,19 @@ module CompletionKit
|
|
|
3
3
|
helper Heroicons::IconsHelper
|
|
4
4
|
layout "completion_kit/application"
|
|
5
5
|
|
|
6
|
+
ONBOARDING_DISMISS_COOKIE = :ck_onboarding_dismissed
|
|
7
|
+
|
|
6
8
|
rate_limit to: CompletionKit.config.web_rate_limit, within: 1.minute,
|
|
7
9
|
with: -> { render plain: "Rate limit exceeded. Please slow down.", status: :too_many_requests }
|
|
8
10
|
before_action :authenticate_completion_kit!
|
|
9
11
|
|
|
10
12
|
private
|
|
11
13
|
|
|
14
|
+
def workspace_ready?
|
|
15
|
+
CompletionKit::Onboarding::Checklist.new.complete? ||
|
|
16
|
+
cookies[ONBOARDING_DISMISS_COOKIE].present?
|
|
17
|
+
end
|
|
18
|
+
|
|
12
19
|
def authenticate_completion_kit!
|
|
13
20
|
cfg = CompletionKit.config
|
|
14
21
|
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
module CompletionKit
|
|
2
|
+
class DashboardController < ApplicationController
|
|
3
|
+
def show
|
|
4
|
+
return redirect_to(onboarding_path) unless workspace_ready?
|
|
5
|
+
|
|
6
|
+
@prompt_count = Prompt.current_versions.count
|
|
7
|
+
@run_count = Run.count
|
|
8
|
+
@dataset_count = Dataset.count
|
|
9
|
+
@metric_count = Metric.count
|
|
10
|
+
@recent_runs = Run.order(created_at: :desc).limit(5)
|
|
11
|
+
|
|
12
|
+
return unless @run_count > 5
|
|
13
|
+
|
|
14
|
+
@activity = DashboardStats.activity
|
|
15
|
+
@worst_metric = DashboardStats.worst_metric(since: 7.days.ago)
|
|
16
|
+
@failures = DashboardStats.failures(since: 7.days.ago)
|
|
17
|
+
@ignored_metrics = DashboardDismissal.metrics
|
|
18
|
+
@ignored_failures = DashboardDismissal.failures
|
|
19
|
+
@prompt_changes = DashboardStats.prompt_changes
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
@@ -1,18 +1,16 @@
|
|
|
1
1
|
module CompletionKit
|
|
2
2
|
class OnboardingController < ApplicationController
|
|
3
|
-
DISMISS_COOKIE = :ck_onboarding_dismissed
|
|
4
|
-
|
|
5
3
|
def show
|
|
6
|
-
cookies.delete(
|
|
4
|
+
cookies.delete(ONBOARDING_DISMISS_COOKIE) if params[:reset]
|
|
7
5
|
@checklist = Onboarding::Checklist.new
|
|
8
6
|
return if params[:reset]
|
|
9
7
|
|
|
10
|
-
redirect_to
|
|
8
|
+
redirect_to dashboard_path if workspace_ready?
|
|
11
9
|
end
|
|
12
10
|
|
|
13
11
|
def dismiss
|
|
14
|
-
cookies[
|
|
15
|
-
redirect_to
|
|
12
|
+
cookies[ONBOARDING_DISMISS_COOKIE] = { value: "1", expires: 1.year.from_now, httponly: true }
|
|
13
|
+
redirect_to dashboard_path, notice: "Setup skipped. Pick it back up from Settings → Getting started any time."
|
|
16
14
|
end
|
|
17
15
|
|
|
18
16
|
def sample_data
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
module CompletionKit
|
|
2
|
+
module Onboarding
|
|
3
|
+
module Concepts
|
|
4
|
+
DEFINITIONS = {
|
|
5
|
+
credential: {
|
|
6
|
+
name: "Provider Credential",
|
|
7
|
+
definition: "An API key for a model provider such as OpenAI or Anthropic. Encrypted at rest and never returned through the API."
|
|
8
|
+
},
|
|
9
|
+
dataset: {
|
|
10
|
+
name: "Dataset",
|
|
11
|
+
definition: "A CSV of real inputs. Each row becomes one test case."
|
|
12
|
+
},
|
|
13
|
+
prompt: {
|
|
14
|
+
name: "Prompt",
|
|
15
|
+
definition: "A versioned template with {{variable}} placeholders. Editing a prompt that has already been run creates a new version, so past results stay reproducible."
|
|
16
|
+
},
|
|
17
|
+
run: {
|
|
18
|
+
name: "Run",
|
|
19
|
+
definition: "One execution of a prompt against a dataset. Stores every output and the judge's scores."
|
|
20
|
+
},
|
|
21
|
+
response: {
|
|
22
|
+
name: "Response",
|
|
23
|
+
definition: "The model's output for a single dataset row, with the judge's reviews attached."
|
|
24
|
+
},
|
|
25
|
+
metric: {
|
|
26
|
+
name: "Metric",
|
|
27
|
+
definition: "An evaluation dimension with its own 1-5 rubric. The LLM judge scores every response against it."
|
|
28
|
+
}
|
|
29
|
+
}.freeze
|
|
30
|
+
end
|
|
31
|
+
end
|
|
32
|
+
end
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
<section class="ck-page-header">
|
|
2
|
+
<div>
|
|
3
|
+
<p class="ck-kicker">Dashboard</p>
|
|
4
|
+
<h1 class="ck-title">Prompt Testing Lab</h1>
|
|
5
|
+
</div>
|
|
6
|
+
<div class="ck-actions">
|
|
7
|
+
<%= link_to "New run →", new_run_path, class: "ck-button ck-button--secondary" %>
|
|
8
|
+
</div>
|
|
9
|
+
</section>
|
|
10
|
+
|
|
11
|
+
<nav class="ck-statbar ck-rise" aria-label="Workspace totals">
|
|
12
|
+
<%= link_to prompts_path, class: "ck-statbar__item" do %>
|
|
13
|
+
<span class="ck-statbar__label">Prompts</span>
|
|
14
|
+
<span class="ck-statbar__value"><%= @prompt_count %></span>
|
|
15
|
+
<% end %>
|
|
16
|
+
<%= link_to metrics_path, class: "ck-statbar__item" do %>
|
|
17
|
+
<span class="ck-statbar__label">Metrics</span>
|
|
18
|
+
<span class="ck-statbar__value"><%= @metric_count %></span>
|
|
19
|
+
<% end %>
|
|
20
|
+
<%= link_to datasets_path, class: "ck-statbar__item" do %>
|
|
21
|
+
<span class="ck-statbar__label">Datasets</span>
|
|
22
|
+
<span class="ck-statbar__value"><%= @dataset_count %></span>
|
|
23
|
+
<% end %>
|
|
24
|
+
<%= link_to runs_path, class: "ck-statbar__item" do %>
|
|
25
|
+
<span class="ck-statbar__label">Runs</span>
|
|
26
|
+
<span class="ck-statbar__value"><%= @run_count %></span>
|
|
27
|
+
<% end %>
|
|
28
|
+
</nav>
|
|
29
|
+
|
|
30
|
+
<% if @activity %>
|
|
31
|
+
<div class="ck-grid ck-grid--cards ck-grid--cards-3 ck-pulse-grid">
|
|
32
|
+
<div class="ck-card ck-stat-card ck-rise" style="--rise-delay: 60ms;">
|
|
33
|
+
<p class="ck-kicker">Activity · last 14 days</p>
|
|
34
|
+
<% activity_max = @activity.map { |d| d[:count] }.max %>
|
|
35
|
+
<div class="ck-sparkline" role="img" aria-label="<%= @activity.sum { |d| d[:count] } %> runs over the last 14 days">
|
|
36
|
+
<% @activity.each do |day| %>
|
|
37
|
+
<span class="ck-sparkline__bar<%= ' is-peak' if activity_max.to_i.positive? && day[:count] == activity_max %>"
|
|
38
|
+
style="height: <%= activity_max.to_i.zero? ? 0 : (day[:count] * 100.0 / activity_max).round %>%"
|
|
39
|
+
title="<%= day[:date].strftime('%b %-d') %>: <%= day[:count] %> run<%= 's' unless day[:count] == 1 %>"></span>
|
|
40
|
+
<% end %>
|
|
41
|
+
</div>
|
|
42
|
+
<p class="ck-stat-card__foot">
|
|
43
|
+
<span class="ck-stat-card__figure"><%= @activity.sum { |d| d[:count] } %></span> runs in the window
|
|
44
|
+
</p>
|
|
45
|
+
</div>
|
|
46
|
+
|
|
47
|
+
<%= render "completion_kit/dashboard/worst_metric_card",
|
|
48
|
+
worst_metric: @worst_metric, ignored_metrics: @ignored_metrics %>
|
|
49
|
+
|
|
50
|
+
<%= render "completion_kit/dashboard/failures_card",
|
|
51
|
+
failures: @failures, ignored_failures: @ignored_failures %>
|
|
52
|
+
</div>
|
|
53
|
+
|
|
54
|
+
<div class="ck-card ck-card--spaced ck-rise" style="--rise-delay: 240ms;">
|
|
55
|
+
<p class="ck-kicker">Prompt changes · version over version</p>
|
|
56
|
+
<% if @prompt_changes.any? %>
|
|
57
|
+
<ul class="ck-improvements">
|
|
58
|
+
<% @prompt_changes.each do |row| %>
|
|
59
|
+
<% gained = row[:delta].positive? %>
|
|
60
|
+
<li class="ck-improvement">
|
|
61
|
+
<%= link_to row[:prompt].name, prompt_path(row[:prompt]), class: "ck-improvement__name ck-link" %>
|
|
62
|
+
<span class="ck-improvement__versions">v<%= row[:from_version] %> → v<%= row[:to_version] %></span>
|
|
63
|
+
<span class="ck-improvement__scores">
|
|
64
|
+
<span class="ck-improvement__from"><%= row[:from_score] %></span>
|
|
65
|
+
<span class="ck-improvement__arrow">→</span>
|
|
66
|
+
<span class="ck-improvement__to"><%= row[:to_score] %></span>
|
|
67
|
+
</span>
|
|
68
|
+
<span class="ck-improvement__delta <%= gained ? 'is-gain' : 'is-loss' %>"><%= gained ? '▲' : '▼' %> <%= '%+.2f' % row[:delta] %></span>
|
|
69
|
+
</li>
|
|
70
|
+
<% end %>
|
|
71
|
+
</ul>
|
|
72
|
+
<% else %>
|
|
73
|
+
<p class="ck-improvements__empty">
|
|
74
|
+
No measured changes yet. Edit a prompt to create a new version, re-run it against the same dataset and metrics, and the score change, up or down, shows up here.
|
|
75
|
+
</p>
|
|
76
|
+
<% end %>
|
|
77
|
+
</div>
|
|
78
|
+
<% end %>
|
|
79
|
+
|
|
80
|
+
<section class="ck-card--spaced">
|
|
81
|
+
<div class="ck-split">
|
|
82
|
+
<h2 class="ck-section-title">Recent runs</h2>
|
|
83
|
+
<%= link_to "View all", runs_path, class: "ck-link" %>
|
|
84
|
+
</div>
|
|
85
|
+
|
|
86
|
+
<% if @recent_runs.any? %>
|
|
87
|
+
<%= render "completion_kit/runs/table", runs: @recent_runs %>
|
|
88
|
+
<% else %>
|
|
89
|
+
<div class="ck-empty">No runs yet. <%= link_to "Create your first run →", new_run_path, class: "ck-link" %></div>
|
|
90
|
+
<% end %>
|
|
91
|
+
</section>
|
|
@@ -0,0 +1,3 @@
|
|
|
1
|
+
<% concept = CompletionKit::Onboarding::Concepts::DEFINITIONS.fetch(key) -%>
|
|
2
|
+
<% term = local_assigns.fetch(:term, "") -%>
|
|
3
|
+
<span class="ck-concept"><%= term %><button type="button" class="ck-concept__toggle" aria-label="What is a <%= concept[:name] %>?" aria-describedby="concept-<%= key %>-pop"><%= heroicon_tag "information-circle", variant: :outline, size: 14, class: "ck-concept__icon", "aria-hidden": "true" %></button><span class="ck-concept__pop" role="tooltip" id="concept-<%= key %>-pop"><span class="ck-concept__name"><%= concept[:name] %></span><span class="ck-concept__body"><%= concept[:definition] %></span></span></span>
|
|
@@ -15,7 +15,7 @@
|
|
|
15
15
|
<body class="ck-app">
|
|
16
16
|
<header class="ck-topbar">
|
|
17
17
|
<div class="ck-wrap ck-topbar__inner">
|
|
18
|
-
<%= link_to
|
|
18
|
+
<%= link_to dashboard_path, class: "ck-brand" do %><%= image_tag "completion_kit/logo.png", alt: "CompletionKit", style: "height: 64px; width: auto;" %><span class="ck-brand__name">Completion<span class="ck-brand__kit">Kit</span></span><% end %>
|
|
19
19
|
|
|
20
20
|
<details class="ck-nav-menu">
|
|
21
21
|
<summary class="ck-nav-menu__trigger" aria-label="Menu">
|
|
@@ -27,21 +27,21 @@
|
|
|
27
27
|
<%= link_to "Metrics", metrics_path, class: request.path.start_with?(metrics_path) || request.path.start_with?(metric_groups_path) ? ck_button_classes(:dark) : ck_button_classes(:light, variant: :outline) %>
|
|
28
28
|
<%= link_to "Datasets", datasets_path, class: active.(datasets_path) %>
|
|
29
29
|
<%= link_to "Runs", runs_path, class: active.(runs_path) %>
|
|
30
|
-
<% settings_active = request.path.start_with?(provider_credentials_path) || request.path.start_with?(tags_path) || request.path.start_with?(onboarding_path) %>
|
|
30
|
+
<% settings_active = request.path.start_with?(provider_credentials_path) || request.path.start_with?(tags_path) || request.path.start_with?(onboarding_path) || request.path.start_with?(api_reference_path) %>
|
|
31
31
|
<details class="ck-settings-menu">
|
|
32
32
|
<summary class="<%= settings_active ? ck_button_classes(:dark) : ck_button_classes(:light, variant: :outline) %> ck-settings-menu__trigger" aria-label="Settings">
|
|
33
|
-
|
|
33
|
+
<%= heroicon_tag "cog-6-tooth", variant: :outline, size: 18, "aria-hidden": "true" %>
|
|
34
34
|
</summary>
|
|
35
35
|
<div class="ck-settings-menu__panel" role="menu">
|
|
36
|
+
<%= link_to "Getting started", onboarding_path(reset: 1), class: "ck-settings-menu__item" %>
|
|
37
|
+
<%= link_to "API", api_reference_path, class: "ck-settings-menu__item" %>
|
|
36
38
|
<%= link_to "Providers", provider_credentials_path, class: "ck-settings-menu__item" %>
|
|
37
39
|
<%= link_to "Tags", tags_path, class: "ck-settings-menu__item" %>
|
|
38
|
-
|
|
40
|
+
<% if main_app.respond_to?(:logout_path) %>
|
|
41
|
+
<%= button_to "Sign out", main_app.logout_path, method: :delete, class: "ck-settings-menu__item" %>
|
|
42
|
+
<% end %>
|
|
39
43
|
</div>
|
|
40
44
|
</details>
|
|
41
|
-
<%= link_to "API", api_reference_path, class: active.(api_reference_path) %>
|
|
42
|
-
<% if main_app.respond_to?(:logout_path) %>
|
|
43
|
-
<%= button_to "Log out", main_app.logout_path, method: :delete, class: ck_button_classes(:light, variant: :outline) %>
|
|
44
|
-
<% end %>
|
|
45
45
|
</nav>
|
|
46
46
|
</details>
|
|
47
47
|
</div>
|
data/config/routes.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: completion-kit
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.5.
|
|
4
|
+
version: 0.5.31
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Damien Bastin
|
|
@@ -242,6 +242,7 @@ files:
|
|
|
242
242
|
- app/controllers/completion_kit/api/v1/tags_controller.rb
|
|
243
243
|
- app/controllers/completion_kit/api_reference_controller.rb
|
|
244
244
|
- app/controllers/completion_kit/application_controller.rb
|
|
245
|
+
- app/controllers/completion_kit/dashboard_controller.rb
|
|
245
246
|
- app/controllers/completion_kit/dashboard_dismissals_controller.rb
|
|
246
247
|
- app/controllers/completion_kit/datasets_controller.rb
|
|
247
248
|
- app/controllers/completion_kit/mcp_controller.rb
|
|
@@ -299,6 +300,7 @@ files:
|
|
|
299
300
|
- app/services/completion_kit/model_discovery_service.rb
|
|
300
301
|
- app/services/completion_kit/ollama_client.rb
|
|
301
302
|
- app/services/completion_kit/onboarding/checklist.rb
|
|
303
|
+
- app/services/completion_kit/onboarding/concepts.rb
|
|
302
304
|
- app/services/completion_kit/onboarding/sample_data.rb
|
|
303
305
|
- app/services/completion_kit/open_ai_client.rb
|
|
304
306
|
- app/services/completion_kit/open_router_client.rb
|
|
@@ -315,6 +317,7 @@ files:
|
|
|
315
317
|
- app/views/completion_kit/dashboard/_eye_off_icon.html.erb
|
|
316
318
|
- app/views/completion_kit/dashboard/_failures_card.html.erb
|
|
317
319
|
- app/views/completion_kit/dashboard/_worst_metric_card.html.erb
|
|
320
|
+
- app/views/completion_kit/dashboard/show.html.erb
|
|
318
321
|
- app/views/completion_kit/dashboard_dismissals/refresh.turbo_stream.erb
|
|
319
322
|
- app/views/completion_kit/datasets/_form.html.erb
|
|
320
323
|
- app/views/completion_kit/datasets/edit.html.erb
|
|
@@ -331,6 +334,7 @@ files:
|
|
|
331
334
|
- app/views/completion_kit/metrics/index.html.erb
|
|
332
335
|
- app/views/completion_kit/metrics/new.html.erb
|
|
333
336
|
- app/views/completion_kit/metrics/show.html.erb
|
|
337
|
+
- app/views/completion_kit/onboarding/_concept.html.erb
|
|
334
338
|
- app/views/completion_kit/onboarding/show.html.erb
|
|
335
339
|
- app/views/completion_kit/prompts/_form.html.erb
|
|
336
340
|
- app/views/completion_kit/prompts/edit.html.erb
|