kaba 0.1.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.dockerignore +3 -0
- data/Dockerfile +33 -0
- data/README.md +27 -3
- data/exe/kaba +5 -1
- data/kaba.gemspec +1 -0
- data/lib/kaba/version.rb +1 -1
- metadata +17 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 73fe981d3fc1a765b78b603f5f04ee8cd23514cc620f601e437363b158362d8b
|
|
4
|
+
data.tar.gz: bac9dcf38f519286ac2d9d1fd2b24ff3f1c2dabf3a0c85a93b761394f6d13db8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5f60669e92bc3851fe7c481d2271d64cca8cfc1c1e67ffeb2e6ce24c0749171366360b330cfa7e9453e1a30876fbfcdc2c41cd42ef5de97a1a3a31e1000d4f35
|
|
7
|
+
data.tar.gz: 2139393a5569187cb8d7b5a32d9a641b33f26b93ac0c8b799ac991aad2db965db8d755d23850895af16f95c15518a8b9f6ca58df6d16e1497763c58d0498a071
|
data/.dockerignore
ADDED
data/Dockerfile
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
FROM ruby:3.3-alpine
|
|
2
|
+
|
|
3
|
+
# Set the working directory to /kaba
|
|
4
|
+
WORKDIR /kaba
|
|
5
|
+
|
|
6
|
+
# Copy the Gemfile, Gemfile.lock into the container
|
|
7
|
+
COPY Gemfile Gemfile.lock kaba.gemspec ./
|
|
8
|
+
|
|
9
|
+
# Required in kaba.gemspec
|
|
10
|
+
COPY lib/kaba/version.rb /kaba/lib/kaba/version.rb
|
|
11
|
+
COPY Gemfile /kaba/Gemfile
|
|
12
|
+
COPY Gemfile.lock /kaba/Gemfile.lock
|
|
13
|
+
|
|
14
|
+
# Install application dependencies
|
|
15
|
+
RUN apk add --no-cache build-base git && bundle install
|
|
16
|
+
|
|
17
|
+
# Copy the rest of our application code into the container.
|
|
18
|
+
# We do this after bundle install, to avoid having to run bundle
|
|
19
|
+
# every time we do small fixes in the source code.
|
|
20
|
+
COPY . .
|
|
21
|
+
|
|
22
|
+
# Install the gem locally from the project folder
|
|
23
|
+
RUN gem build kaba.gemspec && \
|
|
24
|
+
gem install ./kaba-*.gem --no-document
|
|
25
|
+
|
|
26
|
+
RUN rm -rf /kaba
|
|
27
|
+
|
|
28
|
+
# Set the working directory to /workdir
|
|
29
|
+
WORKDIR /workdir
|
|
30
|
+
|
|
31
|
+
# Set the entrypoint to run the installed binary in /workdir
|
|
32
|
+
# Example: docker run -it -v "$PWD:/workdir" kaba
|
|
33
|
+
ENTRYPOINT ["kaba"]
|
data/README.md
CHANGED
|
@@ -1,9 +1,33 @@
|
|
|
1
1
|
# Kaba
|
|
2
|
+
咔吧是一款数据构建工具，使用 Ruby 完成，使用 typechat 作为核心，目的是构建一款能够比较好适配大模型 sft 数据集的工具，整个项目使用起来只需要安装 docker 即可。
|
|
2
3
|
|
|
3
|
-
|
|
4
|
+
> 开源协议：你爱干嘛干嘛
|
|
5
|
+
|
|
6
|
+
## 安装
|
|
7
|
+
|
|
8
|
+
如果你有一个 Ruby 环境可用（且 ruby 版本大于 3.3），你可以使用以下命令全局安装 kaba：
|
|
9
|
+
```
|
|
10
|
+
gem install kaba
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
否则，你可以通过别名运行一个 docker 化版本（将下面的命令添加到你的~/.bashrc、~/.zshrc或类似文件中，以简化重复使用）。
|
|
14
|
+
|
|
15
|
+
```
|
|
16
|
+
alias kaba='docker run -it --rm -v "${PWD}:/workdir" ghcr.io/mjason/kaba:latest'
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## 目录结构说明
|
|
20
|
+
你的项目目录必须有 data 目录
|
|
4
21
|
- data
|
|
5
22
|
- row
|
|
23
|
+
- *.target.json
|
|
24
|
+
- *.input.txt
|
|
6
25
|
- schema
|
|
26
|
+
- *.ts
|
|
27
|
+
|
|
28
|
+
`*`代表文件名，随你喜欢，一般推荐用数字即可，schema 怎么定义直接看 typechat 文档就好了。
|
|
29
|
+
|
|
30
|
+
## 关联项目
|
|
31
|
+
- [lisa_typechat_server](https://github.com/mjason/lisa_typechat_server)
|
|
7
32
|
|
|
8
|
-
|
|
9
|
-
`gem install kaba`
|
|
33
|
+
如果要修改服务地址你有两个方式，一个通过 `.env` 来处理，还有就是自己设置环境变量，变量名 `LISA_TYPECHAT_ENDPOINT`
|
data/exe/kaba
CHANGED
|
@@ -10,10 +10,14 @@ require 'async/http/faraday'
|
|
|
10
10
|
require 'json'
|
|
11
11
|
require "kaba"
|
|
12
12
|
|
|
13
|
+
require 'dotenv'
|
|
14
|
+
Dotenv.load
|
|
15
|
+
|
|
13
16
|
class Application
|
|
14
17
|
class << self
|
|
15
18
|
def connection
|
|
16
|
-
|
|
19
|
+
endpoint = ENV["LISA_TYPECHAT_ENDPOINT"] || "https://lisa-typechat.listenai.com"
|
|
20
|
+
@connection ||= Faraday.new(endpoint) do |faraday|
|
|
17
21
|
faraday.adapter :async_http, clients: Async::HTTP::Faraday::PersistentClients
|
|
18
22
|
faraday.request :json
|
|
19
23
|
end
|
data/kaba.gemspec
CHANGED
|
@@ -36,6 +36,7 @@ Gem::Specification.new do |spec|
|
|
|
36
36
|
spec.add_dependency "async-http-faraday", "~> 0.19.0"
|
|
37
37
|
spec.add_dependency "colorize", "~> 1.1"
|
|
38
38
|
spec.add_dependency "tty-progressbar", "~> 0.18.3"
|
|
39
|
+
spec.add_dependency "dotenv", "~> 3.1"
|
|
39
40
|
|
|
40
41
|
# For more information and examples about making a new gem, check out our
|
|
41
42
|
# guide at: https://bundler.io/guides/creating_gem.html
|
data/lib/kaba/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kaba
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1
|
|
4
|
+
version: 0.2.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- MJ
|
|
@@ -80,6 +80,20 @@ dependencies:
|
|
|
80
80
|
- - "~>"
|
|
81
81
|
- !ruby/object:Gem::Version
|
|
82
82
|
version: 0.18.3
|
|
83
|
+
- !ruby/object:Gem::Dependency
|
|
84
|
+
name: dotenv
|
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
|
86
|
+
requirements:
|
|
87
|
+
- - "~>"
|
|
88
|
+
- !ruby/object:Gem::Version
|
|
89
|
+
version: '3.1'
|
|
90
|
+
type: :runtime
|
|
91
|
+
prerelease: false
|
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
93
|
+
requirements:
|
|
94
|
+
- - "~>"
|
|
95
|
+
- !ruby/object:Gem::Version
|
|
96
|
+
version: '3.1'
|
|
83
97
|
description: 用来做数据集的工具
|
|
84
98
|
email:
|
|
85
99
|
- tywf91@gmail.com
|
|
@@ -88,6 +102,8 @@ executables:
|
|
|
88
102
|
extensions: []
|
|
89
103
|
extra_rdoc_files: []
|
|
90
104
|
files:
|
|
105
|
+
- ".dockerignore"
|
|
106
|
+
- Dockerfile
|
|
91
107
|
- README.md
|
|
92
108
|
- Rakefile
|
|
93
109
|
- exe/kaba
|