gpt-tokenizer 1.0.1 → 1.0.2

package/README.md

@@ -1,4 +1,6 @@
-# `gpt-tokenizer`
+# gpt-tokenizer
+
+[![Play with gpt-tokenizer](https://codesandbox.io/static/img/play-codesandbox.svg)](https://codesandbox.io/s/gpt-tokenizer-tjcjoz?fontsize=14&hidenavigation=1&theme=dark)
 
 `gpt-tokenizer` is a highly optimized Token Byte Pair Encoder/Decoder for GPT-2, GPT-3, GPT-3.5 and GPT-4 designed for JavaScript applications. OpenAI's GPT models utilize byte pair encoding to transform text into a sequence of integers before feeding them into the model. This package is a JavaScript implementation of OpenAI's original Python encoder/decoder, which can be found [here](https://github.com/openai/gpt-2).
 
@@ -28,6 +30,14 @@ As an UMD module:
 <script src="https://unpkg.com/gpt-tokenizer" />
 ```
 
+## Playground
+
+You can play with the package in the browser using the [Playground](https://codesandbox.io/s/gpt-tokenizer-tjcjoz?fontsize=14&hidenavigation=1&theme=dark).
+
+[![GPT Tokenizer Playground](./docs/gpt-tokenizer.png)](https://codesandbox.io/s/gpt-tokenizer-tjcjoz?fontsize=14&hidenavigation=1&theme=dark)
+
+The playground mimics the official [OpenAI Tokenizer](https://platform.openai.com/tokenizer).
+
 ## Usage
 
 ```typescript

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "gpt-tokenizer",
-  "version": "1.0.1",
+  "version": "1.0.2",
   "description": "BPE Encoder Decoder for GPT-2 / GPT-3",
   "keywords": [
     "BPE",
@@ -61,7 +61,7 @@
     "./package.json": "./package.json"
   },
   "main": "esm/main.js",
-  "unpkg": "dist/main.js",
+  "unpkg": "dist/gpt3encoder.js",
   "module": "esm/main.js",
   "source": "src/main.ts",
   "files": [