groq 0.3.0 → 0.3.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: af8963c428e4a7760f76a17a48d6c92cdae50363c96d6e97eef21293a3321beb
-  data.tar.gz: 92d619e893e9fa727c76f42c85f095cc1e874ba78474840f7c6b1d139d691077
+  metadata.gz: 69ebe13e0c0cdc2c38d128a218d68a6da1cf86a6999a78d4b01b9da9b78475c9
+  data.tar.gz: 786a3c8c4c22c683aa646589cfa82687667636a8ea04d24ffa8282ccc2ca2de9
 SHA512:
-  metadata.gz: bc25feb5cfdaf37955932e59f079568ce4f135dcd61eade7d13bf1d62777fec08642f246bbe2b90d4b0c0ffaaa28e9bc18092814cd19b2da3f9c06486d713417
-  data.tar.gz: 2c436ded0ab2152c5625a7b0690942b7d24b71cb36aef56a95984a5faa31ee8b959230f7f5389c4941ed613f19608971a00c415c22b7b6199d04f6a8674f2a14
+  metadata.gz: a723890cf8c2ff7bd0e0c45ea857b790e31894373972a1d13c95ad24db125e9fff2c0e8d1ef3970542c84b8c115bc4d727cc7f40fce79314c58a1984ced2c064
+  data.tar.gz: 5d9f44792162f831da02fdc3e17b464e08be8e02c920c5e38c7e9eb7a0a4dede7f840c7baa732c89c76d5b3d48f07dc39c9bf3da0bd5ff5411b783f9f2dc1fc4

data/.tool-versions

@@ -1 +1 @@
-ruby 3.1.4
+ruby 3.1.5

data/README.md

@@ -60,11 +60,14 @@ JSON.parse(response["content"])
 
 Install the gem and add to the application's Gemfile by executing:
 
-> bundle add groq
+```plain
+bundle add groq
 ```
+
 If bundler is not being used to manage dependencies, install the gem by executing:
 
-> gem install groq
+```plain
+gem install groq
 ```
 ## Usage
 
@@ -76,13 +79,19 @@ If bundler is not being used to manage dependencies, install the gem by executin
 client = Groq::Client.new # uses ENV["GROQ_API_KEY"] and "llama3-8b-8192"
 client = Groq::Client.new(api_key: "...", model_id: "llama3-8b-8192")
 
-Groq.configuration do |config|
+Groq.configure do |config|
   config.api_key = "..."
   config.model_id = "llama3-70b-8192"
 end
 client = Groq::Client.new
 ```
 
+In a Rails application, you can generate a `config/initializer/groq.rb` file with:
+
+```plain
+rails g groq:install
+```
+
 There is a simple chat function to send messages to a model:
 
 ```ruby
@@ -101,8 +110,10 @@ client.chat([
 
 ### Interactive console (IRb)
 
-> bin/console
+```plain
+bin/console
 ```
+
 This repository has a `bin/console` script to start an interactive console to play with the Groq API. The `@client` variable is setup using `$GROQ_API_KEY` environment variable; and the `U`, `A`, `T` helpers are already included.
 
 ```ruby
@@ -160,7 +171,7 @@ As above, you can specify the default model to use for all `chat()` calls:
 ```ruby
 client = Groq::Client.new(model_id: "llama3-70b-8192")
 # or
-Groq.configuration do |config|
+Groq.configure do |config|
   config.model_id = "llama3-70b-8192"
 end
 ```
@@ -318,10 +329,10 @@ The defaults are:
 => 1
 ```
 
-You can override them in the `Groq.configuration` block, or with each `chat()` call:
+You can override them in the `Groq.configure` block, or with each `chat()` call:
 
 ```ruby
-Groq.configuration do |config|
+Groq.configure do |config|
   config.max_tokens = 512
   config.temperature = 0.5
 end

data/examples/README.md

@@ -5,9 +5,9 @@
 Chat with a pre-defined agent using the following command:
 
 ```bash
-bundle exec examples/groq-user-chat.rb
+bundle exec examples/user-chat.rb
 # or
-bundle exec examples/groq-user-chat.rb --agent-prompt examples/agent-prompts/helloworld.yml
+bundle exec examples/user-chat.rb --agent-prompt examples/agent-prompts/helloworld.yml
 ```
 
 There are two example agent prompts available:
@@ -20,14 +20,47 @@ At the prompt, either talk to the AI agent, or some special commands:
 - `exit` to exit the conversation
 - `summary` to get a summary of the conversation so far
 
-### Streaming
+### Streaming text chunks
 
 There is also an example of streaming the conversation to terminal as it is received from Groq API.
 
 It defaults to the slower `llama3-70b-8192` model so that the streaming is more noticable.
 
 ```bash
-bundle exec examples/groq-user-chat-streaming.rb --agent-prompt examples/agent-prompts/pizzeria-sales.yml
+bundle exec examples/user-chat-streaming.rb --agent-prompt examples/agent-prompts/pizzeria-sales.yml
+```
+
+### Streaming useful chunks (e.g. JSON)
+
+If the response is returning a list of objects, such as a sequence of JSON objects, you can try to stream the chunks that make up the JSON objects and process them as soon as they are complete.
+
+```bash
+bundle exec examples/streaming-to-json-objects.rb
+```
+
+This will produce JSON for each planet in the solar system, one at a time. The API does not return each JSON as a chunk, rather it only returns `{` and `"` and `name` as distinct chunks. But the example code [`examples/streaming-to-json-objects.rb`](examples/streaming-to-json-objects.rb) shows how you might build up JSON objects from chunks, and process it (e.g. store to DB) as soon as it is complete.
+
+The system prompt used is:
+
+```plain
+Write out the names of the planets of our solar system, and a brief description of each one.
+
+Return JSON object for each one:
+
+{ "name": "Mercury", "position": 1, "description": "Mercury is ..." }
+
+Between each response, say "NEXT" to clearly delineate each JSON response.
+
+Don't say anything else except the JSON objects above.
+```
+
+The code in the repo uses the `NEXT` token to know when to process the JSON object.
+
+The output will look like, with each JSON object printed (or saved to DB) obly when it has been completely built from chunks.
+
+```json
+{"name":"Mercury","position":1,"description":"Mercury is the smallest planet in our solar system, with a highly elliptical orbit that takes it extremely close to the sun."}
+{"name":"Venus","position":2,"description":"Venus is often called Earth's twin due to their similar size and mass, but it has a thick atmosphere that traps heat, making it the hottest planet."}
 ```
 
 ### Pizzeria
@@ -35,7 +68,7 @@ bundle exec examples/groq-user-chat-streaming.rb --agent-prompt examples/agent-p
 Run the pizzeria example with the following command:
 
 ```bash
-bundle exec examples/groq-user-chat.rb --agent-prompt examples/agent-prompts/pizzeria-sales.yml
+bundle exec examples/user-chat.rb --agent-prompt examples/agent-prompts/pizzeria-sales.yml
 ```
 
 > 🍕 Hello! Thank you for calling our pizzeria. I'm happy to help you with your inquiry. Do you have a question about our menu or would you like to place an order?
@@ -59,3 +92,29 @@ bundle exec examples/groq-user-chat.rb --agent-prompt examples/agent-prompts/piz
 > The conversation started with a customer calling the pizzeria and speaking with an AI assistant. The assistant offered to help with menu inquiries or taking an order. The customer was considering ordering from the menu. The assistant presented the cheapest menu item, Garlic Knots, and asked if the customer wanted to add it to their order. The customer may have been interested in other options as well. The assistant then presented the cheapest pizza option, the Veggie Pizza, and asked if the customer wanted to order it along with the Garlic Knots. The customer agreed, and the assistant took note of the order, which consisted of a Veggie Pizza and 6 Garlic Knots for a total of $18. The assistant asked how the customer would like to pay for their order.
 >
 > 😋 exit
+
+## Two Agents
+
+Here is an example of two agents talking to each other, without any user input.
+
+### Buying Pizza
+
+```bash
+bundle exec examples/groq-two-agents-chatting.rb --agent-prompt examples/agent-prompts/pizzeria-sales.yml --agent-prompt examples/agent-prompts/food-customer.yml
+```
+
+This will proceed for 10 turns, with the agents taking turns to speak. Pass the `-i 15` option to increase the number of turns.
+
+> 🍕 Hello! Welcome to Pizza Palace. Thank you for reaching out to us. How can I assist you today?
+>
+> 😋 Hi! Thanks for having me. I'm actually pretty hungry, so I'm looking to order some food. Can you tell me a bit about your menu? What are some of your popular items?
+>
+> 🍕 I'd be happy to tell you more about our menu. We have a variety of delicious options to choose from. Our most popular items include our Margherita Pizza, Pepperoni Pizza, and BBQ Chicken Pizza. The Margherita is a classic with tomato sauce, mozzarella, and fresh basil. The Pepperoni Pizza is a crowd-pleaser with a generous layer of pepperoni on top. And our BBQ Chicken Pizza has a sweet and tangy BBQ sauce, topped with chicken, onions, and cilantro.
+>
+> We also have some great non-pizza options, such as our Garlic Knots, which are a favorite among our customers. And for dessert, our Cannoli are a must-try - they're filled with creamy ricotta cheese and chocolate chips.
+>
+> What sounds good to you? Would you like me to walk you through our entire menu or is there something specific you're in the mood for?
+>
+> 😋 Mmm, everything sounds delicious! I think I'll go for something a bit hearty. Can you tell me more about the BBQ Chicken Pizza? What kind of chicken is used? And is the pepperoni on the Pepperoni Pizza thick-cut or thin-cut?
+>
+> Also, how would you recommend ordering the Garlic Knots? Are they a side dish or can I get them as part of a combo?

data/examples/agent-prompts/food-customer.yml

@@ -0,0 +1,12 @@
+---
+name: "Food Customer"
+system_prompt: |-
+  You are a hungry customer looking to order some food.
+
+  You can ask about the menu, place an order, or inquire about delivery options.
+
+  When asked about delivery, you say you'll pick up.
+  When asked about payment, you confirm you'll pay when you pick up.
+  You have $25 to spend.
+agent_emoji: "😋"
+can_go_first: true

data/examples/agent-prompts/helloworld.yml

@@ -1,5 +1,6 @@
 ---
-system: |-
+name: "Hello World"
+system_prompt: |-
   I am a friendly agent who always replies to any prompt
   with a pleasant "Hello" and wishing them well.
 agent_emoji: "🤖"

data/examples/agent-prompts/pizzeria-sales.yml

@@ -1,5 +1,6 @@
 ---
-system: |-
+name: "Pizzeria Sales"
+system_prompt: |-
   You are a phone operator at a busy pizzeria. Your responsibilities include answering calls and online chats from customers who may ask about the menu, wish to place or change orders, or inquire about opening hours.
 
   Here are some of our popular menu items:

data/examples/groq-two-agents-chatting.rb

@@ -0,0 +1,124 @@
+#!/usr/bin/env ruby
+#
+# This is a variation of groq-user-chat.rb but without any user prompting.
+# Just two agents chatting with each other.
+
+require "optparse"
+require "groq"
+require "yaml"
+
+include Groq::Helpers
+
+@options = {
+  model: "llama3-8b-8192",
+  # model: "llama3-70b-8192",
+  agent_prompt_paths: [],
+  timeout: 20,
+  interaction_count: 10 # total count of interactions between agents
+}
+OptionParser.new do |opts|
+  opts.banner = "Usage: ruby script.rb [options]"
+
+  opts.on("-m", "--model MODEL", "Model name") do |v|
+    @options[:model] = v
+  end
+
+  opts.on("-a", "--agent-prompt PATH", "Path to an agent prompt file") do |v|
+    @options[:agent_prompt_paths] << v
+  end
+
+  opts.on("-t", "--timeout TIMEOUT", "Timeout in seconds") do |v|
+    @options[:timeout] = v.to_i
+  end
+
+  opts.on("-d", "--debug", "Enable debug mode") do |v|
+    @options[:debug] = v
+  end
+
+  opts.on("-i", "--interaction-count COUNT", "Total count of interactions between agents") do |v|
+    @options[:interaction_count] = v.to_i
+  end
+end.parse!
+
+raise "New two --agent-prompt paths" if @options[:agent_prompt_paths]&.length&.to_i != 2
+
+def debug?
+  @options[:debug]
+end
+
+# Will be instantiated from the agent prompt file
+class Agent
+  def initialize(args = {})
+    args.each do |k, v|
+      instance_variable_set(:"@#{k}", v)
+    end
+    @messages = [S(@system_prompt)]
+  end
+  attr_reader :messages
+  attr_reader :name, :can_go_first, :user_emoji, :agent_emoji, :system_prompt
+  def can_go_first?
+    @can_go_first
+  end
+
+  def self.load_from_file(path)
+    new(YAML.load_file(path))
+  end
+end
+
+# Read the agent prompt from the file
+agents = @options[:agent_prompt_paths].map do |agent_prompt_path|
+  Agent.load_from_file(agent_prompt_path)
+end
+go_first = agents.find { |agent| agent.can_go_first? } || agents.first
+
+# check that each agent contains a system prompt
+agents.each do |agent|
+  raise "Agent #{agent.name} is missing a system prompt" if agent.system_prompt.nil?
+end
+
+# Initialize the Groq client
+@client = Groq::Client.new(model_id: @options[:model], request_timeout: @options[:timeout]) do |f|
+  if debug?
+    require "logger"
+
+    # Create a logger instance
+    logger = Logger.new($stdout)
+    logger.level = Logger::DEBUG
+
+    f.response :logger, logger, bodies: true  # Log request and response bodies
+  end
+end
+
+puts "Welcome to a conversation between #{agents.map(&:name).join(", ")}. Our first speaker will be #{go_first.name}."
+puts "You can quit by typing 'exit'."
+
+agent_speaking_index = agents.index(go_first)
+loop_count = 0
+
+loop do
+  speaking_agent = agents[agent_speaking_index]
+  # Show speaking agent emoji immediately to indicate request going to Groq API
+  print("#{speaking_agent.agent_emoji} ")
+
+  # Use Groq to generate a response
+  response = @client.chat(speaking_agent.messages)
+
+  # Finish the speaking agent line on screen with message response
+  puts(message = response.dig("content"))
+
+  # speaking agent tracks its own message as the Assistant
+  speaking_agent.messages << A(message)
+
+  # other agent tracks the message as the User
+  other_agents = agents.reject { |agent| agent == speaking_agent }
+  other_agents.each do |agent|
+    agent.messages << U(message)
+  end
+
+  agent_speaking_index = (agent_speaking_index + 1) % agents.length
+  loop_count += 1
+  break if loop_count > @options[:interaction_count]
+rescue Faraday::TooManyRequestsError
+  warn "...\n\nGroq API error: too many requests. Exiting."
+  exit 1
+end

data/examples/streaming-to-json-objects.rb

@@ -0,0 +1,87 @@
+#!/usr/bin/env ruby
+
+require "optparse"
+require "groq"
+require "yaml"
+
+include Groq::Helpers
+
+@options = {
+  model: "llama3-70b-8192",
+  timeout: 20
+}
+OptionParser.new do |opts|
+  opts.banner = "Usage: ruby script.rb [options]"
+
+  opts.on("-m", "--model MODEL", "Model name") do |v|
+    @options[:model] = v
+  end
+
+  opts.on("-t", "--timeout TIMEOUT", "Timeout in seconds") do |v|
+    @options[:timeout] = v.to_i
+  end
+
+  opts.on("-d", "--debug", "Enable debug mode") do |v|
+    @options[:debug] = v
+  end
+end.parse!
+
+raise "Missing --model option" if @options[:model].nil?
+
+# Initialize the Groq client
+@client = Groq::Client.new(model_id: @options[:model], request_timeout: @options[:timeout]) do |f|
+  if @options[:debug]
+    require "logger"
+
+    # Create a logger instance
+    logger = Logger.new($stdout)
+    logger.level = Logger::DEBUG
+
+    f.response :logger, logger, bodies: true  # Log request and response bodies
+  end
+end
+
+prompt = <<~TEXT
+  Write out the names of the planets of our solar system, and a brief description of each one.
+
+  Return JSON object for each one:
+
+  { "name": "Mercury", "position": 1, "description": "Mercury is ..." }
+
+  Between each response, say "NEXT" to clearly delineate each JSON response.
+
+  Don't say anything else except the JSON objects above.
+TEXT
+
+# Handle each JSON object once it has been fully streamed
+
+class PlanetStreamer
+  def initialize
+    @buffer = ""
+  end
+
+  def call(content)
+    if !content || content.include?("NEXT")
+      json = JSON.parse(@buffer)
+
+      # do something with JSON, e.g. save to database
+      puts json.to_json
+
+      # reset buffer
+      @buffer = ""
+      return
+    end
+    # if @buffer is empty; and content is not JSON start {, then ignore + return
+    if @buffer.empty? && !content.start_with?("{")
+      return
+    end
+
+    # build JSON
+    @buffer << content
+  end
+end
+
+streamer = PlanetStreamer.new
+
+@client.chat([S(prompt)], stream: streamer)
+puts

data/examples/{groq-user-chat-streaming.rb → user-chat-streaming.rb}

@@ -34,10 +34,6 @@ end.parse!
 raise "Missing --model option" if @options[:model].nil?
 raise "Missing --agent-prompt option" if @options[:agent_prompt_path].nil?
 
-def debug?
-  @options[:debug]
-end
-
 # Read the agent prompt from the file
 agent_prompt = YAML.load_file(@options[:agent_prompt_path])
 user_emoji = agent_prompt["user_emoji"]
@@ -47,7 +43,7 @@ can_go_first = agent_prompt["can_go_first"]
 
 # Initialize the Groq client
 @client = Groq::Client.new(model_id: @options[:model], request_timeout: @options[:timeout]) do |f|
-  if debug?
+  if @options[:debug]
     require "logger"
 
     # Create a logger instance

data/examples/{groq-user-chat.rb → user-chat.rb}

@@ -35,10 +35,6 @@ end.parse!
 raise "Missing --model option" if @options[:model].nil?
 raise "Missing --agent-prompt option" if @options[:agent_prompt_path].nil?
 
-def debug?
-  @options[:debug]
-end
-
 # Read the agent prompt from the file
 agent_prompt = YAML.load_file(@options[:agent_prompt_path])
 user_emoji = agent_prompt["user_emoji"]
@@ -48,7 +44,7 @@ can_go_first = agent_prompt["can_go_first"]
 
 # Initialize the Groq client
 @client = Groq::Client.new(model_id: @options[:model], request_timeout: @options[:timeout]) do |f|
-  if debug?
+  if @options[:debug]
     require "logger"
 
     # Create a logger instance

data/lib/generators/groq/install_generator.rb

@@ -0,0 +1,20 @@
+require "rails/generators/base"
+
+module Groq
+  module Generators
+    class InstallGenerator < Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+
+      def create_groq_init_file
+        create_file "config/initializers/groq.rb", <<~RUBY
+          # frozen_string_literal: true
+
+          Groq.configure do |config|
+            config.api_key = ENV["GROQ_API_KEY"]
+            config.model_id = "llama3-70b-8192"
+          end
+        RUBY
+      end
+    end
+  end
+end

data/lib/groq/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Groq
-  VERSION = "0.3.0"
+  VERSION = "0.3.2"
 end

metadata

@@ -1,29 +1,29 @@
 --- !ruby/object:Gem::Specification
 name: groq
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.3.2
 platform: ruby
 authors:
 - Dr Nic Williams
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2024-04-25 00:00:00.000000000 Z
+date: 2024-07-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: faraday
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '2.0'
+        version: 1.10.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '2.0'
+        version: 1.10.0
 - !ruby/object:Gem::Dependency
   name: json
   requirement: !ruby/object:Gem::Requirement
@@ -123,10 +123,14 @@ files:
 - Rakefile
 - docs/images/groq-speed-price-20240421.png
 - examples/README.md
+- examples/agent-prompts/food-customer.yml
 - examples/agent-prompts/helloworld.yml
 - examples/agent-prompts/pizzeria-sales.yml
-- examples/groq-user-chat-streaming.rb
-- examples/groq-user-chat.rb
+- examples/groq-two-agents-chatting.rb
+- examples/streaming-to-json-objects.rb
+- examples/user-chat-streaming.rb
+- examples/user-chat.rb
+- lib/generators/groq/install_generator.rb
 - lib/groq-ruby.rb
 - lib/groq.rb
 - lib/groq/client.rb
@@ -156,7 +160,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.5
+rubygems_version: 3.3.27
 signing_key:
 specification_version: 4
 summary: Client library for Groq API for fast LLM inference.