kaggle 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 3ad655835decf29a7a46e8b9c1d62a91bf05975bb8a694e3fac92b9f5f141eb7
+  data.tar.gz: '0803369672874a9a8f275a53fca15baeb66b1f2a121b2d5bdf91b39349a3bab1'
+SHA512:
+  metadata.gz: 947d6474751ade9122c0ec9fcb7d7f533a1b23dc9903ba6f59d3e88f20c4b1ec38ff70c53bb242d3a9cafca7f374bee0b307c2fb74ccc52add31fa7555906864
+  data.tar.gz: ced4244587280c337dbab8455ffb133db8181631afed0de72bb2b0c773ea7f32b8eff6f30c0d7a6037d5d5ee59cd245e975f7c5f9bdf436da6c9ed9a490707cb

data/.claude/settings.local.json

@@ -0,0 +1,13 @@
+{
+  "permissions": {
+    "allow": [
+      "Read(//Users/trex22/development/url_categorise/**)",
+      "Bash(chmod:*)",
+      "Bash(mkdir:*)",
+      "Bash(bundle exec rake:*)",
+      "Bash(rake test)"
+    ],
+    "deny": [],
+    "ask": []
+  }
+}

data/.ruby-version

@@ -0,0 +1 @@
+3.4.5

data/CLAUDE.md

@@ -0,0 +1,154 @@
+# Claude Assistant Documentation
+
+This file documents how Claude helped develop this Ruby gem and provides guidance for future development.
+
+## Development History
+
+This Kaggle Ruby gem was created with assistance from Claude (Sonnet 4) on 2025-08-23. The development process followed established Ruby gem conventions and best practices.
+
+## Architecture Decisions
+
+### 1. Gem Structure
+- Followed standard Ruby gem conventions based on successful gems like `url_categorise` and `luno`
+- Used modular architecture with separate modules for constants, client logic, and error handling
+- Implemented comprehensive test coverage using Minitest
+
+### 2. API Design
+- Used HTTParty for HTTP client functionality due to its simplicity and Ruby idioms
+- Implemented authentication via basic auth with username/API key
+- Added configurable paths for downloads and caching to support different deployment scenarios
+
+### 3. Error Handling
+- Created specific error classes inheriting from base `Kaggle::Error`
+- Implemented graceful degradation for network and parsing failures
+- Added comprehensive validation for user inputs
+
+### 4. Testing Strategy
+- Used Minitest with WebMock for HTTP request stubbing
+- Included test coverage reporting with SimpleCov (70% minimum)
+- Added comprehensive test helpers and fixtures
+
+## Key Implementation Notes
+
+### Authentication
+The gem supports two authentication methods:
+1. Environment variables (`KAGGLE_USERNAME`, `KAGGLE_KEY`)
+2. Explicit parameters during client initialization
+
+### Caching Strategy
+- Simple file-based caching for parsed CSV data
+- Cache keys generated from dataset paths
+- Optional cache usage controlled by method parameters
+
+### CSV Parsing
+- Uses Ruby's built-in CSV library for reliability
+- Converts CSV to JSON array of hashes (row objects)
+- Includes comprehensive error handling for malformed files
+
+## Commands for Development
+
+### Setup
+```bash
+bin/setup                  # Install dependencies
+```
+
+### Testing
+```bash
+rake test                  # Run all tests
+rake test TEST=specific    # Run specific test file
+```
+
+### Console
+```bash
+bin/console               # Interactive Ruby console with gem loaded
+```
+
+### Linting
+```bash
+# Note: No specific linter configured yet - add rubocop in future versions
+```
+
+## Future Development Guidelines
+
+### 1. API Expansion
+When adding new Kaggle API endpoints:
+- Add constants to `lib/kaggle/constants.rb`
+- Add methods to `lib/kaggle/client.rb` 
+- Follow existing error handling patterns
+- Add comprehensive tests in `test/kaggle/`
+
+### 2. CLI Enhancement
+The CLI tool (`bin/kaggle`) can be expanded with:
+- More command options and flags
+- Better output formatting
+- Progress indicators for large downloads
+- Configuration file support
+
+### 3. Performance Optimizations
+Consider adding:
+- Concurrent downloads for multiple files
+- Streaming for large files
+- More sophisticated caching strategies
+- Connection pooling for API requests
+
+### 4. Error Handling Improvements
+- Retry logic for transient failures
+- Better error messages with suggested actions
+- Logging capabilities for debugging
+
+## Common Development Patterns
+
+### Adding New API Methods
+1. Add endpoint to constants
+2. Implement client method with proper error handling
+3. Add comprehensive tests
+4. Update CLI if user-facing
+5. Document in README
+
+### Testing Network Interactions
+- Always use WebMock to stub HTTP requests
+- Test both success and failure scenarios
+- Include edge cases like timeouts and malformed responses
+- Verify authentication headers are sent correctly
+
+### Code Style Guidelines
+- Follow Ruby community style conventions
+- Use descriptive method and variable names
+- Keep methods focused and single-purpose
+- Include inline documentation for complex logic
+
+## Troubleshooting
+
+### Common Issues
+1. **Authentication Failures**: Verify credentials and API key permissions
+2. **Download Failures**: Check network connectivity and dataset availability
+3. **Parsing Errors**: Verify file format and encoding
+4. **Path Issues**: Ensure download/cache directories are writable
+
+### Debugging
+- Use `bin/console` for interactive debugging
+- Enable WebMock allow_localhost for integration testing
+- Use Pry for breakpoint debugging in tests
+
+## Contributing Guidelines
+
+When contributing to this gem:
+1. Follow existing code patterns and conventions
+2. Add tests for new functionality
+3. Update documentation (README, CLAUDE.md)
+4. Ensure backward compatibility
+5. Consider performance implications
+
+## Claude-Specific Notes
+
+This gem was developed iteratively with Claude assistance. The AI helper was particularly useful for:
+- Analyzing existing gem patterns and conventions
+- Implementing comprehensive test coverage
+- Creating consistent error handling patterns
+- Structuring CLI tools and bin scripts
+
+Future Claude interactions should:
+- Reference this documentation for context
+- Maintain existing architectural decisions
+- Follow established patterns for new features
+- Update this file with new insights or changes

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at contact@jasonchalom.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Jason Chalom
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,210 @@
+# Kaggle
+A Ruby client for the Kaggle API with support for datasets, competitions, models, and more. See: https://www.kaggle.com/docs/api
+
+This is an unofficial project and still a work in progress (WIP) ... more to come soon.
+
+## Features
+
+- 📊 Download Kaggle datasets programmatically
+- 📄 Parse CSV datasets to JSON format
+- 💾 Configurable caching to avoid re-downloading
+- 🔧 Flexible download and cache paths
+- ⚡ Built-in error handling and validation
+- 🛠️ Command-line interface for quick operations
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'kaggle'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install kaggle
+
+## Setup
+
+You'll need Kaggle API credentials to use this gem. There are three ways to authenticate:
+
+### Option 1: JSON File (Recommended)
+1. Go to your [Kaggle account page](https://www.kaggle.com/account)
+2. Click "Create New API Token" to download `kaggle.json`
+3. Place the file in your project directory or specify the path
+
+### Option 2: Environment Variables
+```bash
+export KAGGLE_USERNAME="yourusername"
+export KAGGLE_KEY="your_api_key"
+```
+
+### Option 3: Direct Credentials
+Pass credentials directly when initializing the client.
+
+### Kaggle JSON File Format
+The `kaggle.json` file downloaded from Kaggle should have this format:
+```json
+{
+  "username": "yourusername",
+  "key": "your_api_key"
+}
+```
+
+## Usage
+
+### Basic Usage
+
+```ruby
+require 'kaggle'
+
+# Option 1: Use kaggle.json file (automatically detected)
+client = Kaggle::Client.new
+
+# Option 1b: Use custom JSON file path
+client = Kaggle::Client.new(credentials_file: '/path/to/kaggle.json')
+
+# Option 2: Use environment variables
+client = Kaggle::Client.new
+
+# Option 3: Use explicit credentials
+client = Kaggle::Client.new(
+  username: 'your_username',
+  api_key: 'your_api_key'
+)
+```
+
+### List Datasets
+
+```ruby
+# List all datasets
+datasets = client.list_datasets
+
+# Search datasets
+datasets = client.list_datasets(search: 'housing')
+
+# Paginate results
+datasets = client.list_datasets(page: 2, page_size: 10)
+```
+
+### Download Datasets
+
+```ruby
+# Basic download
+file_path = client.download_dataset('zillow', 'zecon')
+
+# Download and parse CSV to JSON
+data = client.download_dataset('zillow', 'zecon', parse_csv: true)
+
+# Use caching to avoid re-downloading
+data = client.download_dataset('zillow', 'zecon', 
+                              parse_csv: true, 
+                              use_cache: true)
+```
+
+### Custom Paths
+
+```ruby
+client = Kaggle::Client.new(
+  credentials_file: '/path/to/kaggle.json',
+  download_path: '/custom/downloads',
+  cache_path: '/custom/cache'
+)
+```
+
+### Dataset Information
+
+```ruby
+# Get dataset files list
+files = client.dataset_files('zillow', 'zecon')
+
+# Parse existing CSV file
+data = client.parse_csv_to_json('/path/to/file.csv')
+```
+
+## Command Line Interface
+
+The gem includes a command-line interface:
+
+```bash
+# List datasets
+kaggle list
+
+# Search datasets
+kaggle list "housing"
+
+# Download dataset
+kaggle download zillow zecon
+
+# Download and parse CSV
+kaggle download zillow zecon --parse-csv
+
+# Use custom credentials file
+kaggle download zillow zecon --credentials-file /path/to/kaggle.json
+
+# Use custom paths
+kaggle download zillow zecon --download-path /custom --cache-path /custom/cache
+
+# Show dataset files
+kaggle files zillow zecon
+
+# Show version
+kaggle --version
+```
+
+## Configuration Options
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `credentials_file` | `./kaggle.json` | Path to Kaggle credentials JSON file |
+| `download_path` | `./downloads` | Where to save downloaded files |
+| `cache_path` | `./cache` | Where to cache parsed data |
+| `timeout` | `30` | HTTP request timeout in seconds |
+| `use_cache` | `false` | Use cached parsed data when available |
+| `parse_csv` | `false` | Automatically parse CSV files to JSON |
+
+## Error Handling
+
+The gem includes specific error types:
+
+```ruby
+begin
+  client.download_dataset('invalid', 'dataset')
+rescue Kaggle::AuthenticationError
+  puts "Invalid credentials"
+rescue Kaggle::DatasetNotFoundError
+  puts "Dataset not found"
+rescue Kaggle::DownloadError
+  puts "Download failed"
+rescue Kaggle::ParseError
+  puts "Failed to parse data"
+end
+```
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+### Tests
+
+To run tests execute:
+
+    $ rake test
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/yourusername/kaggle. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+## Code of Conduct
+
+Everyone interacting in the Kaggle project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/yourusername/kaggle/blob/main/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -0,0 +1,10 @@
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'test'
+  t.libs << 'lib'
+  t.test_files = FileList['test/**/*_test.rb']
+end
+
+task default: :test

data/kaggle.gemspec

@@ -0,0 +1,46 @@
+require_relative 'lib/kaggle/version'
+
+Gem::Specification.new do |spec|
+  spec.name = 'kaggle'
+  spec.version = Kaggle::VERSION
+  spec.authors = ['Your Name']
+  spec.email = ['your.email@example.com']
+
+  spec.summary = 'Ruby client for the Kaggle API'
+  spec.description = 'A Ruby gem for interacting with the Kaggle API, including dataset downloads with caching support'
+  spec.homepage = 'https://github.com/yourusername/kaggle'
+  spec.license = 'MIT'
+  spec.required_ruby_version = '>= 3.0.0'
+
+  spec.metadata['allowed_push_host'] = 'https://rubygems.org'
+  spec.metadata['homepage_uri'] = spec.homepage
+  spec.metadata['source_code_uri'] = spec.homepage
+  spec.metadata['changelog_uri'] = "#{spec.homepage}/blob/main/CHANGELOG.md"
+
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      (File.expand_path(f) == __FILE__) ||
+        f.start_with?(*%w[bin/ test/ spec/ features/ .git .circleci appveyor Gemfile])
+    end
+  end
+
+  spec.bindir = 'exe'
+  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
+  spec.require_paths = ['lib']
+
+  spec.add_dependency 'httparty', '>= 0.23'
+  spec.add_dependency 'csv', '>= 3.3'
+  spec.add_dependency 'oj', '3.16.11'
+  spec.add_dependency 'fileutils', '>= 1.7'
+  spec.add_dependency 'rubyzip', '>= 2.0'
+
+  spec.add_development_dependency 'rake', '~> 13.3.0'
+  spec.add_development_dependency 'minitest', '~> 5.25.5'
+  spec.add_development_dependency 'minitest-focus', '~> 1.4.0'
+  spec.add_development_dependency 'minitest-reporters', '~> 1.7.1'
+  spec.add_development_dependency 'webmock', '~> 3.24.0'
+  spec.add_development_dependency 'mocha', '~> 2.4.5'
+  spec.add_development_dependency 'pry', '~> 0.15.2'
+  spec.add_development_dependency 'simplecov', '~> 0.22.0'
+  spec.add_development_dependency 'timecop', '~> 0.9.10'
+end