asimov-brightdata-module 0.0.3-x86_64-linux-gnu

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: a501f60a8fea6f1f6048d4a3d98e24c0dfa3337d0827a1682c55a43a3b4dcf06
+  data.tar.gz: 230b7ec2d7611e456e5e1bc2bebcd91b7bca96a53baeab888220507ec7de5ce1
+SHA512:
+  metadata.gz: 5d7696ae9c6ea76f178dbe4fc02a0088d65c0383fee20e194ce411d3cd714eca71c49878ff6a817a4dfc99872a883ba55157deb115fcc8887f78fb42b24a411f
+  data.tar.gz: 22e68899a4f3be45a00f27a2f99d0b85f916daee3dbb50e7053c44010a618e7a9ce57d7a20a60d7c9c3255fece23f1e8c8f7a7f31b441a790717b155c8030d19

data/AUTHORS

@@ -0,0 +1 @@
+* Arto Bendiken <arto@asimov.so>

data/CHANGES.md

@@ -0,0 +1,17 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## 0.0.3 - 2025-05-29
+### Added
+- Implement JSON-LD output for X profiles
+- Implement pretty-printed JSON output
+### Changed
+- Bump the asimov-module dependency
+
+## 0.0.2 - 2025-05-22
+
+## 0.0.1 - 2025-05-21

data/README.md

@@ -0,0 +1,153 @@
+# ASIMOV Bright Data Module
+
+[![License](https://img.shields.io/badge/license-Public%20Domain-blue.svg)](https://unlicense.org)
+[![Package on Crates.io](https://img.shields.io/crates/v/asimov-brightdata-module.svg)](https://crates.io/crates/asimov-brightdata-module)
+[![Package on PyPI](https://img.shields.io/pypi/v/asimov-brightdata-module.svg)](https://pypi.org/project/asimov-brightdata-module)
+[![Package on RubyGems](https://img.shields.io/gem/v/asimov-brightdata-module.svg)](https://rubygems.org/gems/asimov-brightdata-module)
+[![Package on NPM](https://img.shields.io/npm/v/asimov-brightdata-module.svg)](https://npmjs.com/package/asimov-brightdata-module)
+
+[ASIMOV] module for data import powered by the [Bright Data] web data platform.
+
+## ✨ Features
+
+- Imports structured data from Airbnb, Amazon, Crunchbase, eBay, Facebook,
+  Google, Indeed, Instagram, LinkedIn, Walmart, X (aka Twitter), Yahoo, and
+  YouTube.
+- Collects the raw JSON data via the Bright Data API (requires an API key).
+- Constructs a semantic knowledge graph based on the [KNOW] ontology.
+- Supports plain JSON output as well as [RDF] output in the form of JSON-LD.
+- Distributed as a standalone static binary with zero runtime dependencies.
+
+## 🛠️ Prerequisites
+
+- [Rust] 1.85+ (2024 edition) if building from source code
+
+## ⬇️ Installation
+
+### Installation from PyPI
+
+```bash
+pip install -U asimov-brightdata-module
+```
+
+### Installation from RubyGems
+
+```bash
+gem install asimov-brightdata-module
+```
+
+### Installation from NPM
+
+```bash
+npm install -g asimov-brightdata-module
+```
+
+### Installation from Source Code
+
+```bash
+cargo install asimov-brightdata-module
+```
+
+## 👉 Examples
+
+```bash
+export BRIGHTDATA_API_KEY="..."
+```
+
+### Fetching X Profiles
+
+```bash
+asimov-brightdata-fetcher https://x.com/bright_init   # JSON
+asimov-brightdata-importer https://x.com/bright_init  # JSON-LD
+```
+
+### Fetching LinkedIn Profiles
+
+```bash
+asimov-brightdata-fetcher https://www.linkedin.com/in/orlenchner/
+asimov-brightdata-fetcher https://www.linkedin.com/company/bright-data/
+```
+
+### Fetching Crunchbase Profiles
+
+```bash
+asimov-brightdata-fetcher https://www.crunchbase.com/organization/brightdata
+```
+
+### Fetching Amazon Products
+
+```bash
+asimov-brightdata-fetcher https://www.amazon.com/Master-Algorithm-Ultimate-Learning-Machine/dp/0465094279
+```
+
+## ⚙ Configuration
+
+### Environment Variables
+
+- `BRIGHTDATA_API_KEY`: (required) the [Bright Data API key] to use
+
+## 📚 Reference
+
+### Installed Binaries
+
+- `asimov-brightdata-cataloger`: discovers entities via the Bright Data API
+  _(not implemented yet)_
+- `asimov-brightdata-fetcher`: collects JSON data from the Bright Data API
+- `asimov-brightdata-importer`: collects and transforms JSON into JSON-LD
+  _(not implemented yet)_
+
+### Supported Datasets
+
+Dataset | URL Prefix | JSON | RDF
+:------ | :--------- | :--: | :--:
+Airbnb | `https://www.airbnb.com/rooms/` | ✅ | 🚧
+Amazon | `https://www.amazon.com/` | ✅ | 🚧
+  | `https://www.amazon.com/sp?seller=` | ✅ | 🚧
+Crunchbase | `https://www.crunchbase.com/organization/` | ✅ | 🚧
+eBay | `https://www.ebay.com/itm/` | ✅ | 🚧
+Facebook | `https://www.facebook.com/events/` | ✅ | 🚧
+  | `https://www.facebook.com/groups/` | ✅ | 🚧
+  | `https://www.facebook.com/marketplace/item/` | ✅ | 🚧
+  | `https://www.facebook.com/share/p/` | ✅ | 🚧
+Google | `https://www.google.com/shopping/product/` | ✅ | 🚧
+Indeed | `https://www.indeed.com/cmp/` | ✅ | 🚧
+Instagram | `https://www.instagram.com/` | ✅ | 🚧
+  | `https://www.instagram.com/p/` | ✅ | 🚧
+  | `https://www.instagram.com/reel/` | ✅ | 🚧
+LinkedIn | `https://www.linkedin.com/company/` | ✅ | 🚧
+  | `https://www.linkedin.com/in/` | ✅ | 🚧
+  | `https://www.linkedin.com/jobs/` | ✅ | 🚧
+  | `https://www.linkedin.com/posts/` | ✅ | 🚧
+  | `https://www.linkedin.com/pulse/` | ✅ | 🚧
+Walmart | `https://www.walmart.com/global/seller/` | ✅ | 🚧
+  | `https://www.walmart.com/ip/` | ✅ | 🚧
+X (Twitter) | `https://x.com/` | ✅ | ✅
+Yahoo | `https://finance.yahoo.com/quote/` | ✅ | 🚧
+YouTube | `https://www.youtube.com/@` | ✅ | 🚧
+  | `https://www.youtube.com/watch?v=` | ✅ | 🚧
+<img width="100" height="1"/> | <img width="550" height="1"/> | <img width="50" height="1"/> | <img width="50" height="1"/>
+
+## 👨‍💻 Development
+
+```bash
+git clone https://github.com/asimov-modules/asimov-brightdata-module.git
+```
+
+---
+
+[![Share on X](https://img.shields.io/badge/share%20on-x-03A9F4?logo=x)](https://x.com/intent/post?url=https://github.com/asimov-modules/asimov-brightdata-module&text=asimov-brightdata-module)
+[![Share on Reddit](https://img.shields.io/badge/share%20on-reddit-red?logo=reddit)](https://reddit.com/submit?url=https://github.com/asimov-modules/asimov-brightdata-module&title=asimov-brightdata-module)
+[![Share on Hacker News](https://img.shields.io/badge/share%20on-hn-orange?logo=ycombinator)](https://news.ycombinator.com/submitlink?u=https://github.com/asimov-modules/asimov-brightdata-module&t=asimov-brightdata-module)
+[![Share on Facebook](https://img.shields.io/badge/share%20on-fb-1976D2?logo=facebook)](https://www.facebook.com/sharer/sharer.php?u=https://github.com/asimov-modules/asimov-brightdata-module)
+[![Share on LinkedIn](https://img.shields.io/badge/share%20on-linkedin-3949AB?logo=linkedin)](https://www.linkedin.com/sharing/share-offsite/?url=https://github.com/asimov-modules/asimov-brightdata-module)
+
+[ASIMOV]: https://github.com/asimov-platform
+[Bright Data]: https://brightdata.com/products/web-scraper
+[Bright Data API key]: https://docs.brightdata.com/general/account/api-token
+[JSON-LD]: https://json-ld.org
+[KNOW]: https://github.com/know-ontology
+[NPM]: https:/npmjs.org
+[Python]: https://python.org
+[RDF]: https://github.com/rust-rdf
+[Ruby]: https://ruby-lang.org
+[Rust]: https://rust-lang.org

data/UNLICENSE

@@ -0,0 +1,24 @@
+This is free and unencumbered software released into the public domain.
+
+Anyone is free to copy, modify, publish, use, compile, sell, or
+distribute this software, either in source code form or as a compiled
+binary, for any purpose, commercial or non-commercial, and by any
+means.
+
+In jurisdictions that recognize copyright laws, the author or authors
+of this software dedicate any and all copyright interest in the
+software to the public domain. We make this dedication for the benefit
+of the public at large and to the detriment of our heirs and
+successors. We intend this dedication to be an overt act of
+relinquishment in perpetuity of all present and future rights to this
+software under copyright law.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
+
+For more information, please refer to <https://unlicense.org/>

data/VERSION

@@ -0,0 +1 @@
+0.0.3

data/bin/asimov-brightdata-fetcher

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+require_relative "../lib/asimov-brightdata-module"
+argv0 = ::ASIMOV_Brightdata_Module::Fetcher.path
+if ::Gem.win_platform?
+  ::Kernel.exit(::Kernel.system(argv0, *ARGV) ? 0 : $?.exitstatus)
+else
+  ::Kernel.exec(argv0, *ARGV)
+end

data/bin/asimov-brightdata-importer

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+require_relative "../lib/asimov-brightdata-module"
+argv0 = ::ASIMOV_Brightdata_Module::Importer.path
+if ::Gem.win_platform?
+  ::Kernel.exit(::Kernel.system(argv0, *ARGV) ? 0 : $?.exitstatus)
+else
+  ::Kernel.exec(argv0, *ARGV)
+end

data/lib/asimov-brightdata-module/fetcher.rb

@@ -0,0 +1,23 @@
+# This is free and unencumbered software released into the public domain.
+
+module ASIMOV_Brightdata_Module; end
+
+class ASIMOV_Brightdata_Module::Fetcher
+  def self.path
+    ::File.join(__dir__, "asimov-brightdata-fetcher")
+  end
+
+  def self.execute(...)
+    self.new(...).execute
+  end
+
+  def initialize(*args, env: {})
+    @argv0 = self.class.path.freeze
+    @args = args.map(&:to_s).freeze
+    @env = env.to_h.freeze
+  end
+
+  def execute(*args)
+    ::Kernel.system(@env, @argv0, *(@args + args.map(&:to_s))) ? 0 : $?.exitstatus
+  end
+end

data/lib/asimov-brightdata-module/importer.rb

@@ -0,0 +1,23 @@
+# This is free and unencumbered software released into the public domain.
+
+module ASIMOV_Brightdata_Module; end
+
+class ASIMOV_Brightdata_Module::Importer
+  def self.path
+    ::File.join(__dir__, "asimov-brightdata-importer")
+  end
+
+  def self.execute(...)
+    self.new(...).execute
+  end
+
+  def initialize(*args, env: {})
+    @argv0 = self.class.path.freeze
+    @args = args.map(&:to_s).freeze
+    @env = env.to_h.freeze
+  end
+
+  def execute(*args)
+    ::Kernel.system(@env, @argv0, *(@args + args.map(&:to_s))) ? 0 : $?.exitstatus
+  end
+end

data/lib/asimov-brightdata-module.rb

@@ -0,0 +1,8 @@
+# This is free and unencumbered software released into the public domain.
+
+module ASIMOV_Brightdata_Module
+  VERSION = "0.0.3".freeze
+end
+
+require_relative 'asimov-brightdata-module/fetcher'
+require_relative 'asimov-brightdata-module/importer'

metadata

@@ -0,0 +1,58 @@
+--- !ruby/object:Gem::Specification
+name: asimov-brightdata-module
+version: !ruby/object:Gem::Version
+  version: 0.0.3
+platform: x86_64-linux-gnu
+authors:
+- ASIMOV Protocol
+bindir: bin
+cert_chain: []
+date: 2025-05-29 00:00:00.000000000 Z
+dependencies: []
+description: ASIMOV module for data import powered by the Bright Data web data platform.
+email: support@asimov.so
+executables:
+- asimov-brightdata-fetcher
+- asimov-brightdata-importer
+extensions: []
+extra_rdoc_files: []
+files:
+- AUTHORS
+- CHANGES.md
+- README.md
+- UNLICENSE
+- VERSION
+- bin/asimov-brightdata-fetcher
+- bin/asimov-brightdata-importer
+- lib/asimov-brightdata-module.rb
+- lib/asimov-brightdata-module/asimov-brightdata-fetcher
+- lib/asimov-brightdata-module/asimov-brightdata-importer
+- lib/asimov-brightdata-module/fetcher.rb
+- lib/asimov-brightdata-module/importer.rb
+homepage: https://github.com/asimov-modules
+licenses:
+- Unlicense
+metadata:
+  homepage_uri: https://github.com/asimov-modules
+  source_code_uri: https://github.com/asimov-modules/asimov-brightdata-module
+  changelog_uri: https://github.com/asimov-modules/asimov-brightdata-module/blob/master/CHANGES.md
+  bug_tracker_uri: https://github.com/asimov-modules/asimov-brightdata-module/issues
+  documentation_uri: https://github.com/asimov-modules/asimov-brightdata-module/blob/master/README.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.6.9
+specification_version: 4
+summary: ASIMOV module for data import powered by the Bright Data web data platform.
+test_files: []