fluent-plugin-oci-logging-fluentd-v1.15 1.0.3

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: e274d50ce9b66dc365cc18d13a7cb89c1e036420eb2cb00c6dc9fcd79278829b
+  data.tar.gz: c34fe1818586edca220158d5c7723b03656e623f475fed1e316f4ba9fbc2269c
+SHA512:
+  metadata.gz: 1374a784e0f06a49029fafa23c1fe539a308e670ea6e866f0c0b987d7c393e0fa079f3ff61406abf8b13d39c344f19ed82f01de0b828aacc29bcda41a585e320
+  data.tar.gz: dce98148332b5b8413922a35719c79b718b787dc64e35b4b0d285b436a97c494fd8baa7e3d6bde7f2ee39bb557957f6b75adb0081cf78586dc9cbfcf16a58381

data/README.md

@@ -0,0 +1,136 @@
+# Oracle Cloud Infrastructure Fluentd Plugin
+
+This is the official [fluentd](https://docs.fluentd.org/) plugin for the Oracle
+Cloud Infrastructure (OCI) Logging service. This project is open source, in
+active development and maintained by Oracle.
+
+## Requirements
+To use this fluentd plugin, you must have:
+
+* An Oracle Cloud Infrastructure acount.
+* A user created in that account, in a group with a policy that grants the
+desired permissions. This can be a user for yourself, or another person/system
+that needs to call the API. For an example of how to set up a new user, group,
+compartment, and policy, see [Adding Users](https://docs.cloud.oracle.com/Content/GSG/Tasks/addingusers.htm)
+in the Getting Started Guide. For a list of typical policies you may want to
+use, see [Common Policies](https://docs.cloud.oracle.com/Content/Identity/Concepts/commonpolicies.htm)
+in the User Guide.
+* Ruby version 2.2 or later running on Mac, Linux or Windows.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'fluent-plugin-oci-logging'
+```
+
+And then execute:
+
+```shell
+$ bundle
+```
+
+Or install it yourself as:
+
+```
+gem install fluent-plugin-oci-logging
+```
+
+Besides the plugin, the above commands will also automatically install fluentd,
+as well as the rest of the required ruby dependencies, in your system.
+
+## Configuration
+
+For usage with [instance principals](https://docs.oracle.com/en-us/iaas/Content/Identity/Tasks/callingservicesfrominstances.htm):
+
+```xml
+<source>
+  @type dummy
+  tag test
+  dummy {"test":"message"}
+</source>
+<match **>
+    @type oci_logging
+    log_object_id  ocid1.log.oc1.XXX.xxx
+</match>
+```
+
+For usage with an [API signing key]( https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm):
+
+```xml
+<source>
+  @type dummy
+  tag test
+  dummy {"test":"message"}
+</source>
+<match **>
+    @type oci_logging
+    principal_override user
+    log_object_id  ocid1.log.oc1.XXX.xxx
+</match>
+```
+
+To authenticate as a particular user, you need to [generate an API Signing Key](https://docs.cloud.oracle.com/en-us/iaas/Content/API/SDKDocs/cliconfigure.htm) for that user.
+
+## Logging Setup
+
+Detailed instructions, alongside examples, on how you can setup your logging
+environment can be found in the official [OCI docs](https://docs.cloud.oracle.com/en-us/iaas/Content/Logging/Task/managinglogs.htm).
+Also, to find out how to search your logs, you can check the documentation
+for [log search](https://docs.cloud.oracle.com/en-us/iaas/Content/Logging/Concepts/searchinglogs.htm).
+
+## Documentation
+
+Full documentation, including prerequisites, installation, and configuration
+instructions can be found [here](https://docs.cloud.oracle.com/en-us/iaas/Content/Logging/Concepts/loggingoverview.htm).
+
+API reference can be found [here](https://docs.cloud.oracle.com/en-us/iaas/tools/ruby/latest/index.html).
+
+This documentation can be found installed in your system in the gem specific directory. You can find its exact location by running the command:
+
+```shell
+gem contents fluent-plugin-oci-logging
+```
+
+Alternatively, you can also view it via ruby's documentation tool `ri` with the following command:
+
+```shell
+ri -f markdown fluent-plugin-oci-logging:README
+```
+
+Finally, you can view it by extracting the gem contents (the gem file itself is a tar archive).
+
+
+## Known Issues
+
+You can find information on any known issues with the SDK under the [Issues](https://github.com/oracle/fluent-plugin-oci-logging/issues) tab.
+
+## Questions or Feedback?
+
+Please [open an issue for any problems or questions](https://github.com/oracle/fluent-plugin-oci-logging/issues) you may have.
+
+Addtional ways to get in touch:
+
+* [Stack Overflow](https://stackoverflow.com/): Please use the [oracle-cloud-infrastructure](https://stackoverflow.com/questions/tagged/oracle-cloud-infrastructure) and [oci-ruby-sdk](https://stackoverflow.com/questions/tagged/oci-ruby-sdk) tags in your post
+* [Developer Tools section](https://community.oracle.com/community/cloud_computing/bare-metal/content?filterID=contentstatus%5Bpublished%5D~category%5Bdeveloper-tools%5D&filterID=contentstatus%5Bpublished%5D~objecttype~objecttype%5Bthread%5D) of the Oracle Cloud forums
+* [My Oracle Support](https://support.oracle.com)
+
+## Contributing
+
+This project welcomes contributions from the community. Before submitting a pull
+request, please [review our contribution guide](./CONTRIBUTING.md).
+
+## Security
+
+Please consult the [security guide](./SECURITY.md) for our responsible security
+vulnerability disclosure process.
+
+## License
+
+Copyright (c) 2021, Oracle and/or its affiliates.
+
+This software is dual-licensed to you under the Universal Permissive License (UPL) 1.0 as shown at <https://oss.oracle.com/licenses/upl>
+or Apache License 2.0 as shown at <http://www.apache.org/licenses/LICENSE-2.0>. You may choose either license.
+
+See [LICENSE](./LICENSE.txt) for more details.

data/lib/fluent/plugin/logging_setup.rb

@@ -0,0 +1,290 @@
+# fluent-plugin-oci-logging version 1.0.
+#
+# Copyright (c) 2021, Oracle and/or its affiliates.
+# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
+
+# frozen_string_literal: true
+
+require 'retriable'
+require 'oci'
+
+require_relative 'os'
+require_relative 'logging_utils'
+
+module Fluent
+  module Plugin
+    # Setup code for OCI Logging
+    module PublicLoggingSetup
+      RETRIES = 3
+      USER_SIGNER_TYPE = 'user'
+      AUTO_SIGNER_TYPE = 'auto'
+      LINUX_OCI_CONFIG_DIR = Fluent::Plugin::PublicLoggingUtils.determine_linux_config_path
+      WINDOWS_OCI_CONFIG_DIR = Fluent::Plugin::PublicLoggingUtils.determine_windows_config_path
+      USER_CONFIG_PROFILE_NAME = Fluent::Plugin::PublicLoggingUtils.determine_config_profile_name(LINUX_OCI_CONFIG_DIR, WINDOWS_OCI_CONFIG_DIR)
+      PUBLIC_RESOURCE_PRINCIPAL_ENV_FILE = '/etc/resource_principal_env'
+
+      R1_CA_PATH = '/etc/pki/tls/certs/ca-bundle.crt'
+      PUBLIC_DEFAULT_LINUX_CA_PATH = '/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'
+      PUBLIC_DEFAULT_WINDOWS_CA_PATH = 'C:\\oracle_unified_agent\\unified-monitoring-agent\\embedded\\ssl\\certs\\cacert.pem'
+      PUBLIC_DEFAULT_UBUNTU_CA_PATH = '/etc/ssl/certs/ca-certificates.crt'
+      PUBLIC_DEFAULT_DEBIAN_CA_PATH = PUBLIC_DEFAULT_UBUNTU_CA_PATH
+
+      def logger
+        @log ||= OS.windows? ? Logger.new(WINDOWS_UPLOADER_OUTPUT_LOG) : Logger.new($stdout)
+      end
+
+      ##
+      # Calculate federation endpoints based on metadata and optional inputs
+      #
+      # @param [String] region the region identifier
+      #
+      # @return [String] the federation endpoint that will be used
+      def get_federation_endpoint(region)
+        if region == 'r1'
+          endpoint = ENV['FEDERATION_ENDPOINT'] || 'https://auth.r1.oracleiaas.com/v1/x509'
+        else
+          if @realmDomainComponent.nil?
+            logger.warn('realm domain is null, fall back to OCI Regions')
+            @realmDomainComponent = OCI::Regions.get_second_level_domain(region) if @realmDomainComponent.nil?
+          end
+
+          endpoint = ENV['FEDERATION_ENDPOINT'] || "https://auth.#{region}.#{@realmDomainComponent}/v1/x509"
+        end
+
+        logger.info("endpoint is #{endpoint} in region #{region}")
+        endpoint
+      end
+
+      def get_instance_md_with_retry(retries = RETRIES)
+        Retriable.retriable(tries: retries, on: StandardError, timeout: 12) do
+          return get_instance_md
+        end
+      end
+
+      def get_instance_md
+        # v2 of IMDS requires an authorization header
+        md = get_instance_md_with_url('http://169.254.169.254/opc/v2/instance/')
+        if md.nil?
+          logger.info('IMDS v2 is not available, use v1')
+          md = get_instance_md_with_url('http://169.254.169.254/opc/v1/instance/')
+        end
+
+        if !md.nil?
+          logger.info "Successfully fetch instance metadata for hosts in overlay #{md}"
+          return md
+        else
+          raise StandardError, 'Failure fetching instance metadata, possible '\
+            'reason is network issue or host is not OCI instance'
+        end
+      end
+
+      def get_instance_md_with_url(uri_link)
+        uri = URI.parse(uri_link)
+        http = ::Net::HTTP.new(uri.host, uri.port)
+        http.open_timeout = 2 # in seconds
+        http.read_timeout = 2 # in seconds
+        request = ::Net::HTTP::Get.new(uri.request_uri)
+        request.add_field('Authorization', 'Bearer Oracle') if uri_link.include?('v2')
+        resp = http.request(request)
+        JSON.parse(resp.body)
+      rescue StandardError
+        logger.warn("failed to get instance metadata with link #{uri_link}")
+        nil
+      end
+
+      ##
+      # Calculate logging endpoint from environment or metadata.
+      # Preference is given to the environment variable 'LOGGING_FRONTEND'.
+      #
+      # @param [String] region the region identifier
+      #
+      # @return [String] The logging endpoint that will be used.
+      def get_logging_endpoint(region, logging_endpoint_override: nil)
+        unless logging_endpoint_override.nil?
+          logger.info "using logging endpoint override #{logging_endpoint_override} for testing"
+          return logging_endpoint_override
+        end
+
+        if region == 'r1'
+          endpoint = ENV['LOGGING_FRONTEND'] || "https://ingestion.logging.#{region}.oci.oracleiaas.com"
+        else
+          if @realmDomainComponent.nil?
+            logger.warn('realm domain is null, fall back to OCI Regions')
+            @realmDomainComponent = OCI::Regions.get_second_level_domain(region) if @realmDomainComponent.nil?
+          end
+
+          endpoint = ENV['LOGGING_FRONTEND'] || "https://ingestion.logging.#{region}.oci.#{@realmDomainComponent}"
+        end
+
+        logger.info("endpoint is #{endpoint} in region #{region}")
+        endpoint
+      end
+
+      def get_signer_type(principal_override: nil, config_dir: nil)
+        config_dir ||= OS.windows? ?  WINDOWS_OCI_CONFIG_DIR : LINUX_OCI_CONFIG_DIR
+
+        if (File.file?(config_dir) && principal_override != AUTO_SIGNER_TYPE) || principal_override == USER_SIGNER_TYPE
+          begin
+            logger.info("using #{USER_SIGNER_TYPE} signer type with config dir #{config_dir}")
+            oci_config = OCI::ConfigFileLoader.load_config(
+              config_file_location: config_dir, profile_name: USER_CONFIG_PROFILE_NAME
+            )
+            signer_type = USER_SIGNER_TYPE
+          rescue StandardError => e
+            if e.full_message.include?('Profile not found in the given config file.')
+              logger.warn("Profile #{USER_CONFIG_PROFILE_NAME} not configured "\
+                "in user api-key cli using other principal instead: #{e}")
+              signer_type = AUTO_SIGNER_TYPE
+              oci_config = OCI::Config.new
+            else
+              raise "User api-keys not setup correctly: #{e}"
+            end
+          end
+        else # if user api-keys is not setup in the expected format we expect instance principal
+          logger.info("using #{AUTO_SIGNER_TYPE} signer type")
+          signer_type = AUTO_SIGNER_TYPE
+          oci_config = OCI::Config.new
+        end
+        [oci_config, signer_type]
+      end
+
+      ##
+      # Configure the signer for the logging client call
+      #
+      # @param [String] signer_type the type of signer that should be returned
+      #
+      # @return [OCI::Signer] a signer that is representative of the signer type
+      def get_signer(oci_config, signer_type)
+        case signer_type
+        when USER_SIGNER_TYPE
+          get_host_info_for_non_oci_instance(oci_config)
+          set_default_ca_file
+          OCI::Signer.new(
+            oci_config.user,
+            oci_config.fingerprint,
+            oci_config.tenancy,
+            oci_config.key_file,
+            pass_phrase: oci_config.pass_phrase
+          )
+        when AUTO_SIGNER_TYPE
+          logger.info 'signer type is "auto", creating signer based on system setup'
+          get_host_info_for_oci_instance
+          set_default_ca_file
+          signer = create_resource_principal_signer
+          if signer.nil?
+            logger.info('resource principal is not setup, use instance principal instead')
+            signer = create_instance_principal_signer
+          else
+            logger.info('use resource principal')
+          end
+
+          signer
+        else
+          raise StandardError, "Principal type #{signer_type} not supported, "\
+            "use 'instance', 'resource' or 'user' instead"
+        end
+      end
+
+      def get_host_info_for_non_oci_instance(oci_config)
+        # set needed properties
+        @region = oci_config.region
+        # for non-OCI instances we can't get the display_name or hostname from IMDS and the fallback is the ip address
+        # of the machine
+        begin
+          @hostname = Socket.gethostname
+        rescue StandardError
+          ip = Socket.ip_address_list.detect { |intf| intf.ipv4_private? }
+          @hostname = ip ? ip.ip_address : 'UNKNOWN'
+        end
+
+        # No metadata service support for non-OCI instances
+        @realmDomainComponent = OCI::Regions.get_second_level_domain(@region)
+
+        logger.info("in non oci instance, region is #{@region}, "\
+                    " hostname is #{@hostname}, realm is #{@realmDomainComponent}")
+      end
+
+      def get_host_info_for_oci_instance
+        md = @metadata_override || get_instance_md_with_retry
+
+        @region = md['canonicalRegionName'] == 'us-seattle-1' ? 'r1' : md['canonicalRegionName']
+        @hostname = md.key?('displayName') ? md['displayName'] : ''
+        @realmDomainComponent = md.fetch('regionInfo', {}).fetch(
+          'realmDomainComponent', OCI::Regions.get_second_level_domain(@region)
+        )
+        logger.info("in oci instance, region is #{@region},  hostname is"\
+          " #{@hostname}, realm is #{@realmDomainComponent}")
+      end
+
+      ##
+      # Since r1 overlay has a different default make sure to update this
+      #
+      def set_default_ca_file
+        @ca_file = PUBLIC_DEFAULT_LINUX_CA_PATH if @ca_file.nil?
+        if OS.windows?
+          @ca_file = @ca_file == PUBLIC_DEFAULT_LINUX_CA_PATH ? PUBLIC_DEFAULT_WINDOWS_CA_PATH : @ca_file
+        elsif OS.ubuntu?
+          @ca_file = @ca_file == PUBLIC_DEFAULT_LINUX_CA_PATH ? PUBLIC_DEFAULT_UBUNTU_CA_PATH : @ca_file
+        elsif OS.debian?
+          @ca_file = @ca_file == PUBLIC_DEFAULT_LINUX_CA_PATH ? PUBLIC_DEFAULT_DEBIAN_CA_PATH : @ca_file
+        else
+          @ca_file = @region == 'r1' && @ca_file == PUBLIC_DEFAULT_LINUX_CA_PATH ? R1_CA_PATH : @ca_file
+        end
+
+        if @ca_file.nil?
+          msg = 'ca_file is not specified'
+          logger.error msg
+          raise StandardError, msg
+        end
+
+        # verify the ssl bundle actually exists
+        unless File.file?(@ca_file)
+          msg = "Does not exist or cannot open ca file: #{@ca_file}"
+          logger.error msg
+          raise StandardError, msg
+        end
+
+        # setting the cert_bundle_path
+        logger.info "using cert_bundle_path #{@ca_file}"
+      end
+
+      def create_resource_principal_signer
+        logger.info 'creating resource principal'
+        add_rp_env_override
+        OCI::Auth::Signers.resource_principals_signer
+      rescue StandardError => e
+        logger.info("fail to create resource principal with error #{e}")
+        nil
+      end
+
+      def add_rp_env_override
+        env_file = ENV['LOCAL_TEST_ENV_FILE'] || PUBLIC_RESOURCE_PRINCIPAL_ENV_FILE
+
+        resource_principal_env = {}
+
+        raise 'resource principal env file does not exist' unless File.exist? env_file
+
+        file = File.readlines(env_file)
+        file.each do |env|
+          a = env.split('=')
+          resource_principal_env[a[0]] = a[1].chomp
+        end
+
+        logger.info("resource principal env is setup with #{resource_principal_env}")
+        ENV.update resource_principal_env
+      end
+
+      def create_instance_principal_signer
+        endpoint = @federation_endpoint_override || get_federation_endpoint(@region)
+
+        unless endpoint.nil?
+          logger.info "create instance principal with  federation_endpoint = #{endpoint}, cert_bundle #{@ca_file}"
+        end
+
+        ::OCI::Auth::Signers::InstancePrincipalsSecurityTokenSigner.new(
+          federation_endpoint: endpoint, federation_client_cert_bundle: @ca_file
+        )
+      end
+    end
+  end
+end

data/lib/fluent/plugin/logging_utils.rb

@@ -0,0 +1,207 @@
+# fluent-plugin-oci-logging version 1.0.
+#
+# Copyright (c) 2021, Oracle and/or its affiliates.
+# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
+
+# frozen_string_literal: true
+
+require 'etc'
+require 'securerandom'
+require 'oci'
+
+module Fluent
+  module Plugin
+    # Common utility methods
+    module PublicLoggingUtils
+      PUBLIC_CLIENT_SPEC_VERSION = '1.0'
+      PUBLIC_LOGGING_PREFIX = 'com.oraclecloud.logging.custom.'
+
+      ##
+      # Build the wrapper for all the log entries to be sent to lumberjack.
+      #
+      # @return [OCI::LoggingClient::Models::PutLogsDetails] PutLogsDetails wrapper to be filled with log entries
+      def get_put_logs_details_request
+        request = OCI::Loggingingestion::Models::PutLogsDetails.new
+        request.specversion = PUBLIC_CLIENT_SPEC_VERSION
+        request
+      end
+
+      ##
+      # Build the requests to be sent to the Lumberjack endpoint.
+      #
+      # @param [Time] time Fluentd event time (may be different from event's
+      #                    timestamp)
+      # @param [Hash] record The fluentd record.
+      # @param [String] tagpath The fluentd path if populated.
+      # @param [Hash] log_batches_map List of pre-existing log batch.
+      # @param [String] sourceidentifier The fluentd contianing the source path.
+      #
+      # @return [Array] requests List of requests for Lumberjack's backend.
+      def build_request(time, record, tagpath, log_batches_map, sourceidentifier)
+        log = @log || Logger.new($stdout)
+        content = flatten_hash(record)
+
+        # create lumberjack request records
+        logentry = ::OCI::Loggingingestion::Models::LogEntry.new
+        logentry.time = Time.at(time).utc.strftime('%FT%T.%LZ')
+        begin
+          logentry.data = content.to_json
+        rescue StandardError
+          begin
+            log.warn('ruby "to_json" expected UTF-8 encoding will retry parsing with forced encoding')
+            # json requires UTF-8 encoding some logs may not follow that format so we need to fix that
+            content = encode_to_utf8(content)
+            logentry.data = content.to_json
+          rescue StandardError
+            log.warn('unexpected encoding in the log request, will send log as a string instead of json')
+            # instead of loosing data because of an unknown encoding issue send the data as a string
+            logentry.data = content.to_s
+          end
+        end
+        logentry.id = SecureRandom.uuid
+
+        requestkey = tagpath + sourceidentifier
+
+        unless log_batches_map.key?(requestkey)
+          log_entry_batch = OCI::Loggingingestion::Models::LogEntryBatch.new
+          log_entry_batch.source = @hostname
+          log_entry_batch.type = PUBLIC_LOGGING_PREFIX + tagpath
+          log_entry_batch.subject = sourceidentifier
+          log_entry_batch.defaultlogentrytime = Time.at(time).utc.strftime('%FT%T.%LZ')
+          log_entry_batch.entries = []
+
+          log_batches_map[requestkey] = log_entry_batch
+        end
+
+        log_batches_map[requestkey].entries << logentry
+      end
+
+      ##
+      # Send prebuilt requests to the logging endpoint.
+      #
+      # @param [Hash] log_batches_map
+      def send_requests(log_batches_map)
+        log = @log || Logger.new($stdout) # for testing
+
+        request = get_put_logs_details_request
+
+        request.log_entry_batches = log_batches_map.values
+        begin
+          resp = @client.put_logs(@log_object_id, request)
+        rescue OCI::Errors::ServiceError => e
+          log.error "Service Error received sending request: #{e}"
+          if e.status_code == 400
+            log.info 'Eating service error as it is caused by Bad Request[400 HTTP code]'
+          else
+            log.error "Throwing service error for status code:#{e.status_code} as we want fluentd to re-try"
+            raise
+          end
+        rescue OCI::Errors::NetworkError => e
+          log.error "Network Error received sending request: #{e}"
+          if e.code == 400
+            log.info 'Eating network error as it is caused by Bad Request[400 HTTP code]'
+          else
+            log.error "Throwing network error for code:#{e.code} as we want fluentd to re-try"
+            raise
+          end
+        rescue StandardError => e
+          log.error "Standard Error received sending request: #{e}"
+          raise
+        end
+        request.log_entry_batches.each do |batch|
+          log.info "log batch type #{batch.type}, log batch subject #{batch.subject}, size #{batch.entries.size}"
+        end
+
+        log.info "response #{resp.status} id: #{resp.request_id}"
+      end
+
+      ##
+      # Flatten the keys of a hash.
+      #
+      # @param [Hash] record The hash object to flatten.
+      #
+      # @return [Hash] The updated, flattened, hash.
+      def flatten_hash(record)
+        record.each_with_object({}) do |(k, v), h|
+          if v.is_a? Hash
+            flatten_hash(v).map { |h_k, h_v| h["#{k}.#{h_k}"] = h_v }
+          elsif k == 'log'
+            h['msg'] = v
+          else
+            h[k] = v
+          end
+        end
+      end
+
+      ##
+      # Force all the string values in the hash to be encoded to UTF-8.
+      #
+      # @param [Hash] record the flattened hash needing to have the encoding changes
+      #
+      # @return [Hash] The updated hash.
+      def encode_to_utf8(record)
+        # the reason for using ISO-8859-1 is that it covers most of the out
+        # of band characters other encoding's don't have this way we can
+        # encode it to a known value and then encode it back into UTF-8
+        record.transform_values { |v| v.to_s.force_encoding('ISO-8859-1').encode('UTF-8') }
+      end
+
+      ##
+      # Parse out the log_type from the chunk metadata tag
+      #
+      # @param [String] rawtag the string of the chunk metadata tag that needs to be parsed
+      #
+      # @return [String] take out the tag after the first '.' character or return the whole tag if there is no '.'
+      def get_modified_tag(rawtag)
+        tag = rawtag || 'empty'
+        tag.split('.').length > 1 ? tag.partition('.')[2] : tag
+      end
+
+      ##
+      # Return the correct oci configuration file path for linux platforms.
+      #
+      # @return [String] path to the configuration file
+      def self.determine_linux_config_path
+        managed_agent = '/etc/unified-monitoring-agent/.oci/config'
+        unmanaged_agent = File.join(Dir.home, '.oci/config')
+        config = File.file?(managed_agent) ? managed_agent : unmanage_agent
+        return config
+      rescue StandardError
+        return unmanaged_agent
+      end
+
+      ##
+      # Return the correct oci configuration file path for windows platforms.
+      #
+      # @return [String] path to the configuration file
+      def self.determine_windows_config_path
+        managed_agent = 'C:\\oracle_unified_agent\\.oci\\config'
+        unmanaged_agent = File.join("C:\\Users\\#{Etc.getlogin}\\.oci\\config")
+        config = File.file?(managed_agent) ? managed_agent : unmanage_agent
+        return config
+      rescue StandardError
+        return unmanaged_agent
+      end
+
+      ##
+      # Return the correct oci configuration user profile for auth
+      #
+      # @param [String] linux_cfg Path to linux configuration file
+      # @param [String] windows_cfg Path to windows configuration file
+      #
+      # @return [String] username to use
+      def self.determine_config_profile_name(linux_cfg, windows_cfg)
+        managed_agent='UNIFIED_MONITORING_AGENT'
+        unmanaged_agent='DEFAULT'
+        location = OS.windows? ? windows_cfg : linux_cfg
+        if location.start_with?('/etc/unified-monitoring-agent',  'C:\\oracle_unified_agent')
+          return managed_agent
+        end
+        oci_config = OCI::ConfigFileLoader.load_config(config_file_location: location)
+        oci_config.profile
+      rescue StandardError
+        unmanaged_agent
+      end
+    end
+  end
+end

data/lib/fluent/plugin/os.rb

@@ -0,0 +1,70 @@
+# fluent-plugin-oci-logging version 1.0.
+#
+# Copyright (c) 2021, Oracle and/or its affiliates.
+# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
+
+# frozen_string_literal: true
+
+# Utility class to determine the underlying operating system.
+module OS
+  def self.windows?
+    (/cygwin|mswin|mingw|bccwin|wince|emx/ =~ RUBY_PLATFORM) != nil
+  end
+
+  def self.mac?
+    (/darwin/ =~ RUBY_PLATFORM) != nil
+  end
+
+  def self.unix?
+    !OS.windows?
+  end
+
+  def self.linux?
+    unix? and !mac?
+  end
+
+  def self.ubuntu?
+    linux? and os_name_ubuntu?
+  end
+
+  def self.debian?
+    linux? and os_name_debian?
+  end
+
+  def self.os_name_ubuntu?
+    os_name = 'not_found'
+    file_name = '/etc/os-release'
+    if File.exists?(file_name)
+      IO.foreach(file_name).each do |line|
+        if line.start_with?('ID=')
+          os_name = line.split('=')[1].strip
+        end
+      end
+    else
+      logger.info('Unknown linux distribution detected')
+    end
+    os_name == 'ubuntu' ? true : false
+  rescue StandardError => e
+    log.error "Unable to detect ubuntu platform due to: #{e.message}"
+    false
+  end
+
+  def self.os_name_debian?
+    os_name = 'not_found'
+    file_name = '/etc/os-release'
+    if File.exists?(file_name)
+      File.foreach(file_name).each do |line|
+        if line.start_with?('ID=')
+          os_name = line.split('=')[1].strip
+        end
+      end
+    else
+      logger.info('Unknown linux distribution detected')
+    end
+    os_name == 'debian' ? true : false
+  rescue StandardError => e
+    log.error "Unable to detect ubuntu platform due to: #{e.message}"
+    false
+  end
+
+end

data/lib/fluent/plugin/out_oci_logging.rb

@@ -0,0 +1,105 @@
+# fluent-plugin-oci-logging version 1.0.
+#
+# Copyright (c) 2021, Oracle and/or its affiliates.
+# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
+
+# frozen_string_literal: true
+
+require_relative 'logging_utils'
+require_relative 'logging_setup'
+require 'socket'
+
+module Fluent
+  module Plugin
+    # OCI Logging Fluentd Output plugin
+    class OCILoggingOutput < Fluent::Plugin::Output
+      include Fluent::Plugin::PublicLoggingUtils
+      include Fluent::Plugin::PublicLoggingSetup
+
+      Fluent::Plugin.register_output('oci_logging', self)
+
+      # allow instance metadata to be configurable (for testing)
+      config_param :metadata_override, :hash, default: nil
+
+      # Path to the PEM CA certificate file for TLS. Can contain several CA certs.
+      # We are defaulting to '/etc/oci-pki/ca-bundle.pem'
+      # This can be overriden for testing.
+      config_param :ca_file, :string, default: PUBLIC_DEFAULT_LINUX_CA_PATH
+
+      # Override to manually provide the host endpoint
+      config_param :logging_endpoint_override, :string, default: nil
+
+      # Override to manually provide the federation endpoint
+      config_param :federation_endpoint_override, :string, default: nil
+
+      # The only required parameter used to identify where we are sending logs for LJ
+      config_param :log_object_id, :string
+
+      # optional forced override for debugging, testing, and potential custom configurations
+      config_param :principal_override, :string, default: nil
+
+      attr_accessor :client, :hostname
+
+      helpers :event_emitter
+
+      PAYLOAD_SIZE = 9*1024*1024 #restricting payload size at 9MB
+
+      def configure(conf)
+        super
+        log.debug 'determining the signer type'
+
+        oci_config, signer_type = get_signer_type(principal_override: @principal_override)
+        signer = get_signer(oci_config, signer_type)
+        log.info "using authentication principal #{signer_type}"
+
+        @client = OCI::Loggingingestion::LoggingClient.new(
+          config: oci_config,
+          endpoint: get_logging_endpoint(@region, logging_endpoint_override: @logging_endpoint_override),
+          signer: signer,
+          proxy_settings: nil,
+          retry_config: nil
+        )
+
+        @client.api_client.request_option_overrides = { ca_file: @ca_file }
+      end
+
+      def start
+        super
+        log.debug 'start'
+      end
+
+      #### Sync Buffered Output ##############################
+      # Implement write() if your plugin uses a normal buffer.
+      ########################################################
+      def write(chunk)
+        log.debug "writing chunk metadata #{chunk.metadata}", \
+                  dump_unique_id_hex(chunk.unique_id)
+        log_batches_map = {}
+        # For standard chunk format (without #format() method) 
+        size = 0 
+        chunk.each do |time, record|
+          begin
+            tag = get_modified_tag(chunk.metadata.tag)
+            source_identifier = record.key?('tailed_path') ? record['tailed_path'] : ''
+            content = flatten_hash(record)
+            size += content.to_json.bytesize
+            build_request(time, record, tag, log_batches_map, source_identifier)
+            if size >= PAYLOAD_SIZE
+              log.info "Exceeding payload size. Size : #{size}"
+              send_requests(log_batches_map)
+              log_batches_map = {}
+              size = 0
+            end
+          rescue StandardError => e
+            log.error(e.full_message)
+          end
+        end
+        # flushing data to LJ
+        unless log_batches_map.empty?
+          log.info "Payload size : #{size}"
+          send_requests(log_batches_map)
+        end
+      end
+    end
+  end
+end

data/lib/version.rb

@@ -0,0 +1,22 @@
+# fluent-plugin-oci-logging version 1.0.
+#
+# Copyright (c) 2021, Oracle and/or its affiliates.
+# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
+
+# frozen_string_literal: true
+
+require 'open3'
+
+def number_of_commits
+  tstamp = Time.now.utc.strftime('%Y%m%d%H%M%S').to_s
+  cmd = 'git log --pretty=oneline | wc -l'
+  stdout_str, stderr_str, status = Open3.capture3(cmd)
+  status.success? ? stdout_str.strip() : "rc-#{tstamp}"
+end
+
+module OCILogging
+  MAJOR=1
+  MINOR=0
+  PATCH=3
+  VERSION = ENV['OCI_LOGGING_GEM_VERSION'] || "#{MAJOR}.#{MINOR}.#{PATCH}"
+end

metadata

@@ -0,0 +1,193 @@
+--- !ruby/object:Gem::Specification
+name: fluent-plugin-oci-logging-fluentd-v1.15
+version: !ruby/object:Gem::Version
+  version: 1.0.3
+platform: ruby
+authors:
+- OCI Observability Team
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2023-02-14 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: fluentd
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.15.3
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.15.3
+- !ruby/object:Gem::Dependency
+  name: oci
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.12'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.12'
+- !ruby/object:Gem::Dependency
+  name: retriable
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.1.2
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.1.2
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: mocha
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.9'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.9'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+- !ruby/object:Gem::Dependency
+  name: rubocop-rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
+- !ruby/object:Gem::Dependency
+  name: simplecov
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.17'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.17'
+- !ruby/object:Gem::Dependency
+  name: test-unit
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+description: 'Oracle Observability FluentD Plugins : Logging output plugin for OCI
+  logging'
+email:
+- no-reply@support.oracle.com
+executables: []
+extensions: []
+extra_rdoc_files:
+- README.md
+files:
+- README.md
+- lib/fluent/plugin/logging_setup.rb
+- lib/fluent/plugin/logging_utils.rb
+- lib/fluent/plugin/os.rb
+- lib/fluent/plugin/out_oci_logging.rb
+- lib/version.rb
+homepage: https://docs.cloud.oracle.com/en-us/iaas/Content/Logging/Concepts/loggingoverview.htm
+licenses:
+- UPL-1.0
+- Apache-2.0
+metadata:
+  source_code_uri: https://github.com/oracle/fluent-plugin-oci-logging
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.3.1
+signing_key: 
+specification_version: 4
+summary: OCI Fluentd Logging output plugin following Unified Schema
+test_files: []