datadog-ci 1.19.0 → 1.20.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6c730f78f9fefc725ba4499a38729eff14f5fc48a4a8e6a055ade2b4cedc030a
-  data.tar.gz: 86a9682f7623bc85a8e8a05fd8b1f91ecb12f459e7f2425fd2fb79d4ce217227
+  metadata.gz: b04703fefecfa1e090ce2265ed419e4fa41c170a35c897ab2ac900fe9bca467b
+  data.tar.gz: b94de33937e2116579dcaf36c7a7c4f6040597420b27dff1e0f47641f1966be2
 SHA512:
-  metadata.gz: ebb3fca5a5cf1db85795b017ffefef4bb09560c4c8c65f54b4d4cc88b93016776a861ec5c48bf700bdff4ee18c4416f6aee632b5abd392ae930a1a1b00042f0c
-  data.tar.gz: 9b5b171277dc7bb8510dec4bd332a937431ac4bb3f277c9ad3221dd38f3d5eec27b1c92415aa47c0289259543d2ee21fe8ae1dc532e0f9d9376706a38e59e712
+  metadata.gz: 0bce96aa0950aecccac7b7382c731c62d29035b403231fec3ca7f59bb85768e52e0041cfa709a62ed5054e360979776d37e0a501a587d875f3263e82d455ab14
+  data.tar.gz: 6035f0db0a5a7eb9aae0f51813f330acd3abbe43175000513860abf5ad406f0d76d0aaf8f6e09409f00c251816aae84c7c3757eb56a660fdcefe7e9cbd478d7a

data/CHANGELOG.md

@@ -1,5 +1,16 @@
 ## [Unreleased]
 
+## [1.20.0] - 2025-07-03
+
+### Added
+
+* Add Datadog output to RSpec documentation formatter ([#355][])
+* Add Drone CI support and update CI envs handling ([#349][])
+
+### Fixed
+
+* FIX: send git.tag when requesting library settings if git.branch is not available ([#359][])
+
 ## [1.19.0] - 2025-06-16
 
 ### Changed
@@ -463,7 +474,8 @@ Currently test suite level visibility is not used by our instrumentation: it wil
 
 - Ruby versions < 2.7 no longer supported ([#8][])
 
-[Unreleased]: https://github.com/DataDog/datadog-ci-rb/compare/v1.19.0...main
+[Unreleased]: https://github.com/DataDog/datadog-ci-rb/compare/v1.20.0...main
+[1.20.0]: https://github.com/DataDog/datadog-ci-rb/compare/v1.19.0...v1.20.0
 [1.19.0]: https://github.com/DataDog/datadog-ci-rb/compare/v1.18.0...v1.19.0
 [1.18.0]: https://github.com/DataDog/datadog-ci-rb/compare/v1.17.0...v1.18.0
 [1.17.0]: https://github.com/DataDog/datadog-ci-rb/compare/v1.16.0...v1.17.0
@@ -660,4 +672,7 @@ Currently test suite level visibility is not used by our instrumentation: it wil
 [#327]: https://github.com/DataDog/datadog-ci-rb/issues/327
 [#329]: https://github.com/DataDog/datadog-ci-rb/issues/329
 [#335]: https://github.com/DataDog/datadog-ci-rb/issues/335
-[#339]: https://github.com/DataDog/datadog-ci-rb/issues/339
+[#339]: https://github.com/DataDog/datadog-ci-rb/issues/339
+[#349]: https://github.com/DataDog/datadog-ci-rb/issues/349
+[#355]: https://github.com/DataDog/datadog-ci-rb/issues/355
+[#359]: https://github.com/DataDog/datadog-ci-rb/issues/359

data/README.md

@@ -9,34 +9,25 @@ Learn more on our [official website](https://docs.datadoghq.com/tests/) and chec
 ## Features
 
 - [Test Visibility](https://docs.datadoghq.com/tests/) - collect metrics and results for your tests
+- [Flaky test management](https://docs.datadoghq.com/tests/flaky_management/) - track, triage, and remediate flaky tests across your organization. Quarantine or disable problematic tests to keep known flakes from breaking builds, and create cases and Jira issues to track work toward fixes.
+- [Auto test retries](https://docs.datadoghq.com/tests/flaky_tests/auto_test_retries/?tab=ruby) - retrying failing tests up to N times to avoid failing your build due to flaky tests
+- [Early flake detection](https://docs.datadoghq.com/tests/flaky_tests/early_flake_detection/?tab=ruby) - Datadog’s test flakiness solution that identifies flakes early by running newly added tests multiple times
 - [Test impact analysis](https://docs.datadoghq.com/tests/test_impact_analysis/) - save time by selectively running only tests affected by code changes
-- [Flaky test management](https://docs.datadoghq.com/tests/flaky_test_management/) - track, alert, search your flaky tests in Datadog UI
-- [Auto test retries](https://docs.datadoghq.com/tests/flaky_test_management/auto_test_retries/?tab=ruby) - retrying failing tests up to N times to avoid failing your build due to flaky tests
-- [Early flake detection](https://docs.datadoghq.com/tests/flaky_test_management/early_flake_detection/?tab=ruby) - Datadog’s test flakiness solution that identifies flakes early by running newly added tests multiple times
-- [Search and manage CI tests](https://docs.datadoghq.com/tests/search/)
+- [Test Health](https://docs.datadoghq.com/tests/test_health) - The Test Health dashboard provides analytics to help teams manage and optimize their testing in CI. This includes sections showing the current impact of test flakiness and how Test Optimization is mitigating these problems.
+- [Inspect your tests' logs in Datadog](https://docs.datadoghq.com/tests/correlate_logs_and_tests)
 - [Enhance developer workflows](https://docs.datadoghq.com/tests/developer_workflows)
 - [Add custom measures to your tests](https://docs.datadoghq.com/tests/guides/add_custom_measures/?tab=ruby)
 - [Browser tests integration with Datadog RUM](https://docs.datadoghq.com/tests/browser_tests)
 
-## Installation
-
-Add to your Gemfile.
+## Setup
 
-```ruby
-group :test do
-  gem "datadog-ci"
-end
-```
+- [Test visibility setup](https://docs.datadoghq.com/tests/setup/ruby/?tab=cloudciprovideragentless)
+- [Test impact analysis setup](https://docs.datadoghq.com/tests/test_impact_analysis/setup/ruby/?tab=cloudciprovideragentless) (test visibility setup is required before setting up test impact analysis)
 
 ## Upgrade from ddtrace v1.x
 
 If you used [test visibility for Ruby](https://docs.datadoghq.com/tests/setup/ruby/) with [ddtrace](https://github.com/datadog/dd-trace-rb) gem, check out our [upgrade guide](/docs/UpgradeGuide.md).
 
-## Setup
-
-- [Test visibility setup](https://docs.datadoghq.com/tests/setup/ruby/?tab=cloudciprovideragentless)
-- [Test impact analysis setup](https://docs.datadoghq.com/tests/test_impact_analysis/setup/ruby/?tab=cloudciprovideragentless) (test visibility setup is required before setting up test impact analysis)
-
 ## Contributing
 
 See [development guide](/docs/DevelopmentGuide.md), [static typing guide](docs/StaticTypingGuide.md) and [contributing guidelines](/CONTRIBUTING.md).

data/lib/datadog/ci/contrib/rspec/configuration/settings.rb

@@ -25,6 +25,12 @@ module Datadog
               end
             end
 
+            option :datadog_formatter_enabled do |o|
+              o.type :bool
+              o.env Ext::ENV_DATADOG_FORMATTER_ENABLED
+              o.default true
+            end
+
             # internal only
             option :dry_run_enabled do |o|
               o.type :bool

data/lib/datadog/ci/contrib/rspec/documentation_formatter.rb

@@ -0,0 +1,126 @@
+module Datadog
+  module CI
+    module Contrib
+      module RSpec
+        # Instrument RSpec::Core::Formatters::DocumentationFormatter
+        module DocumentationFormatter
+          def self.included(base)
+            base.prepend(InstanceMethods)
+          end
+
+          module InstanceMethods
+            def example_passed(notification)
+              super
+
+              dd_example_finished(notification.example)
+            end
+
+            def example_failed(notification)
+              super
+
+              dd_example_finished(notification.example)
+            end
+
+            def example_pending(notification)
+              super
+
+              dd_example_finished(notification.example)
+            end
+
+            def dump_summary(notification)
+              super
+
+              return unless @dd_flaky_tests || @dd_quarantined_tests || @dd_disabled_tests || @dd_skipped_by_tia_tests
+
+              summary = "\nDatadog Test Optimization Summary:\n"
+              summary << "#{@dd_flaky_tests} flaky detected " if @dd_flaky_tests&.positive?
+              summary << "#{@dd_quarantined_tests} quarantined " if @dd_quarantined_tests&.positive?
+              summary << "#{@dd_disabled_tests} disabled " if @dd_disabled_tests&.positive?
+              summary << "#{@dd_skipped_by_tia_tests} skipped by test impact analysis " if @dd_skipped_by_tia_tests&.positive?
+              summary << "\n"
+
+              output.puts summary
+            end
+
+            private
+
+            def dd_example_finished(example)
+              @group_level += 1
+
+              dd_retries_output(example)
+              dd_test_management_output(example)
+              dd_test_impact_analysis_output(example)
+
+              @group_level -= 1
+            end
+
+            def dd_retries_output(example)
+              if !example.metadata[Ext::METADATA_DD_RETRIES] ||
+                  !CI::Ext::Test::RetryReason::DATADOG_RETRY_REASONS.include?(
+                    example.metadata[Ext::METADATA_DD_RETRY_REASON]
+                  )
+                return
+              end
+
+              retries_count = example.metadata[Ext::METADATA_DD_RETRIES]
+
+              output.puts(
+                "#{current_indentation}| Retried #{retries_count} times by #{dd_retry_source(example.metadata[Ext::METADATA_DD_RETRY_REASON])}"
+              )
+              results = example.metadata[Ext::METADATA_DD_RETRY_RESULTS]
+              results_output = results.map { |status, count| "#{count} / #{retries_count} #{status}" }.join(", ")
+              output.puts(
+                "#{current_indentation}| Results were: #{results_output}"
+              )
+
+              if results[CI::Ext::Test::Status::FAIL] > 0 && results[CI::Ext::Test::Status::PASS] > 0
+                output.puts(
+                  "#{current_indentation}| Flaky test detected"
+                )
+
+                @dd_flaky_tests ||= 0
+                @dd_flaky_tests += 1
+              end
+            end
+
+            def dd_retry_source(reason)
+              case reason
+              when CI::Ext::Test::RetryReason::RETRY_FAILED
+                "Datadog Auto Test Retries"
+              when CI::Ext::Test::RetryReason::RETRY_DETECT_FLAKY
+                "Datadog Early Flake Detection"
+              when CI::Ext::Test::RetryReason::RETRY_FLAKY_FIXED
+                "Datadog Flaky Test Management"
+              else
+                "Datadog Test Optimization"
+              end
+            end
+
+            def dd_test_management_output(example)
+              if example.metadata[Ext::METADATA_DD_QUARANTINED]
+                output.puts("#{current_indentation}| Test was quarantined by Datadog Flaky Test Management")
+
+                @dd_quarantined_tests ||= 0
+                @dd_quarantined_tests += 1
+              end
+
+              if example.metadata[Ext::METADATA_DD_DISABLED]
+                output.puts("#{current_indentation}| Test was disabled by Datadog Flaky Test Management")
+
+                @dd_disabled_tests ||= 0
+                @dd_disabled_tests += 1
+              end
+            end
+
+            def dd_test_impact_analysis_output(example)
+              if example.metadata[Ext::METADATA_DD_SKIPPED_BY_ITR]
+                @dd_skipped_by_tia_tests ||= 0
+                @dd_skipped_by_tia_tests += 1
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/datadog/ci/contrib/rspec/example.rb

@@ -74,6 +74,24 @@ module Datadog
                       exception: execution_result.pending_exception
                     )
                   end
+
+                  if datadog_configuration[:datadog_formatter_enabled]
+                    if test_span&.is_retry?
+                      metadata[Ext::METADATA_DD_RETRIES] ||= 0
+                      metadata[Ext::METADATA_DD_RETRY_RESULTS] ||= Hash.new(0)
+
+                      metadata[Ext::METADATA_DD_RETRIES] += 1
+                      metadata[Ext::METADATA_DD_RETRY_REASON] = test_span&.retry_reason
+                      metadata[Ext::METADATA_DD_RETRY_RESULTS][test_span&.status] += 1
+                    end
+
+                    metadata[Ext::METADATA_DD_QUARANTINED] = true if test_span&.quarantined?
+                    metadata[Ext::METADATA_DD_DISABLED] = true if test_span&.disabled?
+
+                    if test_span&.skipped_by_test_impact_analysis?
+                      metadata[Ext::METADATA_DD_SKIPPED_BY_ITR] = true
+                    end
+                  end
                 end
               end
 

data/lib/datadog/ci/contrib/rspec/ext.rb

@@ -11,6 +11,15 @@ module Datadog
           DEFAULT_SERVICE_NAME = "rspec"
 
           ENV_ENABLED = "DD_TRACE_RSPEC_ENABLED"
+          ENV_DATADOG_FORMATTER_ENABLED = "DD_TRACE_RSPEC_DATADOG_FORMATTER_ENABLED"
+
+          # Metadata keys
+          METADATA_DD_RETRIES = :dd_retries
+          METADATA_DD_RETRY_RESULTS = :dd_retry_results
+          METADATA_DD_RETRY_REASON = :dd_retry_reason
+          METADATA_DD_QUARANTINED = :dd_quarantined
+          METADATA_DD_DISABLED = :dd_disabled
+          METADATA_DD_SKIPPED_BY_ITR = :dd_skipped_by_itr
         end
       end
     end

data/lib/datadog/ci/contrib/rspec/patcher.rb

@@ -5,6 +5,7 @@ require_relative "../patcher"
 require_relative "example"
 require_relative "example_group"
 require_relative "runner"
+require_relative "documentation_formatter"
 
 module Datadog
   module CI
@@ -20,6 +21,7 @@ module Datadog
             ::RSpec::Core::Runner.include(Runner)
             ::RSpec::Core::Example.include(Example)
             ::RSpec::Core::ExampleGroup.include(ExampleGroup)
+            ::RSpec::Core::Formatters::DocumentationFormatter.include(DocumentationFormatter)
           end
         end
       end

data/lib/datadog/ci/ext/environment/configuration_discrepancy_checker.rb

@@ -0,0 +1,91 @@
+# frozen_string_literal: true
+
+require_relative "../git"
+require_relative "../telemetry"
+require_relative "../../utils/telemetry"
+
+module Datadog
+  module CI
+    module Ext
+      module Environment
+        class ConfigurationDiscrepancyChecker
+          def initialize(ci_provider_env_tags, local_git_tags, user_provided_tags)
+            @ci_provider_env_tags = ci_provider_env_tags
+            @local_git_tags = local_git_tags
+            @user_provided_tags = user_provided_tags
+          end
+
+          def check_for_discrepancies
+            checks = [
+              {
+                left: normalize_value(@ci_provider_env_tags[Git::TAG_COMMIT_SHA]),
+                right: normalize_value(@local_git_tags[Git::TAG_COMMIT_SHA]),
+                type: "commit_discrepancy",
+                expected: "ci_provider",
+                discrepant: "git_client"
+              },
+              {
+                left: normalize_value(@user_provided_tags[Git::TAG_COMMIT_SHA]),
+                right: normalize_value(@local_git_tags[Git::TAG_COMMIT_SHA]),
+                type: "commit_discrepancy",
+                expected: "user_supplied",
+                discrepant: "git_client"
+              },
+              {
+                left: normalize_value(@user_provided_tags[Git::TAG_COMMIT_SHA]),
+                right: normalize_value(@ci_provider_env_tags[Git::TAG_COMMIT_SHA]),
+                type: "commit_discrepancy",
+                expected: "user_supplied",
+                discrepant: "ci_provider"
+              },
+              {
+                left: normalize_value(@ci_provider_env_tags[Git::TAG_REPOSITORY_URL]),
+                right: normalize_value(@local_git_tags[Git::TAG_REPOSITORY_URL]),
+                type: "repository_discrepancy",
+                expected: "ci_provider",
+                discrepant: "git_client"
+              },
+              {
+                left: normalize_value(@user_provided_tags[Git::TAG_REPOSITORY_URL]),
+                right: normalize_value(@local_git_tags[Git::TAG_REPOSITORY_URL]),
+                type: "repository_discrepancy",
+                expected: "user_supplied",
+                discrepant: "git_client"
+              },
+              {
+                left: normalize_value(@user_provided_tags[Git::TAG_REPOSITORY_URL]),
+                right: normalize_value(@ci_provider_env_tags[Git::TAG_REPOSITORY_URL]),
+                type: "repository_discrepancy",
+                expected: "user_supplied",
+                discrepant: "ci_provider"
+              }
+            ]
+
+            git_info_match = true
+            checks.each do |check|
+              if check[:left] && check[:right] && check[:left] != check[:right]
+                Utils::Telemetry.inc(Ext::Telemetry::METRIC_GIT_COMMIT_SHA_DISCREPANCY, 1, {
+                  type: check[:type].to_s,
+                  expected_provider: check[:expected].to_s,
+                  discrepant_provider: check[:discrepant].to_s
+                })
+                git_info_match = false
+              end
+            end
+
+            Utils::Telemetry.inc(Ext::Telemetry::METRIC_GIT_COMMIT_SHA_MATCH, 1, {
+              matched: git_info_match.to_s
+            })
+          end
+
+          private
+
+          def normalize_value(value)
+            return nil if value.nil? || value == ""
+            value
+          end
+        end
+      end
+    end
+  end
+end

data/lib/datadog/ci/ext/environment/extractor.rb

@@ -49,7 +49,9 @@ module Datadog
               Git::TAG_COMMIT_SHA => @provider.git_commit_sha,
               Git::TAG_PULL_REQUEST_BASE_BRANCH => @provider.git_pull_request_base_branch,
               Git::TAG_PULL_REQUEST_BASE_BRANCH_SHA => @provider.git_pull_request_base_branch_sha,
-              Git::TAG_COMMIT_HEAD_SHA => @provider.git_commit_head_sha
+              Git::TAG_COMMIT_HEAD_SHA => @provider.git_commit_head_sha,
+
+              Environment::TAG_PR_NUMBER => @provider.pr_number
             }
 
             # Normalize Git references and filter sensitive data
@@ -57,13 +59,16 @@ module Datadog
             # Expand ~
             expand_workspace!
 
+            # Convert all tag values to strings
+            @tags.transform_values! { |v| v&.to_s }
+
             # remove empty tags
             @tags.reject! do |_, v|
               # setting type of v here to untyped because steep does not
               # understand `v.nil? || something`
 
               # @type var v: untyped
-              v.nil? || v.strip.empty?
+              v.nil? || v.to_s.strip.empty?
             end
 
             @tags
@@ -80,6 +85,7 @@ module Datadog
 
             @tags[Git::TAG_TAG] = Utils::Git.normalize_ref(@tags[Git::TAG_TAG])
             @tags[Git::TAG_BRANCH] = Utils::Git.normalize_ref(@tags[Git::TAG_BRANCH])
+            @tags[Git::TAG_PULL_REQUEST_BASE_BRANCH] = Utils::Git.normalize_ref(@tags[Git::TAG_PULL_REQUEST_BASE_BRANCH])
             @tags[Git::TAG_REPOSITORY_URL] = Datadog::Core::Utils::Url.filter_basic_auth(
               @tags[Git::TAG_REPOSITORY_URL]
             )

data/lib/datadog/ci/ext/environment/providers/appveyor.rb

@@ -74,6 +74,14 @@ module Datadog
               env["APPVEYOR_REPO_COMMIT_AUTHOR_EMAIL"]
             end
 
+            def git_commit_head_sha
+              env["APPVEYOR_PULL_REQUEST_HEAD_COMMIT"]
+            end
+
+            def pr_number
+              env["APPVEYOR_PULL_REQUEST_NUMBER"]
+            end
+
             def git_commit_message
               commit_message = env["APPVEYOR_REPO_COMMIT_MESSAGE"]
               if commit_message

data/lib/datadog/ci/ext/environment/providers/azure.rb

@@ -80,6 +80,14 @@ module Datadog
               env["BUILD_SOURCEVERSIONMESSAGE"]
             end
 
+            def git_pull_request_base_branch
+              env["SYSTEM_PULLREQUEST_TARGETBRANCH"]
+            end
+
+            def pr_number
+              env["SYSTEM_PULLREQUEST_PULLREQUESTNUMBER"]
+            end
+
             def ci_env_vars
               {
                 "SYSTEM_TEAMPROJECTID" => env["SYSTEM_TEAMPROJECTID"],

data/lib/datadog/ci/ext/environment/providers/base.rb

@@ -105,6 +105,9 @@ module Datadog
             def git_commit_head_sha
             end
 
+            def pr_number
+            end
+
             private
 
             def set_branch_and_tag

data/lib/datadog/ci/ext/environment/providers/bitbucket.rb

@@ -65,6 +65,10 @@ module Datadog
               env["BITBUCKET_PR_DESTINATION_BRANCH"]
             end
 
+            def pr_number
+              env["BITBUCKET_PR_ID"]
+            end
+
             private
 
             def url

data/lib/datadog/ci/ext/environment/providers/bitrise.rb

@@ -79,6 +79,10 @@ module Datadog
               # For example, a pull request wants to merge the content of a branch into the branch main. In this case, this Env Var’s value is main.
               env["BITRISEIO_GIT_BRANCH_DEST"]
             end
+
+            def pr_number
+              env["BITRISE_PULL_REQUEST"]
+            end
           end
         end
       end

data/lib/datadog/ci/ext/environment/providers/buddy.rb

@@ -70,6 +70,10 @@ module Datadog
               # from docs: The name of the Git BASE branch of the currently run Pull Request
               env["BUDDY_RUN_PR_BASE_BRANCH"]
             end
+
+            def pr_number
+              env["BUDDY_RUN_PR_NO"]
+            end
           end
         end
       end

data/lib/datadog/ci/ext/environment/providers/buildkite.rb

@@ -89,6 +89,12 @@ module Datadog
               env["BUILDKITE_PULL_REQUEST_BASE_BRANCH"]
             end
 
+            def pr_number
+              return nil if env["BUILDKITE_PULL_REQUEST"] == "false"
+
+              env["BUILDKITE_PULL_REQUEST"]
+            end
+
             def ci_env_vars
               {
                 "BUILDKITE_BUILD_ID" => env["BUILDKITE_BUILD_ID"],

data/lib/datadog/ci/ext/environment/providers/circleci.rb

@@ -72,6 +72,10 @@ module Datadog
               env["BUILD_SOURCEVERSIONMESSAGE"]
             end
 
+            def pr_number
+              env["CIRCLE_PR_NUMBER"]
+            end
+
             def ci_env_vars
               {
                 "CIRCLE_WORKFLOW_ID" => env["CIRCLE_WORKFLOW_ID"],

data/lib/datadog/ci/ext/environment/providers/codefresh.rb

@@ -40,6 +40,14 @@ module Datadog
               env["CF_BRANCH"]
             end
 
+            def git_pull_request_base_branch
+              env["CF_PULL_REQUEST_TARGET"]
+            end
+
+            def pr_number
+              env["CF_PULL_REQUEST_NUMBER"]
+            end
+
             def ci_env_vars
               {
                 "CF_BUILD_ID" => env["CF_BUILD_ID"]

data/lib/datadog/ci/ext/environment/providers/drone.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+require "json"
+
+require_relative "base"
+
+module Datadog
+  module CI
+    module Ext
+      module Environment
+        module Providers
+          # Drone CI: https://drone.io/
+          # Environment variables docs: https://docs.drone.io/pipeline/environment/reference/
+          class Drone < Base
+            def self.handles?(env)
+              env.key?("DRONE")
+            end
+
+            def provider_name
+              Provider::DRONE
+            end
+
+            def job_name
+              env["DRONE_STEP_NAME"]
+            end
+
+            def pipeline_number
+              env["DRONE_BUILD_NUMBER"]
+            end
+
+            def pipeline_url
+              env["DRONE_BUILD_LINK"]
+            end
+
+            def stage_name
+              env["DRONE_STAGE_NAME"]
+            end
+
+            def workspace_path
+              env["DRONE_WORKSPACE"]
+            end
+
+            def git_repository_url
+              env["DRONE_GIT_HTTP_URL"]
+            end
+
+            def git_commit_sha
+              env["DRONE_COMMIT_SHA"]
+            end
+
+            def git_branch
+              env["DRONE_BRANCH"]
+            end
+
+            def git_tag
+              env["DRONE_TAG"]
+            end
+
+            def git_commit_author_name
+              env["DRONE_COMMIT_AUTHOR_NAME"]
+            end
+
+            def git_commit_author_email
+              env["DRONE_COMMIT_AUTHOR_EMAIL"]
+            end
+
+            def git_commit_message
+              env["DRONE_COMMIT_MESSAGE"]
+            end
+
+            def git_pull_request_base_branch
+              env["DRONE_TARGET_BRANCH"]
+            end
+
+            def pr_number
+              env["DRONE_PULL_REQUEST"]
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/datadog/ci/ext/environment/providers/github_actions.rb

@@ -77,8 +77,6 @@ module Datadog
             end
 
             def git_pull_request_base_branch
-              return nil if github_event_json.nil?
-
               env["GITHUB_BASE_REF"]
             end
 

data/lib/datadog/ci/ext/environment/providers/gitlab.rb

@@ -108,6 +108,10 @@ module Datadog
               env["CI_MERGE_REQUEST_SOURCE_BRANCH_SHA"]
             end
 
+            def pr_number
+              env["CI_MERGE_REQUEST_IID"]
+            end
+
             private
 
             def extract_name_email

data/lib/datadog/ci/ext/environment/providers/jenkins.rb

@@ -65,6 +65,14 @@ module Datadog
               env["GIT_BRANCH"]
             end
 
+            def git_pull_request_base_branch
+              env["CHANGE_TARGET"]
+            end
+
+            def pr_number
+              env["CHANGE_ID"]
+            end
+
             def ci_env_vars
               {
                 "DD_CUSTOM_TRACE_ID" => env["DD_CUSTOM_TRACE_ID"]

data/lib/datadog/ci/ext/environment/providers/teamcity.rb

@@ -25,6 +25,14 @@ module Datadog
             def job_url
               env["BUILD_URL"]
             end
+
+            def git_pull_request_base_branch
+              env["TEAMCITY_PULLREQUEST_TARGET_BRANCH"]
+            end
+
+            def pr_number
+              env["TEAMCITY_PULLREQUEST_NUMBER"]
+            end
           end
         end
       end

data/lib/datadog/ci/ext/environment/providers/travis.rb

@@ -61,6 +61,28 @@ module Datadog
             def git_commit_message
               env["TRAVIS_COMMIT_MESSAGE"]
             end
+
+            def git_pull_request_base_branch
+              return nil unless pull_request?
+
+              env["TRAVIS_BRANCH"]
+            end
+
+            def git_commit_head_sha
+              return nil unless pull_request?
+
+              env["TRAVIS_PULL_REQUEST_SHA"]
+            end
+
+            def pr_number
+              return nil unless pull_request?
+
+              env["TRAVIS_PULL_REQUEST"]
+            end
+
+            def pull_request?
+              env["TRAVIS_EVENT_TYPE"] == "pull_request"
+            end
           end
         end
       end

data/lib/datadog/ci/ext/environment/providers.rb

@@ -10,6 +10,7 @@ require_relative "providers/buddy"
 require_relative "providers/buildkite"
 require_relative "providers/circleci"
 require_relative "providers/codefresh"
+require_relative "providers/drone"
 require_relative "providers/github_actions"
 require_relative "providers/gitlab"
 require_relative "providers/jenkins"
@@ -34,6 +35,7 @@ module Datadog
             Providers::Buildkite,
             Providers::Circleci,
             Providers::Codefresh,
+            Providers::Drone,
             Providers::GithubActions,
             Providers::Gitlab,
             Providers::Jenkins,

data/lib/datadog/ci/ext/environment.rb

@@ -3,6 +3,7 @@
 require "datadog/core/telemetry/logging"
 
 require_relative "git"
+require_relative "environment/configuration_discrepancy_checker"
 require_relative "environment/extractor"
 
 require_relative "../utils/git"
@@ -24,6 +25,7 @@ module Datadog
         TAG_NODE_LABELS = "ci.node.labels"
         TAG_NODE_NAME = "ci.node.name"
         TAG_CI_ENV_VARS = "_dd.ci.env_vars"
+        TAG_PR_NUMBER = "pr.number"
 
         module Provider
           APPVEYOR = "appveyor"
@@ -35,6 +37,7 @@ module Datadog
           BUILDKITE = "buildkite"
           CIRCLECI = "circleci"
           CODEFRESH = "codefresh"
+          DRONE = "drone"
           GITHUB = "github"
           GITLAB = "gitlab"
           JENKINS = "jenkins"
@@ -51,9 +54,8 @@ module Datadog
           tags = Environment::Extractor.new(env).tags
 
           # If user defined metadata is defined, overwrite
-          tags.merge!(
-            Environment::Extractor.new(env, provider_klass: Providers::UserDefinedTags).tags
-          )
+          user_provided_tags = Environment::Extractor.new(env, provider_klass: Providers::UserDefinedTags).tags
+          tags.merge!(user_provided_tags)
 
           # Fill out tags from local git as fallback
           local_git_tags = Environment::Extractor.new(env, provider_klass: Providers::LocalGit).tags
@@ -61,6 +63,10 @@ module Datadog
             tags[key] ||= value
           end
 
+          # send some telemetry for the cases where git commit sha is overriden
+          discrepancy_checker = ConfigurationDiscrepancyChecker.new(tags, local_git_tags, user_provided_tags)
+          discrepancy_checker.check_for_discrepancies
+
           ensure_post_conditions(tags)
 
           tags

data/lib/datadog/ci/ext/telemetry.rb

@@ -68,6 +68,9 @@ module Datadog
 
         METRIC_TEST_SESSION = "test_session"
 
+        METRIC_GIT_COMMIT_SHA_MATCH = "git.commit_sha_match"
+        METRIC_GIT_COMMIT_SHA_DISCREPANCY = "git.commit_sha_discrepancy"
+
         TAG_TEST_FRAMEWORK = "test_framework"
         TAG_EVENT_TYPE = "event_type"
         TAG_HAS_CODEOWNER = "has_codeowner"

data/lib/datadog/ci/ext/test.rb

@@ -159,6 +159,12 @@ module Datadog
           RETRY_FAILED = "auto_test_retries"
           RETRY_FLAKY_FIXED = "attempt_to_fix"
           RETRY_EXTERNAL = "external"
+
+          DATADOG_RETRY_REASONS = [
+            RETRY_FAILED,
+            RETRY_DETECT_FLAKY,
+            RETRY_FLAKY_FIXED
+          ]
         end
 
         # possible reasons why a test was skipped

data/lib/datadog/ci/remote/library_settings_client.rb

@@ -81,7 +81,7 @@ module Datadog
                 "service" => test_session.service,
                 "env" => @dd_env,
                 "repository_url" => test_session.git_repository_url,
-                "branch" => test_session.git_branch,
+                "branch" => test_session.git_branch || test_session.git_tag,
                 "sha" => test_session.git_commit_sha,
                 "test_level" => Ext::Test::ITR_TEST_SKIPPING_MODE,
                 "configurations" => {

data/lib/datadog/ci/span.rb

@@ -72,6 +72,12 @@ module Datadog
         tracer_span.get_tag(Ext::Test::TAG_STATUS) == Ext::Test::Status::SKIP
       end
 
+      # Returns the current status of the span.
+      # @return [String, nil] the status of the span ("pass", "fail", "skip"), or nil if not set
+      def status
+        tracer_span.get_tag(Ext::Test::TAG_STATUS)
+      end
+
       # Sets the status of the span to "pass".
       # @return [void]
       def passed!
@@ -165,6 +171,12 @@ module Datadog
         tracer_span.get_tag(Ext::Git::TAG_BRANCH)
       end
 
+      # Returns the git tag extracted from the environment.
+      # @return [String, nil] the tag or nil if not set.
+      def git_tag
+        tracer_span.get_tag(Ext::Git::TAG_TAG)
+      end
+
       # Returns the base commit SHA for the pull request, if available.
       # @return [String, nil] the base commit SHA or nil if not set.
       def base_commit_sha

data/lib/datadog/ci/test_retries/component.rb

@@ -86,6 +86,12 @@ module Datadog
           end
         end
 
+        def record_test_started(test_span)
+          # mark test as retry in the beginning
+          # if this is a first execution, the current_retry_driver is nil and this is noop
+          current_retry_driver&.mark_as_retry(test_span)
+        end
+
         def record_test_finished(test_span)
           if current_retry_driver.nil?
             # we always run test at least once and after the first pass create a correct retry driver

data/lib/datadog/ci/test_retries/driver/base.rb

@@ -11,17 +11,22 @@ module Datadog
             false
           end
 
-          def record_retry(test_span)
+          def mark_as_retry(test_span)
             test_span&.set_tag(Ext::Test::TAG_IS_RETRY, "true")
             test_span&.set_tag(Ext::Test::TAG_RETRY_REASON, retry_reason)
           end
 
+          def record_retry(test_span)
+          end
+
           # duration in float seconds
           def record_duration(duration)
           end
 
           def retry_reason
-            "unknown"
+            # we set retry reason to be external (ie retried outside of datadog)
+            # by default if we don't know why the test was retried
+            Ext::Test::RetryReason::RETRY_EXTERNAL
           end
         end
       end

data/lib/datadog/ci/test_visibility/component.rb

@@ -267,6 +267,8 @@ module Datadog
 
           test_optimisation.mark_if_skippable(test)
           test_optimisation.start_coverage(test)
+
+          test_retries.record_test_started(test)
         end
 
         def on_test_session_finished(test_session)
@@ -293,9 +295,8 @@ module Datadog
           test_optimisation.stop_coverage(test)
           test_optimisation.on_test_finished(test, maybe_remote_context)
 
-          Telemetry.event_finished(test)
-
           test_retries.record_test_finished(test)
+          Telemetry.event_finished(test)
         end
 
         def on_after_test_span_finished(tracer_span)

data/lib/datadog/ci/test_visibility/transport.rb

@@ -19,10 +19,6 @@ module Datadog
       class Transport < Datadog::CI::Transport::EventPlatformTransport
         attr_reader :serializers_factory, :dd_env
 
-        def self.log_once
-          @log_once ||= Datadog::Core::Utils::OnlyOnce.new
-        end
-
         def initialize(
           api:,
           dd_env:,
@@ -72,12 +68,17 @@ module Datadog
 
             encoded
           else
-            message = "Invalid event skipped: #{serializer} Errors: #{serializer.validation_errors}"
-            Datadog.logger.warn(message)
-            CI::Transport::Telemetry.endpoint_payload_dropped(1, endpoint: telemetry_endpoint_tag)
+            message = "Event with type #{serializer.event_type}(name=#{serializer.name}) is invalid: #{serializer.validation_errors}"
+
+            if serializer.event_type == "span"
+              # events of type span are often skipped because of missing resource field
+              # (because they are misconfigured in tests context)
+              Datadog.logger.debug(message)
+            else
+              Datadog.logger.warn(message)
+              CI::Transport::Telemetry.endpoint_payload_dropped(1, endpoint: telemetry_endpoint_tag)
 
-            # log invalid message once as error to internal telemetry
-            self.class.log_once.run do
+              # for CI events log all events to internal telemetry
               Core::Telemetry::Logger.error(message)
             end
 

data/lib/datadog/ci/version.rb

@@ -4,7 +4,7 @@ module Datadog
   module CI
     module VERSION
       MAJOR = 1
-      MINOR = 19
+      MINOR = 20
       PATCH = 0
       PRE = nil
       BUILD = nil

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: datadog-ci
 version: !ruby/object:Gem::Version
-  version: 1.19.0
+  version: 1.20.0
 platform: ruby
 authors:
 - Datadog, Inc.
@@ -139,6 +139,7 @@ files:
 - lib/datadog/ci/contrib/parallel_tests/patcher.rb
 - lib/datadog/ci/contrib/patcher.rb
 - lib/datadog/ci/contrib/rspec/configuration/settings.rb
+- lib/datadog/ci/contrib/rspec/documentation_formatter.rb
 - lib/datadog/ci/contrib/rspec/example.rb
 - lib/datadog/ci/contrib/rspec/example_group.rb
 - lib/datadog/ci/contrib/rspec/ext.rb
@@ -166,6 +167,7 @@ files:
 - lib/datadog/ci/contrib/simplecov/result_extractor.rb
 - lib/datadog/ci/ext/app_types.rb
 - lib/datadog/ci/ext/environment.rb
+- lib/datadog/ci/ext/environment/configuration_discrepancy_checker.rb
 - lib/datadog/ci/ext/environment/extractor.rb
 - lib/datadog/ci/ext/environment/providers.rb
 - lib/datadog/ci/ext/environment/providers/appveyor.rb
@@ -178,6 +180,7 @@ files:
 - lib/datadog/ci/ext/environment/providers/buildkite.rb
 - lib/datadog/ci/ext/environment/providers/circleci.rb
 - lib/datadog/ci/ext/environment/providers/codefresh.rb
+- lib/datadog/ci/ext/environment/providers/drone.rb
 - lib/datadog/ci/ext/environment/providers/github_actions.rb
 - lib/datadog/ci/ext/environment/providers/gitlab.rb
 - lib/datadog/ci/ext/environment/providers/jenkins.rb