ddtrace 0.54.2 → 1.2.0

data/docs/ProfilingDevelopment.md

@@ -6,42 +6,39 @@ For a more practical view of getting started with development of `ddtrace`, see
 
 ## Profiling components high-level view
 
-Components below live inside <../lib/ddtrace/profiling>:
+Components below live inside <../lib/datadog/profiling>:
 
-* `Collectors::Stack`: Collects stack trace samples from Ruby threads for both CPU-time (if available) and wall-clock.
+* `Collectors::OldStack`: Collects stack trace samples from Ruby threads for both CPU-time (if available) and wall-clock.
   Runs on its own background thread.
-* `Encoding::Profile`: Encodes gathered data into the pprof format.
+* `Collectors::CodeProvenance`: Collects library metadata to power grouping and categorization of stack traces (e.g. to help distinguish user code, from libraries, from the standard library, etc).
+* `Encoding::Profile::Protobuf`: Encodes gathered data into the pprof format.
 * `Events::Stack`, `Events::StackSample`: Entity classes used to represent stacks.
-* `Ext::CPU`: Monkey patches Ruby's `Thread` with our `Ext::CThread` to enable CPU-time profiling.
-* `Ext::CThread`: Extension used to enable CPU-time profiling via use of Pthread's `getcpuclockid`.
 * `Ext::Forking`: Monkey patches `Kernel#fork`, adding a `Kernel#at_fork` callback mechanism which is used to restore
   profiling abilities after the VM forks (such as re-instrumenting the main thread, and restarting profiler threads).
-* `Pprof::*` (in <../lib/ddtrace/profiling/pprof>): Converts samples captured in the `Recorder` into the pprof format.
-* `Tasks::Setup`: Takes care of loading our extensions/monkey patches to handle fork() and CPU profiling.
-* `Transport::*` (in <../lib/ddtrace/profiling/transport>): Implements transmission of profiling payloads to the Datadog agent
-  or backend.
+* `Pprof::*` (in <../lib/datadog/profiling/pprof>): Used by `Encoding::Profile::Protobuf` to convert samples captured in
+  the `OldRecorder` into the pprof format.
+* `Tasks::Setup`: Takes care of loading our extensions/monkey patches to handle `fork()`.
+* `HttpTransport`: Implements transmission of profiling payloads to the Datadog agent or backend.
 * `TraceIdentifiers::*`: Used to retrieve trace id and span id from tracers, to be used to connect traces to profiles.
 * `BacktraceLocation`: Entity class used to represent an entry in a stack trace.
 * `Buffer`: Bounded buffer used to store profiling events.
-* `Exporter`: Writes profiling data to a given transport.
-* `Flush`: Entity class used to represent metadata for a given profile.
+* `Flush`: Entity class used to represent the payload to be reported for a given profile.
 * `Profiler`: Profiling entry point, which coordinates collectors and a scheduler.
-* `Recorder`: Stores profiling events gathered by `Collector`s.
-* `Scheduler`: Periodically (every 1 minute) takes data from the `Recorder` and pushes them to all configured
-  `Exporter`s. Runs on its own background thread.
+* `OldRecorder`: Stores profiling events gathered by the `Collector::OldStack`. (To be removed after migration to libddprof aggregation)
+* `Exporter`: Gathers data from `OldRecorder` and `Collectors::CodeProvenance` to be reported as a profile.
+* `Scheduler`: Periodically (every 1 minute) takes data from the `Exporter` and pushes them to the configured transport.
+  Runs on its own background thread.
 
 ## Initialization
 
 When started via `ddtracerb exec` (together with `DD_PROFILING_ENABLED=true`), initialization goes through the following
 flow:
 
-1. <../lib/ddtrace/profiling/preload.rb> triggers the creation of the `Datadog.profiler` instance by calling the method
-2. `Datadog.profiler` is handled by `Datadog::Configuration`, which triggers the configuration of `ddtrace` components
-   in `#build_components`
-3. Inside `Datadog::Components`, the `build_profiler` method triggers the execution of the `Tasks::Setup`
-4. The `Setup` task activates our extensions
-    * `Datadog::Profiling::Ext::Forking`
-    * `Datadog::Profiling::Ext::CPU`
+1. <../lib/datadog/profiling/preload.rb> triggers the creation of the profiler instance by calling the method `Datadog::Profiling.start_if_enabled`
+2. Creation of the profiler instance is handled by `Datadog::Configuration`, which triggers the configuration of all
+  `ddtrace` components in `#build_components`
+3. Inside `Datadog::Components`, the `build_profiler` method triggers the execution of the `Tasks::Setup` task
+4. The `Setup` task activates our extensions (`Datadog::Profiling::Ext::Forking`)
 5. Still inside `Datadog::Components`, the `build_profiler` method then creates and wires up the Profiler as such:
     ```asciiflow
             +------------+
@@ -54,27 +51,32 @@ flow:
     +---------+--+  +-+-------+-+
               |       |       |
               v       |       v
-        +-----+-+     |  +----+------+
-        | Stack |     |  | Exporters |
-        +-----+-+     |  +-----------+
+     +--------+-+     |  +----+----------+
+     | OldStack |     |  | HttpTransport |
+     +--------+-+     |  +---------------+
               |       |
               v       v
-            +-+-------+-+
-            | Recorder  |
-            +-----------+
+    +---------+---+  ++---------+
+    | OldRecorder |<-| Exporter |
+    +-------------+  +-+--------+
+                       |
+                       v
+        +--------------+--+
+        | Code Provenance |
+        +-----------------+
     ```
 6. The profiler gets started when `startup!` is called by `Datadog::Configuration` after component creation.
 
 ## Run-time execution
 
-During run-time, the `Scheduler` and the `Collectors::Stack` each execute on their own background thread.
+During run-time, the `Scheduler` and the `Collectors::OldStack` each execute on their own background thread.
 
-The `Collectors::Stack` samples stack traces of threads, capturing both CPU-time (if available) and wall-clock, storing
-them in the `Recorder`.
+The `Collectors::OldStack` samples stack traces of threads, capturing both CPU-time (if available) and wall-clock, storing
+them in the `OldRecorder`.
 
-The `Scheduler` wakes up every 1 minute to flush the results of the `Recorder` into one or more `exporter`s.
-Usually only one exporter is in use. By default, the `Exporter` delegates to the default `Transport::HTTP` transport, which
-takes care of encoding the data and reporting it to the datadog agent (or to the API, when running without an agent).
+The `Scheduler` wakes up every 1 minute to flush the results of the `Exporter` into the transport.
+By default, the `Scheduler` gets created with the default `HttpTransport`, which
+takes care of encoding the data and reporting it to the Datadog agent.
 
 ## How CPU-time profiling works
 
@@ -100,7 +102,7 @@ To further enable filtering of a profile to show only samples related to a given
 profiler is tagged with the `local root span id` and `span id` for the given trace/span.
 
 This is done using the `Datadog::Profiling::TraceIdentifiers::Helper` that retrieves a `root_span_id` and `span_id`, if
-available, from the supported tracers. This helper is called by the `Collectors::Stack` during sampling.
+available, from the supported tracers. This helper is called by the `Collectors::OldStack` during sampling.
 
 Note that if a given trace executes too fast, it's possible that the profiler will not contain any samples for that
 specific trace. Nevertheless, the linking still works and is useful, as it allows users to explore what was going on their

data/docs/PublicApi.md

@@ -0,0 +1,14 @@
+# Public API
+
+`ddtrace` respects [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
+
+Classes, modules, and methods marked as part of the public API will not introduce
+breaking changes outside of a major version release.
+
+Objects that belong to the public API are marked with the `@public_api` YARD documentation tag.
+When navigating [`ddtrace`'s YARD documentation](https://rubydoc.info/gems/ddtrace), public API
+objects will have an explicit banner informing the user that they are part of the public API contract.
+
+Objects not marked with the `@public_api` tag are not part of the public API contract, and thus
+considered internal to `ddtrace`. These objects can receive breaking changes in minor and patch
+releases.